TFS 2008

The basic process NUnitTFS follows for TFS 2008 runs thus:
  1. Transform NUnit output. An XSL transform is used to take the output from NUnit and transform it into MSTest's .trx format. This transformation generates a unique ID on the root attribute known as the run ID.
  2. Add build information to .trx. An XML element called <Build> is added to the .trx file after the <Times> element. This has two attributes, flavor and platform, that contain details about the build.
  3. Retrieve build ID from TFS. A call to the QueryBuilds method on the TFS's BuildServiceV2 web service is made to retrieve details about the build. From the returned information the unique ID for the build is extracted.
  4. Retrieve the publish service version. A call to the GetVersion method on TFS's PublishTestResultsBuildService2V1 web service. Note that this version isn't used anywhere! The MSTest executable peformed this call when publishing test results so I thought I NUnitTFS had better do it too...
  5. Call PrepareToPublish. The PrepareToPublish method on TFS's PublishTestResultsBuildService2V1 web service is called. We pass the build ID and run ID and TFS returns a directory (on the TFS server) to save the .trx file to.
  6. Save the .trx file. The .trx file is saved in the directory TFS gave us. The name can be specified by the -outputFile command line parameter; the default value is NUnitOutput.trx.
  7. Call PublishRun. The PublishRun method on TFS's PublishTestResultsBuildService2V1 web service is called. We pass the build ID, run ID and name of the file we've saved. TFS then grabs the file and imports it. If something goes wrong at this stage you'll normally get the incredibly helpful Publish_GeneralFailure error.

TFS 2010

The process has changed massively with TFS 2010. The main difference is that there is no copying files around the network; files are uploaded via a web request, which is much nicer. It also now seems to be the case that an XML file has to be uploaded for each test, which isn't ideal...
  1. Transform NUnit output. An XSL transform is used to take the output from NUnit and transform it into MSTest's .trx format. This transformation generates a unique ID on the root attribute known as the run ID. Whilst TFS 2010 has a more 'objecty' approach (i.e. there is a lot more passing of details via SOAP objects) the code interally still works from this .trx file. This is so you can transform the NUnit (or other test framwork!) output however you want using the -x command line switch and a custom XSLT yet still get the file uploading.
  2. Add build information to .trx. An XML element called <Build> is added to the .trx file after the <Times> element. This has two attributes, flavor and platform, that contain details about the build.
  3. Retrieve build information from TFS. A call to the QueryBuilds method on the BuildServiceV3 web service (/Build/v3.0/BuildService.asmx) is made to retrieve details about the build. From the returned information we use the build's URI to link test runs to it.
  4. Check test run doesn't exist. We now check that the test run doesn't already exist via a call to the QueryTestRunByTmiRunId method of the TestResults web service, (/TestManagement/v1.0/TestResults.asmx) passing in the run ID.
  5. Get current user's details. Test runs have to be associated with a user so we make a call to the ReadIdentities method of the IdentityManagementServiceV3 web service (/Services/v3.0/IdentityManagementService.asmx) to get the TeamFoundationId for the current user. (i.e. whatever user your build server is running as)
  6. Create test run. A test run is now created with a call to the CreateTestRun method on the TestResults web service. This is initially created with few details and a state of NotStarted.
  7. Create test results. Basic details of the tests are pulled out of the .trx and test results are created for the run with a call to the CreateTestResults method of the TestResults web service.
  8. Update test run with TMI run and assemblies. TFS 2010 requires an XML file called tmiRun.tr to be uploaded an attached to the test run, along with the assemblies being tested. tmiRun.tr is a stripped down version of the .trx file with the <TestDefinitions> and <Results> elements removed. We make a call to the UpdateTestRun method of the TestResults service, updating the state of the test run to InProgress and giving details of the files to be uploaded. We are given a series of attachment IDs in return. We then upload each attachment by POSTing the file to the file upload handler (/TestManagement/v1.0/AttachmentUpload.ashx) passing the test run details and attachment ID in the form data.
  9. Update test results with details and tmiResult.tr. We then make a call to the UpdateTestResults method of the TestResults service, updating each test result to a state of Completed and including the outcome plus and error messages. We also give details of an attachment for each result, tmiResult.tr, which is another stripped down version of the .trx containing just the <TestDefinitions> and <Results> elements for that test result. TFS returns us some attachment IDs which we then use to upload all the tmiResult.tr_ files via the file upload handler.

Last edited May 23, 2010 at 7:01 PM by MrKWatkins, version 5

Comments

No comments yet.