NUnit for Team Build

New Post: Test results not being published

MrKWatkins — Mon, 19 Oct 2009 09:30:06 GMT

Any chance you could give me the NUnit output for those tests? It might well be a problem with the resultant .trx file. I remember fixing a bug in it a while back where test names over a certain length (256) I think couldn't be uploaded to TFS and it failed with the wonderfully useful "General Failure"... So it's probably something in the resultant .trx that TFS doesn't like. If you can get me the NUnit output I'll have a play and see what I can do.

New Post: NCover for Team Build?

MrKWatkins — Mon, 19 Oct 2009 09:26:34 GMT

I've managed to mangle the NCover results into MSCoverage's XML format. However it turns out that that isn't the format you need to upload to TFS; you need the original binary format. So it might take a little while... I am still opimisitic though, even if I have to write something to run MSCoverage instead...

Source code checked in, #29516

MrKWatkins — Sun, 18 Oct 2009 17:12:25 GMT

Added inconclusive tests to the base line.

Commented Issue: Add Support For Inconclusive Tests [5282]

MrKWatkins — Sun, 18 Oct 2009 17:12:02 GMT

Both NUnit and MSTest have a concept of inconclusive tests; make sure they are transformed correctly.
Comments: ** Comment from web user: MrKWatkins **

Turns out there isn't anything like inconclusive="True" in the NUnit output to make this nice and easy. Inconclusive tests have success="False" on their test-case element just like failures; however they lack a child failure element. Details of inconclusive messages (from Assert.Inconclusive(string)) are not included in the XML. I have raised a bug with NUnit (https://bugs.launchpad.net/nunitv2/+bug/454814) as I feel there should be something to indicate definitely that a test is inconclusive. Will wait for the results of that before adding anything to the XSLT.

Source code checked in, #29511

MrKWatkins — Sun, 18 Oct 2009 16:57:23 GMT

Added inconclusive tests to base line.

Created Issue: Add Support For Inconclusive Tests [5282]

MrKWatkins — Sun, 18 Oct 2009 16:35:37 GMT

Both NUnit and MSTest have a concept of inconclusive tests; make sure they are transformed correctly.

Closed Issue: Descriptions Not Transformed [5281]

MrKWatkins — Sun, 18 Oct 2009 16:05:33 GMT

The XSLT doesn't transform NUnit descriptions from the Description named parameter in the TestAttribute. A fix can be found at http://lindsve.net/2009/01/integrating-nunit-test-result-in-the-tfs-data-warehouse/.
Comments: Resolved with changeset 29503.

Closed Issue: Ignored tests are reported as failures [5111]

MrKWatkins — Sun, 18 Oct 2009 16:05:31 GMT

The NUnitToMSTest.xslt transform causes TFS to report ignored/skipped NUnit tests as failures. This thread (http://nunit4teambuild.codeplex.com/Thread/View.aspx?ThreadId=44099) details a change to the XSLT that fixes this issue. Please could this be added to source control for the next build?
Comments: Resolved with changeset 29503.

Source code checked in, #29503

MrKWatkins — Sun, 18 Oct 2009 16:05:27 GMT

Added baseline project to help with developing the XSLT. Added support for descriptions. Fixed bug whereby ignored tests were reported as failures.

New Post: skipped tests appear as failed

MrKWatkins — Sun, 18 Oct 2009 16:01:53 GMT

I'll be checking this in shortly. Expect a new release soon - want to add a few more things first. I've also subscribed to emails for the project so I know in future when people request things...

Updated Release: Version 1.2

MrKWatkins — Sun, 18 Oct 2009 15:59:36 GMT

Bug fix release.

Created Issue: Descriptions Not Transformed [5281]

MrKWatkins — Sun, 18 Oct 2009 15:53:24 GMT

The XSLT doesn't transform NUnit descriptions.

Source code checked in, #29491

MrKWatkins — Sun, 18 Oct 2009 15:05:43 GMT

Added bindings to TFS.

New Post: skipped tests appear as failed

CtrlAltDel — Fri, 09 Oct 2009 03:05:48 GMT

Works for me too - thanks!

I've taken the liberty of creating a work issue so that this can be tracked more easily.

Created Issue: Ignored tests are reported as failures [5111]

CtrlAltDel — Fri, 09 Oct 2009 03:03:55 GMT

New Post: Test results not being published

CtrlAltDel — Fri, 02 Oct 2009 06:09:32 GMT

Curiouser and curiouser - I Ignored all the failing tests, and now the results are published successfully!

New Post: Test results not being published

CtrlAltDel — Thu, 01 Oct 2009 22:46:49 GMT

OK, so using runas to open a command prompt on the server, and copy a file to the TestResults folder works, as does creating a sub-folder in TestResults, so it doesn't appear to be permissions.

Does TFS log its internals anywhere?

New Post: Test results not being published

CtrlAltDel — Thu, 01 Oct 2009 22:08:17 GMT

We were able to publish when we were playing around with our sandbox project when exploring TFS, but when we set up the for-real project, and used the same code in our TFSBuild.proj, it goes pear-shaped.

The problem is that the specified build destination doesn't exist - TFS is trying to copy the TRX file to \\mtfssyd01\Drops\MyProject\DailyBuild_20091001.4\TestResults\07d90a01-0b11-2d00-91c4-30db1d215203\, but only gets as far as creating \\mtfssyd01\Drops\MyProject\DailyBuild_20091001.4\TestResults\. I'll have a go at it from the command-line as you suggest, but do you think that there is any mileage in patching the Publisher to check that the drop folder exists before copying the file in?

New Post: Test results not being published

rbanks54 — Thu, 01 Oct 2009 21:14:05 GMT

Have you successfully published before?

Can you check that you have write access via the share? Start a command prompt as the build service account and try manually copying the trx file to the location shown in the build log and see if it gets through.

The only other thing that may have happened is that you may have a connection drop or time out during the upload of the test results or the disk may have filled. A "general failure" is a little non-specific, and permission problems are normally reported as specific errors, so I'd suspect something at the O/S level went wrong.

New Post: Test results not being published

CtrlAltDel — Thu, 01 Oct 2009 02:16:54 GMT

Just to clarify that, NUnit4TeamBuild is trying to upload the TRX file, but because the path returned by PrepareToPublish does not exist (due to the missing 07d90a01-0a27-0900-91c4-30db1d215203 sub-folder), the file is not uploaded. The question still remains then - how can I diagnose why TFS isn't creating the final subfolder? Or should NUnit4TeamBuild try to create that last folder if it doesn't exist?