skipped tests appear as failed

Jan 13, 2009 at 4:21 PM
TeamBuild correctly reports that 5 tests failed. But when downloading the test-results into VS a whole bunch more tests appear as having failed. After research we found that these other tests actually had the IGNORE attribute applied or a CATEGORY which has been excluded by the commandline.

This makes it hard to find the tests which _really_ failed.
Feb 6, 2009 at 11:12 PM
The parsing done in the xslt does not account for Executed="False" in the nunit results so it treats anything that was not a success as a failure.

If you modify the xslt node for outcome, we can differentiate the non executed tests:

<style type="text/css"> <!-- body { line-height: normal; margin-top: 4px; margin-right: 4px; font-variant: normal; margin-bottom: 1px; margin-left: 4px } p { margin-top: 0; margin-bottom: 0 } --> </style>
Change the outcome attribute to this:
 
      <xsl:attribute name="outcome">
       <xsl:choose >
        <xsl:when test="@executed='True'">
         <xsl:choose >
          <xsl:when test="@success='True'">
           <xsl:value-of select="'Passed'"/>
          </xsl:when>
          <xsl:otherwise>
           <xsl:value-of select="'Failed'"/>
          </xsl:otherwise>
         </xsl:choose>
        </xsl:when>
        <xsl:otherwise>
         <xsl:value-of select="'NotExecuted'"/>
        </xsl:otherwise>
       </xsl:choose>
      </xsl:attribute>

If you look at the TFSWarehouse.Outcome table, it lists possible values.  I found via trial and error that the TRX loader was happy when using the term with the spaces trimmed out.

If you have other cases for nunit result values for success and executed, you can follow a similar pattern of choose/when tests to generate the appropriate outcome values.  The xslt above suffices for our situation for the moment so that's as far as I took it.

Also note that it is processing each test-case node in the nunit results.

Enjoy.
Toby


Mar 25, 2009 at 10:34 AM
thanks! I have included the changes in my file. It would be nice someone could release a new version with these fixes, though!
Oct 9, 2009 at 4:05 AM

Works for me too - thanks!

I've taken the liberty of creating a work issue so that this can be tracked more easily.

Oct 18, 2009 at 5:01 PM

I'll be checking this in shortly. Expect a new release soon - want to add a few more things first. I've also subscribed to emails for the project so I know in future when people request things...