Running NUnit Tests in TFS 2010 Continuous Build Environment

I tried different ways to run nunit on our build server.
From
"NUnit for Team Build", to "executing nunit-console on cmd" and eventually customizing our project files…
None of those methods made me happy.
image
Figure: Sad - Broken builds and mucking around with XML make dev's unhappy


The best way (and most important *working* way) is by customizing the build template.

By using the customization from Shawn Wallace I was able to run nunit tests on the build server and get a nice output if a test fails.
http://blog.shawnewallace.com/2011/02/running-nunit-tests-in-tfs-2010.html

I am writing this up, because I have 3 comments on his build template change

  1. I removed all 'sap:' attributes from his XML in order to get it working on our TFS 2010
  2. Make sure to customize the path of the nunit-console runner
      Search for nunit-console in the template and replace the path
  3. Add the below XML before the 1st occurrence of '<If Condition="[Not DisableTests]"'

 

The resulting template looks like this

                  
                          <!-- *********** START:  Unit Testing *********** -->
                        
                        
                          <If Condition="[Not DisableTests]" DisplayName="If Not DisableTests" >
                            <If.Then>
                              <Sequence DisplayName="Run Tests" >
                                <Sequence.Variables>
                                  <Variable x:TypeArguments="scg:IEnumerable(x:String)" Name="testAssemblies" />
                                  <Variable x:TypeArguments="mtbwa:TestAssemblySpec" Name="testAssembly" />
                                  <Variable x:TypeArguments="mtbwa:TestAssemblySpec" Name="testAssemblySpec" />
                                </Sequence.Variables>
                                                                
                                <ForEach x:TypeArguments="mtbwa:TestSpec" DisplayName="ForEach&lt;TestSpec&gt;"  Values="[TestSpecs]">
                                  <ActivityAction x:TypeArguments="mtbwa:TestSpec">
                                    <ActivityAction.Argument>
                                      <DelegateInArgument x:TypeArguments="mtbwa:TestSpec" Name="testSpec" />
                                    </ActivityAction.Argument>
                                    <Sequence >
                                      <Sequence.Variables>
                                        <Variable x:TypeArguments="x:String" Name="testAssemblyMatchPattern" />
                                      </Sequence.Variables>
                                      <mtbwa:WriteBuildMessage DisplayName="Write outputDirectory"  Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[&quot;outputDirectory is: &quot; + outputDirectory]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
                                      <Cast x:TypeArguments="mtbwa:TestSpec, mtbwa:TestAssemblySpec"  Operand="[testSpec]" Result="[testAssemblySpec]" />
                                      <mtbwa:WriteBuildMessage DisplayName="Write testAssemblySpec.AssemblyFileSpec" Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[&quot;testAssemblySpec.AssemblyFileSpec is: &quot; + testAssemblySpec.AssemblyFileSpec]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
                                      <Assign >
                                        <Assign.To>
                                          <OutArgument x:TypeArguments="x:String">[testAssemblyMatchPattern]</OutArgument>
                                        </Assign.To>
                                        <Assign.Value>
                                          <InArgument x:TypeArguments="x:String">[String.Format("{0}\{1}", outputDirectory, testAssemblySpec.AssemblyFileSpec)]</InArgument>
                                        </Assign.Value>
                                      </Assign>
                                      <mtbwa:WriteBuildMessage  Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[&quot;testAssemblyMatchPattern: &quot; + testAssemblyMatchPattern]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
                                      <mtbwa:FindMatchingFiles DisplayName="Find Test Assemblies --&gt; testAssemblies"  MatchPattern="[testAssemblyMatchPattern]" Result="[testAssemblies]" />
                                      
                                      <!-- Assign value to treatTestFailureAsBuildFailure -->
                                      
                                      <Assign >
                                        <Assign.To>
                                          <OutArgument x:TypeArguments="x:Boolean">[treatTestFailureAsBuildFailure]</OutArgument>
                                        </Assign.To>
                                        <Assign.Value>
                                          <InArgument x:TypeArguments="x:Boolean">[testSpec.FailBuildOnFailure]</InArgument>
                                        </Assign.Value>
                                      </Assign>
                                      <If Condition="[testAssemblies.Count() &gt; 0]" DisplayName="If Test Assemblies Found" >
                                        <If.Then>
                                          <Sequence DisplayName="Run NUnit" >
                                            <ForEach x:TypeArguments="x:String" DisplayName="Loop through all Test Assemblies" Values="[testAssemblies]">
                                              <ActivityAction x:TypeArguments="x:String">
                                                <ActivityAction.Argument>
                                                  <DelegateInArgument x:TypeArguments="x:String" Name="item" />
                                                </ActivityAction.Argument>
                                                <Sequence >
                                                  <Sequence.Variables>
                                                    <Variable x:TypeArguments="x:String" Name="assemblyName" />
                                                    <Variable x:TypeArguments="x:String" Name="testResultXmlPath" />
                                                    <Variable x:TypeArguments="x:String" Name="variable1" />
                                                  </Sequence.Variables>
                                                  <Assign DisplayName="Get Assembly Name" >
                                                    <Assign.To>
                                                      <OutArgument x:TypeArguments="x:String">[assemblyName]</OutArgument>
                                                    </Assign.To>
                                                    <Assign.Value>
                                                      <InArgument x:TypeArguments="x:String">[item.Substring(item.LastIndexOf("\") + 1)]</InArgument>
                                                    </Assign.Value>
                                                  </Assign>
                                                  <Assign DisplayName="Get Test Result Path" >
                                                    <Assign.To>
                                                      <OutArgument x:TypeArguments="x:String">[testResultXmlPath]</OutArgument>
                                                    </Assign.To>
                                                    <Assign.Value>
                                                      <InArgument x:TypeArguments="x:String">[String.Format("{0}\{1}.ResultXml.xml", logFileDropLocation, assemblyName)]</InArgument>
                                                    </Assign.Value>
                                                  </Assign>
                                                  <mtbwa:WriteBuildMessage DisplayName="Write testResultXmlPath"  Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[&quot;testResultXmlPath: &quot; + testResultXmlPath]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
                                                  
                                                 <!-- Execute NUnit tests -->
                                                  
                                                  <mtbwa:InvokeProcess Arguments="[String.Format(&quot;&quot;&quot;{0}&quot;&quot; /xml:&quot;&quot;{1}&quot;&quot; /nologo /nodots&quot;, item, testResultXmlPath)]" DisplayName="Execute NUnit" FileName="c:\Program Files (x86)\NUnit 2.5.10\bin\net-2.0\nunit-console.exe" >
                                                    <mtbwa:InvokeProcess.ErrorDataReceived>
                                                      <ActivityAction x:TypeArguments="x:String">
                                                        <ActivityAction.Argument>
                                                          <DelegateInArgument x:TypeArguments="x:String" Name="errOutput" />
                                                        </ActivityAction.Argument>
                                                        <mtbwa:WriteBuildError  Message="[errOutput]" />
                                                      </ActivityAction>
                                                    </mtbwa:InvokeProcess.ErrorDataReceived>
                                                    <mtbwa:InvokeProcess.OutputDataReceived>
                                                      <ActivityAction x:TypeArguments="x:String">
                                                        <ActivityAction.Argument>
                                                          <DelegateInArgument x:TypeArguments="x:String" Name="stdOutput" />
                                                        </ActivityAction.Argument>
                                                        <Sequence >
                                                          
                                                          <mtbwa:WriteBuildMessage  Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[&#x9;  stdOutput]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
                                                          
                                                          <!-- if test output contains errors or failures -->
                                                          
                                                          <If Condition="[(Not stdOutput.Contains(&quot;Failures: 0&quot;) And stdOutput.Contains(&quot;Failures:&quot;)) Or&#xA;(Not stdOutput.Contains(&quot;Errors: 0&quot;) And stdOutput.Contains(&quot;Errors:&quot;))]" DisplayName="If there were failed tests" >
                                                            <If.Then>
                                                              <Sequence >
                                                                
                                                                <!-- put build test status in failed state -->
                                                                
                                                                <Assign >
                                                                  <Assign.To>
                                                                    <OutArgument x:TypeArguments="mtbc:BuildPhaseStatus">[BuildDetail.TestStatus]</OutArgument>
                                                                  </Assign.To>
                                                                  <Assign.Value>
                                                                    <InArgument x:TypeArguments="mtbc:BuildPhaseStatus">[Microsoft.TeamFoundation.Build.Client.BuildPhaseStatus.Failed]</InArgument>
                                                                  </Assign.Value>
                                                                </Assign>
                                                                
                                                                <!-- if treatTestFailureAsBuildFailure create build error, else create build warning -->
                                                                
                                                                <If Condition="[treatTestFailureAsBuildFailure]" >
                                                                  <If.Then>
                                                                    <mtbwa:WriteBuildError  Message="[assemblyName &amp; &quot; -&gt; &quot; &amp; stdOutput]" />
                                                                  </If.Then>
                                                                  <If.Else>
                                                                    <mtbwa:WriteBuildWarning  Message="[assemblyName &amp; &quot; -&gt; &quot; &amp; stdOutput]" />
                                                                  </If.Else>
                                                                </If>
                                                              </Sequence>
                                                            </If.Then>
                                                          </If>
                                                        </Sequence>
                                                      </ActivityAction>
                                                    </mtbwa:InvokeProcess.OutputDataReceived>
                                                  </mtbwa:InvokeProcess>
                                                  
                                                  <!-- end EXECUTE NUnit tests -->
                                                  
                                                </Sequence>
                                              </ActivityAction>
                                            </ForEach>
                                          </Sequence>
                                        </If.Then>
                                      </If>
                                    </Sequence>
                                  </ActivityAction>
                                </ForEach>
                              </Sequence>
                            </If.Then>
                          </If>
                        
                        
                          <!-- *********** END:  Unit Testing *********** -->
                        

 

Thanks Shawn!!

Latest Posts

Popular Posts