I am a big fan of "measuring", because what you don't measure you can't improve.
Measuring over time allows us to establish a trend and the trend is often more interesting that the hard numbers.
Trends that might be interesting are:
- Code Coverage: I don't care about 55.3%, 65.7% or even 85% Code Coverage.
I am more interested in the trend over time: How did Code Coverage change since last week? Since last month?
If it changes too much, maybe we have problem…
BTW: If you have > 95% Code Coverage, send me an email… Keen to see your code and your tests! - Max Request/second in stress test: What's the maximum number of requests that we can handle per second in our web application? Did it increase or decrease over time?
Jason Stangroome blogged about a quick XAML extension of the default build template, that fails a build when the warning count increases.
This is great! Make sure to grab it and put it into your build.
And say Thanks to @jstangroome
I just added an output message to show me the current values (see the yellow). Since it was not so easy to do, I include it here.
<!-- Insert this Sequence as one of the last children of the Sequence with DisplayName="Compile and Test" --> <Sequence DisplayName="Fail Build if Warnings increase"> <Sequence.Variables> <Variable x:TypeArguments="mtbc:IBuildDetail" Name="LastBuildDetail" /> <Variable x:TypeArguments="x:Int32" Name="LastWarningCount" /> <Variable x:TypeArguments="x:Int32" Name="WarningCount" /> </Sequence.Variables> <Assign x:TypeArguments="mtbc:IBuildDetail" To="[LastBuildDetail]" Value="[BuildDetail.BuildServer.GetBuild(BuildDetail.BuildDefinition.LastBuildUri)]" /> <Assign x:TypeArguments="x:Int32" To="[LastWarningCount]" Value="[Microsoft.TeamFoundation.Build.Client.InformationNodeConverters.GetBuildWarnings(LastBuildDetail).Count]" /> <Assign x:TypeArguments="x:Int32" To="[WarningCount]" Value="[Microsoft.TeamFoundation.Build.Client.InformationNodeConverters.GetBuildWarnings(BuildDetail).Count]" /> <mtbwa:WriteBuildMessage Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[String.Format("LastWarningCount: {0}. WarningCount: {1} ", LastWarningCount, WarningCount)]" /> <If Condition="[WarningCount > LastWarningCount]"> <If.Then> <Sequence> <mtbwa:WriteBuildError Message="This build has more warnings than the last build. Focus on fixing more build warnings than are introduced." /> <!-- The follow SetBuildProperties line will mark the build as Failed instead of just Partially Successful if the warning count has increased. --> <mtbwa:SetBuildProperties PropertiesToSet="Status" Status="Failed" /> </Sequence> </If.Then> </If> </Sequence>
Figure: Fail build, when Warning count increased since last build
Figure: Build failed cause too many warning
Figure: "View Log" shows you the details: Warnings increased from 0 to 20
Thanks Jason!!
Now I am off to write some XAML for my code coverage numbers, and max req/sec…
CU later
8 comments:
One idea I read somewhere is to fail the build if the number of TODOs rises above some limit (which you gradually decrease over time).
Might be interesting to do the same thing with code metrics e.g. cyclomatic complexity.
@Kent
Good idea!
The count of "TODO" in all files is just a little harder get...
CC would be nice to have as well!
There are many sub-sequences, where exactly is the snippet supposed to go? Can you provide a line number and/or the code below or above?
@Anonymous
See the comment in code:
- - Insert this Sequence as one of the last children of the Sequence with DisplayName="Compile and Test" - -
Apparently, there is a bug in this code. It compares the warning count not considering whether the previous build was successful ;)
@Unit01
Good catch... There are definitely a couple of points where we can improve this build template.
Do you have an improved version?
Is there any way to identify the user who introduced the build warnings and send a mail to the distribution list?
Hi Seetharaam
>> identify the user who introduced the build warnings
With "Build on each Checkin" you can find the "culprit" easily.
With "Rolling builds" it is a bit harder since more checkins get queued up.
>> send a mail to the distribution list?
Use TFS Alerts to do that. Google for your TFS version and "email alert"
Post a Comment