At our yearly 1 week Zühlke web camp a couple of us took the opportunity to share experiences with branching and release techniques.
TL DR;
I had this very interesting discussion with some fellow developers and post here a quick Q&A
Question
>>If a unit test fails - do you fix it by looking at the name of the test, or do you fix it by looking at the unit test?
Answer
By looking at the name of the unit test, you know what is broken, as per definition of "Unit test". (A unit test should test only 1 small unit of work)
So you don't need to actually read and understand the test, which saves you HEAPS of time!!
If you have an exception with stack trace you probably start with that 
Question
>>If you have to change the unit test code, do you then update the name, because it is now out of sync?
Answer
You should ask yourself why you change the unit test code
I think this misunderstanding comes from mixing up: "Unit test" and all other tests, like "Integration test", "System test" and "End to End test"
There is a big difference between a "Unit test" and all other tests, see Definition of Unit test
David started a nice article series: The developer banter - this time about testing
"How do you organise your tests. Do you separate your unit tests, integration tests and UI tests into separate projects?
Do you do anything specific to keep track of your tests?
What naming conventions do you use? Do you run them before a check in or is that what the build server is for?"
I am *very* keen on automating tests, so I was looking into unit testing a WCF service.
If I say "unit test", I mean a fast, in-memory, independent and automated test of a functional unit-of-work in the system.
Unit of work is at maximum one method (on my projects)
With thanks to Roy Osherove
Why should we unit test a WCF service?
I have a class with the following code. This looks OK but isn’t…
public class Lot : INotifyPropertyChanged { private string _actualHighestBidder; public string ActualHighestBidder { get { return _actualHighestBidder; } set { if (_actualHighestBidder.Equals(value) != true) { _actualHighestBidder = value; RaisePropertyChanged("ActualHighestBidder"); } } } -- SNIP -- Implementation of INotifyPropertyChanged omitted
Spot the problem in the code above!!!
Scroll down
…
Scroll down
…
The private variable “_actualHighestBidder” is a string and is null.
Calling Equals on a null variable causes an annoying NullReferenceException.
There are 2 ways to solve this
1. Initialize your private backed field
private string _actualHighestBidder = string.Empty;
2. Test for null
public string ActualHighestBidder { get { return _actualHighestBidder; } set { if ((_actualHighestBidder == null || _actualHighestBidder.Equals(value) != true)) { _actualHighestBidder = value; RaisePropertyChanged("ActualHighestBidder"); } } }
I prefer the first method
but
how do we make sure we are not introducing this bugs in the future?
Note
This bug is very annoying, because we get this exception at runtime!
Idea!
Set all properties to some values with reflection
So that we can have a test like this
[TestMethod()]
public void LotProperties_SetAllProperties_NoException()
{
Lot target = new Lot();
target.SetAllPublicProperties();
target.GetAllPublicProperties();
Assert.IsNotNull(target);
}
My implementation of these 2 methods look like the below…
The method SetAllPublicProperties, sets all properties via Reflection
And the method GetAllPublicProperties does the same but tries to read the values!!
using System; using System.Diagnostics; using System.Reflection; using Microsoft.VisualStudio.TestTools.UnitTesting; namespace Tests.Integration.Utils { public static class ReflectionHelper { public static void SetAllPublicProperties(this object target) { Type targetType = target.GetType(); foreach (MemberInfo memberInfo in targetType.GetProperties()) { //Debug.WriteLine("memberInfo: " + memberInfo.ToString()); PropertyInfo propertyInfo = memberInfo as PropertyInfo; if (propertyInfo != null) { //Debug.WriteLine("propertyInfo: field " + propertyInfo.ToString()); if (propertyInfo.CanWrite) { MethodInfo setMethod = propertyInfo.GetSetMethod(); if (propertyInfo.PropertyType == typeof(string)) { setMethod.Invoke(target, new object[] { "String parameter" }); } else if (propertyInfo.PropertyType == typeof(bool)) { setMethod.Invoke(target, new object[] { true }); } else if (propertyInfo.PropertyType == typeof(Single)) { setMethod.Invoke(target, new object[] { 123 }); } else if (propertyInfo.PropertyType == typeof(Single?)) { setMethod.Invoke(target, new object[] { null }); setMethod.Invoke(target, new object[] { (Single?)123 }); } else { Assert.Fail("Generic Poperty setter not implemented for type: " + propertyInfo.PropertyType); } } } } } public static void GetAllPublicProperties(this object target) { Type targetType = target.GetType(); foreach (MemberInfo memberInfo in targetType.GetProperties()) //System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags. { //Debug.WriteLine("memberInfo: " + memberInfo.ToString()); PropertyInfo propertyInfo = memberInfo as PropertyInfo; if (propertyInfo != null) { //Debug.WriteLine("propertyInfo: field " + propertyInfo.ToString()); if (propertyInfo.CanRead) { MethodInfo getMethod = propertyInfo.GetGetMethod(); object readValue = getMethod.Invoke(target, null); Debug.WriteLine("Value from property: " + readValue); } } } } } }
Use it and extend it to all your needed types!
*Updated* 21 March 2011: Updated how "Coded UI tests" finds a control with info from Balagans Blog - How does “Coded UI test” finds a control?
We need Web functionality tests!!! (and a lot of people are already doing them)
Reasons?
Note from Wikipedia
Stress testing: Simulates increased load, more than expected
Load testing: End to end performance testing under anticipated load
I tried the following 3 players (recommendations from work mates, and others..)
All of these 3 tools provide a recording tool, where you can record your browsing actions and generate code from that.
This is important!!!
No one wants to write these tests manually!
Record your session. Copy & paste to Visual Studio. Done
Bug reported here: http://sourceforge.net/tracker/index.php?func=detail&aid=2934193&group_id=193118&atid=944137
Figure: Simulating a user registration with Seleniumselenium.Open("/Public/default.aspx"); selenium.Click("//div[@id='userlogin']/div/div[2]"); selenium.Click("//div[@id='userstatusLoggedOut']/div/a/span"); selenium.WaitForPageToLoad("30000");
selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_FirstName", "o"); selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_LastName", "g"); selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_Email", "peter@gfader.com"); selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_ConfirmEmail", "peter@gfader.com"); selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_Password", "password!"); selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_ConfirmPassword", "password!"); selenium.Click("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm___CustomNav0_StepNextButtonButton"); selenium.WaitForPageToLoad("30000");
selenium.Click("link=Log Out"); selenium.WaitForPageToLoad("30000");
IE ie = new IE("about:blank"); ie.GoTo("http://REMOVED.ssw.com.au/Public"); ie.Div(Find.ByCustom("innertext", "Member Login")).Click(); ie.Link(Find.ByUrl("http://REMOVED.ssw.com.au/Public/Profile/Register.aspx")).Click();
ie.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$FirstName")).TypeText("Peter"); ie.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$LastName")).TypeText("Gfader"); ie.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$Email")).TypeText("petermenuqggfader.com"); ie.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$Password")).TypeText("petermenuqggfader.com"); ie.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$Password")).TypeText("passmenumenuqword!"); ie.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$ConfirmPassword")).TypeText("passmenuqword!"); ie.Button(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$__CustomNav0$StepNextButtonButton")).Click();
Figure: Simulating a user registration with WatiN
I prefer Seleniums: “selenium.Click” instead of the long string and then seeing the “Click()”
// WatiN watin.TextField(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$CreateUserStepContainer$ConfirmPassword")).TypeText("password, f17"); watin.Button(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$__CustomNav0$StepNextButtonButton")).Click(); // Selenium selenium.Type("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm_CreateUserStepContainer_ConfirmPassword", "password"); selenium.Click("ctl10_MainContentArea_ctl00_ctl00_CreateUserForm___CustomNav0_StepNextButtonButton");
Figure: Code comparison of WatiN with Selenium. Enter text in a password field and click on button
Some input control TypeText() was missing
Finding of elements didn't always work E.g. <a href="xx"> <span> My funkty test </span> </a>
Output from the Selenium Java server (see the 2 seconds of start up gap)
19:11:54.341 INFO - Command request: getNewBrowserSession[*chrome, http://REMOVED.ssw.com.au/Public/, ] on session null 19:11:54.341 INFO - creating new remote session 19:11:54.341 INFO - Allocated session 69581bdd207a4684b61f6c368af906f0 for http://REMOVED.ssw.com.au/Public/, launching... 19:11:54.370 INFO - Preparing Firefox profile... 19:11:56.895 INFO - Launching Firefox...
E.g. ctl10_MainContentArea_ctl00_ctl00_SubTotal
That means you can copy & paste from FireBug directly to Visual Studio
string priceSubTotalOnCheckout = selenium.GetText("ctl10_MainContentArea_ctl00_ctl00_SubTotal");
ctl10_MainContentArea_ctl00_ctl00_SubTotal
needs to be converted to
ctl10$MainContentArea$ctl00$ctl00$SubTotal
string resultText = browser.Element(Find.ByName("ctl10$MainContentArea$ctl00$ctl00$SubTotal")).Text;
Note
Figure: Selenium is coming from the Java world, as you can see this here. No properties :-) --> GetXXX
What about Visual Studio 2010 and Test & Lab Manager
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.TestCase", "http://win-peterG:8080/tfs/DefaultCollection;Peter-Gfader-WebTest", "140", DataAccessMethod.Sequential), TestMethod] public void CodedUITestMethod1() { this.UIMap.Openbrowser(); this.UIMap.Gotostartpage();
this.UIMap.ClickonRegisterNew(); this.UIMap.EnterdetailsfirstnamelastnameemailpasswordParams.Ctl10MainContentAreaEditText = TestContext.DataRow["firstname"].ToString(); this.UIMap.EnterdetailsfirstnamelastnameemailpasswordParams.Ctl10MainContentAreaEdit1Text = TestContext.DataRow["lastname"].ToString(); this.UIMap.EnterdetailsfirstnamelastnameemailpasswordParams.Ctl10MainContentAreaEdit2Text1 = TestContext.DataRow["email"].ToString(); this.UIMap.EnterdetailsfirstnamelastnameemailpasswordParams.Ctl10MainContentAreaEdit3Text1 = TestContext.DataRow["email"].ToString(); this.UIMap.EnterdetailsfirstnamelastnameemailpasswordParams.Ctl10MainContentAreaEdit4Text = TestContext.DataRow["password"].ToString(); this.UIMap.EnterdetailsfirstnamelastnameemailpasswordParams.Ctl10MainContentAreaEdit5Text = TestContext.DataRow["password"].ToString(); this.UIMap.Enterdetailsfirstnamelastnameemailpassword(); this.UIMap.ClickonRegister();
this.UIMap.Searchforbag(); this.UIMap.Add1stbagfoundtoshoppingcart();
this.UIMap.Gotoshoppingcart();
this.UIMap.DoCheckout();
this.UIMap.EnterAddressdetails(); this.UIMap.Placeorder(); }
Figure: Code generated by VS2010 for a registration of a new user, searching for a bag, adding this bad and going through the checkout
UIMap is a huge file generated for you that does all the magic behind the scenes, like finding controls, clicking buttons, …
UIMap: Strongly typed representation of controls
How is Visual Studio finding controls?
public HtmlInputButton RegisterButton { get { if ((this.mRegisterButton == null)) { this.mRegisterButton = new HtmlInputButton(this); #region Search Criteria this.mRegisterButton.SearchProperties[HtmlProperties.Button.Id] = "ctl10_MainContentArea_ctl00_ctl00_CreateUserForm___CustomNav0_StepNextButtonButton"; this.mRegisterButton.SearchProperties[HtmlProperties.Button.Name] = "ctl10$MainContentArea$ctl00$ctl00$CreateUserForm$__CustomNav0$StepNextButtonButton"; this.mRegisterButton.FilterProperties[HtmlProperties.Button.DisplayText] = "Register"; this.mRegisterButton.FilterProperties[HtmlProperties.Button.Type] = "submit"; this.mRegisterButton.FilterProperties[HtmlProperties.Button.Title] = null; this.mRegisterButton.FilterProperties[HtmlProperties.Button.Class] = null; this.mRegisterButton.FilterProperties[HtmlProperties.Button.ControlDefinition] = "id=ctl10_MainContentArea_ctl00_ctl00_Cre"; this.mRegisterButton.FilterProperties[HtmlProperties.Button.TagInstance] = "40"; this.mRegisterButton.WindowTitles.Add("Peter Gfader | Sign Up - Windows Internet Explorer"); #endregion } return this.mRegisterButton; } }
I would probably change that to
public HtmlInputButton RegisterButton { get { if ((this.mRegisterButton == null)) { this.mRegisterButton = new HtmlInputButton(this); #region Search Criteria this.mRegisterButton.FilterProperties[HtmlProperties.Button.DisplayText] = "Register"; this.mRegisterButton.FilterProperties[HtmlProperties.Button.Type] = "submit"; #endregion } return this.mRegisterButton; } }
Note: This would fail if there are 2 buttons with "Register" and of type "Submit" on the page. Which is excellent!
Can VS2010 simulate a FireFox?
See this image for details about the VS2010 platform support
Figure: VS2010 platform support from http://videos.visitmix.com/MIX09/T83M
WebAII looks very promising. I will definitely have a closer look on that tool in the next weeks, months…
Conclusion: Please count the 
Other interesting opinions about Watin and Selenium on testdrivendeveloper
PERSONAL
I would not automate the UI too much because that makes your test very fragile for future changes. That you will have sooner or later!
Create a coded UI test for a whole user interaction with your web app
e.g. Register, Login, Browse, Find, Add to Shopping cart, Update Billing Address, Update Shipping Address, Checkout, Pay, See Order successful
Makes your UI test very fragile!
But could be a “Done” criteria for your SCRUM team: "An acceptance UI test must be written for each user story"
1. Create a coded UI tests for single screens
e.g. Logon screen:
Assert "Username", "Password" boxes and "Login" button are there
2. Create a functional test for testing the actual Login functionality.
I hope that we can finally do row tests ([RowTest] attribute) in Visual Studio 2010, like we can in mbUnit and nUnit.
That would make our data driven unit tests so much easier!!!
But nothing to find on the official VS2010 webpage or Soma’s Blog Search for Rowtest
[TestMethod]
[DeploymentItem(@"TestData\validConditions.txt")]
[DeploymentItem(@"TestData\schema.ini")]
[DataSource("System.Data.OleDb", @"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=|DataDirectory|;Extended Properties='text;FMT=TabDelimited;HDR=YES'", @"validConditions#txt", DataAccessMethod.Sequential)]
public void ValidateRulesEntry_Valid_ValidConditionsFromFile()
{
string condition = TestContext.DataRow[0].ToString();
string returnMessage;
bool successFul = CodeParserTryParseCondition(condition, "No error", out returnMessage);
Assert.IsTrue(successFul, string.Format("Condition: {0}. Error: {1}.", condition, returnMessage));
}
BAD: Input data is in a text file, to view that data I have to open the text file
[TestMethod]
[RowTest]
[Row("Target.IsDirty")]
[Row("Target.IsValid")]
[Row("Target.Description.Contains(""Test"")")]
[Row("(1 != 2)")]
public void ValidateRulesEntry_Valid_ValidConditionsFromFile()
{
string condition = TestContext.DataRow[0].ToString();
string returnMessage;
bool successFul = CodeParserTryParseCondition(condition, "No error", out returnMessage);
Assert.IsTrue(successFul, string.Format("Condition: {0}. Error: {1}.", condition, returnMessage));
}
GOOD: Input data for unit test is attached to unit test itself --> easy to read, understand!
How to create simple data-driven-unit-tests with files from directories
Create the following folders in your solution
\files\valid\
\files\invalid\
copy in those folders your single files to test.
e.g. copy your valid testfiles (that are valid items for a test) to \files\valid\
and yout invalid testfiles to \files\invalid\
Problem
there is no dataprovider (DataSource) for directories (not in vs2005, in vs2008 im not sure)
create in each folder a text file.
\files\valid\data.txt
\files\invalid\data.txt
these text files will contain a directory listing with the single files!
add to your unit test
[DeploymentItem(@"files\invalid\data.txt", @"invalid")]
[DataSource("System.Data.OleDb", @"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=.\files\invalid;Extended Properties='text;FMT=Delimited'", @"data.txt", DataAccessMethod.Sequential)]
string fileName = TestContext.DataRow[0].ToString();
[AssemblyInitialize]
public static void CreateOutputDataTxtFiles(TestContext context)
{
string startDir = Path.Combine(Path.GetDirectoryName(Path.GetDirectoryName(context.TestDir)), @"Siag.XXX.BusinessLogic.Test");
string addDir = @"files\valid";
WriteFilePathsToData(Path.Combine(startDir, addDir), "*.dat");
addDir = @"files\invalid";
WriteFilePathsToData(Path.Combine(startDir, addDir), "*.dat");
}
private static void WriteFilePathsToData(string startDir, string searchPattern)
{
StringBuilder result = new StringBuilder();
foreach (string fileName in Directory.GetFiles(startDir, searchPattern))
{
result.AppendLine(fileName);
}
// add additional directories here
string fileNameData = Path.Combine(startDir, @"data.txt");
File.SetAttributes(fileNameData, FileAttributes.Normal);
File.WriteAllText(fileNameData, result.ToString());
}
i wanted to do unit tests with a text file as input
1 line contains: entity serialized as string, entityName and other parameter
e.g.
1|12|123123,123123,143123|54| Riga1_GPS true
line is delimited by tab
#1
create file in your test project
righevalid.txt
File Properties: --> Copy to Output Directory --> copy always
#2
create file in your test project
schema.ini
File Properties: --> Copy to Output Directory --> copy always
#3
content of schema.ini
[righevalid.txt] Format=TabDelimited
#4
create testmethod
#5
add following attributes to testmethod
[TestMethod()] [DeploymentItem(@"righevalid.txt")] [DeploymentItem(@"schema.ini")] [DataSource("System.Data.OleDb", @"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=.;Extended Properties='text;FMT=TabDelimited;HDR=YES'", @"righevalid.txt", DataAccessMethod.Sequential)] public void LoadValidRigaTest() { } Note DeploymentItem copies files to Test Run Directory
DataSource is a special OleDb Usage with txt Files
#6
use TestContext.DataRow[index] to access your single columns (line) of the text file
TestContext.DataRow[0] = col1 and so on
#7
Run Test
#8
see output Passed or Failed
Doubleclick Result you can see the result of the single lines!
GREAT!