Developer Blog Banter - How do you test your applications?

David started a nice article series: The developer banter - this time about testing

"How do you organise your tests. Do you separate your unit tests, integration tests and UI tests into separate projects?
Do you do anything specific to keep track of your tests?
What naming conventions do you use? Do you run them before a check in or is that what the build server is for?"

Interesting question @DavidBurela

My way of testing changes all the time and currently I am doing it like described below.


Goal of my tests

ASAP something was changed that was not expected as application behavior, we get an alert --> email

I use TFS (msbuild, mstest) as my CI server, with builds on each check in.
I try to write my tests as expectations on how the application works.
I think everyone agrees to automate tests  these days…



I organize my tests similar to Liam, by splitting them into slow and fast but I name with "blurring" names like:

  • Unit tests (fast) and
  • Integration tests (slow and not independent)



Naming of projects

I name my test projects like that

  • Tests.Unit.SalaryService
  • Tests.Integration.SalaryService

Figure: Sample solution structure with a WCF service, a MVC web app, Silverlight clients and some Common projects and even a Silverlight Unit test project
On this blog post I describe how we test our WCF service



Naming of test methods

Regarding naming of my tests I use different approaches:

  1. SUT_Scenario_ExpectedResult    as per Roy Osherove - Naming standards for unit tests
  2. SUT_Context_FeatureName


Examples for those tests

1. SUT_Scenario_ExpectedResult

        public void UserConnectionHelper_StoreNewUserConnection_ReturnsNewObject()
            UserConnectionHelper userConnectionHelper = new UserConnectionHelper();

            var result = 
            userConnectionHelper.StoreNewUserConnection(IrrelevantString, IrrelevantString, IrrelevantLong,
                IrrelevantString, IrrelevantBool, IrrelevantBool, IrrelevantString, IrrelevantBool, IrrelevantBool);


2. SUT_Context_FeatureName

        public void SequentialAuction_LimitBids__OnlyBidFullIncrements()
            FakeLot1.ActualHighestBidder = Bob;
            FakeLot1.ActualPrice = 100;
            FakeLot1.BidIncrement = 50;

            Anna.enters_LimitBid(Lot1, 110);


            Expect_HoldingBidder_AndPrice(Bob, Lot1, 100);


Note: I am not using a BDD framework (yet), but the idea is that our business people can read those tests easily and we can discuss the different scenarios.

I have a [UseForDocumentation] attribute, that I use to automatically generating HTML documentation from these tests with T4 templates. Again I should probably use a BDD framework and let the framework do that for me.
When I generate that documentation, I basically strip out all C# noise like (".", ";", etc) and apply some nice CSS formatting.

This is currently a work in progress, and highly depending from the other devs and our Product owner.
If it works, we keep it ;-)


My current favorite mocking framework is moq over Rhinomocks. I didn't try nSubstitue yet (the syntax looks very nice).



How do we write tests?

Some Test first, some Test after, some TDD style, depending on who and what.

Rule of thumb

  • We try to introduce new features now only if there is enough test coverage around that area.
  • We approach new core business functionality TDD style.


Specials about our testing

#1 Testing Silverlight

We have 1 big problem though: Testing automagically Silverlight clients
We have some tests done in the Silverlight Unit test framework, but those tests are not running on the build server, which means they are almost useless.


#2 Stresstest over night

In the current project we focus a lot on performance of the system (Silverlight clients talking to WCF service). So what we do is:
Every night we run a stresstest.
I start up perfmon with some interesting counters and log those every 15seconds to a database. Then we start up 500 console clients + some Silverlight clients and hammer the application hard for 2 hours.
Next morning we have a nice PDF report (SSRS) in our inbox that tells us how the stresstest went (min, max, avg CPU, IO, Network, Memory, connections, …)
Blog post coming up

#3 UI tests 
No web UI tests in this project (yet), because our MVC views don't have any logic. But we are testing the controllers.


All this is subject to change in the next months, but good to have in a blog post.
Lets see how we do it in 1 year!


necronet said...

Do you have any way to order your test better, i mean my .ordered always show all my unit test an is quite difficult to actually try to search for the testing methods!!

any suggestion?

Unknown said...

Ahh...can only dream of such an environment. Having good tests in place makes you so much more confident when implementing new features or making changes. I just love it, but currently at work I'm the !only! one that does TDD style development or forget TDD, even just automated testing in general...this makes makes it way less effective.. :(

>> "ASAP something was changed that was not expected as application >> behavior, we get an alert --> email"

Good post btw!
Does the TFS do this for u? Damn I want this!! Have to talk to Stoffi ;)

Peter Gfader said...

We do continuous integration with TFS
= Build and run tests on each check-in.

If the build breaks we get
#1 red alert in our tray icon
#2 email saying: "Find PeterG and beat him up"
#3 a twitter update ;-)

Peter Gfader said...

I group tests together by putting tests for 1 SUT in 1 file/class called: SUTTests.cs
SUT:System under test, is a method normally

The naming helps to find tests quite easily.

What do you mean by ".ordered"?
Are you using ordered tests from VS2008?
I would try to avoid that type of test because that means your tests are not independent from other tests.
That makes them very hard to maintain, understand, execute automatically and trust.

More about specific unit tests here

Post a Comment

Latest Posts

Popular Posts