Friday, 7 January 2011

Concordion Extensions

Nigel Charman has started a concordion-extensions sub-project that adds some useful extra features to Concordion, such as the ability to add screenshots to the Concordion output so you can more clearly see what state the application is in when something fails.

Another valuable extension inserts a little icon into the output and when you hover over it, it shows logging output that occurred during the test. This is a neat solution to a dilemma that I've sometimes encountered: "If I don't write down the steps, how can I be sure the test has been implemented right?"

We want to write high-level tests that express the business requirements without reference to a particular implementation, so we can change the implementation later. To do this, we hide the test's implementation steps in its accompanying Java fixture. But some people aren't comfortable navigating and reading Java code. If you're a non-developer tester trying to make sure the system is well-tested, it can be scary to "trust" the developers to implement your tests correctly.

The temptation is therefore to explicitly write into the test specification the steps required to execute the test. The trouble is by encoding the "how" you end up locking the test into a specific implementation. The lock-in problem is exacerbated by duplication across tests. When you're detailing all the steps, you find that many tests require a similar set of steps.

Our conflict cloud looks something like this:

Testing Conflict

Some testing frameworks, such as Robot and FitNesse, allow you to pull out common set-up but this is a compromise. Effectively what you're doing is programming, but not in a proper programming language. The intentions of your tests become lost amongst increasingly complex interweaved test scripts.

Nigel's solution allows you to write the tests in a way that retains a clear separation between intent and implementation, yet allows non-developers to be reassured that the test has been implemented correctly.

Testing Conflict Resolved


clarke said...

David, You write so well! I'm jealous...

Really interesting post.


David Peterson said...

Thanks Clark. I thought the cloud might catch your eye :-)