Tuesday, 14 October 2008

Label verbose items to simplify back-references

This is quite an obvious little pattern, but I think it's still worth documenting.

Some repetition is harmless
When I'm writing an example of a required behaviour, I find that I often have to refer to something I've already mentioned. For example:

Given the following names: Charles, Naomi, Melissa, Arnold
Searching for 'ar' should find: Charles, Arnold

Here, I have mentioned the names Charles and Arnold twice: once when describing the context and once when describing the expected outcome. This isn't a problem because I purposely kept the names short to make the example easy to follow.

But repetition can become awkward
If the values are long or complicated then the examples can become hard to read and maintain. For example:

Given the following addresses:
18 Cedar Row, Enfield, Middx, EN8 9TT
"The Grange", 177 Hounslow Road, Epping, Essex, HA10 4PL
2b Armsfield Avenue, London, SE1 2BN
80 Commercial Street, Aberdeen, AR19 1TB
Searching for 'ar' should find:
18 Cedar Row, Enfield, Middx, EN8 9TT
2b Armsfield Avenue, London, SE1 2BN
80 Commercial Street, Aberdeen, AR19 1TB

So, use labels for verbose values
Labelling each item allows you to use the label, instead of the value, whenever you need to refer to it.

Given the following addresses:
(1) 18 Cedar Row, Enfield, Middx, EN8 9TT
(2) "The Grange", 177 Hounslow Road, Epping, Essex, HA10 4PL
(3) 2b Armsfield Avenue, London, SE1 2BN
(4) 80 Commercial Street, Aberdeen, AR19 1TB
Searching for 'ar' should match addresses: (1), (3), and (4)

I call them labels or pseudo-identifiers to distinguish them from real identifiers in the system under test. For example, in the implementation we might have a database table called Address with an identifier column called addressId. But the identifiers we use in the examples are merely a way of making the example easier to read. They are not a reflection of any particular database model. Any mapping between real identifiers and pseudo-identifiers must always be handled behind the scenes in the fixture code.

The labels don't have to be numeric. They could be letters (A, B, C) or other more meaningful short strings.

Labels can also help you to hide details
Another advantage of labels is that they can let you gloss over unnecessary details. For example:

Example acceptance test showing use of pseudo-identifiers

Using labels for bookings (1, 2) and for booking references (A, B) means we don't need to go into any details of what we're booking or what the booking references look like. If the format of the booking references is important, we can cover that in another test, but, in this test, we're not locking ourselves into any particular format. All we're doing is demonstrating the rule: "Each booking is given a unique booking reference that can be used to look-up the booking."

The fixture code will have the following methods:

public class IdentifyingBookingsTest extends ConcordionTestCase {

public void makeBooking(String bookingNumber, String bookingRefAlias) {

public String lookupBooking(String bookingRefAlias) {

Having this extra level of indirection may make the fixture code work harder, but that is not a problem. It is much better to put complexity and implementation-related assumptions in the fixture code, where we have powerful refactoring tools at our disposal, than to complicate the specification and make it more fragile.

Tuesday, 7 October 2008

Friday, 2 May 2008

Concordion for Ruby and Maven

Concordion for Ruby
Ben Goodspeed has written a very neat Ruby port of Concordion and packaged it as a gem. There are currently some minor syntactical differences between the Java version and the Ruby version because the Ruby version doesn't do lookahead variable bindings yet (Ben's working on that) but the core functionality is already implemented (set, execute, assertEquals, and verifyRows).

Maven Integration
Several people requested that the Java version of Concordion be added to Maven repositories and thanks to José Manuel Beas, Wang Yi Zhou, Craig Walls, and others, this is now done (instructions for usage with Maven are here). Concordion 1.3.0 has the same functionality as 1.2.0 but with Maven support.

Saturday, 15 March 2008

Concordion 1.2.0

I've released a new version of Concordion.

  • Support for full set of JUnit4.4 annotations (@Before, @After etc.)
  • New command: assertTrue
For example, this instrumented specification:
The first name <span concordion:set="#firstName">Bob</span>,
<span concordion:assertTrue="#firstName.startsWith(#letter)">
starts with the letter <b concordion:set="#letter">B</b></span>.
Will result in the following output:

A failure would look like this:

Thursday, 13 March 2008

Big Hairy Tables

I like Keith Braithwaite's "software gauges" metaphor. As he explains, a gauge is a shortcut for deciding whether something passes or fails some criteria. For example, if your bag fits inside the metal cage at an airport then it can be taken onboard as hand luggage. You don't need to use a tape measure. What's interesting about gauges is that you don't necessarily need to be able to articulate "the rules"; if you need to know the rules you can infer them from the gauge.

However, I'm not convinced with what Keith says about trader spreadsheets making good gauges. Obviously he's had success with them, so they definitely can be made to work, but would they have worked even better another way? Basically what it boils down to is that I like tables; I just don't like big hairy tables!

If the hand-luggage cage can be taken as an example of good practice then some of the properties of a good gauge appear to be: sturdy and reliable; correct enough for all practical purposes; obvious; unambiguous; quick to use; and simple to understand by the gauge user.

The spreadsheets Keith showed me contained a large number of sparsely populated columns of denormalised data with lots of magic numbers and magic strings. This doesn't seem to stack up well against the list of desirable traits for a gauge. They're not designed with the gauge user in mind. If your domain model doesn't fit the gauge, is it because the model is wrong? Or the gauge is wrong? Or your interpretation of the gauge is wrong? With so many moving parts, it's got to be difficult to work out, even with help from the traders.

Isn't it better to treat the spreadsheets as a starting point rather than an ending point? And then have someone skilled at analysis and abstraction work with the traders and extract smaller, more practical gauges. There's nothing to stop you holding these gauges in spreadsheets too. I have a lot of respect for the customer, but I don't see why, just because the traders have written a spreadsheet, you can't help them improve it. They've got their skills you've got yours.

Sunday, 9 March 2008

Acceptance Test Driven Development

Most of the examples on the Concordion website are technical in nature, so I've put together a short business-focused example. Please take a look.

I'm not trying to push Concordion on you. You can do something similar with other test frameworks. It's really the approach that I'm trying to get across: focusing the acceptance tests on goals, not solutions; and decomposing behaviours, keeping each test as isolated and simple as you can.

While I'm here, let me join in the 2x2 matrix fun:

2x2 Matrix: Abstract vs Concrete / Goal vs Solution

The user-interface (UI) of an application is a solution, not a goal. I often see people writing test scripts in terms of direct user interface interactions (e.g. using a record/playback/verify tool like Selenium) unaware that by doing so they're locking themselves into a particular design.

If, instead, they hid the scripting behind goal-oriented acceptance tests they would leave themselves a lot more freedom to change the solution. I guess not all teams need that kind of freedom. But I wonder how many even know there's a choice?

Saturday, 9 February 2008

What do I mean by a "Scripting DSL"?

I have had a couple of e-mail questions recently, from people who have read the Concordion documentation, asking for more detail about what I mean by a DSL (Domain-Specific Language) for scripting, as shown in this diagram:

I think it's easiest to explain with a realistic implementation of a fixture:

public String getGreetingFor(String firstName) {
User user = new UserBuilder()
BrowserSession session = new BrowserSession();
new UserAdminAction(session).setUp(user);
new LoginDriver(session).login(user);
return new HomePageDriver(session).scrapeGreeting()

In this case, the DSL is made up of the Builder, Action and Driver classes. These implement and hide all the dirty work of setting up the user and navigating around the website.

The Driver classes extend an AbstractDriver class. The abstract base class provides protected methods for clicking on links, scraping the contents of elements, typing text, selecting radio buttons etc.

public abstract class AbstractDriver {

private BrowserSession session;

public AbstractDriver(BrowserSession session) {

protected HtmlPage getHtmlPage() {
return session.getHtmlPage();

protected HtmlElement getElement(String id) {
return getHtmlPage().getHtmlElementById(id);

protected String scrapeText(String id) {
return getElement(id).asText();

protected void click(String id) {
((ClickableElement) getElement(id)).click();

// etc.


The subclasses use these methods, but do not reveal them publicly. The public interface is at a higher level of abstraction.

public class HomePageDriver extends AbstractDriver {

public HomePageDriver(BrowserSession session) {

public String scrapeGreeting() {
return scrapeText(HomePage.GREETING_ID);

public void clickSearch() {

// etc.

Actions encapsulate a series of driver clicks to perform some higher-level action and remove duplication across tests. For example, behind the scenes the UserAdminAction may use a LoginDriver, UserAdminPageDriver, UserDetailsPageDriver, LogoutDriver. Actions may be nested.

You should also read Nat Pryce's series of blog posts on Test Data Builders.

Sunday, 27 January 2008

Concordion 1.1.0

I've released a new version of my Java acceptance testing framework: Concordion. The main new feature is support for JUnit 4 test runners.

For example this Demo.html specification:

<html xmlns:concordion="http://www.concordion.org/2007/concordion">
The greeting for user <span concordion:set="#firstName">Bob</span>
will be:
<span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span>

Can be run with the following JUnit 4 test case:
import org.concordion.integration.junit4.ConcordionRunner;
import org.junit.runner.RunWith;

public class DemoTest {

public String greetingFor(String firstName) {
return String.format("Hello %s!", firstName);

And will result in the following output:
Concordion 1.1.0

Saturday, 26 January 2008