Nigel Charman has taken my story delivery diagram and expanded it to cover the development cycle and project lifecycles, in a talk to the Agile Professionals Network (APN) in Wellington.
You can't get much further from the UK than New Zealand, but Nigel and I seem to share close views about the role and importance of automated acceptance testing.
The worked example in Nigel's presentation is better than the examples in the Concordion tutorials. I think I'm going to have to steal it.
Georges Polyzois has published the second installment in his series about Concordion:
It's cool to see how people are using the tool in real-life.
I gave a brief talk on Concordion at the "Agile Acceptance Testing Tools Round-up" event organised by Gojko Adzic in London last night. For those who attended, or those who didn't and are interested, these are my slides.
The main point I was trying to make is that clean code and clean tests are readable. It's hard to make things too readable. Readable acceptance tests lead to readable fixtures lead to readable code. And the opposite applies too. Once readability starts to slip, things get horrible fast.
"Unmaintainability" is rather an abstract thing to look for, whereas poor readability is much more obvious. Readability in everything from build files to tests, should be a relentless focus for a team. Unreadable code or documents should not be tolerated because whatever you tolerate becomes the norm - it's the broken windows theory - if no-one seems to care, there's little incentive for others to take care.
Strangely, it wasn't until I put together this presentation that the importance of readability really grabbed me.
P.S. For those that attended the talk, as a next step can I suggest you download concordion-kickstart-1.3.1.zip. It's an Eclipse project with a single "Hello World!" test that will hopefully show you just how simple Concordion is to use. If you like it, then you can follow the tutorial and read more about the technique.
Concordion 1.3.1 was released last month with a handful of new and noteworthy features. There was an announcement on the Yahoo mailing list, but I wanted to wait for the Concordion.NET port to catch up before making an announcement on this blog.
For those who didn't know, there are now versions of Concordion for:
So, if you're using Cucumber in Ruby, or FitNesse in .NET check out the alternative. No other acceptance testing framework lets you create such readable acceptance tests as Concordion's. The key advantage in having readable tests is that when you come back in three months time and wonder what the heck you were doing, you might actually have a clue.
Nat Pryce's discussion of blinkered thinking, reminded me of an incident in Paris (France), in August 2003, that my wife still likes to tease me about. It was in the middle of a heatwave. I think the temperatures reached something like 39 or 40C and we ended up spending most of our time in the fully air-conditioned Pompidou Centre.
One of the reasons we'd chosen to go to Paris was that I wanted to go on the Pari Roller which is a huge street skate that runs every Friday night. It starts at ten o'clock at night and finishes about one in the morning.
In the evening, it was still really, really hot, but I figured I might as well give the skate a go, since I'd gone to all the trouble of carrying my skates over. I sweated buckets and got through two big bottles of water, but I made it round and really enjoyed it. When I got back to the hotel I was knackered and drenched with sweat. I just wanted to strip off, have a cold shower and go to bed. I went into the bathroom and turned on the cold tap. The water was warm. I guessed the pipes or the tank on the roof had been heated by the sun. I ran the tap for a long time, but it still didn't go cold. If anything it got hotter. I ended up having a shower in quite hot water and went to bed hot, sweaty and grumpy. Luckily I was so tired from the skate I managed to get some sleep.
The next morning, I mentioned to my wife that the cold tap hadn't been remotely cold.
"Were you using the tap marked 'C'?"
"Yes, of course."
"I see. Did you think about maybe trying the one marked 'F' instead?"
I'm integrating with Spring Security at the moment. It seems to be a well-designed framework, but I got myself confused this morning trying to implement some methods of the UserDetails interface:
boolean isAccountNonExpired()
Indicates whether the user's account has expired.boolean isAccountNonLocked()
Indicates whether the user is locked or unlocked.boolean isCredentialsNonExpired()
Indicates whether the user's credentials (password) has expired.boolean isEnabled()
Indicates whether the user is enabled or disabled.
I imagine the designers wanted all the answers to be true for the normal "happy" situation. But my brain couldn't cope with all the negatives, so I ended up writing a second set of positively-named methods to call from the negative ones. :-)
public boolean isAccountNonExpired() {
return !isAccountExpired();
}
... etc.
In case you weren't aware, GWT 1.6 Milestone 1 has recently been released. The biggest change is that projects are now laid out using a standard WAR file structure. This makes hosted mode much better because you now have full control over the web.xml file. They've also switched from Tomcat to Jetty internally (Jetty's start-up time is quicker) and they've added a button to restart the server without having to exit and restart hosted mode. Very convenient.
I'm combining GWT with Tapestry 5, in my current project, so these changes are very helpful. I can now run them together in hosted mode.
I'm writing this as a reminder to myself and potentially to give a headstart to anyone else making a similar evaluation.
Google App Engine is seriously tempting. It removes the headache of scaling to millions of users by letting you run web applications on Google's infrastructure. The trouble is, you have to write the applications in a very specific way designed to be scaled using Google technologies.
For most Web 2.0 startups, I don't think this trade-off is worth it, for the following reasons:
If all you're trying to do is distribute something, say a Flash application, or to host a simple Facebook plugin with barely any server-side logic then Google App Engine might work out well. It will get you to market quicker and with fewer headaches. However, if you're trying to start-up a serious Web 2.0 business, with unknown future directions and the potential to be acquired, I'd say Google App Engine is probably not a good move. The lock-in reduces your options and is likely to make your company less attractive to anyone but Google. Unfortunately, that means you're just going to have to bite the bullet and handle scalability yourself. :-(
This is quite an obvious little pattern, but I think it's still worth documenting.
Some repetition is harmless
When I'm writing an example of a required behaviour, I find that I often have to refer to something I've already mentioned. For example:
Given the following names: Charles, Naomi, Melissa, Arnold
Searching for 'ar' should find: Charles, Arnold
Here, I have mentioned the names Charles and Arnold twice: once when describing the context and once when describing the expected outcome. This isn't a problem because I purposely kept the names short to make the example easy to follow.
But repetition can become awkward
If the values are long or complicated then the examples can become hard to read and maintain. For example:
Given the following addresses:18 Cedar Row, Enfield, Middx, EN8 9TTSearching for 'ar' should find:
"The Grange", 177 Hounslow Road, Epping, Essex, HA10 4PL
2b Armsfield Avenue, London, SE1 2BN
80 Commercial Street, Aberdeen, AR19 1TB18 Cedar Row, Enfield, Middx, EN8 9TT
2b Armsfield Avenue, London, SE1 2BN
80 Commercial Street, Aberdeen, AR19 1TB
So, use labels for verbose values
Labelling each item allows you to use the label, instead of the value, whenever you need to refer to it.
Given the following addresses:(1) 18 Cedar Row, Enfield, Middx, EN8 9TTSearching for 'ar' should match addresses: (1), (3), and (4)
(2) "The Grange", 177 Hounslow Road, Epping, Essex, HA10 4PL
(3) 2b Armsfield Avenue, London, SE1 2BN
(4) 80 Commercial Street, Aberdeen, AR19 1TB
I call them labels or pseudo-identifiers to distinguish them from real identifiers in the system under test. For example, in the implementation we might have a database table called Address with an identifier column called addressId. But the identifiers we use in the examples are merely a way of making the example easier to read. They are not a reflection of any particular database model. Any mapping between real identifiers and pseudo-identifiers must always be handled behind the scenes in the fixture code.
The labels don't have to be numeric. They could be letters (A, B, C) or other more meaningful short strings.
Labels can also help you to hide details
Another advantage of labels is that they can let you gloss over unnecessary details. For example:
Using labels for bookings (1, 2) and for booking references (A, B) means we don't need to go into any details of what we're booking or what the booking references look like. If the format of the booking references is important, we can cover that in another test, but, in this test, we're not locking ourselves into any particular format. All we're doing is demonstrating the rule: "Each booking is given a unique booking reference that can be used to look-up the booking."
The fixture code will have the following methods:
public class IdentifyingBookingsTest extends ConcordionTestCase {
public void makeBooking(String bookingNumber, String bookingRefAlias) {
...
}
public String lookupBooking(String bookingRefAlias) {
...
}
}
Having this extra level of indirection may make the fixture code work harder, but that is not a problem. It is much better to put complexity and implementation-related assumptions in the fixture code, where we have powerful refactoring tools at our disposal, than to complicate the specification and make it more fragile.
Millions of software projects still aren't being run in an agile way. What can we do to widen agile adoption? Let's look at marketing strategy...
First: eXtreme Programming
In the beginning there was eXtreme Programming. Actually, Tom Gilb created an iterative method at least 10 years before that, called "Evo", which was (and is) in many ways better than XP, but, being based around metrics, it didn't have much sex appeal and never took off. Whereas the "eXtreme" branding appealed to innovators, or at least got them talking, and helped to spread the word.
Then: Rebranding as Agile
The word "eXtreme" may have appealed to innovators but it didn't appeal to their managers, so a few clever people came up with a new word that would serve to encompass not just XP but a whole genre of methods. You might not want to be extreme, but who wouldn't want to be agile?
This marketing masterstroke led to early adopters (media, telecoms, IT firms etc.) beginning to take on agile practices.
Today: The Chasm
Unfortunately, at this point, agile adoption began to stutter. This is where we are now, and where we have been for the last few years. The vast majority of organizations still aren't using agile methods. It's like a chasm has opened up and we haven't worked out how to cross it.
So, what do we do?
Clarke Ching believes the chasm is a mirage and that another rebranding ("EverydayAgile") is all that's needed to cross it. Brian Marick, on the other hand, thinks the chasm is so wide we might as well throw in the towel and retreat to more innovative companies.
My view is that the issue is real and is too important for us to simply give up on. If you consider how much time and money is wasted around the world on software projects that deliver late and don't actually do what the customer needs, the mind boggles. We have been "uncovering better ways of delivering software"; it seems criminal not to help others do the same.
I'm not convinced that a rebranding is necessary but, if it is, I think "Lean" or "JIT" might be better candidates than "EverydayAgile". I can see the idea of "EverydayAgile" is to try to make agile seem less daunting, but I don't think this is the problem. Organizations aren't scared off by the word "agile". Quite the opposite! They're drawn to it, to the extent that they'll claim to be "agile" when they're absolutely not. They see "agile" as a synonym for quality. If we were to rebrand, the purpose should be to stop the brand being diluted to the point that no-one knows what it actually means. However, though aging, the agile manifesto still seems to be doing a sufficiently good job of maintaining a sense of the brand values.
What I think we should do is follow Geoffrey Moore's advice about crossing the chasm:
"The fundamental principle for crossing the chasm is to target a specific niche market as your point of attack and focus all your resources on achieving the dominant leadership position in that segment."
Where should we focus?
I suggest we focus on investment banks. They're on the left-hand edge of the early majority. They're financially-savvy and IT-heavy, so an improvement will be noticed, but they are also large and bureaucratic. They pay well, they're influential, they want to adopt agile, but they just can't work out how.
Rather than generic Agile conferences, we should organise a string of global conferences about Agile Investment Banking, we should have our big-wigs talk to their big-wigs, build agile tool support to meet their specific needs, and focus our entire community's effort on helping them to become agile.
After we have demonstrated a track-record of success in investment banks, we will have crossed the chasm, and convincing everyone else should be a lot easier. That's the theory anyway...
Sometimes when you're testing, it's easy to get carried away with little edge cases and forget about things that are much more important.
I saw a real-life example recently at a conference: The security safe in my hotel bedroom was designed to take a laptop and had a small hole in the top for passing in a power cable, so you could charge your machine while you were out of the room. It seemed like a clever idea, but then the exploratory tester in me took over.
Two minutes later I had successfully broken in without the keycode: I bent a wire coat hanger (handily provided by the hotel), poked it through the hole, hit the little red reset button on the inside edge of the door, keyed-in a new 6 digit code, and watched as the door swung open.
Shielding around the reset button might have solved that issue, but in fact the whole idea was fundamentally flawed: The way the hotel room worked was that the lights and electric power sockets in the room only switched on when you placed your door-card into a slot by the door. That meant that even if you left your laptop in the safe with its power cable attached, it wouldn't be charging while you were out.
This is why customer-level acceptance tests are so important. You can unit test your 6 digit security lock to your heart's content, but it's a complete waste of time if you're not actually solving the customer's problem.
Concordion for Ruby
Ben Goodspeed has written a very neat Ruby port of Concordion and packaged it as a gem. There are currently some minor syntactical differences between the Java version and the Ruby version because the Ruby version doesn't do lookahead variable bindings yet (Ben's working on that) but the core functionality is already implemented (set, execute, assertEquals, and verifyRows).
Maven Integration
Several people requested that the Java version of Concordion be added to Maven repositories and thanks to José Manuel Beas, Wang Yi Zhou, Craig Walls, and others, this is now done (instructions for usage with Maven are here). Concordion 1.3.0 has the same functionality as 1.2.0 but with Maven support.
I enjoyed David Anderson's article "Stop Negotiating, Start Collaborating" but the title doesn't make sense to me. Surely collaboration is a form of negotiation?
The way I see it is that there is a scale of trust underpinning any negotiation: At the low-trust end, you have lawyers involved, you're not budging an inch, and your aim is to hang your opponent out to dry. At the high-trust end, you're chatting over a cup of coffee. You've both got goals, but you're not going to screw each other over because the long-term relationship matters more to both of you than anything else. At this end of the scale you're still negotiating but it's a collaborative effort; you're trying to find solutions that will work out well for both of you.
Negotiation isn't the problem. Lack of trust is the problem and there are no quick fixes. Building trust is about building relationships and that takes time.
Transparency helps - in that it's tricky to build a relationship with someone who won't give any information away - but too much transparency can actually make decisions harder:
It would be ridiculous to expect the candidate to say that, but, if he did, how do you resolve the situation? Whatever you do, someone's going to feel unhappy knowing they could have done better out of the deal.
People aren't robots. There's psychology at play and personal goals and emotions and that means, no matter what you might wish, collaboration is still negotiation, but a high-trust form.
I've released a new version of Concordion.
<p>Will result in the following output:
The first name <span concordion:set="#firstName">Bob</span>,
<span concordion:assertTrue="#firstName.startsWith(#letter)">
starts with the letter <b concordion:set="#letter">B</b></span>.
</p>
I like Keith Braithwaite's "software gauges" metaphor. As he explains, a gauge is a shortcut for deciding whether something passes or fails some criteria. For example, if your bag fits inside the metal cage at an airport then it can be taken onboard as hand luggage. You don't need to use a tape measure. What's interesting about gauges is that you don't necessarily need to be able to articulate "the rules"; if you need to know the rules you can infer them from the gauge.
However, I'm not convinced with what Keith says about trader spreadsheets making good gauges. Obviously he's had success with them, so they definitely can be made to work, but would they have worked even better another way? Basically what it boils down to is that I like tables; I just don't like big hairy tables!
If the hand-luggage cage can be taken as an example of good practice then some of the properties of a good gauge appear to be: sturdy and reliable; correct enough for all practical purposes; obvious; unambiguous; quick to use; and simple to understand by the gauge user.
The spreadsheets Keith showed me contained a large number of sparsely populated columns of denormalised data with lots of magic numbers and magic strings. This doesn't seem to stack up well against the list of desirable traits for a gauge. They're not designed with the gauge user in mind. If your domain model doesn't fit the gauge, is it because the model is wrong? Or the gauge is wrong? Or your interpretation of the gauge is wrong? With so many moving parts, it's got to be difficult to work out, even with help from the traders.
Isn't it better to treat the spreadsheets as a starting point rather than an ending point? And then have someone skilled at analysis and abstraction work with the traders and extract smaller, more practical gauges. There's nothing to stop you holding these gauges in spreadsheets too. I have a lot of respect for the customer, but I don't see why, just because the traders have written a spreadsheet, you can't help them improve it. They've got their skills you've got yours.
Most of the examples on the Concordion website are technical in nature, so I've put together a short business-focused example. Please take a look.
I'm not trying to push Concordion on you. You can do something similar with other test frameworks. It's really the approach that I'm trying to get across: focusing the acceptance tests on goals, not solutions; and decomposing behaviours, keeping each test as isolated and simple as you can.
While I'm here, let me join in the 2x2 matrix fun:
The user-interface (UI) of an application is a solution, not a goal. I often see people writing test scripts in terms of direct user interface interactions (e.g. using a record/playback/verify tool like Selenium) unaware that by doing so they're locking themselves into a particular design.
If, instead, they hid the scripting behind goal-oriented acceptance tests they would leave themselves a lot more freedom to change the solution. I guess not all teams need that kind of freedom. But I wonder how many even know there's a choice?
I have had a couple of e-mail questions recently, from people who have read the Concordion documentation, asking for more detail about what I mean by a DSL (Domain-Specific Language) for scripting, as shown in this diagram:
I think it's easiest to explain with a realistic implementation of a fixture:
public String getGreetingFor(String firstName) {
User user = new UserBuilder()
.withReasonableDefaults()
.withFirstName(firstName)
.build();
BrowserSession session = new BrowserSession();
new UserAdminAction(session).setUp(user);
new LoginDriver(session).login(user);
return new HomePageDriver(session).scrapeGreeting()
}
public abstract class AbstractDriver {
private BrowserSession session;
public AbstractDriver(BrowserSession session) {
this.setSession(session);
}
protected HtmlPage getHtmlPage() {
return session.getHtmlPage();
}
protected HtmlElement getElement(String id) {
return getHtmlPage().getHtmlElementById(id);
}
protected String scrapeText(String id) {
return getElement(id).asText();
}
protected void click(String id) {
((ClickableElement) getElement(id)).click();
}
// etc.
}
public class HomePageDriver extends AbstractDriver {
public HomePageDriver(BrowserSession session) {
super(session);
}
public String scrapeGreeting() {
return scrapeText(HomePage.GREETING_ID);
}
public void clickSearch() {
click(HomePage.SEARCH_LINK_ID);
}
// etc.
}
I've released a new version of my Java acceptance testing framework: Concordion. The main new feature is support for JUnit 4 test runners.
For example this Demo.html specification:
<html xmlns:concordion="http://www.concordion.org/2007/concordion">
<body>
<p>
The greeting for user <span concordion:set="#firstName">Bob</span>
will be:
<span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span>
</p>
</body>
</html>
import org.concordion.integration.junit4.ConcordionRunner;
import org.junit.runner.RunWith;
@RunWith(ConcordionRunner.class)
public class DemoTest {
public String greetingFor(String firstName) {
return String.format("Hello %s!", firstName);
}
}
Being an agile developer in a largely waterfall organisation has reminded me of a game we played in one of my MBA lectures where we were put into groups and asked to solve a problem.
One guy in one of the groups (unbeknownst to us) had been primed by the lecturer to disagree with every idea that his group came up with. At the end of the session, that group had markedly better results than all the other groups. The constant voice of dissent had helped them to uncover their assumptions.
Of course, before revealing the results, the lecturer asked each group to vote a person out and the disagreement-guy was unanimously given the boot.
You're in the middle of implementing some new functionality when you stumble across some poorly written code. Fixing it properly will take some time and, for the stuff you're working on, you can get away with making a couple of tweaks. What should you do? Refactor it now? Or leave it for another time? If so, when?
Douglas Squirrel told me about a neat solution that I hadn't heard of before. The idea is that the first time you come across some smelly code you write a comment that says FIXME. Each time you encounter the code again, or someone else does, an extra X is added so it becomes FIXXME, FIXXXME etc. By the time the comment screams FIXXXXXXXME, you know what you have to do!
I will be speaking about Acceptance Test-Driven Development at the UNICOM agile conference in London, 12/13 February 2008. The conference has a good line-up of speakers including Duncan Pierce, Keith Braithwaite, and Rachel Davies. Early-bird fees are on offer till 18th January.
Update: If you're interested in attending, contact me. I can probably get you a discount.
I've been working with Tapestry 5, for a few weeks now, and I'm loving it. It's a big step up from Tapestry 4 and I think it could become a realistic competitor to Ruby on Rails. It has similar characteristics: convention over configuration and an almost religious devotion by its creator, Howard Lewis-Ship, to driving out duplication.
Sometimes he possibly goes too far. For example, if your page class is /article/ArticleEdit the duplication of the word "article" is removed from the URL, so the URL becomes /article/edit. It seems like an improvement, but actually introduces ambiguity and has caused confusion, judging by queries on the mailing list.
At the moment, you'd be taking a risk using Tapestry 5 in a production application. It's still "alpha" and unstable both in terms of the API, which hasn't fully settled down, and in terms of defects: There are lots of minor, and a handful of more serious, defects reported in JIRA. But if you're developing a non-mission-critical web application you might want to give it some serious consideration. It's already very good and only going to get better.
