What's wrong with the Agile Manifesto?

I've never liked the Agile Manifesto. I'm not blaming the authors because I know it must have been a nightmare trying to come to a consensus, and the manifesto has served a useful purpose bringing like-minded people together under a single brand. It is slightly ironic that the wording hasn't changed in 10 years, but there you go. I'm not brave enough to propose an alternative yet. In this article, I'm just going to explain where I think the logic is faulty.

Here is the wording of the manifesto:

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

It doesn't distinguish agile from "slapdash"

The aim of the manifesto seems to be to try and distinguish agile from waterfall. Unfortunately, agile's worst enemy isn't waterfall but the "we don't write documentation, so we're agile" brigade. And the manifesto doesn't defend agile very well against this threat. Perhaps there's something in the twelve principles but who on earth reads them? Certainly not the slapdash developers claiming to be agile.

It mixes up goals and solutions

"We value working software over comprehensive documentation". OK, but surely whatever software development method you select, working software is the ultimate goal, isn't it? Some people believe that by writing comprehensive documentation you are more likely to produce working software. That's part of their solution. But you can't go round comparing goals with solutions. It's nonsensical. Perhaps what the manifesto authors wanted to convey was that "the code is the documentation", or something along those lines, but couldn't agree to write it, so plumped for the goal as their solution. That's cheating!

The dilemmas are easy to solve

The manifesto poses dilemmas and then tries to position agile at a certain point in each trade-off continuum. But why are we compromising? If both sides have value (as the manifesto plainly says), why don't we find solutions that resolve the conflicts? Let's take each one in turn:

I don't know whether the value of documentation is to do with maintainability, improving clarity or good governance. The point I'm trying to make is that there does not have to be any compromise between those needs and agile methods.

So what is agile then?

I'm not sure. Please can you give me feedback on this post and your ideas. Leave a comment, or blog about it, or tweet me (@davidp99). Thanks.

FlowChain - Ideal company structure?

"If you could start from a clean slate, how would you structure a large company?"

A little Twitter conversation with Simon Baker (@energizr) about product streams, reminded me of a talk at last July's Agile Coaches Gathering, where Bob Marshall (@flowchainsensei) answered the question above by describing his "FlowChain" concept.

This is my interpretation of what he said (from memory) so it may not be quite right and he's probably refined it since then.

Running the whole company in an "agile" way

Essentially FlowChain scales agile practices up to the company level. There is a single (large) development team that works from a prioritised backlog of MMFs (Minimal Marketable Features). I'm not sure who manages the backlog. I've called it a "product strategist", but that's my name for it not Bob's.

The team works on multiple value streams. Each value stream has its own operations staff.

Zooming in:

Extreme flexibility

What I like about FlowChain is the flexibility it provides. If a value stream is doing well, resources can quickly be redeployed to capitalise on its success. Alternatively, new products (value streams) can be developed. In most organisations, moving developers from one project to another is a major upheaval, and that puts pressure on the project to be a success. When you can move developers quickly to something else, it relieves the pressure and allows the company to tackle more risky (but potentially more lucrative) opportunities.

Note that in a large company, the pool of resources for developing products could be very large, and there may therefore be many MMFs in progress at the same time. Bob didn't explain how to resolve resource conflicts, sequencing etc. We may have to wait for his book...

Prototypes, Spikes and Tracer Bullets

Here's an attempt to classify various development techniques and the sequence in which they tend to be used.

Don't kid yourself

"Our team's agile, but we haven't released yet because some of the teams we depend on aren't agile."

Fine, then you're not agile!

BDD: Concrete Examples Aren't Enough

The Given/When/Then style of Behaviour Driven Development (BDD), favoured by Cucumber and JBehave, puts a lot of context in the examples that I claimed was unnecessary clutter, but I had quite a few comments that puzzled me until it dawned on me there's another difference in my approach compared with the Given/When/Then style.

I always state the required behaviour in a sentence or two before giving the examples. Each behaviour is described by a specification like this:

Maybe it looks like a lot of work to write, but it isn't really. The structure is standard and each part is just a sentence or two.

Whereas in the Given/When/Then approach the business rule describing the behaviour generally isn't made explicit. The reader is expected to guess the rule from the examples, so naturally they have to have more context.

Occasionally it's difficult for the domain expert to immediately state the abstract rule and easier to start with concrete examples, but once the expert has explained a few examples then we can usually begin to take an educated guess at the rules behind them—"Ah OK, so the rule is X?"—and then have a useful discussion.

But if we don't make the rule explicit we only have the concrete examples, so they have to be made much more verbose—and potentially implementation-specific—so that readers can correctly interpret them.

It is true that Given/When/Then style examples are much better at avoiding implementation lock-in than test scripting ("click this, click that..."), but I would argue that context-free examples with explicitly stated rules are even better, both in terms of avoiding lock-in and in terms of readability (how long it takes the reader to understand the behaviour expected).

Object-Oriented Example

For ages I treated objects as glorified data structures to be operated on by procedural code. It took me a long time to get out of the procedural mindset and into an object-oriented mindset.

Well-written object-oriented code helps you deal with complexity by hiding internal details so that you can operate at ever higher levels of abstraction. Object-oriented code increases flexibility – objects can be plugged together in different ways like Lego™ pieces – and it keeps duplication and churn to a minimum by keeping data and operations in the same place instead of spreading them throughout the code-base.

The recent discussion on the GOOS mailing list has made me realise that others are also struggling with the transition. Nat Pryce has posted a couple of blog entries today, and I'd like to pick up the same theme with an example in the hope it might help.

Let's start with a simple PopGroup class that implements the role of SongPerformer (in Java this would be an interface):

To construct a valid PopGroup you need a Singer, a Drummer and a Keyboardist. These are the PopGroup's dependencies.

The responsibility for coordinating the various activities to perform songs is the responsibility of the PopGroup object, but the actual acts of singing, drumming and keyboard playing are separated out. This is the single responsibility principle combined with dependency injection.

This approach makes the design highly flexible because we can plug together all kinds of variations. As long as the dependencies we inject implement the right interfaces we can make the pop group perform a song.

For example:

If we don't have a DrumMachine, we can substitute a PhysicalDrummer and the PopGroup will still function the same.

Our PhysicalDrummer object depends on interfaces not implementations. If we're desperate we could grab a passer by, a dustbin and a broom handle and create a drummer:

Unfortunately the BroomHandle class does not implement the Drumstick interface. We could make it implement Drumstick, but the BroomHandle is in a different domain. It's in the domain of sweeping, rather than musical performances. We don't really want to couple its class to an interface from the musical performances domain, so we create an adapter to map between the two. This is known as a ports and adapters architecture.

To make things easier to understand we can wrap the objects into a class that has a name that better expresses the intent.

Now we've hidden all the complexity and we're operating at a higher-level of abstraction.

In reality, a ports and adapters architecture is not so much mapping between entirely unconnected domains as mapping lower-level domains to higher-level domains, so that the technical detail is hidden within simpler more abstract concepts.

Concordion Extensions

Nigel Charman has started a concordion-extensions sub-project that adds some useful extra features to Concordion, such as the ability to add screenshots to the Concordion output so you can more clearly see what state the application is in when something fails.

Another valuable extension inserts a little icon into the output and when you hover over it, it shows logging output that occurred during the test. This is a neat solution to a dilemma that I've sometimes encountered: "If I don't write down the steps, how can I be sure the test has been implemented right?"

We want to write high-level tests that express the business requirements without reference to a particular implementation, so we can change the implementation later. To do this, we hide the test's implementation steps in its accompanying Java fixture. But some people aren't comfortable navigating and reading Java code. If you're a non-developer tester trying to make sure the system is well-tested, it can be scary to "trust" the developers to implement your tests correctly.

The temptation is therefore to explicitly write into the test specification the steps required to execute the test. The trouble is by encoding the "how" you end up locking the test into a specific implementation. The lock-in problem is exacerbated by duplication across tests. When you're detailing all the steps, you find that many tests require a similar set of steps.

Our conflict cloud looks something like this:

Some testing frameworks, such as Robot and FitNesse, allow you to pull out common set-up but this is a compromise. Effectively what you're doing is programming, but not in a proper programming language. The intentions of your tests become lost amongst increasingly complex interweaved test scripts.

Nigel's solution allows you to write the tests in a way that retains a clear separation between intent and implementation, yet allows non-developers to be reassured that the test has been implemented correctly.