Pushing the waterfall uphill

The ideas behind agile devel­op­ment method­olo­gies are not new. They were more for­mally stated in The Agile Man­i­festo. The ideas behind more basic iter­a­tive devel­op­ment are even older, and have their own trade-offs. Both con­tain the idea that you can not know every­thing upfront. The agile method­ol­ogy states this idea explicitely:

Wel­come chang­ing require­ments, even late in
devel­op­ment. Agile processes har­ness change for
the customer’s com­pet­i­tive advantage.

Iter­a­tive, on the other hand, allows for change between iter­a­tions, though not within them in any mean­ing­ful fash­ion. Each has it’s advan­tages, and largely they are cul­tural dif­fer­ences. Some orga­ni­za­tions sim­ply aren’t capa­ble of one or the other, and that’s OK.

The project I’m work­ing on now is attempt­ing to be iter­a­tive. We are try­ing very hard to be iter­a­tive, but the dif­fi­culty is that the client is absolutely insis­tent that we define every sin­gle require­ment up front before we do any­thing. For exam­ple, we are devel­op­ing use cases to doc­u­ment the exter­nal actor’s inter­ac­tions with the sys­tem. A sim­ple enough idea, and one that is very use­ful in a large sys­tem such as this. The prob­lem is two-fold how­ever. First, we are expected to com­plete the use case in one iter­a­tion and never touch it again. Touch­ing it again requires a con­tract change. Bril­liant. Equally bad, but more sub­tly so, is the fact that there is entirely too much detail in the use cases.

To me, use cases rep­re­sent value when they “tell a story” about how an actor (user) inter­acts with the sys­tem. They are told clearly from the actor’s per­spec­tive, and dis­cuss how the actor accom­plishes their goal, but not how the sys­tem works under­neath. Unfor­tu­nately, here we are putting in detailed enti­ties, attrib­utes and ser­vice (SOA) calls that will be con­sid­ered part of the con­tract. This makes it nearly impos­si­ble to “do the right thing” when the time comes, and forces an inor­di­nant amount of design detail into a period of time where not enough infor­ma­tion exists to make the decision.

In the end, it feels like a water­fall process, but with use­less labels applied. You can pre­tend it’s “agile”, or it’s “iter­a­tive”, but when the client looks at you in a meet­ing, and with a straight face and no sense of irony says: “This project is too big to be agile” you know some­thing is going to go ter­ri­bly wrong. The larger the project, the more impor­tant it is to be agile and adap­tive. The scope is entirely too big to ever know every­thing up front. When they say they want to define every sin­gle require­ment before begin­ning the design, and will not allow revis­it­ing of the require­ments, you know it’s going to be a long uphill battle.

There has to be a bet­ter way. I’m no Mar­tin Fowler, but this just doesn’t feel like a good idea.