Agile Myth #2: "Agile is Just About Project Management"

This is my third post on a multi-part series entitled, "Agile Myths and Misconceptions". It's based on the talk I gave at the first PSIA Softech Philippine Software Engineering Conference.  I am striving to correct 12 common misconceptions about Agile Software Development.

Agile has equal or more emphasis on engineering than project management. Just check out the authors of the Agile Manifesto - a large number of them are thought leaders in nuts-and-bolts software engineering: Kent Beck is the creator of JUnit and Test-Driven Development, Martin Fowler has written several books on Design Patterns and Refactoring, Robert Martin has written several books on Object-Oriented Design and code quality... If you check out Extreme Programming, which was the first Agile process to gain popularity, and precedes Agile itself, it leans more towards engineering practices, such as TDD, Continuous Integration, Coding Standards and Pair Programming. If you attended an early Agile training course, it was actually a programming workshop where you pair up with another programmer around a workstation and write code!

Most of the Agile authors were aware that even if you do the project management side well, if you ignore the engineering side, a project will eventually be in quagmire. Consider what Martin Fowler said in his blog:

There's a mess I've heard about with quite a few projects recently. It works out like this:

• They want to use an agile process, and pick Scrum 
• They adopt the Scrum practices, and maybe even the principles
• After a while progress is slow because the code base is a mess

What's happened is that they haven't paid enough attention to the internal quality of their software. If you make that mistake you'll soon find your productivity dragged down because it's much harder to add new features than you'd like. You've taken on a crippling Technical Debt and your scrum has gone weak at the knees.

Here's an illustration of what Martin Fowler is saying:

So initially, a team's velocity increases with each iteration, but as the project progresses, the team's velocity starts to steadily degrade, even if people are working just as hard or harder.

Here's a breakdown of what's happening to the team. Let's pretend that a team's capacity is like a pipe. Requirements go in one end, some development happens inside, and then features come out the other end. Just like any pipe, the team's capacity is finite.

Initially, all of the team's capacity is used to turn requirements into features, but later on, naturally, some bugs are found, so part of the team's capacity is now taken up by bug fixing.

That situation is still fine, but if the team neglects the engineering practices, you end up with Technical Debt. In a nutshell, Technical Debt is difficulty in working code where the design and code quality is bad. The code is very difficult to understand, it's hard to find bugs, a lot of things need to be modified for every change, and the code is very fragile. It's a lot like real debt - when you don't pay your credit card bill, the interest payments eat up your capacity to pay for day-to-day things like food and rent. With technical debt, if the team doesn't take the effort to clean up their code, write automated tests, integrate often, apply proper design approaches, measure code quality, and other engineering practices, eventually the code becomes harder and harder to work with, and just wrestling with the code takes up more and more of the team's capacity, just like interest from debt takes up one's capacity to spend. What's more, since the code is more fragile, the project has more bugs, and so even more of the team's capacity is used up in bug fixing.

Just like real debt, if technical debt is not addressed, it just grows - the more code you add to the system, the harder and harder it just gets to work with. Eventually, the majority of the team's capacity is taken up by just technical debt and bug fixing. Very little actual creation of new features comes out of the development team, if any at all. 

I'm not exaggerating in any way. One of the clients I consulted for had development teams in three countries - with none of them producing any new features for months! The teams were bogged down with such a bad codebase that they couldn't change any code without creating a large number of bugs.

Again, the way you avoid this situation is through proper engineering practices. If you're just starting out with engineering practices, I think a good place to start would be at coding practices. If you're a Jave programmer, good book for this is Robert Martin's "Clean Code". 

After that, you might want to move to learning Test-Driven Development. My recommended TDD book for Java developers is "Test-Driven" by Lasse Koskela. After that, read up on proper practices for version control and Continuous Integration. For those of us working on legacy code, which is the majority I would think, a good reference is "Working Effectively with Legacy Code" by Michael Feathers.

For those further along, you might want to start exploring approaches to design. Domain-model design is a good place to start. Some recommended references are "Applying UML and Patterns" by Craig Larman, "The Data Model Resource Book" by Len Silverston, "Domain-Driven Design" by Eric Evans, and "Implementing Domain-Driven Design" by Vaughn Vernon.

For the other areas of design, you can check out the various design patterns books around such as Head First Design Patterns, Patterns of Enterprise Application Architecture, Enterprise Integration Patterns and Service Design Patterns.

The next myth I will tackle is Agile Myth #3: "Agile is Short Milestones".