Waterfall vs. Agile development: A case study

If you could do the same project with the same group of people, first using a waterfall methodology and next using an Agile methodology, what do you think the results would be? Well, though not the exact same project, two very similar projects were done with the same development teams, one using waterfall, and the other using Agile, which yielded some interesting findings.

The situation

Bill Holst, president and principal consulting software engineer for Prescient Software Engineering, managed two projects for Colorado Springs Utilities -- an electric project and a gas project. Both projects were distribution design systems very similar in scope. Holst contracted out the development work and used the same team for both projects. The electric project was a fixed price bid and was done using a traditional waterfall approach. The gas project was done next with the same development team -- a team that was new to Agile -- with T&M (Times and Materials) pricing. In both cases the customers of the project were field engineers.

The waterfall approach dictates that requirements are complete before coding begins. Typically, once the requirements phase has completed, the users don’t get involved again until the user acceptance test (UAT) phase nears the end of the project. With an Agile approach, the users remain involved throughout the lifecycle, with regular reviews and updates to the requirements.

Using waterfall yields unsatisfactory results

Holst felt that the electric project, which was done first, had many deficiencies. There was a long lag time between expected delivery and actual delivery. The software didn’t match what the customers felt they had asked for. There was a lot of churn throughout the project with disappointing results. Frustration was felt all around.

Initial transition: “We suck at Agile”

The team knew they needed to do something differently, so they decided to try an Agile approach for the gas project. They hired some Agile trainers to coach them through the transition, but there was a lot of initial team resistance. The field engineers didn’t like it, thinking the methodology was too touchy-feely. “We want to build a system and these guys are talking about commitment,” was the general sentiment. Holst joked that they team suggested getting coffee mugs that said, “We suck at Agile.”

The project suffered through many problems. The test cases didn’t match the logic, which was convoluted and confusing. Six iterations went forward, but a look at velocity and burn-down charts showed that progress was getting worse rather than better. The team made a tough decision: Stop and regroup.

The turnaround

The team decided to use the 7^th iteration to re-examine the requirements. The developers took two weeks off and the field engineers who had defined the requirements using pseudo-code, recognized that their original requirements needed to be redone. They took the original 800 lines of pseudo-code and, now, with a better understanding of the system, what was working and what wasn’t, were able to rework it down to around 200 lines of pseudo-code in much more logical way. This redesign of the requirements was the turning point of the project.

This change in direction was only possible because of the Agile nature of the project. “The cost to change the system this far downstream would have been too high [using the traditional model],” says Holst. Having the more flexible pricing model, along with the ability to change requirements midstream was a major key to success.

The results

With the reworked requirements, the team was set to start practicing Agile in the manner it was meant to be practiced. Velocity increased as the backlog decreased and, even with the two-week delay to regroup, the project ended up being finished early and under budget.

In all respects, the project was a success. In comparing it to the first project, if this were a competition, the Agile project would have scored higher in every category. The quality was better, and the test cases were cleaner. “There were fewer test cases, but more coverage,” said Holst. Usability was much better, too. “Usability is a key ‘ility’ that’s hard to define in requirements,” said Holst, but joked that “like pornography, you know it when you see it.”

The bottom line was that the Agile project came in early, under budget, and the users loved it.

Keys to success

Holst attributes success to two important factors: Agile training and the regroup effort.

“We hired some consulting training to come in and ground the team in Agile methodology.” This was important because none of the project team had worked with Agile before, so getting the training and having the necessary support was key.

Regarding the regroup effort, Holst says, “We had a major shift in what the product team wanted about halfway through the project and we were able to reform the project and redo logically what we wanted the software to do mid-stream.” Holst believes that the flexibility provided with an Agile methodology that allowed for this regroup was one of the primary keys to success.

Will Agile always prevail?

Even though in this case Agile was the clear winner if this had been a competition between the Agile and Waterfall methodologies, it does not mean that every Agile project will succeed or that Agile is clearly the better methodology. Even the Agile project started out very poorly and had the team continued down that path, that project most likely would have had results just as poor as the waterfall project. However, the ability to inspect and adapt, taking time to regroup and rework requirements, allowed this team to get back on track.

How does the team feel? Let’s just say they want a new coffee mug slogan. They’re no longer saying, “We suck at Agile,” but instead, “Let’s do it again.”