I've always leaned toward the agile development approach, but I didn't know all the terminology until reading Mark Cohn's book, User Stories Applied. I recently sat down to tout the virtues of Agile development to a friend via email, and what follows is pretty much exactly what I wrote.

There are lots of companies using agile for beginning-to-end large projects, and it has some pretty solid benefits for both the client and the producer. The approach is that first we all acknowledge that it's simply impossible to know all of the requirements of a project from the outset. We have a big picture idea of what the project will be, but we will only know specific details once the client gets a chance to see working functionality and has time to work with the software and further refine their ideas about how exactly the functionality will work. The project beings with a big planning session, just as with a waterfall project, and ideally we will have all the developers, designers, and stakeholders sitting around the same table during the initial planning. User stories are written for all the functionality that can be thought of, but we don't waste time going into mind numbing detail, we just get everything out in the form of "as a [user role] I want [feature] so that [benefit]" (this is the essence of a user story). Once we're fairly confident we have all the user stories taken care of, the developers will give estimates to the complexity of the user stories, and therefore about how long they will take. The estimates aren't contractual in any way, they're just a guage of how long we think the items will take at this point. 

From this point, the customer then prioritizes all the user stories from most important to least important functionality (the "backlog") and then based on the estimates given for those stories, puts together a sprint's worth of user stories. During the sprint planning meeting, (which occurs before each sprint, so every two weeks) the developers ask all of the questions they have about the specifics of those user stories and adjust their estimates accordingly. If it appears we can do more than what was selected, the customer selects another user story to add, but if the discussion reveals the tasks are more complicated, then the customer can remove a story or split stories into smaller stories to make room to do it all in two weeks. At the end of the sprint, those user stories are to be completely finished, with all associated design and functionality, rigorously tested. Ideally, automated tests will also be written for those user stories to prevent regression of that functionality in future sprints. The goal of the sprint is to deliver shippable, production ready code for all user stories. Herein lies one of the biggest benefits for the customer: They have a true picture of how much of their project is done, instead of having large waterfall projects where "90%" of the project is completed at the end of the budget, but in fact 50% of the actual work remains because their are bugs and regressions and features that have been missed.

At the end of the sprint, the customer reviews all of the functionality that's been delivered and the cycle continues with another sprint planning meeting and another sprint kicked off. The burn down charts now come into play to show the momentum of the team against the estimates for all of the functionality, and over the course of the first 2-3 sprints, we get a much more accurate picture of when the project will be completed and what the cost will be, based on actual work, and showing functional code. This is far more accurate than the waterfall approach where we don't have enough information to give estimates and wind up taking a shot in the dark about how complex the system will actually be.

There's another huge advantage to this especially in the Drupal world. Because Drupal comes with so much pre-baked functionality, the first several sprints can focus on just using existing functionality instead of spending long hours perfecting something like the registration process. This allows the customer to see what already exists and get huge amounts of functionality very quickly. They can then prioritize custom coding and designs based on the functionality that already exists in Drupal, rather than reinventing the wheel and potentially doubling the cost of a project when the existing functionality of Drupal or contributed modules would have actually suited the customer just fine, but they didn't know how Drupal did it out of the box. So unless the customer is already deeply familiar with Drupal and all the major contributed modules, the agile method really shines in this perspective.

To sum it up, I feel like the old way of doing things (Waterfall development) is broken. The appeal of it is very strong because it offers lots of guarantees and potentially fixed prices to the customer, but the truth of it is that the contracts and fixed bids don't reflect reality, and I recently read that because of that, two thirds of all software projects either fail completely or go over time, over budget, or both. Agile development is about a relationship of trust and partnership that is built between a customer and a development team. It's not about carefully laid contracts, It's about building something together that is the best it can possibly be. Most importantly, agile development mirrors and adapts to the reality that fixed scope doesn't truly exist in web development as ideas, technology and best practices are always evolving.

var switchTo5x = true;stLight.options({"publisher":"dr-53823897-181c-4461-ae09-5adff87aeeb"});

View the discussion thread.