Building software in a waterfall approach where there is a big milestone release at the end of the process, isn't conducive to success in the modern world, according to top executives from the NYSE and GE.
Speaking at the Zendcon PHP developers conference, Robert Kerner, vice-president at New York Stock Exchange (NYSE) told developers in no uncertain terms, that the waterfall approach doesn't work.
Kerner joined the NYSE a week before a major new software platform was set to roll out, after two years of development and millions of dollars of investment. The platform never saw the light of day, as Kerner killed the effort as it didn't meet expectations and instead set out to build a new platform using an agile iterative approach to releases.
"I started pulling on the thread and found that the waterfall they had setup was pretty much a big lie," Kerner said. "A waterfall kind of is a lie -- not that anyone intends to make it a lie, it's just that any kind of development is iterative, whether you admit it or not. No one writes requirements and specifications to the level of detail that you need."
Kerner noted that business needs change and software development needs to change along with them. One key tenant of the NYSE's move to agile has been an embrace of open source technologies including PHP and the Drupal content management system.
"Our intent from the beginning was to start small and have incremental successes for the business," Kerner said. "That's what agile is all about, delivering incremental business value as you go and not making the business wait two years to get something out of the millions that they are investing."
Kerner added that as part of the NYSE's agile process they now have code sprints that last for two weeks. According to Kerner, every two weeks his team is delivering business value.
"The best part is that we don't have to be finished," Kerner said. "The business knows that we're an agile shop and we're going to deliver something and they're very comfortable giving us the room that we need to grow."
The GE Way
For GE (General Electric), agility is all about simplifying the software development process. Matt Merchant, CTO, GE Corporate, delivered a keynote at the Zendcon conference explaining that through agility and simplification his goal is to produce better quality software at a faster pace.
Cost is a key metric for Merchant who noted that if you can fix a bug at the design stage it costs less to fix than if the same bug exists in production.
"I know you can't write defect-free code, but you can sure as hell try," Merchant said.
Merchant noted that GE started with a waterfall approach where they would have one major release a year. All software development processes were done with a ticket process which generated at least 3,000 tickets a month to migrate code into production.
In an effort to help streamline the process, GE adopted a lean six sigma approach to process management. The goal according to Merchant, was to reduce the number of tickets and to get rid of non-value add sign-offs for approvals.
GE also undertook a process to standardize the tools and systems that its' developers were using in order to further simply development. Merchant noted that GE was able to reduce their development time on large projects by 39 percent, by cleaning up the development process.
"You've got to break down the barriers that are setup within an organization," Merchant said.
Merchant noted that with a big company like GE, people tend to fall back on process to ensure that things don't get out of hand from a compliance point of view.
"What happens then over the course of 10 to 15 years is that those processes create such a bureaucracy that you have no ability to actually deliver anything," Merchant said.
Merchant offered a number of suggestions for organizations seeking to simplify their development process. He suggested that a standardized toolset be used and that security as well quality assurance people are brought in at the beginning of the process. Merchant also noted that at GE they use virtualization as core component as well.
"Simplification can drive good behavior," Merchant said. "It can make you stand back and ask why am I doing that again?"
Merchant also suggested that software development organizations take small incremental steps. He suggested that organization start with their bases processes and understand why they're doing what they're doing.
"Keep asking why do I have these steps?" Merchant said. "Why can I not simplify this stuff down and make it a lot easier?"