SDLC Methodologies: From Waterfall to Agile

What are SDLC Methodologies?

SDLC Methodologies are processes and practices used by software development teams in order to successfully navigate the Software Development Life Cycle (SDLC).

We’re not just here to provide you with an exhaustive list of obscure SDLC methodologies. Instead, we’re going to set the record straight on SDLC Methodologies. On the web, you’ll find articles that will define and explain a long list of SDLC Methodologies and give a brief summary of each so you can “choose” which is best for your project. It seems simple and harmless enough, but this is not how SDLC methodologies are used in the professional software development world.

Most legacy SDLC methodologies aren’t even taught in University or bootcamp classrooms. Instead, today’s classes teach Agile frameworks like Scrum and Kanban. Despite this, SDLC methodologies have indeed evolved greatly over time, to the point where once-ubiquitous methodologies like Waterfall have become obsolete and irrelevant other than serving as the history that helps us understand the birth of Agile.

Today, the dominant SDLC methodology used by professional software organizations is Agile along with the many Agile frameworks like Scrum and Kanban that extend its principles beyond software development.
To understand the story of SDLC Methodologies, it is best to look at them chronologically. And while there are a number of methodologies that have been tried, all of them except the Agile family has fallen out of use today. You could even say we live in a Post-Agile world.

The First SDLC Methodology - The Waterfall Method - 1970s to 90s

The first SDLC methodology to take hold in software development was the Waterfall method. Associated with Winston W. Royce, It was first introduced in a paper he wrote and used it as an example of what a bad methodology looks like: "I believe in this concept, but the implementation described above is risky and invites failure." Despite his warnings and guidance, the Waterfall methodology quickly became the standard and stayed that way for over 20 years.

Waterfall is broken down into phases, and other modern methodologies can even pull from these phases and utilize them, these phases are:

• Requirement Analysis
• Planning
• Architectural Design
• Software Development
• Testing
• Deployment
• Maintenance

According to the Waterfall method, the software development process goes through all the SDLC phases with no overlapping and consists of a single development cycle. According to the fact that it is a linear-sequential life cycle model, any phase in the development process can begin only if the previous one is complete. Teams are large and everyone on the team (business analysts, architects, developers, tests, operations, etc.) all work within their own silos.

After the entire architecture, data structures, and functional designs are ready, the development team starts coding the software. Only after all code is written can integration and validation start. This means that the code is not tested before the Testing phase and only unit tests are executed during development.

Finally, the software finishes testing and is deployed to production and for the first time, where users are able to take it for a test drive. The Waterfall method can take several months or even years to complete, which means that if it doesn’t meet user expectations, changes are extremely slow and expensive. In many cases, defects never get fixed at all.

Likewise, due to the lack of feedback from customers or other stakeholders during the design and development process, it was quite common for Waterfall teams to build unnecessary or under-used features, leading to wasted time, effort, and money.

As technology leaders of the 1990s began realizing that the Waterfall method had a tendency to produce lengthy and costly business outcomes, they started seeking more flexible alternatives.

Alternative Methodologies Come and Golean

Waterfall was showing its age, and it never really worked well, to begin with. As a result, pioneers in software developed novel methodologies aiming to either improve or replace Waterfall.

Methodologies like Prototyping, Iterative, Spiral, V-Shape, came and went, and more modern frameworks like Scrum, XP (Extreme Programming), and Kanban were developed around the same time as the standard we use today, Agile. In fact, a lot of folks that signed the Agile Manifesto were XP creators and users.

Understanding some of these now-outdated models helps us better understand how Waterfall transitioned into Agile:

The Prototyping Model

An outdated methodology that is no longer in active use, it served its purpose as one of the earliest alternatives to Waterfall, dating back to the mid 1970s. The Prototype method revolves around the creation of a low fidelity prototype for the purposes of collecting early feedback from prospective users. From there, prototypes are evolved into final software requirements.

The Iterative Model

The Iterative methodology was an early precursor to Agile. It emphasized iterative and incremental action. Its earliest reported use was as part of NASA’s Project Mercury in the early 1960s.

With the Iterative Model, only the major requirements are known from the beginning. Based on these, the development team creates a quick and cheap first version of the software. Then, as additional requirements are identified, additional iterations of the software are designed and built.  Each iteration goes through all the phases of the SDLC and these cycles are repeated until completion. It was common for the team to work on several SDLC phases at the same time.

Spiral Model

The Spiral Method is described by Barry Boehm in his 1986 paper “A Spiral Model of Software Development and Enhancement.” The Spiral Model boils down to a meta-model, which evaluates the specific risk profile of the project before recommending an approach that blends aspects of the other popular methodologies of the day, including Iterative and Waterfall. As such, it rejects a one size fits all approach to process model adoption.

V-Shape Model

The V-Shape model is named after its two key concepts: Validation and Verification. In the Verification Phases, requirements and designs are created. Each Validation Phase has a corresponding Verification Phase, where testing and user acceptance occurs. These two phases are linked together by the Implementation (or coding) phase.

The New Millenium: Agile Takes Over

With no single methodology presenting a suitable alternative to Waterfall, which was woefully too slow and risky, 17 pioneers in software engineering gathered to create the Agile “Software Development” Manifesto on February 11th, 2001.

Agile is the mainstream methodology used in modern software development, and expands its influence beyond coding into many aspects of product development, from ideation to customer experience.

The Agile methodology breaks a project down into multiple cycles, each passing through some or all of the SDLC phases. The focus is on people and how they work together to get the project done. Agile calls for continuous collaboration between team members and stakeholders with regular cycles of feedback and iteration.

The Agile Manifesto’s 4 Core Values

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

Agile Roles

Agile Roles assign responsibilities to members of the team. They are different than positions as a single person can take on multiple Agile roles depending on the scope of the project. Conversely, multiple people can share the same role.
Here are some of the roles you could see in an Agile project:

• Product Owner - The Product Owner, also known as the “voice of the customer”, defines the product vision based on all insights, feedback, and ideas gathered. He or she is the owner of the product requirements and works closely with the development team to communicate the vision by documenting it in short narratives called User Stories. User Stories typically include a name, description, reference to any external documents, and an explanation of how to test the implementation. Product Owners often maintain a backlog of User Stories if there are too many to be executed concurrently.
• Scrum Master - Similar to a project manager, this role is all about making sure the team is following Agile principles, values, and processes.
• Team Member - All members of the development team have different skills and collaborate together to build functional software. Teams can include QA engineers, business analysts, designers, database engineers, and more depending on the project scope.

Advantages of Agile Methodology

• Deliver software well-tailored to an ever-growing understanding of customer demands
• Software is deployed more quickly and improved more regularly
• Better code hygiene including style, readability, and structuring
• Flexible and adaptable process enables pivots or changes mid-project
• Doesn’t require a complete list of requirements upfront
• Makes room to act on organizational learning as the project progresses
• Transparency and continuous communication with involved stakeholders

Agile Frameworks

Organizations can choose to adopt a single Agile framework or they can combine elements of multiple frameworks to suit the needs of the project and characteristics of the team. The most popular Agile frameworks are:

Scrum

Scrum is a very popular Agile framework characterized by continuous collaboration, frequent deliveries, and special development cycles called ‘Sprints’. Scrum revolves around the following checkpoints:

• Planning meetings, in which the team identifies and discusses the Sprint priorities.
• Commitment meetings, in which the team reviews the backlog of user stories to determine how much effort it involves and how much work can be done during the upcoming Sprint.
• Daily standup meetings, which are notably short meetings that ensure everyone is aligned. In this regard, each team member communicates updates on story status, blockers, or concerns.
• Demo meetings, which the team attends at the end of each Sprint to show the functionalities implemented during the current sprint to the Product Owner.
• Retrospective meetings, which are also hosted at the end of each Sprint to discuss lessons learned, what went well, and what needs improvement.

Scrum introduces the Scrum Master role to the Agile method. The Scrum Master’s job is to manage and improve processes, help the team stay authentic to Agile values, and focus on maximizing productivity. A good Scrum Master ensures that the process and progress are transparent to all stakeholders.

Kanban

Kanban is a scheduling system framework for the Agile-eque Lean methodology. It doesn’t have its roots in software development, but synergizes very well with Agile and has become a staple of Agile teams.

Kanban got its start in lean manufacturing, where Toyota applied the same “just in time” principles that supermarkets use to manage inventory stock levels based on customer demand. Kanban, meaning signboard in Japanese, uses cards to track and support the production system by visually showing the steps within the process and how long each step is taking using cards.

Kanban has a host of benefits when applied to Agile. You can limit WIP, focus on cycle time, and utilize just-in-time practices.

Kanban is sometimes compared to Scrum, which are similar in some ways, but are distinct frameworks:

• Scrum utilizes fixed length Sprints cycles while Kanban is about continuous flow
• Scrum is role focused, while Kanban doesn’t utilize roles
• Scrum measures velocity, while Kanban focuses on cycle time

In the Kanban framework, the team creates a visual representation of their tasks and statuses by using sticky notes on a physical whiteboard or by using a dedicated software application. Tasks are moved through predefined stages such as To-Do, In Progress, In Review, or Complete.

A few examples of popular Kanban productivity apps:

• Trello
• Jira
• Proofhub
• Zoho Projects
• ZenHubs

Extreme Programming

Extreme Programming (XP) is an Agile framework focused on project flexibility and writing high quality, well-tested code. The official Extreme Programming website states that XP improves a software project in 5 key ways:

• Communication
• Simplicity
• Feedback
• Respect
• Courage

Extreme Programming is best known for the following:

• Pair programming is a technique where two programmers share the same workstation and create software together. One acts as the driver and the other one as the navigator, then they switch roles. When paired, code review can take place instantly, and defects are more likely to be identified and corrected immediately. Pair programming encourages mentorship, knowledge sharing, and learning. And while it may take more time to produce new code when two developers work on the same task, the resulting code is higher quality with less defects.
• Unit and functional testing are emphasized in XP. Tests are to be comprehensive and automated, reducing technical debt and ensuring code can confidently be validated and re-used.
• Continuous communication between programmers and stakeholders to gather and act upon their input, feedback, and change requests. XP requires an “extended development team” that may include business managers, customers, and other key stakeholders.

Lean

Lean isn’t a software development methodology. Lean’s origins go back to a manufacturing production method invented in the 1930s, officially given a name in the 80s, and more-formally defined in the 90s. Lean is a system that focuses on making more with less. Many have more-recently discovered that Lean works extremely well with software development, especially Agile.

While Agile focuses on delivering continuous value, the goal of Lean is to increase the speed and decrease the cost of product development. With Lean, the highest risks are wasted time and effort. Lean discourages multitasking and encourages team members to focus on what’s important in the present moment. By doing this, the waste associated with unnecessary documentation, meetings, or planning are eliminated.

Lean focuses on the following “just in time” principles:

• Eliminating waste in cost, scope, and scheduling
• Amplifying learning
• Taking decisions as late as possible
• Fast delivery
• Empowering the team
• Building integrity
• Optimizing the entire project

DevOps

DevOps is not technically an SDLC methodology but it does share the goal of maximizing software project success and includes Agile-inspired concepts.

On Wikipedia, DevOps is defined as “a set of practices that combines software development and IT operations. It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. DevOps is complementary with Agile software development; several DevOps aspects came from Agile methodology.”

DevOps, just like Lean, can work alongside Agile to create an infrastructure that eliminates the barriers slowing development and delivery of the final software product. DevOps brings deployment and operation of the software fully into the Agile development process in the same way Agile brought testing and business analysis into software development. Ultimately, the team is empowered to be self-sufficient and take ownership of software development, shipping, and support. They use Continuous Delivery (CD) for frequent releases and to maintain a well-tested and high-quality codebase.

History of DevOps

The DevOps movement started around 2008. The constant pressure to make rapid changes plus the emergence of a new wave of infrastructure automation allowed non-specialists to enter the space and highlighted the need for cross-functional collaboration.

New expectations around delivering more-regular software changes were a big motivation for creating DevOps. Desktop applications were being replaced by web and mobile applications, and instead of delivering physical media (CDs or DVDs), companies began providing Software as a Service (SaaS) over the web. As the industry’s challenges evolved, DevOps offered a solution.

Advantages of DevOps

• Software development teams are self-sufficient; shipping and maintaining software without depending on the IT or technical operations teams.
• The deployment process is automated and optimized. A junior developer can learn to safely deploy, with less effort.
• Teams implement Continuous Integration / Continuous Delivery (CI/CD).
• Using the right tools, engineers save time on deployment so they can focus on coding.
• Feedback loops integrated throughout the entire process.

Conclusion

SDLC methodologies have changed dramatically since the decades of Waterfall method dominance. These changes began intensifying in the 1990s as developers scrambled to find a method that would eliminate the shortcomings of Waterfall, exploring options like Spiral, V-Shape, Iterative, and Prototype. But as the new millennium arrived, software developers began to realize that Agile provided the flexibility and scalability that software organizations required. Agile has since been enhanced by frameworks that extend its principles into every aspect of product and software development, from ideation to deployment.

‍