Understanding DevOps, its evolution, and life cycle

This section focuses on the evolution of DevOps and lists phases or critical practices that form the DevOps life cycle.

Revisiting DevOps evolution

Let's take a step back and think about how DevOps has evolved. Agile software development methodology refers to a set of practices based on iterative development where requirements and solutions are built through collaboration between cross-functional teams and end users. DevOps can be perceived as a logical extension of Agile. Some might even consider DevOps as an offspring of Agile. This is because DevOps starts where Agile logically stops. Let's explore what this means in detail.

Agile was introduced as a holistic approach for end-to-end software delivery. Its core principles are defined in the Agile Manifesto (https://agilemanifesto.org/), with specific emphasis on interaction with processes and tools, improving collaboration, incremental and iterative development, and flexibility in response to changes to a fixed plan. The initial Agile teams primarily had developers, but it quickly extended to product management, customers, and quality assurance. If we factor in the impact of the increased focus on iterative testing and user acceptance testing, the result is a new capacity to deliver software faster to production.

However, Agile methodology creates a new problem that has resulted in a need for a new evolution. Once software is delivered to production, the operations team are primarily focused on system stability and upkeep. At the same time, development teams continue to add new features to a delivered software to meet customers' dynamic needs and to keep up with the competition.

Operators were always cautious for the fear of introducing issues. Developers always insist on pushing changes since these were tested in their local setup, and developers always thought that it is the responsibility of the operators to ensure that the changes work in production. But from an operator's standpoint, they have little or no understanding of the code base. Similarly, developers have little or no understanding of the operational practices. So essentially, developers were focused on shipping new features faster and operators were focused on stability. This forced developers to move slower in pushing the new features out to production. This misalignment often caused tensions within an organization.

Patrick Debois, an IT consultant who was working on a large data center migration project in 2007, experienced similar challenges when trying to collaborate with developers and operators. He coined the term DevOps and later continued this movement with Andrew Shafer. They considered DevOps as an extension of Agile. In fact, when it came to naming their first Google group for DevOps, they called it Agile System Administration.

The DevOps movement enabled better communication between software development and IT operations and effectively led to improved software with continuity being the core theme across operating a stable environment, consistent delivery, improved collaboration, and enhanced operational practices with a focus on innovation. This led to the evolution of the DevOps life cycle, which is detailed in the upcoming sub-section.

DevOps life cycle

DevOps constitutes phases or practices that in their entirety form the DevOps life cycle. In this section, we'll look at these phases in detail, as shown in the following diagram:

Figure 1.1 – Phases of the DevOps life cycle

There are six primary phases in a DevOps life cycle. They are as follows:

• Plan and build
• Continuous integration
• Continuous delivery
• Continuous deployment
• Continuous monitoring and operations
• Continuous feedback

The keyword here is continuous. If code is developed continuously, it will be followed with a need to continuously test, provide feedback, deploy, monitor, and operate. These phases will be introduced in the following sections.

Phase 1 – plan and build

In the planning phase, the core focus is to understand the vision and convert it into a detailed plan. The plan can be split into phases, otherwise known as epics (in Agile terminology). Each phase or epic can be scoped to achieve a specific set of functionalities, which could be further groomed as one or multiple user stories. This requires a lot of communication and collaboration between various stakeholders.

In the build phase, code is written in the language of choice and appropriate build artifacts are created. Code is maintained in a source code repository such as GitHub, Bitbucket, and others.

Phase 2 – continuous integration

CI is a software development practice where developers frequently integrate their code changes to the main branch of a shared repository. This is done, preferably, several times in a day, leading to several integrations.

Important note

Code change is considered the fundamental unit of software development. Since development is incremental in nature, developers keep changing their code.

Ideally, each integration is triggered by an automated build that also initiates automated unit tests, to detect any issues as quickly as possible. This avoids integration hell, or in other words, ensures that the application is not broken by introducing a code change or delta into the main branch.

Phase 3 – continuous delivery

Continuous delivery is a software development practice to build software such that a set of code changes can be delivered or released to production at any time. It can be considered an extension of CI and its core focus is on automating the release process to enable hands-free or single-click deployments.

The core purpose is to ensure that the code base is releasable and there is no regression break. It's possible that the newly added code might not necessarily work. The frequency to deliver code to production is very specific to the organization and could be daily, weekly, bi-weekly, and so on.

Phase 4 – continuous deployment

Continuous deployment is a software development practice where the core focus is to release automated deployments to production without the user's intervention. It aims to minimize the time elapsed between developers writing new line(s) of code and this new code being used by live users in production.

At its core, continuous deployment incorporates robust testing frameworks and encourages code deployment in a testing/staging environment post the continuous delivery phase. Automated tests can be run as part of the pipeline in the test/stage environment. In the event of no issues, the code can be deployed to production in an automated fashion. This removes the need for a formal release day and establishes a feedback loop to ensure that added features are useful to the end users.

Phase 5 – continuous monitoring and operation

Continuous monitoring is a practice that uses analytical information to identify issues with the application or its underlying infrastructure. Monitoring can be classified into two types: server monitoring and application monitoring.

Continuous operations is a practice where the core focus is to mitigate, reduce, or eliminate the impact of planned downtime, such as scheduled maintenance, or in the case of unplanned downtime, such as an incident.

Phase 6 – continuous feedback

Continuous feedback is a practice where the core focus is to collect feedback that improves the applica/service. A common misconception is that continuous feedback happens only as the last phase of the DevOps cycle.

Feedback loops are present at every phase of the DevOps pipeline such that feedback is conveyed if a build fails due to a specific code check-in, a unit/integration test or functional test fails in a testing deployment, or an issue is found by the customer in production.

GitOps is one of the approaches to implement continuous feedback where a version control system has the capabilities to manage operational workflows, such as Kubernetes deployment. A failure at any point in the workflow can be tracked directly in the source control and that creates a direct feedback loop.

Key pillars of DevOps

DevOps can be categorized into five key pillars or areas:

• Reduce organizational silos: Bridge the gap between teams by encouraging them to work together toward a shared company vision. This reduces friction between teams and increases communication and collaboration.
• Accept failure as normal: In the continuous aspect of DevOps, failure is considered an opportunity to continuously improve. Systems/services are bound to fail, especially when more features are added to improve the service. Learning from failures mitigates reoccurrence. Fostering failure as the normal culture will make team members more forthcoming.
• Implement gradual change: Implementing gradual change falls in line with the continuous aspect of DevOps. Small, gradual changes are not only easier to review but in the event of an incident in production, it is easier to roll back and reduce the impact of the incident by going back to a last known working state.
• Leverage tooling and automation: Automation is key to implement the continuous aspect of CI/CD pipelines, which are critical to DevOps. It is important to identify manual work and automate it in a way that eventually increases speed and adds consistency to everyday processes.
• Measure everything: Measuring is a critical gauge for success. Monitoring is one way to measure and observe that helps to get important feedback to continuously improve the system.

This completes our introduction to DevOps where we discussed its evolution, life cycle phases, and key pillars. At the end of the day, DevOps is a set of practices. The next section introduces site reliability engineering, or SRE, which is essentially Google's practical approach to implementing DevOps key pillars.