Understanding the role of DevOps in delivering value

At this point, you should have a pretty good handle on what the term value means in the context of Lean-Agile and VSM practices. In this section, we'll explore how DevOps supports the delivery of value. We'll start with a quick introduction to the basic concepts behind DevOps and how it helps instantiate at least some of the values and principles of Agile.

Delivering value in IT

Mature IT organizations install the values and principles outlined in the Manifesto for Agile Software Development to improve their focus on the customer, speed of delivery, and efficiencies. Organizations do not need to integrate or automate their Agile-based SDLC and operations-oriented processes to achieve significant benefits. On the other hand, those organizations that do can rapidly accelerate their pace of delivering new value.

Moreover, IT organizations receive even more significant benefits by improving communications and integration between development and operations. The feedback from operations helps development teams create a more VA, sustainable, and higher-quality product. The collaborations also help the development teams deploy products that are easier to deploy, configure, secure, and roll back when necessary.

It's not wrong to think of DevOps as a reengineering of the IT function. How much reengineering is dependent upon the current state of the IT organization. Those organizations still practicing traditional Waterfall-type practices require more significant changes to their processes and culture than those who practice agile-based approaches such as Scrum or eXtreme Programming (XP). Agile-based methodologies already reengineer traditional SDLC processes from a linear-sequential development life cycle process to an iterative and incremental development cycle, from a practical standpoint.

At first glance, the individual activities of Waterfall and Agile look kind of similar. For example, both approaches include the following type of work:

• Planning
• Requirements gathering and analysis
• Design and architecture
• Development
• Testing
• Customer reviews and acceptance
• Product release and deployment

The primary difference is that Waterfall treats these activities across singular projects as a linear-sequential and plan-driven process. The traditional model lengthens the overall lead time to deliver previously identified value, which often creates a host of problems in finding and fixing bugs. Late deliveries may effectively deliver what customers thought they wanted but not provide what they need at the delivery time. The organization must justify, fund, and initiate new projects before adding new enhancements and correcting defects (previously unidentified customer requirements) in the product, which usually pushes the new work out into the next fiscal year. By then, it is too late.

In contrast, Agile treats SDLC activities as a cyclical process that does not stop until the ROI for continued development no longer supports the cost of adding new value. Agile-based practices release new value incrementally across multiple iterative development cycles as a frequent and repetitive pattern. Short delivery cycles keep the code small between testing, making bugs and defects easier to find and resolve. Frequent customer reviews and team retrospectives help ensure the team stays continuously focused on their customers' current needs, priorities, and requests for improvements.

Though the Agile Manifesto speaks to CD concepts, it does not promote an integration or automation strategy. Those ideas came later. Instead, the Agile Manifesto speaks about improving collaborations and communications across the development function. DevOps started as a strategy to improve collaborations and communications between development- and operations-oriented teams. In its current form, DevOps promotes IT agility through collaboration, integration, and automation across IT to rapidly deliver customer value, higher quality, and less stress.

Automating value across IT

DevOps extends the Agile model in two important ways. First, DevOps extends beyond the traditional SDLC processes to link the activities and people within IT operations. Second, DevOps evolved in lockstep with CI and automation capabilities, which further accelerated the ideation-to-value-delivery lead time while simultaneously improving the quality of delivery.

Business process improvement (BPI) and BPR specialists know that it doesn't make sense to automate a critical business process until after analyzing and implementing the desired process improvements. Ideally, the organization takes a lean approach to map their current as-is and desired to-be value streams to determine what the new process needs to look like and then executes the work to make the desired transformations.

An adage is that automating a flawed process simply makes it put out a lousy result more quickly. In other words, if an organization is already not delivering value, automating the process produces the wrong result more efficiently and more rapidly. In other words, automating a flawed process only generates more waste more quickly.

So, before any IT organization attempts to integrate and automate its SDLC and operations activities, they first need to understand which issues they need to resolve. We'll look at these issues in the next section.

Collaborating across IT development and ops

Conceptually, DevOps began as a strategy simply to improve collaborations between IT development and operations teams. The collaborations helped development teams see the issues operations teams faced when deploying their products. Here are some example issues:

• Without proper collaboration, engineering and test environments may not adequately mimic production environments, causing a host of problems, such as the following:

  a. Production environment configuration instructions may be inadequate.

  b. Testing may not have the same applications installed as the production environments, making it difficult to see configuration, application programming interface (API), and integration conflicts.

  c. Performance testing in engineering and test environments may not adequately evaluate the loads and stresses encountered in production environments.

  d. Rollback and failover instructions developed in engineering and test environments may not work correctly in production environments.

• Development teams may not pay sufficient attention to security concerns in production environments.
• Operations teams lack a practical way to make their concerns known and to make their needs a priority within development teams.
• Operations, through their helpdesk and customer support functions, have the most direct knowledge of customer issues and desired enhancements.

In effect, IT development teams have both internal and external customers they must support. However, when development teams focus only on external customers, they are unaware of the significant technical debt accumulations that severely impact the operations teams.

Now that we've identified the communications and collaboration issues between development and operations, and the fact that development has two customers (internal and external), we can identify typical IT value streams. So, that is the topic of the next section.

Defining IT value chains and value streams

IT organizations are free to define IT value streams in any way they choose, so long as they identify their internal and external customers and take the time to organize and assess their activities to maximize customer-centric value. The VSM chapters provide detailed instructions on mapping as-is and to-be processes from the perspective of value and then monitor and analyze delivery performance against organizational improvement objectives.

On the other hand, an IT organization does not have to start from scratch—for example, the Open Group provides its IT4IT Reference Architecture (that is, IT for IT) as a standard reference architecture and value chain-based operating model for managing IT businesses. The Open Group subscribes to Michael Porter's definition of a value chain as a classification scheme for the complete set of primary and supporting activities that contribute to the life cycle of creating net value of a market offering. In the Open Group's vernacular, an IT organization is a value chain.

In this context, the Open Group defines a value stream, describing the critical activities for a discrete area within the IT value chain. The value stream activities create new net value units within the IT product or service as it progresses through its life cycle. In other words, each value stream activity within a sequence is value-adding. Otherwise, the activity shouldn't exist, or at least should not exist in its current form.

IT4IT defines four primary IT value streams, outlined as follows:

• Strategy to portfolio: Drive IT portfolio to business innovation.
• Requirement to deploy: Build what the business needs, when it needs it.
• Request to fulfill: Catalog, fulfill, and manage service usage.
• Detect to correct: Anticipate and resolve production issues.

Accelerating Agility

One of the Manifesto principles for Agile Development is that Agile processes promote sustainable development indefinitely and constantly. However, the practical reality is that those objectives are challenging to achieve. Customers always seem to have more immediate needs than the team can take on, and the team feels pressure from executives and marketing and sales staff to deliver everything at once and right now. Those pressures don't go away with Agile.

To be sure, development teams look like heroes when they incrementally deliver value that customers what, and when customers can see a timely result from their high priority requests. Still, the earlier sentence stands: "Customers always seem to have more immediate needs than the team can take on."

The integration and automation capabilities employed in a mature DevOps environment substantially accelerate the pace of delivery. In their book Accelerate: Building and Scaling High Performing Technology Organizations, the authors (Forsgren, Humble, and Kim, 2018) compare the metrics of high-performing IT development teams, using DevOps capabilities, to the metrics of the low-performing teams that did not. The results are stunning, as we can see here:

• 46 times more frequent code deployments
• 440 times faster lead time from committing code to deployment
• 170 times faster mean time to recovery (MTTR) from downtime
• Five times lower change failure rate (one-fifth as likely for a change to fail)

This book addresses the fundamental mechanisms of DevOps. Specifically, Chapter 5, Driving Business Value through a DevOps Pipeline, introduces the complexities and challenges of developing CI/CD and DevOps pipelines. Then, in Section 3, Installing DevOps Pipelines - To Compete in Our Digital Economy, of this book, we'll dive into four strategies to develop your DevOps pipelines.

We have now completed our discussions on accelerating Agility and are now going to move on to understand why and how VSM and DevOps are complementary practices.