This book will focus on implementing and scaling DevOps in large enterprises. Before we get into the specific challenges of an enterprise, we need to have a common understanding of DevOps.

Somewhere, businesses and their leaders must have thought that it was a good idea to put developers and operators into one team. In essence, DevOps is the development and operations stages working as one team, on the same product and managing it. You build it, you run it.

DevOps has gained a lot of momentum over the past decade, especially in enterprises. But implementing DevOps turned out to be quite difficult. The reason for this is that enterprises are not organized in a structure that works for DevOps. From the last century onward, most enterprises outsourced a lot of their IT. Most of the IT muscles of a major enterprise are therefore still with system integrators and software houses. DevOps becomes more difficult when development is done by a software house and operations is outsourced to a system integrator.

DevOps starts with the business. By bringing teams together into a development and operations environment that traditionally work in silos, an enterprise can speed up development and release new products and services. The rationale behind this is that less time is needed to do handovers between development and operations. Also, by removing the barrier between development and operations, the quality of products will improve since DevOps includes quality assurance, testing, and security. Customer feedback is continuously evaluated and included in new iterations of the product.

The benefits of DevOps are as follows:

• It brings business, development, and operations together, without silos.
• Enterprises can respond faster to demands from the market because they're absorbing continuous feedback.
• Products are continuously improved and upgraded with new features, instead of planning for major next releases.
• Through automation in DevOps pipelines, enterprises can reduce costs in terms of both development and operations and, at the same time, improve the quality of their products.

It starts with the business and thus the starting point is the enterprise architecture. This is where the business goals are set and we define how these goals will be met. IT delivery is key to meeting these goals. In large enterprises, the architecture also defines the IT delivery processes and the demarcation between these processes. We will look at IT delivery and its processes in more detail in the next section.

Understanding IT delivery in enterprises

As we mentioned at the beginning of this section, large enterprises typically have an operating model that is based on outsourcing. This makes implementing DevOps more complicated. The enterprise architect will have to have a very clear view of the demarcation between the different processes and who's responsible for fulfilling these processes. Who is responsible for what, when, and why? The next question is, how does it map to DevOps?

First, we need to understand what the main processes are in IT delivery. These processes are as follows:

• Business demand: A business needs to understand what the requirements are for a product that it delivers. These requirements are set by the people who will use the product. Customers will demand a product that meets a specific functionality and quality. The architecture must focus on delivering an end product that satisfies the needs of the customers of an enterprise. IT delivery is a crucial part of delivering an end-product. In DevOps, an assigned product owner makes sure that the product meets the requirements. The product owner will have to work closely with the enterprise architect. In the Creating a reference architecture section, we will learn that the enterprise architecture and DevOps are complementary.
• Business planning: Once the demand is clear, the product needs to be scoped. In DevOps, product teams typically start with a Minimum Viable Product (MVP), a first iteration of the product that does meet the requirements of the customer. When designing the MVP, processes need to be able to support the development and operations of that product. Hence, business planning also involves quality management and testing, two major components of IT delivery. This needs to be reflected in the architecture.
• Development: In DevOps, the product team will work with user stories. A team must break down the product into components that can be defined as deliverables. For this, we must have a clear definition of the user story. A user story always has the same format: As a [function of the user] I want to [desire of the user] so that I [description of the benefits a user will get if the function has been delivered and the goal is achieved]. The key of any user story is its acceptance criteria, or the Definition of Done (DoD). When is the product really finished and does it meet the goals that have been set? In Chapter 3, Architecting for DevOps Quality, you will learn more about the DoD.

One important remark that must be made is that when we refer to a product, we are talking about a product that is code-based.

Tip

There's one major movement in IT delivery: everything in IT is shifting to code. It's one of the main principles of The Modern DevOps Manifesto: Everything is code. It applies to applications, but also to infrastructure components such as network devices, servers, and storage devices. Therefore, DevOps not only includes software development and operations for applications, but also for infrastructure with Infrastructure as Code and Configuration as Code. Public clouds such as AWS, Azure, and Google Cloud Platform play a significant role in these developments.

In other words, the team is developing code: application code, Infrastructure as Code, and also test code. A developer will work on a specific piece of code that has been defined in the product backlog. The whole end product – for instance, an application – has been broken down into product backlog items (PBIs), where each developer will work on a PBI. As soon as a piece of code is ready, it needs to be tested on itself, but also as a component of the end product. Due to this, in development, code needs to be merged. This merging process is triggered by a pull request, where the developer requests to have the code merged and joined to the end product, thus fulfilling the user story. This is done using pipelines.

In Chapter 2, Managing DevOps from Architecture, we will discuss setting up and managing pipelines, both for application development and for infrastructure.

We can divide the whole DevOps cycle into two major phases called deployment and operations, as follows:

• Deployment: In this stage, the code is tested and validated so that it matches the user story. It will now be deployed to the production state. Testing and releasing to production is a process that, ideally, is automated in the pipeline, as is integration. Before the code is actually pushed to production, it also needs to be merged with configuration. Think of security packages that need to be applied to components that run in production. In the test and quality process, the full package – application code and infrastructure components – needs to be validated as "ready for production". The result should be a "live" product. If, when you're performing testing and validation, bugs, flaws, or violations of security rules are discovered, the product will be sent back to an earlier stage in the development process.
• Operations: After deployment, the live product needs to be operated on. For this, enterprises work according to IT Service Management (ITSM) principles. The fact that operators are in the same team as developers doesn't mean that the ITSM processes are not valid anymore. An example is when incidents occur and the incident management process must be triggered. In operations, we distinguish between the following main processes:

  a) Request fulfillment

  b) Incident management

  c) Problem management (postmortem)

  d) Configuration management

  e) Change management

But DevOps adds something to this; that is, continuous integration and continuous delivery (CI/CD):

• Continuous integration (CI): CI is built on the principle of a shared repository, where code is frequently updated and shared across teams that work in the cloud environments. CI allows developers to work together on the same code at the same time. The changes in the code are directly integrated and ready to be fully tested in different test environments.
• Continuous delivery (CD): This is the automated transfer of software to test environments. The ultimate goal of CD is to bring software to production in a fully automated way. Various tests are performed automatically. After deployment, developers immediately receive feedback on the functionality of their code.

CI/CD requires a feedback loop to make it continuous. It needs feedback about the delivered products and services. This is then looped back to the developers and from there, new iterations are planned to improve the product or service.

This works well if an enterprise controls the full cycle, but large enterprises have outsourced a vast number of activities to other companies. The rationale behind this sourcing strategy is typically because a certain activity is not perceived as a core activity, and it can be done more cost-effectively by a company that specializes in such activities.

However, enterprises have gone through a massive change over the last decade. IT has become more and more important and, in some cases, has become a core activity. Banks are a good example. Banks are IT companies nowadays, and the output of their IT delivery is financial products. Due to customer demands, releases of these products with new features have become more frequent, with up to several releases per day. The consequence of this is a major shift in IT delivery itself.

The next few sections will discuss how IT delivery works in sourcing models and how it impacts successfully implementing DevOps.

IT delivery in sourcing models

In this section, we will look at the sourcing model in large enterprises. This can be quite complicated, but if we learn to think in terms of sourcing tiers, it becomes more tangible and comprehensible. This is the target enterprise model, as shown in the following diagram:

Figure 1.1 – Target enterprise model

Using this model, we can break down IT delivery into three tiers:

• Tier 1: Strategic level. This is the tier for enterprise governance. The enterprise defines the strategic business goals that are translated in the enterprise architecture. The overall architecture principles are the outcome of the enterprise architecture and drive the IT architecture, including DevOps. We will discuss this further in the Creating the reference architecture section.
• Tier 2: Tactical level. This the tier where the IT architecture is developed, including DevOps. It's also the tier where service-level agreements (SLAs) and key performance indicators (KPIs) are defined to measure the outcomes of IT delivery. You will learn more about this in the Understanding SLAs and KPIs in DevOps section.
• Tier 3: Operational or services level. At this level, the components of the architecture are detailed, including the interfaces to the various suppliers and service providers. The agreements that are defined in tier 2 must be adopted at this level so that all involved developers and operators work in the same way, with the same tools and with the same understanding of the goals. In the Understanding DevOps components section, we will learn more about this.

In practice, we see service providers also acting on tier 2; for instance, if they are involved in larger programs spanning multiple products. Tier 2 then becomes the orchestration level, where a provider is responsible for aligning different streams in the lower tier. The key takeaway is that tier 1 should always be the enterprise level, where the overall governance and architecture is defined.

In this section, we learned that a lot of enterprises have outsourced larger parts of their IT and that this can complicate the process of implementing DevOps. We learned that the strategy for the entire enterprise is at tier 1, the highest tier. DevOps sits on the lowest tiers, where projects are actually executed. This is the tier where the enterprise interfaces with sourcing companies. However, this only works if we have a clear view of the architecture. We will discuss this in the next section.

In the previous sections, we looked at the different processes in IT delivery, how it is integrated with DevOps, and how this is executed in sourcing models. We have learned that it starts with a clear architecture that clearly defines the processes.

Any DevOps architecture will have to address planning, development, integration, deployment, and operations. But we have to keep in mind why we are doing DevOps, which is to achieve business goals in a faster, more agile way where we continuously improve products. The DevOps architecture does not stand on its own; it has to be linked to the enterprise architecture.

An enterprise architect will most likely start from The Open Group Architecture Framework (TOGAF). TOGAF is globally accepted as the standard for enterprise and business architecture. It uses the Architecture Development Method (ADM) to draft the architecture. The ADM is shown in the following diagram:

Figure 1.2 – The ADM cycle in TOGAF

Just like DevOps, the ADM is a cycle – except for the preliminary phase, which is where the need for an architecture is formulated. This has to be done at the tier 1 level in the sourcing model that we discussed in the previous section, IT delivery in sourcing models. The strategy and enterprise architecture is always set at tier 1.

The core of ADM is the sequence in architecture design, which is business first, and setting the principles and requirements for the actual solutions. These principles drive the architecture for data usage, the applications, and finally the technology that will be used. This is important because architecture is not about technology in the first place. Technology is purely the ability to achieve business goals at the enterprise level.

ADM assumes that the architecture is not static. It changes as soon as the business requirements change. If the business demands change, there will likely be a need to adapt the architecture and the forthcoming solutions. This is where TOGAF and DevOps meet, since the two frameworks complement each other.

The following table shows where the enterprise architecture and DevOps are complementary. To put it very simply, the enterprise architecture sets the strategic business goals, where DevOps translates this at a more tangible, tactical level and really tells us how to achieve these goals by developing products and executing operations. The following table shows the main differences between the enterprise architecture (EA) and DevOps:

In the next section, we will study the DevOps principles.

Understanding the DevOps principles

The enterprise architecture is executed on tier 1, the strategic level. This is where the goals are set for the entire enterprise. The next level is tier 2, where DevOps teams will translate the goals into product features and start developing. DevOps teams will have to work according to a set of principles.

In this section, we will look at the main principles for DevOps. In this book, we will use the six principles from the DevOps Agile Skills Association (DASA):

• Customer-centric action: Develop an application with the customer in mind – what do they need and what does the customer expect in terms of functionality? This is also the goal of another concept, domain-driven design, which contains good practices for designing.
• Create with the end-result in mind: How will the product look when it is completely finished?
• End-to-end responsibility: Teams need to be motivated and enabled to take responsibility from the start to the finish of the product life cycle. This results in mottos such as you build it, you run it, and you break it, you fix it. One more to add is you destroy it, you rebuild it better.
• Cross-functional autonomous teams: Teams need to be able to make decisions themselves in the development process.
• Continuous improvement: This must be the goal – to constantly improve the product.
• Automate as much as possible: The only way to really gain speed in delivery and deployment is by automating as much as possible. Automation also limits the occurrence of failures, such as misconfigurations.

Adhering to these principles will lead to the following architecture statements, which are at the core of DevOps:

• Automation: Following the principle of "everything is code," the next step is "automate everything." With automation, the amount of time between testing and deployment will be significantly reduced, enabling a faster release process. But automation will also lead to less manual interaction and therefore less errors.
• Collaboration: Two of the six principles are cross-functional autonomous teams and end-to-end responsibility. This can only be achieved by collaboration. Development and operations will have to work very closely together to speed up the delivery of releases. Although this is also a cultural change, collaboration requires a common toolset that supports collaboration.
• Integration: Development and operations come together, but also, business and IT come together. In DevOps, we integrate the business demands with IT delivery using user stories. Code is integrated with new functionality that is coming out of business demand. That demand is changing faster these days, so development needs to keep up by means of CI. This will lead to changes in operations as well – they will need to adopt these new developments at the same speed. With that, integration is the core of DevOps.
• Portfolio and configuration management: Automation and integration require a clear portfolio that contains the building blocks that can be automated easily. These building blocks are artifacts in the ADM cycle, and they represent packages of functionality that can be used to fulfill a requirement. A building block is reusable and replaceable; therefore, it must be clearly and specifically defined. Better said, the configuration of the building blocks needs to be well documented and brought under the control of configuration management. If done well, these building blocks will also have clear interfaces so that they can be fully automated.

In this section, we looked at the IT delivery processes and how they impact DevOps. We learned that IT delivery is driven by business demand and that this business demand is the starting point for any architecture. This is included in the TOGAF framework for the enterprise architecture. After that, we mapped the enterprise architecture to DevOps principles.

In the next section, we will merge the DevOps principles for the architecture and the IT delivery principles into a reusable reference model.

Working with the DevOps architecture reference model

The final step is to merge the DevOps principles into one model for our reference architecture. The model contains two circles. The outer circle is the product circle, while the inner circle represents the operational activities. As a logical consequence, the outer circle is governed by the enterprise itself.

The inner circle is about actually delivering the products using DevOps. There are interfaces between the outer and inner circle: collaboration, automation, integration, and configuration management.

The reference model is shown in the following diagram:

Figure 1.3 – The DevOps architecture reference model

In the outer circle, the business goals are translated into the architecture. From the architecture, a portfolio is created with building blocks to create products and services. Products are released to the market and adopted, but due to changing demands, there will be requests for changes. These changes will drive enterprise planning and ultimately change the business goals, meaning that the business will constantly have to adapt to changing demands. This is the field of enterprise architecture.

The plans and the actual builds are executed in the inner circle. In this circle, the product is broken down into product backlog items that will be developed and eventually operated on by DevOps teams. These teams do not operate by themselves, but on triggers from the outer circle. That's what the interface layer is about – it's the interface between the business and the execution teams doing IT delivery. There's collaboration between architecture and development. Releases should be automated as much as possible, requests and changes must be integrated with planning and the backlog of the DevOps teams, and builds that are pushed to production must be monitored and brought under the control of configuration management so that the architecture and portfolio stay consistent in case of changes.

Let's have a look at how this would work in practice by plotting personas into the model. This will result in a DevOps workflow for enterprises, as shown in the following diagram:

Figure 1.4 – DevOps workflow for enterprises

Here, we have created a model where the enterprise has full control over its portfolio and products. Yet, it can improve quality and speed up delivery by working with combined, multidisciplinary teams – even those that come from different suppliers.

In the next section, we will study the final, lowest tier in our model and discover various DevOps components.

So far, we've learned how to start defining the architecture, looked at the architecture principles for DevOps, and drafted a reference architecture model. The next step is to look at the different components within DevOps. In this section, we will learn what components must be included in a DevOps architecture. This is tier 3 of our target enterprise model – the level where all the activities are executed.

The following diagram shows all the components that will be discussed briefly:

Figure 1.5 – The DevOps life cycle

The reason that this has been presented as an infinite loop – or a pretzel – is because feedback from the live product that is managed by ops (operations) will be continuously looped back to dev (development) to improve the product.

The different components are as follows:

• Plan
• Create (in some DevOps models for components, this is referred to as Code and Build)
• Test (in some models, this is referred to as Verify or Validate)
• Preprod (in some models, this is referred to as Pre-release)
• Release
• Configure
• Monitor

At this level, interoperability is crucial. Remember that large enterprises will likely work with several service providers, fulfilling parts of the IT delivery process. When we want all these companies to work together in a DevOps way, we need to make sure that the processes and tools are aligned. Next, we need to have a common understanding of the various activities that are executed as part of these processes. The key term here is consistency. All DevOps components must be defined and implemented in a consistent way. Every developer and operator must work according to the same definition and with the same components.

The main question is, in what stage should ops already be involved? The answer is, at the earliest stage possible, so indeed in the plan phase. Ops plays a key role in defining how products can be managed once they've gone live. They should set requirements and an acceptance criterion before going live. If a developer builds something that can't be managed by ops, the product will fail, and business demands will not be met.

The following table breaks down the components into activities:

In Chapter 2, Managing DevOps from Architecture, and Chapter 3, Architecting for DevOps Quality, we will dive deeper into this and how architects can improve their designs for these components using CI/CD pipelines to enable automation, collaboration, and integration.

In the next section, we will discuss the drivers for architecture from a business perspective, as laid down in SLAs and KPIs.

In the Understanding IT delivery in enterprises section, we learned that in DevOps, IT delivery and IT service management processes are still valid. Typically, enterprises contract SLAs and KPIs to fulfill these processes so that these enable the business goals. If one of the processes fails, the delivery of the product will be impacted and as an ultimate consequence, the business will not achieve its goals, such as an agreed delivery date or go live release of the product. Hence, understanding SLAs and KPIs is important for any architect. This is why it is included in the sourcing model that we discussed in the IT delivery in sourcing models section.

Service-level agreements are positioned between the tactical processes of DevOps and the strategic level at the enterprise level where the goals are set. SLAs and KPIs should support these goals and guide the DevOps process.

The six most important metrics that should be included in SLAs for DevOps are as follows:

• Frequency of deployments: Typically, DevOps teams work in sprints, a short period of time in which the team works on a number of backlog items as part of the next release of a product. The KPI measures how often new features are launched on a regular basis. Keep in mind that releases of new features can be scheduled on a monthly (often spanning multiple sprints), weekly, or even daily basis.
• Deployment time: The time that elapses between the code being released after the test phase to preproduction and ultimately production, including the ready state of the infrastructure.
• Deployment failure rate: This refers to the rate of outages that occur after a deployment. Ideally, this should be zero, but this is not very realistic. Deployments – especially when the change rate is high – will fail every now and then. Obviously, the number should be as low as possible.
• Deployment failure detection time: This KPI strongly relates to the previous one. Failures will occur, but then the question is, how fast are these detected and when will mitigating actions to resolve these issues be taken? This KPI is often also referred to as Mean Time to Recovery (MTTR). This is the most important KPI in DevOps cycles.
• Change lead time: This is the time that elapses between the last release and the next change to occur. Subsequently, it is measured in terms of how long the team will need to address the change. Shorter lead times indicate that the team works efficiently.
• Full cycle time: The total time that elapses for each iteration or each deployment.

This list is by no means exhaustive. Enterprises can think of a lot of different metrics and KPIs. But the advice here is to keep things simple. Keep in mind that every metric that is included in any contract needs to be monitored and reported, which can become very cumbersome. One more thing to remember is that the most important metric sits at the business level. Ultimately, the only thing that really counts is how satisfied the customer of the business is or, better said: what's the value that's delivered to the end customer?

In the final section of this chapter, we will elaborate on the term value by explaining the VOICE model.

DevOps teams need to deliver value to the end customer. The VOICE model, as defined by the IT company Sogeti, addresses this. VOICE stands for Value, Objectives, Indicators, Confidence, and Experience. The idea behind this model is that any IT delivery should deliver value to someone – typically, the end customer of a business. Value sets the objectives for IT delivery and these objectives are measured using indicators. Indicators also measure whether the pursued value will be achieved.

Confidence is about the indicators and if they contain relevant information to confirm that IT delivery actually results in the targeted value. Lastly, experience tells us if the delivered system is fulfilling the business demands and which improvements will lead to more business value. With that, the cycle starts over again.

This model is shown in the following diagram:

Figure 1.6 – The VOICE model

Since VOICE also involves looping feedback back to the beginning of the cycle with the aim of improving products and adding more value to the business, the model can be used for DevOps projects.

In Chapter 3, Architecting for DevOps Quality, we will explore VOICE in more detail.

This chapter was the introduction to DevOps for architects. We learned that the enterprise architecture sets the architecture principles for the entire enterprise by using the TOGAF methodology. The business goals are defined at the enterprise level. DevOps projects and teams are concerned with IT delivery and fulfilling the business' demands by building, deploying, and running IT systems.

DevOps needs to adhere to the business goals and, therefore, with the enterprise architecture. Yet, DevOps features a specific architecture that enables CI/CD in IT systems. Because of this, we learned about the six DevOps principles and how these are applied to a reference model in which the enterprise still has full control of the products, but multidisciplinary teams can work autonomously on them.

Next, we looked at the different DevOps components and KPIs to measure the outcomes of DevOps projects. The key takeaway from this is that every project needs to add to a better user experience and thus add business value. Due to this, we briefly studied the VOICE model.

In the next chapter, we will learn more about automation, collaboration, and integration by designing CI/CD pipelines.

1. True or false: DevOps brings business, development, and operations together, without silos.
2. The DevOps principles lead to four key attributes in the architecture for DevOps. One of them is automation. Name the other three.
3. What does the acronym CI/CD stand for?
4. Ideally, every deployment succeeds, but in practice, some deployments will fail. The time that is needed to detect this failure and start mitigating actions is an important KPI in DevOps contracts. What is the commonly used term for this KPI?

• The Modern DevOps Manifesto: https://medium.com/ibm-garage/the-modern-devops-manifesto-f06c82964722
• Quality for DevOps Teams, by Rik Marselis, Berend van Veenendaal, Dennis Geurts and Wouter Ruigrok, Sogeti Nederland BV.