Measuring DevOps

Once DevOps practices and principles are implemented, the next step is to find out whether these DevOps practices and principles are providing any tangible benefits to the organization. To find the impact of DevOps on delivering changes to customers, appropriate monitoring, audit, and collection of metrics should be developed and deployed. This telemetry should be measured on an ongoing basis. Also, there should be regular baselining of data for effective comparisons in future. After implementing DevOps, the metrics should be captured over a period of time and then compared with the baseline. This comparison of data should uncover intelligence about effectiveness of DevOps in the organization and appropriate corrective measures should be undertaken.

Some of the important metrics that should be tracked are as follows:

Metrics	Impact
Number of deployments	If the number of deployments is higher prior to DevOps implementation, it means that continuous integration, continuous delivery, and deployments favor the overall delivery to production.
Number of daily code check-ins/pushes	If this number is comparatively high, it denotes that developers are taking advantage of continuous integration and the possibilities for code conflict and staleness are reduced.
Number of releases in a month	A higher number is testimony the fact that there is higher confidence in delivering changes to production and that DevOps is helping to do that.
Number of defects/bugs/issues on production	This number should be lower than pre-DevOps implementation numbers. However, if this number is considerable, it reflects that testing is not comprehensive within continuous integration and the continuous delivery pipeline and needs to be further strengthened. Quality of delivery is also low.
Number of failures in continuous integration	This is also known as broken build. This indicates that developers are writing improper code.
Number of failures in the release pipeline / continuous deployment	If the number is high, it indicates that code is not meeting feature requirements. Also, automation of environment provisioning might have issues.
Code coverage percentage	If this number is less, it indicates that unit tests do not cover all scenarios comprehensively. It could also mean that there are code smells with higher cyclomatic complexity.