Clinical Trials and Software Process
Note: this article first appeared here
In the Accelerate book, researchers explain several metrics which they have shown will measure the performance of a DevOps organization, and crucially, drive performance of the organization as a whole. I will explain why this is important, using an analogy with your risk of a heart attack.
In 2018 Nicole Forsgren, Jez Humble and Gene Kim released Accelerate: The Science of Lean Software and DevOps to detail their research into DevOps. They identified a causal link between DevOps organizations which score well on a number of metrics and success in the marketplace. Good scores on those metrics are in turn driven by successful implementation of DevOps practices. The book explains which metrics their research has found to be particularly significant.
If you’ve been following the DevOps movement you won’t be surprised to learn that these practices include Continuous Integration and Deployment Automation. As the research proceeds they continue to refine our understanding of which practices are most significant and which metrics are most useful to measure. I think it is really encouraging that researchers are applying proper science to this problem. This relationship they’ve discovered, between DevOps practices, metrics and organizational performance, is really useful for helping leaders in all kinds of organizations to make more informed decisions about how to do their work.
Avoiding a heart attack
I like to compare it with the relationship between your risk of a heart attack, your cholesterol levels, and your lifestyle. A low risk of heart attack is what you’re trying to optimize for, just as an organization might try to optimize for creating profit, increasing shareholder value, or reducing suffering in the world. Each organization will have some kind of ultimate aim, but as an employee, or even a leader, it is probably not easy to influence directly.
Measuring your cholesterol levels is a way to assess your risk of a heart attack. In much the same way, measuring the DevOps metrics detailed in Accelerate is a way to assess your organization’s chances of successfully meeting its goals. The lifestyle choices you make will, of course, affect your cholesterol levels, and consequently your heart attack risk. Similarly, the practices you use for development and operations will affect the values you score on Accelerate’s DevOps metrics, and hence your chances of achieving your goals. Crucially, the feedback loop is much faster for the metric than for the ultimate goal.
Lifestyle matters
I watched a documentary recently where they had four couples each take up a new diet for one month. Before and after they measured all sorts of things: weight, fat percentage, and a host of biomarkers, including cholesterol. One of the couples changed to quite an extreme Low-Carb diet, where they ate lots of meat and dairy, no fruit, and only a restricted selection of vegetables. After only one month it was noticeable how much higher their cholesterol levels had become. They were still quite young, so their risk of heart attack was still relatively low, but you could see if they persisted with this diet for the long term, the risk was going to become significant.
I think you can use the Accelerate DevOps metrics in a similar way. You can measure them relatively easily for your organization, and get an idea of where you stand. If you have poor values it doesn’t necessarily mean you are in imminent danger of company failure, but it doesn’t look good for the long term. If you make changes to the way you build software, for example by introducing Continuous Integration or investing in deployment automation, you should be able to see changes in your metrics relatively quickly. Hopefully this will encourage you to press on with more productive practices and become more likely to achieve your organization’s aims.
