Second Wave DevOps

14 years ago, I sat in the audience and watched John Allspaw and Paul Hammond explain how they worked together to deploy Flickr 10+ times a day - the moment what would become DevOps was born. Like everyone else who had the good fortune to see it, I was stunned - at their articulation of the cultural problems that were in our way, and the technology solutions they had developed to solve their problems.

Watching that talk again in 2023, I’m proud of how much we changed the industry. The idea that developers and operations folks must work together is the norm, and has been for a long time. Things like using source control, shared observability, feature flags, dark launching, continuous integration, and continuous delivery are widely considered best practices. If you know what it was like to work in a large enterprise in 2009, it was nowhere close to as good as today. We did that. I’m incredibly proud of being a part of the DevOps movement, and you should be too. Disruptive innovation with such a positive impact on people doesn’t happen like that very often.

The DevOps movement has always been clear about our objectives - to bring a culture of collaboration to the way we build and operate our software, alongside tooling to “enable the fast flow of planned work into production (e.g., performing tens, hundreds, or even thousands of code deploys per day), while achieving world-class stability, availability, and security.” (taken from the DevOps Handbook’s introduction - the first paragraph, no less. Thanks, Gene, Jez, Patrick, and John)

But after 14 years of effort - 88% of the respondents to the State of DevOps report in 2022 are unable to deploy more frequently than once a week, often once every six months. What’s the deal? Were our ambitions simply too high? Is the enterprise just too complex? Do people not want to collaborate? Are we not DevOps-ing hard enough?

Here is what I know: it wasn’t for lack of conviction. If there is one thing we have unequivocally nailed it is the culture of trust and collaboration at the heart of DevOps work. This is how people want to work. We’re not failing because we don’t want it enough.

Let’s face facts: our implementation is what’s letting us down. What worked for John and Paul in 2009 is, in broad strokes, exactly what we have been asking every single DevOps practitioner to do since. We’ve replaced all the individual tools in the system multiple times (look at the CNCF Cloud Native Landscape for the evidence): less automated infrastructure, more infrastructure as code; less monitoring, more observability; less data centers, more cloud; less svn, more git; less virtual machines, more docker; less capistrano, more kubernetes; less hudson, more github actions. The problem isn’t that we haven’t optimized each individual part of the system enough. We’ve built more efficient tooling at every step. But the way the whole system is put together? The experience of using it? That’s basically identical to how it was in 2009, and it’s the reason we’re stuck.

As the complexity of the environments we tried to improve grew, we never stopped to ask ourselves if the system as a whole would work for everyone. We didn’t design it for our aspirations, but for the immediate issues we had to solve. We took the techniques we had in 2009, and we pushed them harder. We refined them. We replaced some of them. But we never sat down and looked at them holistically and asked if the whole adds up to the sum of its parts when the scale of our ambition reaches every technology company on the planet. We just figured since it worked that way for Paul and John, it’ll work that way for everyone if we just try hard enough.

We know now what the fate in store for the majority of us will be if we follow everything that the DevOps handbook tells us to do, the way it tells us to do it: we will improve, but we will ultimately fall short. People are writing non-ironic blog posts about how it’s time to go back to dev being dev, and ops being ops; about how DevOps is dead. The failure of our implementations has put the success of our culture change at risk. We let it happen at our peril - I remember what it was like before DevOps - it was unequivocally worse for everyone involved.

I still believe that everyone in technology deserves to work in a culture of collaboration and trust, deploying software hundreds or thousands of times a day, while achieving world-class stability, availability, and security. If we want it to come true - not to be consigned to history as another quaint idea co-opted by industry - we have to start breaking the rules of how the system is designed.

The best evidence you have is your own day-to-day experience. Doing “DevOps work” is unquestionably the worst part of building a modern application. It’s full of tiny papercuts, indignities we suffer in our toolchains, our feedback loops, and our software. It’s a city of brutalist buildings filled with sharp-edged couches pretending to be comfortable. Think of all the advances in how we interact with tools in other domains - then take a look at the way you build, deploy, and operate your software, at all the crazy gyrations you use to glue it all together - and ask yourself why you accept it.

If we want to keep the cultural improvements we’ve gained and realize the full potential of what we can accomplish together, we need a second wave of DevOps tools. A reinvigorated movement of builders, who are willing to explore what’s possible when we are no longer constrained by the way the system worked in 2009. We can fix the user experience of doing DevOps work. We can re-imagine what it’s like to collaborate on application deployment, infrastructure automation, and observability. Or we can continue to ossify around the status quo.

System Initiative is our attempt to break those rules and dream up an approach that makes the daily experience better - starting with what we know best, infrastructure automation, but hopefully growing into so much more. I’m excited and proud of it. But I know it won’t be enough. If we’re going to accomplish what we set out to achieve 14 years ago, it’s going to take all of us together to build Second Wave DevOps tools. What will you build when you stop being constrained by what has come before? None of this existed when we started. We built it all. Every choice was made by someone exactly like you. If we want to change our fate, we have to change the system itself. Get ambitious again. Don’t wait for anyone else to do it for you. Go do it. Let’s build Second Wave DevOps together, and finish what we started.

To learn more about System Initiative, head over to our home page and watch our short demo video.