Since it was introduced in 2009, the DevOps mindset has encouraged organizations to put resources behind improving metrics like lead time, deployment frequency, change failure rate, and mean time to recovery (MTTR). This approach worked for many leading engineering organizations to develop, deliver, and ship software faster and more efficiently than ever before. Some were even able to deploy hundreds or thousands of times a day. In doing so, they began delivering customer value at an unthinkable speed vs the good old days of chucking code over the fence.
But while DevOps turbocharged productivity and efficiency for some organizations, its adoption fell short of the high expectations set by others. And although many teams are now sailing happily along their DevOps journey, too many still remain stuck in the middle. Most are frustrated, some burnt out, and many unable to cross what we at Humanitec calls the “DevOps Mountain of Tears”.
Recently, platform engineering has garnered a serious amount of hype as the possible next step for DevOps. It involves forming dedicated platform teams to build internal developer platforms (IDPs) that enhance software delivery and lift many DevOps limitations. But already the discipline is proving to be much more than just hype. IDPs have powered initiatives at Nike and Starbucks, helped GitHub accelerate its infrastructure growth, and shown how organizations can thrive at scale.
This article will explore the relationship between platform engineering and DevOps and how turning your DevOps program into a product can improve engineering outcomes across the board. We’ll also discuss the notion that if platform engineering really is the answer to faster time to market and revenue growth,why aren’t all enterprises succeeding with it?
The cognitive load conundrum
When it comes to entering the world of DevOps, one of the biggest problems resulting from DevOps adoption is excessive developer cognitive load. Partly to blame is the proliferation of cloud-native technologies and with it, the need to manage multiple cloud platforms. This added layer of infrastructure complexity demands more effort to integrate and maintain a potential myriad of services and tools from different cloud providers. Additionally, there are literally thousands of tools and frameworks to learn and use, as illustrated by the Cloud Native Interactive Landscape (CNCF). With all this to take on and take in, how can developers possibly keep up? And how do they find time to do their most important job: delivering features? Sadly, teams often embark on their cloud journeys underestimating the amount of developer cognitive load these complex tools and setups amount to.
DevOps workflows often fail to define and separate roles. This can result in an unproductive “let the devs handle it” mindset; no matter what the “it” is. Developers are expected to become gurus of everything from Kubernetes and infrastructure as code to running services on their own. Which is why cognitive (over)load is one of the biggest reasons behind failed DevOps adoption.
Having developers do ops is more attractive for velocity and empowering development teams to experiment. One approach could be to embed a DevOps engineer in every development team that performs infrastructure duties, so they are closely aligned and centrally positioned to follow standards and procedures. Terraform modules, guidelines, guardrails, peer reviews, and templates can be centrally coordinated to follow standards in this mode. The problem with this approach is that it relies more on teams who follow best practices—and it’s more resource intensive. This can be mitigated with guardrails or notifications, but it’s hard to cover all cases. It’s also challenging to keep up with updated templating and guidelines, which then introduces even more cognitive load.
Such blurred lines mean that today’s developers must be ready to tackle it all. Beyond handling support tickets, they must eat, breathe, and sleep Kubernetes. As a result, many have become “shadow” ops team members. A typical symptom of this is experienced back-end developers constantly being asked to handle infrastructure-related work for their less-experienced colleagues. They rarely find time to code or manage their other responsibilities, which ultimately ends up hurting the organization’s overall productivity.
Take a product mindset, and don’t reinvent the wheel
When it comes to the core principles at the heart of platform engineering, taking a product mindset is a fundamental tenet. Instead of simply creating software to meet a specification or putting production fires, successful platform engineers follow product management best practices and treat their platform-as-a-product. Developers are their customers, whose needs take the spotlight when it comes to creating the platform.
What platform-as-a-product doesn’t mean is buying something that claims to cover the entire software development life cycle. Unlike using PaaS or end-to-end DevOps solutions, designing a platform-as-a-product provides full control over your toolchain—letting you integrate both legacy and new tools as needed. It also allows you to define workflows, whether UI- or git-based, so that your end IDP product meets the exact needs of your organization and your developers.
So instead of trying to second guess what developers want or hopping on the latest tech hype train, effective platform engineers should just ask developers what they need. And LISTEN. Although this attitude shift may seem simple, it can result in new streamlined systems that heal long-standing pain points vs heightening DevOps friction or Ops overhead.
Humanitec’s Aeris Stewart hits the nail on the head, explaining that “Where DevOps creates too much cognitive load for developers, platform engineering seeks to alleviate it by finding the right level of abstraction and paving golden paths.” It takes all the different tech and tools that you have floating around and binds them into a golden path that alleviates cognitive load on the individual contributor. It enables self-service for engineers on an IDP. By empowering people with autonomy, IDPs support highly efficient work practices through structured processes, standardization, and appropriate levels of abstraction.
Many organizations build IDPs specifically to keep their developers from having to reinvent the wheel. Because platforms cover the operational necessities of an entire application’s life cycle, developers can concentrate on creating services and apps by building on common frameworks, not merely tweaking the systems that deliver them. It’s vital for platform engineers to avoid reinventing the wheel and creating or rebuilding the whole thing from scratch. As a platform team, the real focus should be on where you can deliver the most value va what the ideal stack is. How can you take what's out there—both commercial stuff like Humanitec or open source stuff like Argo and Backstage—and combine them into something that makes sense for your whole team? When you have this figured out, your job as a platform team really becomes focused on last-mile optimization.
Help shape the future of platform engineering
One of the many awesome things about platform engineering is that when it comes to tooling, the discipline is wide open. Each business has its own set of unique needs and challenges. Each platform can be equally as unique in design.
The same goes for platform engineering roles, with recent research showing under 23% of the respondents surveyed held platform engineering titles—despite the fact they worked on platforms (most were SREs, software architects, or other engineers). As platform engineering awareness expands, DevOps and platform roles will continue growing in popularity, so we also expect DevOps titles to evolve and more accurately reflect changing responsibilities.
In short, platform engineering is new enough that people still have the chance to forge their own roles, carve new career paths, and help shape the future of this incredibly important discipline.
To learn more from leading DevOps experts and expand your network, join platformegineering.org; the largest community of platform engineers out there.