For many engineering teams, velocity—a measurement of effort expended by developers, usually as story points completed in a sprint—feels like the best way to measure output across stacks of projects and initiatives. But from the perspective of a leader who is tasked with bridging the gap between tech execution and business objectives, it doesn’t tell the full story. If the velocity is high, why aren’t business goals being met around revenue, retention, or sales?
“It's not actually business impact,” Ben Matthews, Senior Director of Engineering at Stack Overflow, says in our latest episode of Leaders of Code. “To be clear, I'm still a fan of velocity as an introspective data point. That is not the goal, but it's a way to maybe diagnose a team.”
When tech leaders look past output and towards impact, businesses flourish and teams become healthier. But if velocity is just a tool and not a goal, how do you measure real success for engineering teams? The answer lies in understanding the core objectives of the organization, and how engineering teams contribute to them. “How can we move beyond velocity and something like that and get to real business impact?” Dan Lines, co-founder and COO of LinearB, asks during the episode. “Our customer base and the people that I talk to, they're on the hook for productivity. So I think the first thing that we need to do is decide what is real business impact.”
This article explores topics first shared by Ben Matthews and Dan Lines in the latest episode of Leaders of Code, where they discuss how to unblock developer productivity and move beyond velocity.
The pitfalls of velocity
From a planning and accountability perspective, velocity gives teams a clean way to measure output vs. effort. It can help them plan for sprints and prioritize long-term productivity targets. It can even help with accountability, allowing teams to rightsize their work and communicate it cross-departmentally.
The issues begin when it is used as the sole metric of success for teams, as it fails to reveal the nuances necessary for high-level strategic thinking and positioning by leadership. It sets up a standard that over-emphasizes pure workload rather than productive effort towards organizational objectives.
Teams can “move fast” without actually delivering meaningful outcomes. When this happens, quality is often what’s sacrificed for the sake of numbers. Corners are cut, tests and reviews are skipped, all while velocity gets higher. DX lists velocity as one of the five flawed productivity metrics for developers, alleging that teams may inflate their numbers to appear more efficient, undermining the ultimate goal of helping teams estimate and communicate their work.
Teams can “move fast” without actually delivering meaningful outcomes.
When the goal communicated by leadership is simply to hit higher and higher numbers each sprint, can we blame developers? As Matthews wrote in a previous Stack Overflow blog post, “All team members, not just developers, begin to realize they aren’t making good decisions; they are going down the wrong path but this arbitrary velocity number is the one defining what is right or wrong.”
And on teams that emphasize velocity over all other considerations, burnout is an inevitability. Engineers are asked to do more and more without truly understanding where their work is making a difference. According to a Deloitte report, 75% of IT, engineering, and business leaders agree that developer experience is of key importance to the success of their overall business. It’s clear to many leaders that when engineering teams begin to feel burnout, the ripples of that are felt throughout the organization.
Velocity oversimplifies the complex and creative work that engineering teams are doing, pigeon-holing the collaborative process into simple numbers that fail to tell a whole story of the health of an organization. But that doesn’t mean velocity has no place on well-functioning teams.
Velocity as a tool, not a goal
What velocity provides is a single snapshot of a development team. “A lot of people view that as a metric for the team,” Matthews says on the LinearB Dev Interrupted Podcast, “but we really view it as a data point of many other things.” As a diagnostic tool, it can help leaders better gauge when strategy is working or where more resources need to be placed within a team.
Velocity oversimplifies the complex and creative work that engineering teams are doing.
When a team’s velocity is high, engineering leaders should look closely at how the team works to understand what’s driving that efficiency. If velocity falls below expectations, it may be time to revisit goals and adjust strategy accordingly. It's also important to recognize that both instances may be influenced by unavoidable business factors—sickness, seasonal slowdowns, or leadership changes can all impact the raw numbers reflected in a team’s velocity.
Ultimately, velocity isn’t the whole picture; it’s one indicator engineering leaders can look at to better understand what’s happening on their teams. So if velocity is not the goal, what is?
Measuring success for developers
When considering how to measure success on development teams, McKinsey warns against oversimplifying measurements, encouraging leaders to understand the complexities of engineering to better support the creative and collaborative processes of their teams. Where are teams investing their time? Where are bottlenecks forming? Are teams hitting benchmarks aligned with industry standards? For Google’s recent EngSat survey, Speed, Ease, and Quality were all key markers for both developer satisfaction and productivity.
However, visibility on the leadership side into development teams is not enough. Developers also need to understand where their work is impacting and supporting companywide goals and initiatives. When development teams are siloed and asked to perform based on pure numbers, their ability to impact the overall business is stifled.
When leadership works with their engineering teams to find solutions to business challenges, they create a highly visible value stream between each individual developer and the customer at the end of the line. For engineering-forward organizations, developer experience and satisfaction is a top priority, so factors like transparency and recognition of work go a long way towards developer well-being.
Perhaps most vital is for business and tech leaders to create roadmaps of success for engineers that clearly align with the goals of the overall business. LinearB cofounder and COO Lines acknowledges that these business goals can differ wildly between businesses: “For some of the leaders that I work with, real business impact might be as simple as, we got to get to production faster…Other ones might we're actually having issues in production. Our customers are a little angry at us right now. Real business impact is improving our quality.”
Tying engineering OKRs directly to product or business objectives allows teams to work towards impact and not just output. For Matthews, bringing engineering teams into the process itself is pivotal: “You're a stakeholder like everyone else. We're putting you in meetings with users. We're putting you in meetings with sales and customer support. And that really, I think, brings the business impact more holistically to all of engineering.”
Implementation of new metrics
Like any strategy change, moving beyond output towards impact is not without pitfalls. It’s important that leaders regularly review how well they are measuring team contribution and adjust accordingly.
Teams also need to be given the full context of what they are being measured on and why, and given the space to acclimate to these new reporting metrics. In a thought experiment by Itamar Gilad, engineering teams that spent time researching customers, setting clear goals, and testing ideas had a higher positive impact ratio on features year over year.
As new insights emerge, leadership should celebrate successes, and understand dips as ways to improve, rather than push their teams towards increasing output without reflection.
While velocity is a great diagnostic tool for leaders, it fails to give the whole picture of the health and productivity of a development team. As technology becomes more complex, recognizing the creative and collaborative work of developers is more important than ever for developer experience. And by moving past pure output, development teams are able to support organization-wide objectives and make a real impact on the health and success of the business.