Skip to content

Outcome Before Output: Why Velocity Is the Wrong Number

Completed tickets are output, but not yet evidence of impact. Why velocity misleads in management reporting and how to align teams with outcomes instead.

≈ 11 min readSoftwareProduct
// Sprint-Report, wie er im Statusdeck landet
const sprint42 = {
  storyPointsCommitted: 34,
  storyPointsCompleted: 38,
  ticketsDone: 17,
  velocityTrend: "up",
};

Exactly one statement follows from this object: the team completed more points than it had committed to. Whether a user reaches their goal faster as a result, whether a system runs more reliably, whether revenue is generated anywhere – there is no field for that here, and you can’t compute one in after the fact. Still, exactly this curve travels into reports sprint after sprint, gets compared between two teams and occasionally ends up on a board slide next to revenue.

The question that actually counts isn’t “How many story points did the team complete?” but “What changed for the user, for the system, for the business as a result?” That’s the difference between output and outcome. And if you blur it, you systematically optimize a team for being busy instead of having an impact.

The dividing line runs between an observed activity and the demonstrable change that follows from it.

Activity and impact answer different questions

  • Observation · What was built or completed?
  • Examples · Features, tickets and story points
  • Use · Planning and tracking the work
  • Limit · Doesn’t yet prove any change for the user
  • Observation · What changed in behavior or results?
  • Examples · Activation, time to first value and error rate
  • Use · Checks whether a solution contributes to the goal
  • Limit · Needs context and a robust measurement

This keeps velocity useful as an internal planning measure without turning it into a statement about productivity or business value.

Velocity measures predictability, not productivity#

Let’s start at the source, because velocity has gotten a bad reputation it doesn’t deserve. Almost two decades ago, Martin Fowler described precisely what the number was meant for: a calibration tool for estimates. The principle is called “Yesterday’s Weather” – we assume that a team will get about as much done in the next sprint as in the last one. That keeps plans realistic without anyone making heroic commitments.

Story points are a team-specific baseline. They mean something different for team A than for team B, because they are estimated relative to each context. That’s exactly why comparing two teams by their velocity isn’t just imprecise but simply meaningless – you compare two different units of measurement and pretend they are the same.

And then the predictable happens. As soon as velocity becomes a management target, it gets gamed. Teams estimate more generously, split tickets more finely, declare minor things a story. The curve rises, delivery stays the same. That isn’t malice, it’s Goodhart’s Law in its purest form: “When a measure becomes a target, it ceases to be a good measure.” Any number you elevate to a measure of success stops being honest at that very moment.

So I don’t want to demonize velocity at all. Internally, as an aid for planning and forecasting, it’s completely legitimate and useful. The mistake isn’t in the number but in promoting it to a measure of success. That’s an important distinction that tends to get lost in the heat of the “velocity is evil” debates.

True output is impact – and you can’t measure it directly#

Fowler’s truly radical point comes next: “True output is business value.” A team’s real yield isn’t the number of features but the value created. His example sums it up: a team that delivers fewer features but generates ten million in profit is more productive than one with more features and five million in value.

The catch is sobering. Business value lags behind the investment, often by years. What a team builds today pays off – if at all – only later, intertwined with the market, sales, timing and a dozen other factors. An immediate, clean measurement of productivity is therefore simply impossible. Fowler’s honest conclusion: “We have to admit our ignorance.” We can’t measure productivity directly. Period.

That sounds discouraging, but it’s liberating. Because if the perfect number doesn’t exist, we stop looking for it and turn to what we can actually influence and observe: the impact of our work on users and the system.

Outcome instead of output: the core of modern product work#

This is exactly where the product side comes in. In her work on continuous discovery, Teresa Torres draws a clean distinction: outputs are what we build – features, screens, endpoints. Outcome is the change in user behavior that creates value for the customer and the business. If you declare outputs the goal, you produce a “disconnect” between the team’s activity and the value delivered. The team is fully utilized, the board is full, and still nothing moves for the user.

Marty Cagan puts the same observation even more sharply: “Empowered teams get outcomes, feature teams get output.” A feature team is handed a roadmap and works through it – output. An “empowered” team gets a problem and an outcome goal and is allowed to figure out for itself how to get there. The difference isn’t cosmetic. It decides whether people think along or just work through a list.

The tool I have had the best experience with here is the Opportunity Solution Tree. At the root sits exactly one outcome, for example “increase the activation rate by X percent.” Below it branch out opportunities – concrete user problems or needs that contribute to this outcome. Below those come solutions, and at the very bottom the experiments we use to check whether a solution actually works. The tree forces the question the velocity curve never asks: Does what we are building right now contribute to the root at all?

graph TD
    subgraph output["Output view (vanity)"]
        B[Backlog] --> S[Sprint]
        S --> V["Velocity chart<br/>many completed tickets"]
        V --> U["up and to the right"]
    end
    subgraph outcome["Outcome view"]
        O["Outcome<br/>activation rate +X%"] --> OP1[Opportunity A]
        O --> OP2[Opportunity B]
        OP1 --> SOL[Solution]
        SOL --> EXP[Experiment]
    end
    V -. "does the output move<br/>the root?" .-> O

The diagram shows the two worlds side by side. On the left, the output pipeline that reliably points upward. On the right, the tree that makes impact traceable. The dashed arrow in between is the real management question – and exactly this arrow is missing almost entirely from the reports I get to see.

Vanity metrics: the engineering version of self-deception#

Eric Ries coined the term vanity metrics for this kind of number in the Lean Startup context. Sign-ups, page views, downloads – numbers that go “up and to the right” and feel good without proving real progress. Velocity and completed tickets are the engineering version of exactly this pattern. They rise reliably, they create a good feeling, and they say nothing about whether we have gotten closer to the goal.

On the other side are actionable metrics: numbers that show a cause-and-effect relationship and lead to a decision. For Ries, the unit of progress isn’t “feature shipped” but “validated learning” – confirmed learning about what actually works for the user. The framework for this is called innovation accounting, the cycle Build-Measure-Learn.

In practice, this means: you don’t instrument what you built, you instrument what the user does in response. An event that tracks an outcome looks different from one that counts output.

// Actionable: measures what actually moves the user
track("activation_completed", {
  userId,
  cohort,
  timeToValueSeconds, // how fast does the user reach first value?
});

// Vanity: feels like progress, says nothing about impact
// track("features_shipped", { count });
// track("story_points_done", { total });

The decisive step comes next: you roll out the feature behind a flag and compare the outcome delta between two cohorts. Does the new solution measurably increase the activation rate, or not? That’s Build-Measure-Learn in one line of infrastructure – and it shifts the question of success from “did we build it?” to “did it work?”

Better proxies: DORA and fitness functions#

At this point, someone usually chimes in: But there’s DORA. The four key metrics from the Accelerate research by Forsgren, Humble and Kim – Deployment Frequency, Lead Time for Changes, Change Failure Rate and Time to Restore Service – are a big step forward. They deliberately measure team outcomes instead of individual outputs. Explicitly no lines of code, no story points, no utilization, because those are siloed activity numbers. And they balance speed against stability, so you can’t optimize one number at the expense of the whole system.

DORA helps, with one caveat: these metrics are proxies too, not real outcomes. Deployment frequency isn’t user impact. A team can deploy fifty times a day and still miss what’s needed. DORA measures the ability to deliver quickly and stably – an excellent indicator of delivery health, but not the goal itself. If you confuse the two, you have just replaced one vanity number with four.

At the architecture level, there is an instrument that is particularly close to my heart, because it makes outcome executable: fitness functions from Evolutionary Architecture by Ford, Parsons and Kua. A fitness function is an objective, automated assessment of an architectural property. Instead of checking off “performance improved” in a ticket, you write a test that checks the system’s effect itself:

def test_checkout_latency_stays_within_budget():
    """Architectural fitness function: checkout stays fast under load."""
    result = load_test(endpoint="/checkout", concurrent_users=200)

    assert result.p99_ms < 500       # latency budget, not a story point
    assert result.error_rate < 0.01  # reliability is the real outcome

The beauty of it: success is no longer a status update here but a measurable system property that is either maintained or not. The test is green because the system delivers the promised effect – not because someone moved a story to “done.” Consumer-Driven Contracts work in a similar way, for example with Pact: a test fails when a service breaks a promise to its consumer. That, too, is a test close to the outcome – it checks the actual effect at the interface instead of the mere existence of code.

How to really align teams with outcomes#

When I set this up in an organization, I follow a simple process. It’s deliberately lean, because the whole point of the exercise is to replace process overhead with clarity, not to create new ceremonies.

flowchart LR
    A["Define the outcome<br/>one goal, measurable"] --> B["Opportunities<br/>map user problems"]
    B --> C["Experiments<br/>behind feature flags"]
    C --> D["Measure impact<br/>cohort delta"]
    D -->|"works"| E["scale"]
    D -->|"does not work"| B

It starts with exactly one outcome, phrased as a measurable change in behavior – not “ship the checkout redesign” but “reduce the checkout abandonment rate.” The idea of a North Star Metric fits here and, at the goal level, OKRs: the objective describes the direction, and the key results are the outcome signals by which we recognize progress.

The second step collects the opportunities: Where do users actually drop off, where does friction arise? Only then come solutions, and they aren’t rolled out and checked off but tested as an experiment behind a flag. If the cohort delta shows an effect, we scale. If it doesn’t, we go back to the opportunities – without a guilty conscience, because a disproven experiment is validated learning and therefore a result in itself.

An important caveat belongs here, otherwise you walk into the next trap. Goodhart applies to outcome KPIs too. An activation rate can be gamed as well, and the time lag from Fowler’s observation remains: impact takes time, and attribution is rarely clear-cut. If you set an outcome goal, you have to factor in this fuzziness instead of defining it away – otherwise you construct false causalities and are no further along than with the velocity chart.

Conclusion#

The common thread through all of this is simple: output is necessary, but impact is the goal. Completed tickets, story points and velocity describe work and planning – not whether anything improved for users or the business. As an internal planning aid, they are legitimate; as a measure of success for management, they steer in the wrong direction, because a target number gets optimized and being busy doesn’t yet prove impact.

The way out isn’t a better single number but an attitude. We align teams with a measurable outcome, make impact visible through actionable metrics, fitness functions and experiments, and treat DORA as what it is – a good proxy for delivery capability, not the goal. In my experience, the result isn’t a heavier process but a lighter one: shorter cycles, because we abandon wrong paths early; less friction, because teams are allowed to decide for themselves; and higher quality, because success depends on impact and not on the status color of a ticket.

Fowler’s sober admission remains the best starting point: we can’t measure productivity directly. But we can stop celebrating the wrong number and start asking the right question – does what we build move the root at all?

Further reading#

Revised and translated from German with the help of AI.