Product Delivery Playbook >> Insights >> A Guide to DORA Metrics and Their Role in End-to-End Delivery

DORA (DevOps Research and Assessment) metrics are a set of four key performance indicators (KPIs) developed by Google's DORA team to measure the effectiveness and efficiency of software delivery and DevOps performance. These metrics are widely considered the industry standard and provide a data-driven way to assess how well a team is delivering software.

The four core DORA metrics are:

1. Deployment Frequency:

• What it is: How often a team successfully deploys code to production.

• What it measures: The velocity and agility of a team. A high deployment frequency indicates that a team is able to release small, incremental changes quickly and with confidence. This is a key indicator of a mature CI/CD pipeline and the ability to respond rapidly to market demands.

• Why it's important: Frequent, small deployments are less risky, easier to troubleshoot, and allow for faster feedback loops from users.

2. Lead Time for Changes:

• What it is: The time it takes for a code commit to be successfully running in production.

• What it measures: The overall efficiency of the software delivery pipeline, from the moment a developer commits code to its live release. This metric reflects the team's ability to respond to business needs.

• Why it's important: A short lead time indicates a streamlined process with minimal bottlenecks, such as manual approvals, long-running tests, or cumbersome code reviews.

3. Change Failure Rate:

• What it is: The percentage of deployments that result in a failure in production, requiring a hotfix, rollback, or other remediation.

• What it measures: The stability and reliability of the software delivery process.

• Why it's important: This metric provides a crucial counterbalance to deployment frequency. A low change failure rate shows that a team is maintaining quality and not sacrificing stability for speed.

4. Time to Restore Service (MTTR - Mean Time to Recover):

• What it is: The average time it takes to recover from a production incident or failure.

• What it measures: The resilience of the system and the effectiveness of a team's incident response and recovery procedures.

• Why it's important: A low MTTR demonstrates that a team is prepared for failures and can quickly restore service, minimizing the impact on users and the business.

Why DORA Metrics are Okay...

DORA metrics are an excellent starting point for any organization looking to improve its software delivery. They provide a common language and a proven framework for measuring DevOps performance.

• Evidence-based: DORA metrics are backed by years of research from the DevOps Research and Assessment team. The studies have consistently shown a strong correlation between these metrics and high organizational performance, including profitability and customer satisfaction.

• Balances speed and stability: By including both velocity metrics (Deployment Frequency, Lead Time) and stability metrics (Change Failure Rate, Time to Restore Service), DORA prevents teams from focusing on speed at the expense of quality.

• Identifies bottlenecks: Tracking DORA metrics can highlight specific pain points in the delivery pipeline. For example, a high Lead Time for Changes might point to a need for more automation in testing or a more efficient code review process.

• Fosters a culture of continuous improvement: DORA metrics provide a baseline from which teams can set goals and track their progress over time. This encourages a data-driven approach to improving processes.

...But Not the Right Metrics for True End-to-End Delivery

While DORA metrics are powerful for measuring the technical aspects of software delivery, they have limitations when it comes to understanding the full "end-to-end" value stream.

The key issue is that DORA metrics focus on the pipeline, not the product. They measure the efficiency of the technical delivery process, but they don't answer the most critical business questions.

Here’s why DORA metrics fall short of being a true end-to-end delivery guide:

1. They are not tied to business outcomes. DORA metrics tell you how fast and reliably you are shipping code, but they don't tell you if that code is valuable. You could have an "Elite" DORA score by deploying hundreds of times a day, but if those deployments are for features nobody uses or don't solve a customer problem, the business value is zero.

• The Missing Link: DORA metrics don't track business-level KPIs like revenue, customer retention, user adoption, or market share. A true end-to-end view requires connecting the technical metrics to these business outcomes.

2. They don't measure the entire value stream. The DORA framework starts at the "code commit" and ends at the "deployment." This leaves out a significant portion of the end-to-end delivery process, including:

• The ideation and discovery phase: How long did it take to decide what to build?

• The planning and design phase: How much time was spent on creating a product backlog, writing user stories, and designing the solution?

• The feedback and learning loop: Did the deployed feature actually have the intended impact? Did we learn something that will inform the next iteration?

• The "what to build" problem: The metrics don't provide insight into how well the team is aligned with the business strategy or if they're working on the right things.

3. They can be gamed (Goodhart's Law). Goodhart's Law states, "When a measure becomes a target, it ceases to be a good measure." If you incentivize teams purely on DORA metrics, they may find ways to boost their numbers without actually improving the underlying process. For example:

• Teams might break down changes into smaller, less meaningful commits just to increase Deployment Frequency, even if it creates more overhead.

• They might cherry-pick which "failures" to report to lower their Change Failure Rate.

4. They are a lagging indicator for many organizational improvements. While DORA metrics are excellent for measuring the effect of a change, they don't tell you what to change. For example, a poor Lead Time for Changes might be caused by a lack of automated testing or an organizational structure that discourages collaboration. DORA metrics show the symptom, but they don't diagnose the root cause.

The Solution: A Holistic, Value Stream Management Approach

To get a truly end-to-end view of delivery, you need to supplement DORA with other metrics and frameworks, such as Value Stream Management (VSM).

A VSM approach expands on DORA by:

• Mapping the entire value stream: This includes everything from the initial customer request to the final delivery and measurement of business value.

• Adding "Flow Metrics": These metrics track the entire flow of work, including features, defects, and technical debt. Key Flow Metrics include:

  • Flow Velocity: How many items are completed over a period of time.

  • Flow Time: The total time from "work start" to "work complete."

  • Flow Efficiency: The ratio of "active work time" to "wait time."

• Connecting to business outcomes: VSM links the work being done (features, bugs) directly to the business value they are intended to create.

In conclusion, DORA metrics are an invaluable tool for measuring and improving the technical efficiency of your software delivery pipeline. They are a must-have for any DevOps team. However, they are not a silver bullet for end-to-end delivery. To fully understand and optimize your entire value chain, you must use DORA in conjunction with a broader, business-centric approach that connects your technical performance to the actual value you are creating for customers and the business.

Further reading;

• Manage variables

• Burndown Charts

• A Guide to DORA Metrics and Their Role in End-to-End Delivery

• Cycle time

• FACT