How do we actually know if the company is already Agile?
Or the longer version for this type of question …
How do we know that all the investments made in Agile transformation, from hiring consultants, transforming DevOps, to providing Agile training for employees, have provided optimal return on investment (ROI)?
This question has frequently come up from company executives to me since 2024. That’s because Agile transformation surged during the pandemic era (2020–2023). Essentially, today these executives want to know:
“Has all the investment into Agile transformation really paid off?”
If we want to know whether our company is agile, we must measure it.
Agile Is Not Just About Speed
Imagine you're an Olympic sprinter. How do you know you're getting faster and ready for the next race?
You measure your speed. Constantly.
Ironically, many companies don’t measure their agility. They rely on assumptions, guesses or superficial indicators like:
The team is using JIRA;
The team holds regular Zoom Daily Scrum;
The organization uses the Spotify model structure (squads, tribes, guilds);
Most employees are Agile certified;
These indicators are not only cosmetic, they’re also qualitative and don’t truly reflect whether a company is agile. A runner doesn’t assume they’re faster just because they’re wearing the most expensive running shoes, right?
Many companies also assume agile is just about being fast. But moving faster in the wrong direction (e.g., off a cliff) is obviously not useful.
Agile must always be tied to increasing business value.
The Project Triangle Is Overrated
Many companies still use the project triangle (on-time, on-scope, on-budget) as their measure of success, even post-Agile transformation.
No wonder they don’t feel any change has occurred, because they’re still using the same mindset and indicators. Often, such companies simply practice mini-waterfalls every two weeks. It’s no surprise many developers in these organizations feel Agile is worse than Waterfall. Some even rant that "Agile is Dead".
In Agile, success is defined by increasing value. When we're agile, the project triangle is overrated because:
A product delivered on-time might already be irrelevant upon release, as the market could’ve shifted or a competitor may have seized the momentum.
Features delivered on-scope might not be needed by customers, or may be rarely used.
A product delivered on-budget doesn’t automatically deliver optimal ROI because the product may generate little revenue or lack customer interest.
As an analogy:
In 2019, my wife and I went on holiday to Venice. We had planned our itinerary six months in advance. But it rained heavily for three straight days while we were there. All our outdoor plans had to change.
We had to be Agile or the trip would’ve been wasted. One unplanned but memorable activity was attending a Vivaldi classical concert at the house where he once worked. That wasn't on our itinerary, but it made the trip special.
Moral of the story:
Agility is more important than sticking to the plan. Maximizing value and staying relevant is the goal.
If Not the Project Triangle, Then What?
Use the Evidence-Based Management (EBM) framework from Scrum.org. You can read the details here.
To explain EBM metrics, I’ll use the same visual I present in Product Owner or Evidence-Based Management workshops.
EBM consists of four Key Value Areas (KVA). To make it easy, I’ll use the analogy of taking a road trip by car.
1. Current Value
→ The business value currently delivered (like the blue dot on Google Maps)
Example metrics: Revenue per employee, Product cost ratio, Customer usage index, Customer satisfaction.
Note: There are many more metrics that can be used to measure the product's current value, what is listed in the EBM guide and also the one above are just some examples. You should work together with the people in your organisation to define the list of metrics for measuring the product's current value.
2. Unrealised Value
→ Goals and growth potential (like the red pin on Google Maps)
Example metrics: Market share, ROI, Top-of-mind awareness, Desired customer experience, Total cost of ownership
3. Time to Market
→ Delivery speed (like the speedometer)
Example metrics: Lead time, Deployment frequency, Mean time to value, Mean time to restore. These metrics are from DORA metrics.
4. Ability to Innovate
→ The Ability to Innovate metrics are used to describe the fundamental problems within the company that cause slow Time to Market (like the car's engine condition).
Example metrics: Technical debt, On-product index, Employee engagement, Context switching time, Focus factor, Frequency of production incidents, Defect trends.
Mapping Key Value Areas to Scrum Roles
I categorize the four KVAs into two broader groups and map them to Scrum accountabilities:
1. Business Value (Current Value & Unrealised Value)
Accountability: Product Owner
"The Product Owner is accountable for maximizing the value of the product resulting from the work of the Scrum Team." — Scrum Guide
2. Delivery Capability (Time to Market & Ability to Innovate)
Accountability: Scrum Master and Developers
"The Scrum Master is accountable for the Scrum Team’s effectiveness."
"Developers are accountable for instilling quality by adhering to a Definition of Done and holding each other accountable as professionals." — Scrum Guide
These roles must collaborate. They cannot work in silos. If one party fails in their accountability, some aspect of the product will suffer.
The entire Scrum Team is accountable for creating a valuable, useful Increment every Sprint.
Relationships Between the Key Value Areas
The four KVAs are interconnected:
Faster delivery (measured by Time to Market) should lead to higher value (measured by Current Value). But speed alone means nothing if it doesn't increase value.
If the product is delivered fast and with high quality, but value doesn't increase, the Product Owner is accountable, possibly due to poor market analysis or failing to prioritize strategically.
If the product is delivered fast by compromising quality, then the Scrum Master and Developers are accountable.
If Time to Market is slow, we need to inspect Ability to Innovate.
For example:
Team members might be spread thin across multiple products (Context Switching).
Sprints may be interrupted by incidents (measured by Frequency of Production Incidents), caused by growing technical debt.
In such cases, Scrum Master and Developers must first improve Ability to Innovate, then Time to Market will follow.
EBM metrics help us see who is delivering value, who is accountable, and who is just going through the motions. It also tells us what we've done and what we should do next.
Analogy: Ability to Innovate & Time to Market
Ability to Innovate is a crucial KVA to improve organizational agility. But many companies ignore it and only focus on Time to Market. Yet it's often the root cause of slow delivery.
Let me explain with another analogy:
A week before a long road trip from Brisbane to Sydney, my car broke down. It had major engine trouble, bad gearbox, failing wheel bearings, and worn brakes.
Of course, I wanted to drive fast on my trip. But doing so in a severely broken car would be dangerous for my family. Even if I pushed the car, I wouldn’t be able to drive fast. When there is sudden braking, it may lead to disaster.
So, before the trip, I took the car to the mechanic. Only then could I drive safely and quickly.
Likewise, many companies want to go fast (Time to Market), but fail to fix fundamental issues (Ability to Innovate):
Teams multitasking on multiple products;
Mounting technical debt left unpaid;
Focus on feature quantity over quality;
Constantly compromising the Definition of Done;
Lack of technical excellence as a culture;
Ability to Innovate cannot be ignored. It’s a shared accountability between Scrum Master and Developers. So every time a company achieves Unrealised Value, it should also invest in improving Ability to Innovate.
Never expect to go fast if you don’t address the real reasons for slow delivery.
From the EBM visual above, we can also see that agility has no finish line. As I often say to my workshop participant, we can always be more agile than yesterday. The fact is, agility is a company’s capability to continuously increase value over time.
I've built a prototype EBM dashboard that you can use as inspiration and starting point. Companies should build their own EBM dashboards tailored to their context using tools like:
Power BI
Tableau
Pentaho
And remember ... if you don’t have metrics, you’re just making assumptions. And assumptions are dangerous. As dangerous as driving a car without a speedometer. So start measuring agility in your company. Starting now.
If your company would like to hold an Evidence-Based Management (EBM) workshop facilitated by me to align understanding on how to measure agility and reduce assumptions in decision-making across all levels, please use the contact form to reach out to me.