EBM Metrics

Last post 11:28 am August 21, 2019
by Harshal Rathee
4 replies
Author
Messages
05:26 pm August 19, 2019

We are trying to implement metrics to support our movement to EBM. https://scrumorg-website-prod.s3.amazonaws.com/drupal/2019-05/EBM_Guide%20January_2019.pdf 

We would like to hear what other people/companies are using for the different metrics. What are you using (or would you use)? Thanks for your help!

  • Current Value (CV)
  • Time-to-Market (T2M)
  • Ability to Innovate (A2I)
  • Unrealized Value (UV)
05:51 pm August 19, 2019

I think the answer to this could vary greatly depending on your product and business context. The appendix of the EBM guide has a lot of great examples as a starting point. I can give you an example of how one might use some of this information in order to run experiments and create value though. 

Under Current Value of a product a Product Owner may look at something like 'usage rate' of it's features. A Product Owner may review an under performing feature and consider several paths. 

1) Retire the feature to simplify the product. Perhaps the team has tried several things and it's still underused. 

2) Work with the development team to make the feature more robust or user friendly so it's more appealing. 

3) Work with sales/marketing to run campaigns to build user awareness of the feature and why they should consider using it. If the users aren't aware of the benefits and have a desire to use it a more robust feature may not net the desired value. 

Utilizing the EBM framework the team should be able to use the information gathered to see if their marketing campaigns or feature enhancements obtained the value they were seeking. It's important to know how you're going to measure success and focus on improving a little at a time to better track the outcome back to the activity, especially if you're dealing with lagging indicators. 

 

02:07 pm August 20, 2019

Nice thread !!

I am also quite keen to apply EBM metrics in my organization but to be honest i don't know where to start with. We have some measurements similar to KVAs in EBM like Current value, time to market and ability to innovate. But since we have some problem in our measurement criteria & they seem not to be reliable, which developed my interest in EBM.

But how to realize our criteria effectively into KVAs is what i find challenging. Is there a simple way to start with ? has anyone applied it successfully.  

03:40 pm August 20, 2019

Harshal, 

In terms of what to measure and how to measure it reliably...

I'm a firm believer that not all of the metrics should be determined in a silo for a given product. As a Product Owner, I would look to my development team to help me come up with important areas of measurement, especially when it comes to time to market and ability to innovate. 

Let's take time to market as an example. I'm defining this as the time the Scrum Team adds a user story to the Sprint Backlog up until it's released to production. If I ask the development team members to put the Sprint Backlog add date and Release to production date of every user story into an excel sheet I will likely have inaccurate results for a few reasons.

  1. Team members are responsible for manually updating  this information. Even with a tool they are responsible for moving work tasks in a timely manner. 
  2. If I didn't include the team members in the decision despite asking them to make changes to their daily behavior I will likely be met with resistance. Changes to daily behavior requires discipline and a strong desire to do so. 

If we're working together, I'm more likely to get buy in for the metrics and the activity needed to gather them. Down the road the team could always add in a PBI to find a solution to automate the metrics in order to increase reliability and decrease manual work. 

If you're starting from nothing I'd try to keep it as simple as possible and work to expand incrementally, especially if manual work is required to capture the metrics. People need time to work through the change process. Including them in the decision making will likely help your speed of adoption. 

 

11:28 am August 21, 2019

Thanks Tony for response !!

I'm a firm believer that not all of the metrics should be determined in a silo for a given product. As a Product Owner, I would look to my development team to help me come up with important areas of measurement, especially when it comes to time to market and ability to innovate. 

Yes , currently its not about any kind of decisions. Its more about to understand the EBM metric itself like checking with other products and companies using this metric, getting some suggestions to start with or sharing experiences. I know it will definitely differ from product to product but still will be beneficial just to know to get more clarity. 

If you're starting from nothing I'd try to keep it as simple as possible and work to expand incrementally, especially if manual work is required to capture the metrics. 

I agree, a simple start should be the best way to proceed.