"Metrics" in a Scrum Team
Premise: I am aware that each company, context, team and products are different, we cannot give a generalized answer.
The question: After one year of "Agile transformation", what would be some "metrics"/"KPIs" showing (or not) that we are "doing better" than before? Maybe "more Agile"? (spoiler: we might have no comparable data about the previous year, probably "measuring/comparing" should have started at the very beginning). Should those metrics be common for several teams (for example within the same "Product/Product Area") or any team should have theirs? (I believe that autonomy, creativity and experimentation would be damaged by this uniformization).
How I would answer: Perhaps, as we are talking about a mindset, we do not really have "data" but we should compare the way we were thinking before (also values and principles), which problems we had (and we solved) and focus on the ones which should be solved. The fact that we were not measuring or having low transparency on Jira are something that changed positively. I think EBM could also help answer this question.
What do you think?
Perhaps it would be a good idea to ask shareholders what they think about transformation? Can they see the value it added? If not, what sort of metrics they would prefer? Additionally, you can't go wrong with delivered/not delivered Sprint Goals, which IMO should always be no.1. If you didn't use Sprint Goals, perhaps User Stories delivery rate (considering they were written as something that added value).
If you don't have much data to extract from Jira I am not sure if there's a lot you can do besides surveying stakeholders and Scrum Team and see if those answers satisfy your needs.
In my opinion the best metric is stakeholder satisfaction. Are they getting what they need when they need it? Do they feel included in the solution? Are they providing feedback on a regular basis?
Considering that the company you work for is a stakeholder, is the product a providing a worthy return on investment?
Your comment about Evidence Based Management is a great idea.
After one year of "Agile transformation", what would be some "metrics"/"KPIs" showing (or not) that we are "doing better" than before?
How do you know that the transformation you refer to has even happened? Perhaps some "agile" vocabulary has just been layered over the same-old same-old as a kind of veneer. If so, the hunt for metrics and KPIs will not compensate for a lack of focus on empirical outcomes.
A few metrics that I've used in previous assignments to assess Agile growth:
- Cycle Time (how long does it take for a concept/suggestion to become working software?). This can reflect a better responsiveness to customer needs and ability to deliver
- # of Unplanned Items per Sprint (ex: prod issues, external requests, attempts to introduce new work into sprint). This can reflect improved quality, and the organization's ability to support Focus
- Sprint Goals met. Is the team consistently meeting their commitment each sprint?
Ultimately it goes back to the "why" of the transformation. Perhaps the transformation was undertaken due to customer complaints, scaling of the organisation, etc. I would suggest seeking out those metrics, as sprint velocity and burndown charts may not mean much to all of your stakeholders.
Of course, transformation are also kicked off because people feel its the right thing to do, in that case I would still look for customer centric metrics that can be related back to a more efficient delivery team.
Aside from the metrics showing a successful ( or otherwise ) transformation, the classic scrum metrics such as burndowns and velocity charts, etc are always necessary to tweak and tune your approach.