Where does using velocity as a measurement come from?
There have been many discussions on this forum where this topic has been discussed. I have participated in many of them. I wanted to get a thread started completely on this topic so that future forum visitors may find it. Plus I would really like to get others input on this.
I looked through the scrum guide. No where is velocity mentioned. All I actually found associated to measuring teams productivity is this statement in the section on Product Backlog under a sub-title of Monitoring Progress Towards Goal.
Various projective practices upon trending have been used to forecast progress, like burn-downs, burn-ups, or cumulative flows. These have proven useful. However, these do not replace the importance of empiricism. In complex environments, what will happen is unknown. Only what has already happened may be used for forward-looking decision-making.
And this one in the Sprint Backlog section sub-section Monitoring Sprint Progress.
At any point in time in a Sprint, the total work remaining in the Sprint Backlog can be summed. The Development Team tracks this total work remaining at least for every Daily Scrum to project the likelihood of achieving the Sprint Goal. By tracking the remaining work throughout the Sprint, the Development Team can manage its progress.
As I have stated multiple times, I really don't like the idea of using velocity because it is based on guesses. I prefer to use throughput because it is based on actual work. Burn-down and burn-up charts based on tasks is useful for monitoring sprint progress. Those same charts based on stories can help provide history on stories per sprint. But I have found that throughput is a better indicator to use in predicting how long it might take to get the story ordered 23 in the product backlog because you may not have your backlog estimated that deeply. I also prefer to monitor story completion in sprints because task can be of varying sizes and complexity. I have seen a team burn through 15 tasks in one day and the same team get through 4 tasks in another sprint. But that team's rate of throughput has been much more consistent.
I'd really like to get some discussion from others on this because I am always looking to find new and better ways. I also know that what is working for me now, may not work for me in a future job. Having as much perspective as possible stored away is something I value.
Like you Daniel, this is solely based on my "better" practice approach gained from a number of years of experience, and should not be taken as a "How To" guide for this topic.
- I personally do not like to measure task completion. I promote the idea of defining tasks, as that level of detail reveals scope and complexity that may not be unearthed otherwise. However, I like to stay very close to "item completion" as the main measurement for teams to gauge their progress.
- To build on your example, a team can complete 15 tasks in a day, yet not complete one item. The same team can complete 4 tasks in one day, and complete 1-2 items. I would place a significantly higher level of importance and value on the latter, and not the former
- Velocity is a metric that is mined post-sprint, but should only be used for planning purposes only. To me, it is a valid metric (along with team capacity) for the team to determine what they may be able to complete in a given sprint
- I also believe throughput (number of items completed) is another valid metric to be calculated and used for planning purposes, both by the Development Team and the Product Owner
Most practices and tools used in Scrum are not mentioned or prescribed in the Scrum Guide, but teams do find value in them. For me, it all depends on what helps the Scrum Team deliver quality deliverables to the business on a consistent basis.
I prefer to use throughput because it is based on actual work.
@Dan, I am assuming that your referring to throughput derived from Kanban i.e. stories completed / sprint or duration. Correct, me if my understanding is incorrect, but you would need similarly sized stories to keep your throughput stable, right? Is there something that you currently follow to ensure that the size is similar?
In some ways, isn't that the same concept of velocity? i.e. relative to previously done work?
This is an interesting subject and I like the idea of throughput but just wanted a discussion around this.
@Steve Yes, I am using the Kanban version of throughput. From what I have learned on the measure, it is not dependent on similar sized stories at all. It is purely a measure of how many stories (or in Kanban sometimes referred to as cards) can be completed during a timebox. Just as velocity is not based on having similar sized stories, throughput does not take into account the size or complexity. Yes, it is relative to previously done work but it is based on an actual number and not a total of estimation (i.e. guesses at the size and complexity of the story).
Just as with velocity, your team will over time start to refine stories into similar sizes. If you track velocity over time, you will find that it will vary widely in the beginning and then start to reach a steady state. In the times I have done this, I found that eventually the estimated story points usually start to go down significantly as the team gets better as refining stories. However, I have also found that the throughput will stabilize much faster than the velocity. I can't explain it but it is what I see time and again. It could just be the teams I have worked with but I have seen it at more than one company with multiple teams.
I have recently started tracking cycle time also just to see if I can find any kind of correlation. Since we are doing Scrum in 2 week sprints, I am curious to see if cycle time has also adjusted based on the time box boundaries. Bit too early for me to draw any conclusions. Trying to get historical data on individual stories from Jira is time consuming. I do see from the cumulative flow diagrams that there is a consistent pattern but want to get a bit more granular on that.
I am very interested in what others have seen. I do not by any means think I have all the answers so the more perspective I can get the better I can be.
Our experiences are certainly different, but reading the above, mine resembles Timothy's. For reasons put forward by him and Steve, I have found out "throughput" not to be that relevant, or helpful for the teams I worked with.
Velocity is too easy not to use. Being simple, it offers the advantage of fast analysis and decision-making.
Some people push(ed) towards measuring cycle time. What's the benefit I genuinely ask? I'm not as experienced as others with Kanban, and perhaps for that reason I fail to see the whole picture.
Cycle time can be incredibly revealing, as it places the focus squarely on how long an item resides within each stage of the process workflow, and on how long it lives overall.
Think of it as another metric that highlights where improvement may be needed from a process or flow perspective, independent from team-based metrics like velocity and capacity.
Suppose you identified that several items spent a majority of their time in the QA stage of the overall workflow. What would that data reveal to you? What would an overall cycle time of an item tell you if it was 3 months? 6 months? 1 year?
Don't laugh - I've created cycle-based metrics where overall cycle time exceeded 1 year. Management though did not want to hear how incredibly wasteful and inefficient their current workflow process was. Hard to argue with raw data, though.
I've found it helpful for the team to track story completion in a sprint burndown chart. The goal is to have a consistent feature-burndown. As we live in an imperfect world, we still have handoffs on team-internal QA for test and documentation. But with the devs continually delivering throughout the sprint, we can avoid a QA bottleneck towards the end of the sprint. So I guess we're talking about throughput?
For longer-term planning, I think the release burndown in JIRA could be helpful. But if Daniel's story number 23 is part of the planned release, it needs to be, at least, roughly estimated.
I'm looking forward to attending a PSK course next year, as I need to get a handle on Cycle Time and Throughput. I've always been sceptical about the power of velocity. Especially the connotations derived by a management still grasping at the straws of Control over everything.
Thank you for adding more depth, Timothy, helps a lot!
Velocity in optional (i sow somewhere not sure exactly where :( ) and only useful if projects do not change, when you move the team to new project the complexity rises and velocity falls and again and again ... until they work on a backend system they are familiar with ...
I agree that kanban trough put and cycle times are great, ... i think next part in kanban books is the monte carlo simulation for predictions using those troughputs and cycle times ...