Estimation
There is one thing that always gets me in the problematic situation - estimation.
When is it done exactly?
Let's say I have 5 user stories in product backlog. When do I estimate them - during current sprint while the team is working on current user stories? If not, then how do I know how many Story Points each has? Is it done during Sprint Planning? Then it's not enough time.
Since each iteration has full cycle - design/development/testing, then during design we may find that there is diviation from what we thought the complexity is. We estimated as 3 sp, but it's actually 5, thus we cannot complete the sprint on time.
I can't figure this out. Can someone help or point me to a resource that describes this in details? In theory everything sounds very simple, in practice we always bump in sprints that we don't finish on time or we underallocate to finish on time, which isn't right as well.
It is possible to provide accurate estimation.
Why does it matter if you estimated 3 and at the end of the effort, it turned out 5? How did you arrive that because it is now 5, you cannot complete the sprint on time?
*Sorry i meant it is impossible to provide accurate estimation
The Scrum Guide states that:
Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog.
So the act of estimation is performed when the Product Backlog Item is refined. Going into the Sprint Planning session, you should have enough Product Backlog Items refined to populate a Sprint. However, based on the most recent Sprint Review, the order of the Product Backlog may change or new items may be added, so there may be some need to refine some Product Backlog Items around the Sprint Review or Sprint Planning sessions if the work is at the top of the Product Backlog.
It's expected that the estimate may deviate from reality as the team takes on the work represented by the Product Backlog Item - it's why it's an estimate. But the other aspects of refinement, such as collaborating on the details with the Product Owner, should help the Development Team gain enough of an understanding of what needs to happen. The Definition of Done also plays a role in determining what work needs to happen. Over time, as the team learns more about the environment, their estimates should get better.
I'd also point out that the Scrum Guide doesn't define what "estimate" means. A common rule of thumb is that any given Product Backlog Item should fit in a single Sprint - it could be sufficient for the team to agree that the work could be taken on and completed in a Sprint. This means that ideas such as "No Estimates" are fully compatible with Scrum.
As Thomas points out, estimation is done in Backlog Refinement so you have enough "Ready" Product Backlog for the next Sprint.
When is it done exactly?
Is there a Definition of "Done" to spell this out for the Development Team and guide them so they can estimate?
I can't figure this out.
Have you asked the Development Team to use their creative wisdom to improve estimation? Have you encouraged them to run some experiments? Take this to the Development Team, and facilitate the conversation in the Sprint Retrospective. Maybe story points work, maybe there are better ways such as T Shirt sizes, or just breaking the work down into same sized PBIs that can fit in a Sprint.
Well, the first thing we should put on the table is that estimations are something we can not predict, I mean, we can say 3 story points (or whatever way of estimation the team has) and at the end of the sprint see if we are confortable with that estimation or not, but not to change the previous estimation (it doesn't make any sense for something is already done) and for future estimations for similar tickets adjust them.
Regarding "when" we need to estimate the tickets for me the answer is obvious, during the sprint, the conversation over PBI need to happens in our dailiy basis, then with this conversation we refine/breakdown our tickets so that we can give this estimation (if we decided to give some sort of it). If the team doesn't have time to keep this kind of conversation the problem is not the estimation itself, it is widely worst.
Thank you everone for your responses. I come from waterfall and trying to adjust. Our organization is stragling with transofrmation and sprint planning/capacity is a big problem right now. Here is how we do it now.
Product Owner > Spec > User Stories > Product Backlog
We start a sprint based on previously estimated user stories
During the sprint we estimate what's in the queue of product backlog so we can decide what we're taking for the next sprint.
Is this correct?
==========
Also, I'm suspecting that our user story is way too complex, but PO cannot break it into smaller details as he doesn't have sufficient technical knowledge.
Example of chat feature. User story received from PO "when someone types on one end, the other user sees 'typing...' "
Is this too much for a user story for one sprint of 2 weeks? If you think of it, there is:
a ballon of "typing"
receiving data from server - typing started, stoped, aborted
error conditions
on the server itself there is plenty of work
Who is responsible for such decomposition of a Product User story?
We start a sprint based on previously estimated user stories
This is generally right. Going into Sprint Planning, you have refined Product Backlog Items (which may or may not be user stories). Well-refined PBIs tend to meet the INVEST criteria. The refinement is an ongoing activity throughout every Sprint as the Product Owner maintains the Product Backlog. You may need to do some refinement between Sprint Review and the next Sprint Planning should there be changes as a result of the review, but generally speaking, items at the top of the Product Backlog should be refined.
During the sprint we estimate what's in the queue of product backlog so we can decide what we're taking for the next sprint.
The estimates are one input into Sprint Planning, yes. Using information such as the refined and ordered Product Backlog, the team's recent past performance, a forecast for the team's capacity for the upcoming Sprint, the Development Team and the Product Owner can collaborate on identifying a Sprint Goal, selecting Product Backlog Items for the Sprint, and performing an appropriate level of planning on how to achieve the Sprint Goal.
Also, I'm suspecting that our user story is way too complex, but PO cannot break it into smaller details as he doesn't have sufficient technical knowledge.
That may be a possibility, but the Product Owner doesn't work in isolation. During refinement, the Development Team is engaged and asking questions. They may also see ways to decompose the work. Perhaps there is uncertainty that needs to be resolved - either product uncertainty that needs input from stakeholders (if the Product Owner doesn't have answers) or technical uncertainty that may need learning and experimentation. If a particular Product Backlog Item is too big, the team can find ways to slice it. Typically, vertical slices are preferred, but that's not always feasible.
Who is responsible for such decomposition of a Product User story?
It's primarily a collaboration between the Product Owner and one or more members of the Development Team. However, the Scrum Master may be asked to facilitate events as necessary. Depending on the background of the Scrum Master, they may have additional insights into the product management or the technical side and can provide guidance on good practices and techniques for different ways to refine and decompose Product Backlog Items.
The sprint refinement is an ongoing process. Some teams prefer to have 1-2 meetings during the sprint where the team or part of the team reviews the PBIs and asks questions for the PO or helps the PO with understanding the technical side, so the stories are ready for the sprint Planning.
Only the development team should decide how much work they can do during a sprint. If a story is too big and the team agrees that it can't be finished in one iteration, then it's good to break it down.
The development team can help the PO with this, and they can negociate how much of the functionality can be delivered. In your example, they could have a story for building the chat box, and another one for adding the error handling, etc.
Who is responsible for such decomposition of a Product User story?
- Who has to understand these items, so they can be completed to "Done" standard in pursuit of a Sprint Goal?
- Who will be held to account if the value of the product, described by such items, is not maximized?
- Who ought to coach and facilitate the various practices which optimize Scrum?