Failed User Stories During Sprint
Me and my team recently started implementing Scrum for a new project and we are all new to it, so bear with me...
One issue that has come up in our first couple sprints is we either create a user story that ends up being more work than believed, or we create one but upon looking into it further realize that we need to create other user stories / tasks to do first.
We've been trying to add good acceptance criteria and definitions, but a lot of this is 'research' or 'investigative' tasks at this point in the process, so it's not as straight forward as does the finished work does task A and task B or not.
This sprint, we had a task to look at the prototype data model / schema which was written for MySQL and determine if it is sufficient or needed to be expanded on. Upon looking into it, however, we realized maybe we should revisit the thought of using MySQL at all, or switch to NoSQL / PostgreSQL or some alternative better suited for big data. This is a much bigger task and the effort involved, although necessary, no longer fits into our sprint and this task.
So, we can remove that data model user story from sprint (maybe defer until later in case we do go with MySQL), but now I need something to do instead of it. So I start doing the investigation for the new stories (research nosql, look at cassandra, hadoop ecosystem, etc.)
What is best practice here? In the middle of the sprint do we just drop the initial user story? Do we create new ones and add them to the sprint? I'm still doing work and there is plenty to do, whether it's on the old story or new ones to make, but from a reporting point of view I'm not sure what to do. Dropping the old user story and not replacing it with new ones will have our velocity significantly lower, even though I was doing work. It will also not track any work as being completed on the user story, although I did start before other pre-requisites came up. However, I also hear changing a sprint in the middle isn't good, and we would definitely be doing that if we create new user stories, assign story points to them, and bring them into the sprint mid-sprint so that our reporting is more accurate (again, I'm already doing the investigative work on the new stories, since it needs to be done.)
What is the best route forward? I realize some may be caused by improper planning up front and we 'should've known' more about the user story in advance, or we should have investigated the data storage solution earlier on, but inevitably these things will come up. How do we stay on track with reporting when they do, especially early on when we may have more of them?
You have kind of answered your own question here:
> I realize some may be caused by improper planning up front and we 'should've known' more about the user story in advance, or we should have investigated the data storage solution earlier on,
Backlog refinement (which used to be called backlog grooming), is a practice that will help mitigate much of the pain you're suffering from, as you can use the refinement process to identify these unknowns ~ 2 sprints before they occur. Then, as you surface the unknowns, do tasks in the current sprint to remove the unknowns such that the unknowns are removed by the time the story enters a later sprint for implementation.
^^^ Be sure to see other links at the bottom of this one
As Charles says, backlog refinement should address much of the uncertainty about the work you induct into future Sprints.
However, I suspect that there is a deeper issue here. I noticed that you talked about "reporting" 3 times and didn't mention a Product Owner once. Scrum isn't about reporting, it's about delivering increments of value to the Product Owner each sprint.
In your case, ftom what you describe, strong product ownership is not being demonstrated. A PO should be exerting a healthy pull on the team for delivery. That seems to be absent, in so far as the team drifts away from delivery during a sprint and towards other activities such as late analysis and reporting. So I'd suggest taking a critical look at product ownership and at how effectively it is being represented.
Thanks to both of you - glad I found a forum where others can offer suggestions since internally I'm not sure how much I trust our Scrum dev. I know our engineering department follows scrum but they still run 18 months to get changes pushed, so I'm not so sure I want to follow in their footsteps.
This brings up two questions:
1. How many sprints in advance are planned? My understanding is during sprint planning we add items to the upcoming sprint - is this the case? Should we ever put items in sprints a couple out (even if they may change)?
I understand the concept of backlog grooming (we have a weekly session set up for it), and maybe it can be improved, but I'm not entirely sure better grooming will always alleviate this issue. In this case, the user story / stories (investigate prototype schema, document current situation, determine if it needs to be built on / expanded, document finalized schema) was all pretty well defined and we knew just what to do. However, it was based on false assumptions that did happen to show themselves during the process of actual work. This is what ultimately lead to needing to shift gears mid-sprint.
2. For reporting, it's not so much reporting to management / product owner, but only internally within our small dev group. Like I said, this is our first project using Scrum, so we're trying to wrap our heads around velocity so we can plan better sprints moving forward. But we had assigned 5 or 8 points to the initial user story, (and similar situation last sprint), and then it ends up not being completed for whatever reason. Now our velocity will be 5-8 points short (a fair amount) of what we aimed to hit, but if we complete the task next sprint (say it was 75% complete in sprint 1 but failed) then all 5 or 8 points will land in sprint 2. Thus sprint 1 looks like we significantly underperformed, and sprint 2 will look like we did awesome. Long term, taking average velocity should help, but early on for new scrum teams this makes things difficult.
> How many sprints in advance are planned?
> My understanding is during sprint planning
> we add items to the upcoming sprint - is this the case?
Yes. Sprint Planning is only done for the current sprint which has just started.
> Should we ever put items in sprints a couple
> out (even if they may change)?
Yes, but not in Sprint Planning since that should focus on the work to be inducted in the new sprint. Making tentative plans for future sprints can be done:
1. during backlog grooming, if there are no inter-team dependencies on the foreseen work.
2. during a Sprint Review when the work remaining is considered, assuming that this would not cause the review's timebox to be exceeded, and again assuming that there are no inter-team dependencies.
3. during release planning, if multiple teams are involved and if those teams are in fact likely to have integration dependencies.
> ...we knew just what to do. However, it was based on false assumptions...
Clearly then, the team *didn't* know "just what to do". They only thought they did.
Charles suggested actioning some tasks in the current Sprint for investigating the work to be done in future ones. Sometimes technical investigation of this nature is referred to as a "development spike". You need to start doing spike work like this, and if there is enough of it you may need to reduce the sprint's story point budget in order to accommodate the time and effort needed.
> Long term, taking average velocity should
> help, but early on for new scrum teams this
> makes things difficult.
I agree it can be a hard knock for a new team but it teaches them an important lesson. Work is not done until it is Done (i.e. meets the Definition of Done). I would not be inclined to make any compromises in this area.