Skip to main content

Should we size items if acceptance criteria is not well defined or ambiguous?

Last post 06:18 pm May 13, 2020 by Ian Mitchell
8 replies
08:15 pm May 11, 2020

Hi Everyone,

I have an issue as SM that I have been dealing with and I am really not sure if the approach is the the correct one.



Background:



Before I started in the company, BIs and POs have not been trained writting user stories, so they would write an idea, and then they would fail the test to the programmer in UAT testing since they would use the code change to give feedback. If this is what they meant or not. There were no acceptance criteria and no sizing of stories (used Kanban)



So we shifted to Scrum in order to work more focussed on a batch on items planned for a sprint based on velocity. 



This introduced sizing and I tried to train the PO and BIs on writting acceptance criteria on items. With sizing we introduced talks as: amount of work, complexity and risk on the items before working on them.



How we work now: 



So initially structured meetings as follows which lead to much less fails in failing UAT testing by PO or BAs afterwards:



We as a dev team (whole team, 4 programmers and tester) would look at the backlog, read the cases and then size the items we think we understand and have a clear acceptance criteria and set them as Ready for development. But on the other items we will write questions to do to the PO or BAs. We use 1 hour a week for this.



Then another day we look at the not understood cases with the PO and BIs and ask them the questions previously written on the ticket. and if we understand enough we size the items and set them ready for development. This takes an additional hour per week.



So is this procedure correct? Or should we size the items even if we do not have a complete or ambiguous acceptance criteria and increase the level of risk and uncertainity so we give more user story points and we learn on them as we work on them during the sprint? 



The process became a bit tedious for the programmers, specially now that with the Corona virus situation we are all doing this from home using "Teams" and sharing desktop. Before this process was more fluent as it was face to face. 



So I wonder if there are more effective ways to do this? Is it correct to involve the whole team to review the backlog items to see if they are understood so we size? or maybe not?



Sometimes since there are some questions on the cases, and we go back and forward with them, we do not manage to size more than 7 or 8 cases on a 1 hour session. 



The splitting of the meetings to do some pre analysis in one meeting with the whole team before we have another meeting with the PO only with cases we did not understand,  is an standard approach? 



In this post from the forum, in step 3 it mention as a question to ask when we size:

  • Are the User Stories Ready, have proper acceptance criteria (the what, not the how)?



https://www.scrum.org/forum/scrum-forum/7297/steps-manage-product-backlog 





But on this other article, it is talking about sizing even if we do not have all the answers so risk and uncertainilty should be taken into account. But should this be applied for the acceptance criteria ("The what?)  as well or only it is refered to the how? 



It is said among other things:

  • Remember that it’s okay to discover the answers over the course of the sprint and while the team is working.

https://spin.atomicobject.com/2019/07/18/scrum-sizing-stories/ 



I would really appreciate some feedback on how you structure your refinement meetings and sizing user stories in an effective way.



Thanks in advance!


08:34 pm May 11, 2020

Based on your description, I'd look at how you're doing refinement.

It looks like the team is spending about 2 hours a week performing refinement, at least in meetings. However, the Scrum Guide suggests that about 10% of the Development Team's capacity is used in refinement, but this isn't a timebox. Given your team of 4 and assuming a 40 hour week, you're spending about 2 hours a week (or about 5% of the team's capacity) on refinement. I'd suggest doubling this, but this isn't necessarily in meetings. Individuals or small groups may work on refinement outside of meetings.

There's no mandated way for how to go about refinement. Different teams that I've worked with have found different approaches that work well. Some teams prefer to do most of the refinement as a large group in meetings. Others synchronize at brief, regular meetings but work individually or small groups (often with the Product Owner) to understand the work. I'm sure there are numerous other ways to do refinement.

When you can size work and the level of ambiguity depends on your team's and organization's tolerance for risk. Although you likely can't identify all issues before you start working, the amount of definition that you need is related to the risk that you are willing to accept. Spending more time to do up-front work can reduce the uncertainty and likely lead you to better estimates for the work, but this can also be wasted effort. There's no one-size-fits-all solution, but I do recommend that teams always account for risk in estimates. Both uncertainty as well as identified risks would increase the estimate.


10:58 pm May 11, 2020

My opinion is that sizing (story points, t-shirt sizes, etc.) is unnecessary. It's just estimation. And that is very project management.

Unfortunately, if your management is using sizing to measure tasks completed or performance, you may not have a choice but to do as you're told.

But if you do have a choice, I would ask the development team members to select refined PBIs (Product Backlog Items) to fit into a one to two week sprint (I'm a champion for one week sprints). No sizing necessary.


05:47 am May 12, 2020

My opinion is that sizing (story points, t-shirt sizes, etc.) is unnecessary. It's just estimation. And that is very project management.

I disagree with this. Estimating is meant as an indicator of relativity and can be used as input whether the items should be split up further, has high risk, uncertainty etc. That is why the Scrum Guide is mentioning forecast instead of commitment (where commitment is the term fitting in project management).

Besides that, how would you rate the level of interaction between PO and Dev Team? 


08:14 am May 12, 2020

Thanks for your input Thomas

Others synchronize at brief, regular meetings but work individually or small groups (often with the Product Owner) to understand the work. 

I thought of this solution, like a Diverge and Converge approach?  The only issue would be that on one way or another we have to meet as a group afterwards so size the case, that would mean that a person (not sure if developer that woked on it to refine it or PO) should present it to the team and do the sizing right? Would not that maybe encourage some ownership over the case for an especific programmer or small group on the case and most likely that one of them take the case afterwards in the sprint? As far as I understand when we size we do it as a whole since not individuals are owner of cases but the whole team, so anyone should take the case in the sprint. But on these approaches I tend to fear that the team will work in silos. 



Some thoughts about that?



So far the issue about the acceptance criteria I learned that a quite high level of undertanding about it should be done before setting the case Ready for development, if not programmers tend to program their interpretation of the case and then the case gets failed. Also when we review the acceptance criteria or description of the case as a "group" in order to size, always someone comes with some question about it that makes as realize that in some cases the requirement is ambiguous and we will have problems if we do not ask more questions to the PO.. Individually that might not happen.



But of course again, to always meet as a group remotely to do deep analysis, could en up being tedious for the team... 





 


08:19 am May 12, 2020

Hi Adams, thanks for your input:

But if you do have a choice, I would ask the development team members to select refined PBIs (Product Backlog Items) to fit into a one to two week sprint (I'm a champion for one week sprints). No sizing necessary.

But if we do not size, when you ask the development team what to fit into one week sprint, don't you have the same discussion as when you size? When you size you discuss mainly: Amount of work, complexity and Risk/Uncertainity of the case, the 3 factors converge into a number. So the number is not that important (even though help us to predict) it is important the discussion that triggers the sizing.



Before doing sizing programmers, read each item when they got it assigned and did not realized these 3 factors until they started working on them.. and that acceptance criteria was ambiguios .. they learned it the hard way, by getting a failded case because of misunderstanding requirements. 


08:25 am May 12, 2020

Hi Sanders,



I agree with you thoughts about estimating.

Regarding:

how would you rate the level of interaction between PO and Dev Team?

That was an issue the POs or BAs where not that available to us, always meetings, they have some manager roles outside the team, so I came with this idea of 1st look at the backlog only the dev team to see if we could pre analize cases, size the ones we can and come with questions to the PO or BAs  when we have the meeting with them so we make it more effective. That is why the 2 meetings.. But of couse it can be a bit tedious to do this week by week.  So I was looking to see if someone does it more effectively, specially at these times that we have to do everyhing remotely do to the pandemic situation. 


03:22 pm May 12, 2020

Going to give my standard spiel about sizing/estimates.  All you are doing is making a Scientific Wild A** Guess (SWAG) based on the information you have available at the time you are doing the exercise.  Now to some of your questions:

Or should we size the items even if we do not have a complete or ambiguous acceptance criteria and increase the level of risk and uncertainity so we give more user story points and we learn on them as we work on them during the sprint? 

Why not size everything?  There is nothing that says just because you put a size on it, you have to work it.  If you are using the story point fibonacci sequence, at level is the team comfortable pulling a story into a Sprint?  Let's assume that an 8 is large as the team is comfortable and you have a number of 13, 20, 40 in your backlog.  Those need to be refined further.  But 8 or less is deemed "ready" by your team. You now have a way of identifying ready vs needs more refinement. 

 Is it correct to involve the whole team to review the backlog items to see if they are understood so we size? or maybe not?

Yes it correct to involve everyone but it is also correct that a smaller group can discuss and then bring information back to the larger group.  The premise of estimating is so that the entire team has a common agreement on how much effort they feel is needed to accomplish the need of the Product Backlog Item.  So involve everyone but use the people effectively for the purpose at hand. 

Sometimes since there are some questions on the cases, and we go back and forward with them,

That sounds like a pretty effective refinement session. I wish more of my teams had these type of interaction. But in the interest of new ideas, have you considered having everyone on the squad spend 1 hour a week reviewing the items in the Product Backlog on their own and posting questions related to them?  If everyone is do the same exercise, some of the discussion you have been saving for a meeting can be accomplished without the need for a meeting. I've had teams that were able to completely refine items through the use of comments on Jira tickets.  

Refinement of the Product Backlog Items is a group activitiy but it does not have to be done in a group gathering. 

 


06:18 pm May 13, 2020

So we shifted to Scrum in order to work more focussed on a batch on items planned for a sprint based on velocity. 

I'm not sure that's a good reason to shift to Scrum. A better one would be to meet incremental goals when work is complex, incompletely understood, and emergent.

Is your team comfortable spending up to 10% of its time on refinement, so emergence is better understood and managed? If not, why not?


By posting on our forums you are agreeing to our Terms of Use.

Please note that the first and last name from your Scrum.org member profile will be displayed next to any topic or comment you post on the forums. For privacy concerns, we cannot allow you to post email addresses. All user-submitted content on our Forums may be subject to deletion if it is found to be in violation of our Terms of Use. Scrum.org does not endorse user-submitted content or the content of links to any third-party websites.

Terms of Use

Scrum.org may, at its discretion, remove any post that it deems unsuitable for these forums. Unsuitable post content includes, but is not limited to, Scrum.org Professional-level assessment questions and answers, profanity, insults, racism or sexually explicit content. Using our forum as a platform for the marketing and solicitation of products or services is also prohibited. Forum members who post content deemed unsuitable by Scrum.org may have their access revoked at any time, without warning. Scrum.org may, but is not obliged to, monitor submissions.