Forums

By posting on our forums you are agreeing to our Terms of Use.

Please note that the first and last name from your Scrum.org member profile will be displayed next to any topic or comment you post on the forums. If you have left the first and last name fields blank on your member profile, your email address will be displayed instead.

All user-submitted content on our Forums may be subject to deletion if it is found to be in violation of our Terms of Use. Scrum.org does not endorse user-submitted content or the content of links to any third-party websites.

Do or do not include UAT within the sprint
Last Post 02 Mar 2016 02:28 AM by Piotr Gorak. 21 Replies.
  •  
  •  
  •  
  •  
  •  
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
Pablo Rossi
Basic Member
Basic Member
Posts:134
Pablo Rossi

--
25 Jun 2014 03:36 AM
    I see some teams struggling with the following scenario.
    2 week sprint. At the end of the sprint they deliver tested increments according to their DOD. By tested I mean, tested by the development team and agreed by the PO.

    However this does not mean it is potential shippable, because after the increment has been delivered, end-users (not the PO) will be testing it as well. (User Acceptance Testing)
    These users can be everyone within the organisation and most of them can't be included in the Development team due to their primary job.

    What would be the best option in this case?
    Ludwig Harsch
    Basic Member
    Basic Member
    Posts:272
    Ludwig  Harsch

    --
    25 Jun 2014 04:02 AM
    Can these users be included in the Review? This would be the best place to collect feedback from them for the next sprint planning. But this would not solve the problem that there is obviously a difference in how the Scrum Team understands the product backlog items, and how the end-users understand them. It is probably necessary for the PO to work with the stakeholders more intensively, so he can adjust their expectations and the Scrum team's understanding.
    Ian Mitchell
    Veteran Member
    Veteran Member
    Posts:1451
    Ian Mitchell

    --
    25 Jun 2014 04:06 AM
    A Product Owner must be in a position to accept an increment and to determine whether or not it is releasable. Nobody else (not even end users) has these responsibilities in Scrum.

    What does your PO think about the position he or she is in, and how the product ownership role can be better fulfilled?

    > These users can be everyone within the organisation and most of them
    > can't be included in the Development team due to their primary job.

    That's fair enough, one would not typically expect end users to be developers. It would be more reasonable to expect the PO to be able to represent the interests of these and other stakeholders, and to ensure that these interests are reflected in the understanding of Done. What does the PO think about about how this representation might be improved?
    Ilia Pavlichenko
    New Member
    New Member
    Posts:71
    Ilia Pavlichenko

    --
    25 Jun 2014 05:49 AM
    Very common situation that I observe in companies all the time. That just means your DoD is not complete. If for potentially releasable increment you need a UAT it should be included in DoD.
    How this can be done? It's a matter of discussion with PO first of all.
    Muthaiya Nallalam Parasuraman
    New Member
    New Member
    Posts:16
    Muthaiya Nallalam Parasuraman

    --
    25 Jun 2014 12:08 PM
    The sprint review process is to get feedback from the stakeholders on the potential increment. This is where the UAT will find its suitable place.

    1) How long can the stakeholder wait without the potential release ?
    2) How much authority the stakeholder has given to the PO with respect to UAT(PO cannot be an expert in legal, aerospace and medical industry at the same time).
    3) What is the nature of the product ? Is UAT necessary to consider the potential product as shippable ? If No, then PO can collect their feedback on the released product and add it to the PBL ?
    4) Will the organisation be able to accept the sprint review as UAT ? Some stakeholders will not agree with the possibility to timebox anything.
    5) What is the nature of the UAT itself ? Some system might take 4 days to test a full flow and approve it. In this case the release planning will be done on a possible staging server , where the PO will be responsible to get the feedback into the PBL. Sprint review can still be useful to decide what is done and what can be potentially released.
    Pablo Rossi
    Basic Member
    Basic Member
    Posts:134
    Pablo Rossi

    --
    26 Jun 2014 07:17 AM

    Posted By Ian Mitchell on 25 Jun 2014 05:06 AM
    A Product Owner must be in a position to accept an increment and to determine whether or not it is releasable. Nobody else (not even end users) has these responsibilities in Scrum.

    The PO can "accept" the increments, but not approve it. Simply because the PO can't represent all the end-users of the product. The product has many different modules and each module represents a department with users specialized in this module.


    What does your PO think about the position he or she is in, and how the product ownership role can be better fulfilled?

    The PO understands that he can't oversee every module because he is not an overall specialised. This application is that big that there are specialists for every module.
    Pablo Rossi
    Basic Member
    Basic Member
    Posts:134
    Pablo Rossi

    --
    26 Jun 2014 07:20 AM

    Posted By Illya Pavlichenko on 25 Jun 2014 06:49 AM
    Very common situation that I observe in companies all the time. That just means your DoD is not complete. If for potentially releasable increment you need a UAT it should be included in DoD.
    How this can be done? It's a matter of discussion with PO first of all.

    It is not included in the DOD because it is not possible. If we include the UAT part in the DoD then this would mean that we'll need to include end-users within the team which is not possible.

    My client is building a big highly complex application which consists of many modules. Every module represents a department with specialized users on board. The PO can't oversee/ approve every item simply because he is not an overall specialist. (no one is)

    Benjamin Korth
    New Member
    New Member
    Posts:16
    Benjamin Korth

    --
    26 Jun 2014 09:29 AM
    Hello P.Ross,

    I would recommend to do UAT (executed by the User) at the end of a Sprint but not during the Sprint.
    It doesn't matter if you do Scrum or not, if you gather Feedback during the development phase it will lead to never ending stories.
    Furthermore UATs should not be included in the DOD:
    1. The user should only review what is done (Transparency)
    2. Only the PO should decide what the product should look like. He has to find the best compromise for all the different user concerns.

    When the Sprint starts all the PBIs should be clear enough to start an implementation. If there are wrong decisions made by the PO this can be fixed in the next Sprint, but for the time the requirements are set.
    Even if the product is not of the value to the user as expected, it should be in a deliverable state. This does not mean that it has to be devlivered, neither that it has value to the enduser.

    So regarding your concerns, if the product is tested (except the UAT) and the PO has accepted it, it is shippable!
    It is done!

    Find a process to handle the upcomming Feedback from the UATs and endusers (e.g. Incedent Management). Integrate this Feedback in the PB.

    Kind regards

    Benjamin
    Ilia Pavlichenko
    New Member
    New Member
    Posts:71
    Ilia Pavlichenko

    --
    26 Jun 2014 09:43 AM
    It's simple. You cannot change Scrum because something is "not possible" or "extrememly hard to do" in your company. On the contrary - YOUR ORGANIZATION changes to fit Scrum and its rules.
    Scrum Guide clearly states that by the end of the Sprint the increment should be potentially shippable. If to 'ship' means UAT for your company then complete DoD would certainly include it.
    Ian Mitchell
    Veteran Member
    Veteran Member
    Posts:1451
    Ian Mitchell

    --
    26 Jun 2014 10:34 AM
    I'm afraid Illya is right. If the PO "can't" fulfil one or more of his responsibilities, or if the DoD "can't" be improved for some reason, then this is evidence that the organization has some distance to go.

    In those cases where attempts are made to change Scrum rather than the organization, the usual result, unfortunately, is very little change.
    Pablo Rossi
    Basic Member
    Basic Member
    Posts:134
    Pablo Rossi

    --
    26 Jun 2014 05:00 PM

    Posted By Illya Pavlichenko on 26 Jun 2014 10:43 AM
    It's simple. You cannot change Scrum because something is "not possible" or "extrememly hard to do" in your company. On the contrary - YOUR ORGANIZATION changes to fit Scrum and its rules.

    So if the company is telling you that it's just not possible to include the UAT within the Sprint. I bet this happens a lot. So what would you do?

    Lecturing that it's stated in the Scrum guide is not going to help this case...
    Pablo Rossi
    Basic Member
    Basic Member
    Posts:134
    Pablo Rossi

    --
    26 Jun 2014 05:04 PM

    Posted By Ian Mitchell on 26 Jun 2014 11:34 AM
    I'm afraid Illya is right. If the PO "can't" fulfil one or more of his responsibilities, or if the DoD "can't" be improved for some reason, then this is evidence that the organization has some distance to go.

    In those cases where attempts are made to change Scrum rather than the organization, the usual result, unfortunately, is very little change.

    I agree, but why would you do if your are in company where this (including UAT in the DOD) is just not supported. Doing UAT after the Sprint before it can go live is a mandatory step

    What would you do/say to convince them
    Ian Mitchell
    Veteran Member
    Veteran Member
    Posts:1451
    Ian Mitchell

    --
    26 Jun 2014 06:09 PM
    What I would do is to calculate, or at least estimate, the amount of waste being incurred through rework. It is the cost to the company of not having an adequate DoD or product ownership, and the subsequent need to repay technical debt.

    You may want to check out this recent thread:

    https://www.scrum.org/Forums/aft/870
    Rob Stebbens
    New Member
    New Member
    Posts:1
    Rob Stebbens

    --
    01 Jul 2014 02:59 PM
    So if the UAT takes five weeks to come back and is then outside of the sprint timeframe - DoD can then never be achieved?

    That scenario doesn't make much sense.

    Surely the UAT is entirely dependent on common sense, what is workable within your specific infrastructure and your environment.

    If I were to rely on my customers to sign off on a UAT for the DoD I'd never release anything. If I thought I could reliably include it I would.

    Isn't the point to produce something that CAN be released? If the User then performs an acceptance test and doesn't like it, then they don't accept that release, then the reasons why need to be understood and approached in the next increment. It's up to the product owner to ensure that this acceptance happens more often than not, as it's his/her job to make sure the users needs are met. If the ratio of rework gets too high, then your facing some serious issues.

    I'm thinking that, our good friends Microsoft released Vista as a desktop environment and everyone pretty much hated it. The next increment was Windows 7, which was to some degree slightly better. :-) The point is people chose not to go live with Windows Vista and they can do the same with your increment.


    Ilia Pavlichenko
    New Member
    New Member
    Posts:71
    Ilia Pavlichenko

    --
    02 Jul 2014 02:22 AM
    Rob, if UAT takes 5 weeks - imagine how much waste they have! It's all about bureaucracy procedures and lack of agility in a company. The question every Scrum Master decides on his/her own is - would I challenge the status quo? Good Scrum implementation always causes organizational changes. It's not about "they do that dev. thing"
    Muthaiya Nallalam Parasuraman
    New Member
    New Member
    Posts:16
    Muthaiya Nallalam Parasuraman

    --
    02 Jul 2014 02:34 AM
    if UAT is taking 5 weeks , my gut feeling is , consultancy is involved. They are delivering once in 3 months with "stage boundaries" . this is also lack of Continuous XXX

    Scrum master has a job to do.
    Ian Mitchell
    Veteran Member
    Veteran Member
    Posts:1451
    Ian Mitchell

    --
    02 Jul 2014 03:38 AM
    > Surely the UAT is entirely dependent on common sense, what is workable
    > within your specific infrastructure and your environment.

    Scrum implementation is hard. It can involve changing an organization's infrastructure and environment, as well as challenging its received wisdom and cultural norms. These have an unfortunate tendency to masquerade as "common sense".

    > Good Scrum implementation always causes organizational changes

    Correct. Moreover these changes should be deep and pervasive across the enterprise, and persistent enough to overcome organizational gravity.
    Bartek Kobylecki
    New Member
    New Member
    Posts:43
    Bartek Kobylecki

    --
    19 Feb 2016 04:37 AM
    I'm trying to understand how UAT fit in Scrum. That's a great thread and it caused me to raise a question here:

    How UAT are different from gathering data about users behaviour after an increment release, so that UAT must be part of DoD?

    In web development you have 3 main sources for product changes: user requests, behavioural analytics data and business goals. Let's assume that business goals are constant.

    Aren't UAT just a way to gather data about hypothesis that had driven the development? Therefore it's the matter of risk acceptance level whether to include them in DoD or not. Isn't it?
    Ian Mitchell
    Veteran Member
    Veteran Member
    Posts:1451
    Ian Mitchell

    --
    20 Feb 2016 01:10 AM
    > How UAT are different from gathering
    > data about users behaviour after an
    > increment release, so that UAT must be part of DoD?

    Scrum makes no prescription about how testing ought to be conducted. There is however an expectation that each Sprint Increment will be fit for potential release. That level of quality is asserted in the Definition of Done.

    Any "user" involvement must be seen in the context of either work that is in progress by the Development Team and therefore undone, or work which is productionized and therefore considered Done.

    > Aren't UAT just a way to gather data
    > about hypothesis that had driven the development?

    If UAT in this case refers to Done work that has been released, then yes. A popular term for this is Validated Learning.

    If UAT refers to undone work that is in progress by the Development Team, then no. In this case it would be better to say that it gathers data about an hypothesis that *is driving* development (present tense).
    Bartek Kobylecki
    New Member
    New Member
    Posts:43
    Bartek Kobylecki

    --
    20 Feb 2016 04:06 AM
    > If UAT in this case refers to Done work that has been released, then yes. A popular term for this is Validated Learning.

    Yes, I was referring to Done work.

    Thanks.
    Rik Pennartz
    New Member
    New Member
    Posts:6
    Rik Pennartz

    --
    01 Mar 2016 09:40 AM
    @Pablo,
    I find this article by Peter Hundermark very useful for your situation:
    http://www.agile42.com/en/blog/2011...rt-1-of-2/

    "Trying to take a too big step in this direction is unlikely to appeal to either party, so start small. If there is a team doing UAT, continue with this process but ask for a single user to join a team. The output of the team augmented by the user should continue to be tested by the UAT team. Run the exercise for three sprints and compare the quality before and after."
    Piotr Gorak
    New Member
    New Member
    Posts:2
    Piotr Gorak

    --
    02 Mar 2016 02:28 AM
    Pablo,

    Something is telling me you ARE actually releasing your product at the end of each sprint. Please correct me, if I'm wrong, but from you posts I understand that you:

    a) develop a product that is used internally in your organization
    b) give it to the end users at the end of each sprint
    c) probably "giving" includes a deployment that is an official deployment of a new product release

    Please clarify, but if this is correct, then this is no different than just releasing what you have to production and getting feedback from real users.

    Piotr

    You are not authorized to post a reply.


    Feedback