I thought this question from a recent Professional Scrum Master delegate highlighted some common issues facing Scrum teams. I've reproduced the question with their permission, and answered below.
Hi Duncan,
I took your Scrum Training class a couple of months back. You mentioned if we had any questions, we could email you and I have an interesting situation I found myself in regarding QA's in our Scrum teams.
During a retrospective, one of my more senior QA's mentioned that he would like to try spending time at the beginning of our 3 week sprints creating automated tests for the completed work on the previous sprint. He would spend 3 to 5 days doing this, by which time the Development team would have something ready for him to start testing in the sprint. His argument is that this makes our regression tests at the end of each sprint more robust and we're building a better library of automated test.
A colleague doesn't believe this follows Scrum, because they are not focusing on the Sprint Goal if they are working on tests for items that sit outside of the sprint. I disagree with this, because regression tests, to me, are part of every goal, though it's not written out as such.
We tried it for a sprint and the QA says he's found it to be really effective and he's been able to write 8 test scenarios, where previously, his average has been 3 to 4 scenarios per sprint.
I can see the benefits, but is my colleague right? Does this go against the Scrum methodology?
Dear Felix,
Thanks for getting in touch. There are many principles, and practices in Agile and Scrum techniques, and sometimes they can seem to overlap and contradict one another. Here is my take on the situation and how the principles apply.
Focus, Commitment and Sprint Goals
There are always lots of things to do, with stakeholders competing to get their stuff done first, engineers keen to explore novel solutions, and many more opportunities to explore than time to explore them. For this reason, its good practice to regularly decide what to focus on, and to commit to that thing until it’s done. This happens at all planning levels:
- A vision is agreed for substantial change to be delivered over 3-6 months
- A sprint goal is agreed for a valuable improvement to be delivered over 1-4 weeks
- A plan is made at the daily scrum for what can be achieved over the next 24 hours
This is great because when competing requirements are proposed, teams can reiterate their commitment to the current. It helps individuals to focus when deciding what to do next. When problems arise, such as unplanned or emergency work, engineers can ask themselves “Given this new information, how can we adjust our plans in order to still deliver our goal or vision?”. If the goal or vision has become obsolete, it can be changed, but this is done is an explicit and transparent way.
Accountability to Deliver Value
Scrum is a framework for delivering complex products of the highest possible value. The value of a product is subjective, but is usually increased by characteristics including:
- Features that allows users to easily get stuff done
- Low total cost of owning, maintaining, developing, testing, and deploying
- Happy customers who feel loyal to the product and are likely to stay
- Happy staff who feel loyal to the company and are likely to stay
The Product Owner has a strategic accountability to maximise the value of the product in these areas and any others that they identify. This is done by collaborating with the development team to organise the product backlog with this objective in mind. The development team is accountable for maximising the value they deliver every sprint, and to achieve this objective they need to improve their engineering practices, and become a more effective team. There is a clear separation of authority between these two roles, with the product owner representing the business interests and the development team representing the implementation, technology, and quality. They are, however, part of a team with a common interest in the value of the product. This means that they should collaborate together both on user-facing requirements, as well as technical capabilities such as automating manual regression tests.
Transparency and Done
Scrum requires a potentially shippable increment at the end of each Sprint. if the regression tests are part of their definition of done then rolling the testing into the next sprint means they aren’t achieving this principle of Scrum.
It is expected that the quality defined in the Definition of Done should increase over time, and a common example of this is to start from a position of limited test automation, and evolve to a situation where all features have automated tests built within the Sprint, and a feature is not considered done without one. The effect of this is to reduce the cost of release arising from manual testing, and the integrity of the product is improved as mistakes are less likely.
If a product doesn’t have good automated test coverage, and this requires a significant amount of effort, then the team may raise this with the Product Owner with a recommendation to enhance the product. One way to do this would be to add these technical enhancements to the Product Backlog, so that the Product Owner is able to understand the value in time saved per release, reduced defect rates, etc. This also maintains transparency with stakeholders, who can see what work is being done.
Recommended approach
Delivering value often requires to balancing of long-term and short-term benefits, such as building a maintainable system in the long-term vs delivering a user-facing feature this Sprint. I’d recommend you raise this issue at the next Retrospective and try to come up with an approach that means your team is able to get on top of their test strategy, figure out how much effort it would be to automate any remaining manual tests, and come up with a balanced plan that everyone is happy with.