Best Practices for Testing Process in Scrum
I am always struggling to put Test process in Scrum. What is the best practices for Testing Process to put in Scrum framework ?
1) Unit Test
2) Regression Test
3) Functional Test
4) Integration Test
Scrum doesn't say anything about testing because it's "a framework for developing, delivering, and sustaining complex products". It transcends the type of product that you are developing, and what types of testing you need to do and how testing fits into the process is different for producing different types of products.
Even when the product is a software product, how you incorporate testing is different. For example, I've worked in regulated industries - aerospace and pharmaceutical. In order to maintain compliance with regulations, there are rules put around testing. You can use Scrum and be in compliance with the required regulations. However, if you did not need to maintain compliance, you may not want your testing to look like it does for a team or organization that does need to maintain compliance.
So, consider the Scrum Guide. It says that you have a Definition of Done for every Product Backlog Item and Increment. In order for a Product Backlog Item to be done, what kinds of testing must be performed? In order to an Increment to be done and potentially releasable, what kinds of testing must be performed? Given a Sprint cadence, how do you fit the required test design, development, and execution into your Sprint cycle? The answers are going to vary widely.
This is one of the challenges I faced when I transitioned my project from the waterfall model to Scrum. The test team did not want to write test cases unless the team have the output, and by the time Dev has an output worth testing, it is too late in the scrum to start testing.
We tackled the issue by:
Breaking each selected product backlog item into many small subtasks which can be delivered and tested.
Taking the help of testing team during integration testing itself, so that they can identify potential issues in advance. To do this, the testing team starts writing test cases based on each subtask, which will fail if the outcome is not met. This, in turn, gives the development team an advance notification of a potential issue.
Now, my testing team helps dev team during the integration testing and they test each sub-story deployed in test environments, which helps us to wrap up the testing before the sprint review meeting.
I will not say the transition was easy, but at the end, it was worth it.
Why is testing and development handled by different teams? Why separate these skills out?
Testing should be factored during sprint cycle.
As per definition Development team are cross functional team and they should be able to test the solution, during the sprint cycle.
As per Ian question, Development team should have the necessary skills to turn Sprint Backlog into a releasable increment at the end of the sprint that meet the Scrum Team Definition of DONE, all type of testing that are require to produce such increment should be included in the Scrum Team Definition of DONE such Unit Testing, Regression Testing, Functional Testing, Integration Testing (Automation Testing). Although this may varies from team to team which means team with less experience may require Testers to be included in the team. This type of practice will give development team capability to practice how to write different type test
In my experience, in large and complex projects, when we demand a high level of quality, even leaving aside the tests required by regulations, we generally generate a large number of tests in which We invest a lot of effort to meet the requirements, which surely leads to delays or low value delivery due to the high consumption of the team capacity.
In most cases, we put more effort into the test development than in the development of the value / feature.
Is there any way or method to manage or balance this situation?
I remember my previous development director doing a scrum session with the teams.
He made us (Devs and QA) write the different things that (1) developer and (2) QA do in every sprint on post-it notes and paste on a white board. He then ask us to remove which we think should be done by both Developer and QA.
After the session, there was only 1 note left, coding.
All other aspects we wrote down (e.g. Testing, clarifying requirements, documenting features or discussing items) can be done by both Devs and QA.
If your QAs are the only ones doing testing, it will be a bottleneck at some point in the sprint.
One of the main conditions to run a successful agile development, to have a cross functional development team, has high skills such as testing, write testing scenarios, Devops etc ...
What I'm planning to do is to have QA team write test cases according to Acceptance criteria/DoD for each story,
so Dev team will take the responsibility of the testing for each case in the story,
and take this effort in consideration in sprint planning,Test cases will be part of DoR for the story,
this is for Unit Testing,
We also need an overall Exploratory testing round for the whole sprint on integration environment,
which will be conducted by QA team after the sprint and before the release.
I see a couple big challenges. In Scrum, a cross functional Development Team has all the resources needed to complete a 'Done' Increment by the end of a Sprint. There are no separate Dev and QA teams.
We also need an overall Exploratory testing round for the whole sprint on integration environment, which will be conducted by QA team after the sprint and before the release
If work to is needed after a Sprint in order to release (sometimes called a hardening Sprint), that is an anti-pattern in Scrum, and ill advised. What you are stating is that the Increment is not 'Done' at the end of a Sprint, and the team is missing one of the most important aspect in all of Scrum. As a Scrum Master you should not tolerate this impediment, and bring this to the Dev Team to include all testing in the Definition of "Done" and ask them how they can get to "Done" every Sprint, including 'exploratory testing'.
Run every Sprint as if it is your last Sprint. Finding surprises and quality issues after a Sprint is over hurts focus, and costs more. The later a problem is found, the more expensive it it to fix.
Also the DOR (Definition of Ready) is another anti-pattern, which leads to gates and contractual thinking, and how Agile is that?
All the members of the dev team are responsible for delivering the done increment, the definition of done should be strong enough to cover the quality as inbuilt. Inbuilt quality means the increment should be tested at each level. The biggest challenge industries face when they are less focused on automation. If automation has been done at each level of the development process (unit test, API test, and UI test level) and being improved in each sprint. It saves time, makes developers and QA skills set work together.
1: Unit test - Automation does not take much time
2: API - Automation does not take much time.
3: UI automation does take time - But when we use docker container services and run on the AWS platform, we create virtual machines on demand and run all the UI in very little time. (For example on AWS Fargate service, we can create 256 virtual computers on-demand, each with 3 CPU and run 500 UI test cases in 5 minutes ).
By this given approach, the team does not wait for the regression testing being done at the end of the sprint.
Scrum testing is testing done in Scrum methodology to verify the software application meets requirements. Scrum Testing also involves checking non-functional parameters like security, usability, performance, etc. There is no active role of Tester in the Scrum Process.
Thanks Thomas Owens and Ashutosh Kumar Rai. I found it useful explanation. Development and testing go hand-in-hand in Agile Scrum. Hence the role defined is "Development Team", there is nothing like 'Testing Team' as per Scrum guide!!!
As per Scrum guide 2017: Characteristics of 'Development team' :-
- Scrum recognizes no titles for Development Team members, regardless of the work being performed by the person;
- Scrum recognizes no sub-teams in the Development Team, regardless of domains that need
to be addressed like testing, architecture, operations, or business analysis; and,
- Individual Development Team members may have specialized skills and areas of focus, but
accountability belongs to the Development Team as a whole.
Fitting Testing into Scrum or Agile is still a struggle, with the fact that Scrum does not advise having separate Roles as "Tester/QA".
Traditionally we have had QA Role and that has got into our DNA and within every project team a Testing team is included by default.
The same person, part of Scrum Development team, doing Coding and Testing, is something entirely deviating from the way things have been done until now. Literally if we ask a Coder (Developer) to do Testing (Functional Testing), they wouldn't even like it. Because they are Coders (Developers) and that is how it has always been.
I haven't really found a solution to this. Unless a Developer really changes his/her mindset and agrees to do Testing as well, we cannot stick to the principle of the Development team being Cross Functional.
Developers are explained in this manner in the current Scrum Guide
The specific skills needed by the Developers are often broad and will vary with the domain of work. However, the Developers are always accountable for:
Creating a plan for the Sprint, the Sprint Backlog;
Instilling quality by adhering to a Definition of Done;
Adapting their plan each day toward the Sprint Goal; and,
Holding each other accountable as professionals.
Previous versions contained statements to indicate that a Development Team would have all the skills necessary to do their work and that no titles were recognized.
Most companies these days have job titles Software Engineer and Quality Assurance Engineers. Within the people that have the Software Engineer title you will have people that specialize in front-end, back-end, database, and other specialities. So if it helps, think of the Developers in a Scrum Team being all those people with Engineer in their titles.
@Shashi Kar, I think that your difficulty in understanding is because you still see two teams, a development team and a quality assurance team, instead of a single team of people that have the ability to do all the work needed. When you see it as two teams you will usually create a handoff and separation of duties. A Scrum Developer team all work on the same thing...to accomplish the Sprint Goal. Embed QA on your Sprint Teams and let them work together.
@ Daniel Wilhite: Thank you for the reply.
Let me explain a little more on my struggle. Typically, before we called it Scrum or anything. We have monthly releases. In which, we had 2 weeks of coding and then 1.5 weeks of Testing. During the 1.5 weeks of Testing, the coders, keep fixing the Bugs and then the release. This was kind of waterfall.
Now if we need to apply this in Scrum and we have a 2 weeks Sprint. When does the Coding happen and when does the testing happen.