Skip to main content

Is it necessary to maintain screenshots during the test execution?

Last post 03:21 pm May 23, 2017 by Paul Fiala
8 replies
09:54 am May 16, 2017

Is it necessary to maintain screenshots during the test execution?

We had a internal process audit for our agile project (scrum) and the audit team has asked us to maintain screenshots for each and every story that we are testing. This would help us to justify that we have covered the acceptance criteria during our testing.

But we have not agreed to the process change, as this goes against the agile principles - keep it simple and collaborative delivery.

Could someone please help us to understand is this the same process that you follow in your scrum project?

Is it really necessary to maintain test execution screenshots for an agile project?

Thanks.,


03:47 pm May 16, 2017

Agile should avoid unnecessary complexity, but it doesn't mean "without documentation."  You mention that you don't feel this goes against the agile principles, but doesn't documentation support transparency and accountability for the development team?

Perhaps there is a better Definition of Done your team could use that doesn't involve tons of screenshots?


06:31 pm May 16, 2017

How does the team currently ensure that all of the acceptance criteria have been met, and why isn't the team's process thought to be justified for the purposes of auditing?


09:45 pm May 16, 2017

Good topic :)

As a practice, our company has required screen shots as evidence of testing as we have been doing manual functional testing.   And I am just guessing that has come as input from our auditing firm since they carry a fair amount of weight here.

The recent question that has come up is the same required if we are doing automated functional testing.   Real quick google search on the topic today does not provide a clear response for best practices.   I did find someone suggest logs from the testing if not screen shots.


07:15 am May 17, 2017

As everything related to auditing it depends on what the auditors expect. According to what standard/system/... are you being audited?

In my experience automated testing is usually evidenced through testing logs. Most automated testing frameworks generate those by default anyway. Depending on the standard your being audited under, this may require the Development Team to review that the automated test really does test the acceptance criteria in question. This can usually be done informally.

If the auditors have concerns you need to talk about it with them in the interest of openness (which is as we all know a Scrum value). Find out what it is they really need and find a way that works for both sides. This will involve your team making suggestions, because sometimes auditors are unaware of the possibilities at your disposal. If they're used to screen shots they may ask for those because they're used to it. If you show them how test logs will also give them evidence of testing, they might agree with you.


09:48 am May 17, 2017

I would have thought a copy of the logs would suffice to prove that the tests have passed. Or if they really want screenshots, can you not run all tests at once and then screenshot the final page showing they all passed?


01:49 pm May 17, 2017

It isn't the responsibility of auditors to prescribe how an agile team should evidence its work. Auditors should instead seek to assure that the method a team uses is being applied consistently and to the level of quality being claimed. The wider organization may set certain standards, but it is not up to auditors to define what is done.

- For manual testing (such as exploratory tests) the team may decide to record screen-shots, or to describe the scenario being tested and check it off, or to author an automated script which is added to the regression pack. Auditors would assess the clarity of whatever policy they had and the consistency with which it is applied.

- For automated testing, logging ought to be in effect and sufficient for the purposes of auditing. If it isn't then the level of automation is inadequate.

 


01:26 pm May 21, 2017

It's not required to maintain screenshots for test execution but we also have this process in our scrum projects. All scenarios are executed when new functionality is added to product. This makes sure that old functionality still works when new functionality is added. However it's tiresome to take screenshots repeatedly. We are planning to start automated UI testing to find regressions easily.


03:21 pm May 23, 2017

Thanks for your comments all, am passing along internally as part of the conversation here.


By posting on our forums you are agreeing to our Terms of Use.

Please note that the first and last name from your Scrum.org member profile will be displayed next to any topic or comment you post on the forums. For privacy concerns, we cannot allow you to post email addresses. All user-submitted content on our Forums may be subject to deletion if it is found to be in violation of our Terms of Use. Scrum.org does not endorse user-submitted content or the content of links to any third-party websites.

Terms of Use

Scrum.org may, at its discretion, remove any post that it deems unsuitable for these forums. Unsuitable post content includes, but is not limited to, Scrum.org Professional-level assessment questions and answers, profanity, insults, racism or sexually explicit content. Using our forum as a platform for the marketing and solicitation of products or services is also prohibited. Forum members who post content deemed unsuitable by Scrum.org may have their access revoked at any time, without warning. Scrum.org may, but is not obliged to, monitor submissions.