Data Integration development and Scrum

Last post 08:14 pm July 29, 2019
by Timothy Baffa
3 replies
09:30 am July 26, 2019

I work as a BA within a team that predominantly works on Data Integration activities.  We have a vendor system, into which we load different types of data, from a variety of sources.  We validate the data and create a golden copy.  We export data from it, with differing content and formats,  it to a variety of target systems.

Traditionally the BA’s have written mapping documents (using Excel) to describe the data import/export requirements.  These documents are fairly tightly controlled and aligned to the software release versions.

As a team we have recently moved to a Scrum way of working and User Stories have been developed which include the mapping document.  To keep the stories small they are not entirely independent; there is a basic story and then subsequent refinements.

Not a great example but here goes:

Story 1: Send of file of all invoices to system Z.

Story 2: Only send invoices from a subset of suppliers to system Z.

Story 3: Only want to send invoices that haven’t been fully paid to system Z.

Story 4: Only send an invoices over a certain value weekly, all other are sent monthly.

You get the gist.  So the mapping document would normally include these additional criteria specified in stories 2,3 and 4 but I do not want to include these in the mapping document for Story 1 but I also do not want to have 4 versions of this mapping document that I need to keep updated (for example an error in version for story 1 would have to be corrected in four versions).

How would others approach this issue ? I’m all for following the “working software over comprehensive documentation” paradigm but our dev and test teams are difficult to budge.

03:46 pm July 26, 2019

Why do you think any of these constitute user stories? How do they promote discussion, and the emergence of scope? What is the problem with the old way-of-working that you are trying to solve, and why do you think Scrum might help?

08:03 am July 29, 2019

To be fair, the decision to use Scrum as a way of working is decision that was imposed on the team and I'm not sure exactly what problems were being targeted. Sadly with all the Agile hype it's seen as a magic bullet and also a means of furthering one's career by having a CV full of Agile projects. Having used Scrum elsewhere at another organisation, I'm not convinced that it is appropriate for this type of work.

However this is what I'm faced with and so I'll press on.

08:14 pm July 29, 2019


One of the popular guidelines to creating PBIs is that each item should be able to stand on its own (independent).   Does attaching Mapping Documents, instead of documenting the requirements and acceptance criteria in each item,  support the independence of each PBI?

I see nothing wrong with your 4-story example.   In my opinion, you seem to be creating unnecessary complexity by attaching common mapping documents (requirements) to multiple PBIs.   

Perhaps any proposed Mapping Document change should be managed by a separate PBI that pertains specifically to that change?