Skip to main content

Agile and Data Warehouse

Last post 05:50 pm December 15, 2020 by Narayan Krishnan
4 replies
01:57 am December 10, 2020

Anyone here who uses on Agile on Data Warehousing projects? We use Informatica and the dev time for some of the complex workflows takes longer than what can be completed in 2 weeks. Completing it in small parts doesn't work since the whole thing needs to be completed to be able to test. Looking to see if anyone else has any experience in this area.

 

Thanks in advance


05:22 pm December 10, 2020

From the current version of the Scrum Guide's section describing the Sprint

They are fixed length events of one month or less to create consistency. 

Two week Sprints are just a common practice but there is nothing to say you can't go longer or shorter.  If Sprint length is the only thing holding you back, you have options.


05:56 pm December 10, 2020

We use Informatica and the dev time for some of the complex workflows takes longer than what can be completed in 2 weeks. Completing it in small parts doesn't work since the whole thing needs to be completed to be able to test.

Scrum is for developing and sustaining complex products, where more is unknown than is known. Each Sprint is an opportunity to mitigate uncertainty and risk. How about empirically testing some of the assumptions that go into those "complex workflows"?


03:10 pm December 11, 2020

I'd approach this situation like this:

  • First, I'd consider what Ian Mitchell said. Is Scrum appropriate for your context? Scrum is best suited for environments that have higher amounts of uncertainty, ambiguity, and risk and the practices that are part of or enabled by Scrum allow you to mitigate these issues. However, if your effort is clear, obvious, and well-defined with a lot of constraints or little room for freedom, then Scrum will likely add overhead. That doesn't mean that you can't get something out of some Scrum practices, but the framework as a whole probably isn't that useful.
  • Second, I'd consider what Daniel Wilhite suggested. Scrum allows for Sprints of up to 1 month. If Scrum is appropriate for you, can you deliver something in a month or less? Can you use that thing that you make to get feedback. I'm not of the mindset that you need to deploy your changes every Sprint, but you do need to be able to show what you're working on to stakeholders, get confirmation that what you are doing is correct, and realign on the desired outcomes and the path to get there. Scrum gives you the framework to do that.
  • Third, I'm unconvinced when someone tells me that the whole thing needs to be completed in order to test. Nearly every complex problem has process steps and you can test, demonstrate, and get feedback on steps to make sure that you are solving the right problem in the right way. Perhaps everything needs to be completed to make it worth deploying, but that doesn't mean that you can't get intermediate feedback along the way.

04:08 pm December 15, 2020

Thanks for all the feedback. I really appreciate it.


By posting on our forums you are agreeing to our Terms of Use.

Please note that the first and last name from your Scrum.org member profile will be displayed next to any topic or comment you post on the forums. For privacy concerns, we cannot allow you to post email addresses. All user-submitted content on our Forums may be subject to deletion if it is found to be in violation of our Terms of Use. Scrum.org does not endorse user-submitted content or the content of links to any third-party websites.

Terms of Use

Scrum.org may, at its discretion, remove any post that it deems unsuitable for these forums. Unsuitable post content includes, but is not limited to, Scrum.org Professional-level assessment questions and answers, profanity, insults, racism or sexually explicit content. Using our forum as a platform for the marketing and solicitation of products or services is also prohibited. Forum members who post content deemed unsuitable by Scrum.org may have their access revoked at any time, without warning. Scrum.org may, but is not obliged to, monitor submissions.