My Big Design Up Front problem

Last post 11:34 am December 30, 2014
by Ian Mitchell
4 replies
04:45 pm December 22, 2014

I would hardly call myself a veteran of Agile or Scrum, but I have been involved in a few projects that used variants of the methodology. I would classify myself as the product owner, if I had to pick a role.

Here is my situation:

We ran two major projects concurrently. Both were very similar. Technically, we should have built one underlying platform to provide a basis for both products, but we didn't and my regret for not having forced this is running very high.

We used elements of Scrum in both cases. Because each project was massive, we needed two vendors. Because I had a good perspective on the overall vision and had lived the hell of not having these systems, I had a reasonably well abstracted sense of the pillars of the platform we needed to develop. When I say "pillars", I mean the basic abstractions that, if acknowledged, could create a shared understanding and a shared vocabulary and (ultimately) a whole lot less code. Unfortunately, when I began to work with Vendor 1 they had no interest in hearing about my pillars. They wanted to work from a simple vision and hundreds of user stories that made no attempt to abstract any concepts that could add to comprehension, achieve efficiencies through reuse, etc.. They claimed that emergent design and refactoring would do the trick. Well.... it is two years later and we still don't have time to refactor and the complexity of the poorly abstracted code means we probably never will.

In the second project with another vendor, I created a 10 sentence document that provided a vision and, more importantly a set of abstractions and a vocabulary for the product. The document defined some abstractions/entities that I insisted we reference in our stories. So, when I talked about a "Clinical Model Score" for a user, or a "User Message", or a "Tailoring Pipeline" everyone knew what it meant. Even more important, we didn't labor with emergent design and, instead, just coded these pillars or constructs early on (in about 30 days) and then built on top of them. I improperly called this "iteration zero" at the time. Basically, we agreed to a framework.

The second project was far more successful than the first and I attribute it to having the pillars of the platform identified early. These were easy abstractions to make for a business person like myself, but they would have been nearly impossible for our teams to have come up with through their normal Scrum process. Refactoring is a great theory - and has worked for superficial aspects of our products, but not for critical abstractions that I am calling pillars or building blocks.

I certainly see the benefits of Scrum, and I'm not attempting to debate them. I also see issues with Big Design Up Front, but my design phase is about 2 weeks of thinking and 30 days of coding for a project of about 5 person years. My problem is that I am heading into another project with Scrum purists and I'm hoping to encourage the approach that has worked for me in the past. Here is what I am looking for:

-I know that there are a number of variants of Agile and Scrum.
-Is there a variant that accommodates the approach I took in the second project?
-If not, is there a good book or site that debates this in a balanced way? I have a basic understanding of Emergent Design, but I don't agree with it on its own and am looking for something that mixes this with Small Planning Up Front.

Sorry for the long post, but this is on the verge of becoming a multi-million dollar problem for my company.

03:22 am December 23, 2014

> Is there a variant that accommodates the approach I took in the second project? 

I believe that Scrum at Scale supports this approach. A skilled team may implement a foundation architecture which Scrum Teams can leverage in future sprints and develop further on an emergent basis.

If an architectural foundation is not in place, then the Product Owner should ensure that the Product's architecture is refactored appropriately by the team. Scalability and sustainability are product qualities for which the PO is accountable. Failure to observe this may incur technical debt and can limit return on investment.

05:10 am December 23, 2014

NB a foundation architecture should implement at least one or two choice features of business value. This is partly to demonstrate that such value can in fact be delivered, and partly to validate key architectural assumptions.

This technique is sometimes referred to in the wider literature as stovepipe prototyping, or as firing a "tracer bullet" through the architecture.

08:39 am December 28, 2014

Hi Stephen,

There is nothing in Scrum that would stop you using your second approach. In fact, I think your first approach is not Scrum compliant. If you are PO, you work on stories, you order the backlog, you accept or reject the work,you are accountable for the value... Story should be written in the way the PO understands them and then devs can ask PO for more explanation. If you say "As a Robosupadupa I want Sipadipaklipa to perform in less than 10 seconds so that repadepatepa can glupazupadupa more for less" and your acceptance criteria is "Given mikatikasika then glupazupadupa should be blank" the development team cannot change it. What they can do is ask you to explain what those mean. If you have stories that explain your architecture, sort of, Scrum is not against those being on top.

There are some great books on stories, I like Gojko Adzic's approach to stories and I have a feeling you will too.

11:34 am December 30, 2014

> I think your first approach is not Scrum compliant. 

The first approach *could* be Scrum compliant, in so far as the framework does not rule out a completely emergent architecture. The problem in this case seems to have been the vendor's claim that "emergent design and refactoring would do the trick". In truth it must be the other way around...the "trick" being to make sure that emergent design and refactoring are actually done. They don't take care of themselves and must be budgeted and planned into every Sprint.