Should high-level planning exist - please help?
I have been involved in software development for 30 years. Developer, development manager, program manager, Director, etc. I have been coaching teams for ~3 years. I've always struggled with this concept and still do as a coach. I think I need a 12-step process.
Scenario- a team is given a product to develop. They are provided a number of high-level "features" directly from their end-customer. In most cases, the organization would like to know cost and schedule. In a worst-case scenario a schedule is given ("We're hoping to complete in 18 months.") and the cost is then assumed (the team(s) cost $XX over 18 months). Not ideal, but reality. Is this a feasible product to invest in? Is there an ROI? When can we forecast revenue (because we are a public company)? Etc. These are always real questions in real companies.
Should the team do high-level refinement of the features, understanding them just enough such that they can then do high-level sizing? Maybe we can get the organization to agree that the estimating, at this point, could be as much as 2X the team's estimate. But, once we break those features down, in add'l Refinement sessions, into smaller pieces of functionality (call them Stories if you'd like) so that we can create our prioritized Backlog, then we'll be able to gather data indicating if our high-level estimate is accurate. If not, we'll be agile and make the necessary adjustments (reduce the scope, add a team, etc.). It's not a commitment. It's a forecast based on what we know today (we're always going to know more tomorrow).
Then, as we're developing and releasing functionality, and hopefully delivering value as we go, we can track if our assumptions were anywhere near accurate. Are we progressing as planned (is our velocity as predicted)? Is the Backlog growing? Do we need to reevaluate what's really required in Release 1? Basically, break the product development down into releases and track progress via a release burnup chart. Is this OK? I just envision a team that follows the book, releases value continuously and then, after 18 months they discover they've only released 50% of what was expected. Now what?
Thanks for listening.
These are always real questions in real companies.
They aren't necessarily the best ones for those companies to be asking. Scrum is not a reductionist and prescriptive approach to requirements management, it is an empirical and experiential one. Those companies have new opportunities that can be leveraged. Better questions might be:
- how do we validate our assumptions
- how can we learn to be predictable rather than predictive
- what are the smallest experiments we can run
- how do we minimize the leap-of-faith we take before obtaining an outcome
- how might we maximize value quickly and take an early bath
- how do we learn to build the right thing at the right time
The approach to higher-level planning which you outline is not necessarily wrong, but it should always be done with respect to such considerations, and in that spirit each Sprint ought to be conducted as if it might well be the last.
If the organization has more work than they can take on at once, they need to make decisions about what work to do based on some criteria. Understanding approximately how long it will take to start realizing value, if the cost of building will likely exceed the value, or what skills may be needed on a team that designs and implements the feature are all pieces of information that could help with that.
Doing some level of high-level refinement and design activities to help give a good enough answer to the questions is appropriate. However, when working in a complex space, there are factors that need to be considered when using the information out of the high-level refinement. Because you are looking far into the future, the work could be invalidated, perhaps before it's even started. The team may start and discover previously unknown unknowns. Work done before it is absolutely necessary comes with risk, but it could reduce other risks.
I believe this is independent of building your Product Backlog. What you are effectively doing at this point is defining, in potentially vague terms, broad sets of work that could be captured as Product Backlog Items, but making a decision on where to invest your discovery efforts and how to order the Product Backlog for delivery. The team can still release value continuously, once it's been put into the Product Backlog.