How does estimation work without limiting design possibilities?
I've encountered the following challenge many times within diverse companies, although it depends on the people and their "needs".
The setting is the usual Product Backlogs Items in the form of user stories, describing the user value to be solved. We don't want to prematurely define, i.e. restrict, design possibilities which are being developed during the sprint when tacking the PBI at hand.
But this approach poses a problem for the estimation, even if relative. Granted, if the to be refined PBI is quite similar to other ones, the estimation might be feasible even without the details. But if the PBI is something new of its kind, the estimation is quite difficult to do because it obviously depends on the design/implementation.
Do you have good pracitces for this?
We don't want to prematurely define, i.e. restrict, design possibilities which are being developed during the sprint when tacking the PBI at hand.
You don't want to prematurely develop them either. The more you do, the greater the leap-of-faith taken before getting an empirical outcome. Any excess investment might be wasted.
Think in terms of developing an MVP. What is the smallest thing you can do to validate the assumptions being made? As long as the Developers believe this work can be Done during a Sprint, it can be estimated and made ready for Sprint Planning.
If you estimate, and I'd strongly encourage teams to consider alternatives to estimation, then you need to do some refinement before estimation. In fact, you may want to do all your refinement before estimation.
The Scrum Guide defines refinement as "the act of breaking down and further defining Product Backlog items into smaller more precise items" and says that refinement is "an ongoing activity to add details, such as a description, order, and size" to the Product Backlog Items. Until you have made your Product Backlog Items precise enough by clarifying the scope and breaking them down, you won't be able to estimate them.
To define and break down the Product Backlog Items, you'll have to do some amount of design. However, this is the hard part. You don't want to invest too much time in up-front design. However, if you do too little up-front design, then you may have an incomplete understanding of the work and risks. It's about doing just enough design to bring the uncertainty and risk inherent in doing work to an acceptable level.
I'd start by trying to understand the tolerance for risk of the development organization as well as key stakeholders. This can help give an idea of how much design to do as part of refinement and how much can be deferred to the Sprint where the work is picked up.
This is a good question, and this scenario often occurs. It relates to a predictive project management mindset still being used. Predictive management relies on upfront estimates to form a baseline, which is then tracked and managed through change processes. Just to stress this is NOT wrong, just different from an Agile approach.
In Agile approaches, estimation is deliberately kept high-level or deferred until the “last responsible moment.” Agile assumes plans and estimates will change, so we don’t invest too much time in making them perfect. Tracking is done empirically as Sprints are completed. Progress is measured through delivered increments and value received, rather than tracking against a schedule baseline or initial estimates.
This doesn’t mean Agile can’t be tracked, it is just tracked differently. Predictive approaches use longer upfront planning and baselines, while Agile starts development sooner and gathers real tracking and feasibility information from actual work delivered early. Early Sprint data provides empirical evidence of what is achievable or feasible.
Teams can still estimate, but they should accept that estimates will evolve. Refinement closer to delivery and empirical data from completed work are more reliable than early estimates and trying to lock things down too early. In your case, it seems the original estimates are being taken too seriously, which may discourage innovative approaches that might take longer.