How often have you heard the word Empiricism in connection with Scrum? I believe almost every time someone says Scrum, they do bring up Empiricism. How often does this word show up in the Scrum Guide? Just 5 times and that too without much explanation. So, the real question here is - how many people who bring up Scrum and Empiricism in the same breath, really understand - what Empiricism is all about or they simply mug up that Empiricism requires three key pillars of Transparency, Inspection and Adaptation.
The Myth of Control
This myth comes from a naive understanding that in software product development, when a set of predefined inputs are provided, the same results could be achieved every time. This idea was based on the assumption that all processes and tasks can be defined in utmost detail, they will not change or evolve and it gave a sense of control.
For over half a century, the IT world has operated with this myth. Mostly, the warning was overlooked, when the “Waterfall” came into existence1. Even though we may be able to define the processes, none get into so much detail that a repeatable output could be ensured; especially when we are dealing with “complex”2 environments.
This has never been true. Atleast, IMHE, 95% of the times the details evolved or changed.
Where does the word Empiricism originate from?
To understand the origins we will have to explore the realm of Industrial Processes. There is a “Defined”3 process and then there is an “Empirical”4 process.
Defined Process:
A “Defined” process is one, where the process control model requires that every aspect of work is completely detailed out and understood. Given a prescribed set of inputs the process will generate the same result every time.
Here is a flow chart of how I make tea.
If this process is started and allowed to execute to its completion, it will always result in exactly the same result - a cup of Black Tea with Sugar or a cup of Black Tea with Ginger and Sugar. No matter where I perform this process. There is no alternative outcome to this defined process.
Empirical Process:
An “Empirical” process is applied when there is not enough clarity or the results can vary even with the same set of inputs. Empirical process control models do not start by establishing a concrete approach to achieve a specified result. Instead, an empirical approach states - given a set of inputs, in order to achieve a desired outcome, WHAT is the best possible approach or process.
Empiricism, is derived from the Empirical Process Control model and encourages continuous Inspection and Adaptation of process through a set of Control Systems, especially in environments where the process itself could not be properly defined or generates results that are unpredictable.
The process applied here to convert the inputs to desired outputs evolves based on the Inspection and Adaptation applied through the control model.
Scrum and Empiricism
Mike Beedle, in his book5 shares a conversation with Babatunde where the latter highlights that software product development is intellectually intensive, requires too much thinking and creativity and thus a defined process is not suitable.
Also, Software Product Development is inherently a complex6 domain problem. There are many dependencies between various modules, teams and even vendors. And often there is no guarantee that these dependencies will be predictably addressed. To address these dependencies and uncertainties, an empirical approach becomes apt.
From the Scrum Guide:
The Scrum framework is purposefully incomplete,[...] Various processes, techniques and methods can be employed within the framework. Scrum wraps around existing practices or renders them unnecessary. Scrum makes visible the relative efficacy of current management, environment, and work techniques, so that improvements can be made.
Scrum, through its various events, artifacts and accountabilities provides the desired “Control Model” required for an Empirical process.
The events - Sprint Planning, Daily Scrum, Sprint Review and Sprint Retrospective provide a formal place to enable Empiricism via Inspection and Adaptation of the artifacts - Product Backlog, Sprint Backlog and Increment by the accountabilities - Product Owner, Developers and Scrum Master along with their key stakeholders.
Conclusion:
Empiricism in Scrum is not a collection of buzzwords but it is an effective control mechanism that drives and enables the Scrum Team to create valuable and useful products.
A Scrum Team and its stakeholders need to understand that Scrum is not a mere toolkit of practices and metrics but it is a change mechanism, which would uncover the inefficacy of an existing system. Scrum when used effectively can evolve the processes, mechanisms and frameworks to create products that deliver desired outcomes and not just complete activities.
Reference:
1: Managing the Development of Large Software Systems - Dr. Winston W. Royce.
2 & 6: Complex Environments: Cynefin Framework - Dave Snowden
3 & 4: Process Dynamics, Modelling and Control - Babatunde “Tunde” Ogannaike
5: Agile Software Development with Scrum - Mike Beedle
P.S. If you are interested in learning more about Scrum then you are welcome to join one of Scrum classes details are available here.
You can also subscribe to my YouTube channel and Agilemania's channel.