Scrum Success Evidence
"Scrum is founded on empirical process control" where "knowledge comes from experience and making decisions based on what is known". What evidence is available to support the framework in its purpose?
Most organisations fall short of 'proper' Scrum, where the implementation of the framework is to the letter of the guide. In fact, I personally have no evidence that any organisation has managed it - although lots claim to be and/or are on the road to it. How do we know, though, that it is fit for purpose? What is the framework itself based on? What data helped shape the framework?
Why is it even top companies like Google, Apple, Spotify, etc. have a variant of the Scrum framework? Even if they are willing to accept that the benefits of Scrum won't be gained because they're not actually practising Scrum, why have they decided to take that route? Is it because there is not enough evidence supporting successful implementation of Scrum or does it not fit their needs?
Is the necessary appetite and desire to implement Scrum fully really lacking on such a wide scale? Why? What can be done to make the benefits more transparent?
Many organizations have a project rather than a product focus, and consequently find great difficulty in expressing what value is. That makes value a hard thing to maximize no matter how empirical they aspire to be.
Agreed, Ian, but why is it so many organisations don't have a product focus? Is it a lack of understanding? Fear? Lack of evidence of success? Perhaps a chicken and egg scenario, but particularly with Scrum, who is proving the framework to be worth it and how? Even organisations who are arguably at the front of agile working don't seem to implement Scrum completely.
why is it so many organisations don’t have a product focus?
The short answer is “organizational gravity” or inertia, and insufficient willingness to overcome it and change, although that is arguably just putting a name on the several issues you allude to. All of these things can be a factor along with organizational politics and its consequent dysfunctions.
I see the problem stemming from the fact that most companies try to "transform" rather than "start". If a company starts up with a product focus they are much more successful with Scrum. But when you take an established company that has historically worked in a project based way, transforming to Scrum is very difficult. It requires that the entire corporation change the way they work. That is difficult to and not impact their revenues for a period of time. And it has to start at the top of the organization and trickle down the hierarchy. If you start at the bottom, it is more of a revolution or coup than a transformation and history shows how successful those have been.
I did work for a company that transformed successfully to Scrum. But they stopped all business for a period of time, hired some Agile Coaches to help, and actually started their transformation at the C-staff level, then built down the organizational ladder. In the end, they truly had a corporate appreciation for Scrum and had flipped the "management triangle". That was several years ago and I am not sure if they kept that culture but at the time it was a success.
If you attempt to transform a department/division of a company to Scrum, you will end up with some variation of Scrum because the rest of the organization is not going to understand, appreciate, or know how to interact.
My $.02 worth at least.