Based on my experience working in agile environments since 2010, I did some research and had general observations on the topic. Regardless of the business domains or product area, there are some common rules that may devastate your efforts toward agility.
First of all, we need to underline that the Agile Transformation never ends. This is an ongoing process of continuous improvements, running inspection and adaptation. However, the outcomes might be insufficient in some cases.
Secondly, in some organizations, the process of Agile Transformation takes many years, yet still, the results are unsatisfactory. People related to this process might ask themselves:
"Why is that? What could happen? What went wrong?" but they often stop striving for Agility, even blaming Agile ("We are different. Agile is not designed for us.").
Even if the current results are disappointing, the reflection process and then taking some actions are crucial in other organizations. This is the first step to improvement.
What's more, I observed the huge difference in Agile adoption in many companies depending on the department or product. It means that several products or whole departments have been working and acting with Agile in mind, where value delivered for customers was exceptional. Simultaneously, in other enterprise departments, they were stuck and didn't see too many benefits from agility.
There are several reasons I observed why Agile Transformations might fail or just creep. Interestingly, all these situations have a common denominator. The list I have created undoubtedly does not cover all cases. This file shows the most common reasons that may impede agility in organizations. Awareness of how the environment is set is the key to understand and make these impediments transparent.
Indifferent engagement of C-level, VP, senior directors in the process
The attitude regarding Agile adoption that is represented by top management impacts the whole process. Disengaged management is the most common reason for my ranking. There were not too many examples in my career when the bottom-up Agile transformation was successful. Usually, top management is at some point aware of Agile activities in the company, Scrum adoption, although they leave it to the teams. One of the frequent reasons for this behavior is that top management is not acquainted with the understanding that Agility is beneficial for business, product, and the most important – customers/users.
They consider Agile, Scrum and Lean to be things that might improve delivery and teams' productivity.
Let's imagine the situation when a large number of Scrum Masters reported the same impediment. It becomes an organizational impediment. How would you resolve it when decision-makers are not interested in it? What would be the impact on teams' engagement and product delivery?
Active management that fosters Agility and empirical way, and actively participates in the whole process is a secret ingredient that makes the transition more realistic.
Low focus on value and customers
Another observation I have made is a strong focus on delivery, productivity, technology and effectiveness. There would be nothing wrong if we added value and customer-oriented mindset to the top of this list. We may deliver plenty of features, yet still not provide any significant value.
Hierarchy and culture
Hierarchy in the organization can harm trust, employee satisfaction, engagement, culture, creativity, motivation, self-organization, agile leadership, and transformation by the fear, pressure, formality, traditional management, lack of transparency, politics, or negative consequences.
The company culture reflects the values and beliefs that are enacted. A company's positive culture reflects better customer satisfaction, employee satisfaction, and revenue (based on research described in Harvard Business Review).[1]
Low understanding of empiricism
In complex environments, the only answer is empiricism. This means frequent inspection, adaptation, and transparency.
That concept is crucial to enable agility in the organization.
Lack of support from middle-management
Sadly, the middle-management level’s attitude might be another reason why Agile Transformations fail. Lack of support or even blocking ideas and changes is widespread. Often, this behavior is based on fear of losing a job or power or control. On the contrary, during the process of adopting Agile, I regularly observed the managers were missing. They are empowered enough to help remove impediments to enable Agile in teams, business, top-management, customers, and many more.
Project-oriented organization (instead of a product)
Projects and traditional, predictive mindsets are still dominant, especially in large organizations. There is a strong attachment or maybe a habit to conventional project management. Products are rarely defined. This approach leads to an iterative waterfall (scrumfall, wagile, etc.) as projects impose fixed time, scope, and budget, thus little opportunity to inspect and adapt and deliver the most desired value to customers.
Low level of transparency and trust
This is one of the most critical blockers on the way towards Agility and should be immediately discovered and revealed. In such circumstances, any change is tough to proceed.
Vague reasons why they want to implement Agility
Several companies I have met could not clearly answer the question: What is the reason you want to make a transition to Agile? Quite often, the reason is that they want to be faster to the market or want to be Agile like other companies.. I do not blame them for these reasons. I only want to make it clear that deliberate reasons may cause your transformation to be more successful.
If companies already use Scrum, the framework is usually mechanically implemented and understood
Misunderstandings when it comes to the Scrum framework are often met in companies. There is a strong focus on velocity, story points, perfectly written user stories, outputs delivery instead of delivering valuable, potentially releasable Increment, and customer satisfaction.
A mechanical implementation of Scrum (versus empirical Professional Scrum) always leads to the so-called Zombie Scrum, which means that there are no benefits from Scrum; the organization is focused on outputs rather than outcomes.[2]
An organizational structure that makes delivery even more complex
How the company is organized, structured, and how they define their delivery flow has a tremendous impact on many aspects. This is visible, especially in organizations where end-to-end product delivery must go through many departments and/or teams. This approach usually requires bureaucratic processes that are simply an unnecessary overhead. Such organizations are typically focused on projects rather than products. A product-oriented structure could be a solution and dedicated teams focused on the same product (reducing context switching).
War between departments
Quite a prevailing situation is a disagreement and blaming game that happens between business departments and IT, in IT internally, in business internally, or other combinations. I have seen these situations several times and I considered them as a huge impediment towards agility.
Traditional contracts that shape the process
These contracts consist of the detailed, fixed scope, fixed time and budget. In such circumstances, particularly when the contract is signed up for a few years ahead, it is challenging to work in an Agile manner without strong customer engagement and agreement to change the way of cooperation.
Limited collaboration with customers and users
Occasionally, organizations prevent teams from direct cooperation with customers. They limit engagement, product and customers' understanding, fast reaction to the market, users' needs by blocking this close collaboration.
Low employees' engagement in changes
Employees' commitment to transformation, their willingness to work in the Agile environment is essential. The Agile Transformation must be performed on all levels in the company. When the level of employee engagement in changes is low, this can ruin all your efforts toward Agility. Usually, the reason is poor understanding of the empiricism needed in the complex environment and implementation of the mechanical Scrum framework. However, it might also be caused by the general culture of the organization.
Teams self-organization is not welcomed
There is a deep reluctance and misunderstanding when it comes to letting teams self-organize. Usually, self-organization is considered as chaos or a fear of being redundant (losing power).
Inability to innovate
A huge technical debt, sporadic releases, lack of continuous integration, etc., might be another obstacle in the organization's way toward agility.
Low tolerance for experiments and learning
People are not allowed to do any probing, then inspecting and responding. Even short (and low cost) experiments are not welcomed.
Impediments are not solved
Even if the organization is aware of generally known impediments that are serious obstacles, no one feels responsible for solving them.
I was hoping you could give me a detailed plan for the Agile Transformation
The Agile adoption process is also empirical. We need to have an important goal(s). A detailed long-term plan is not realistic as we learn more and more over time. We need to inspect and adapt frequently.
Reports from an Agile audit are ignored
Frequently, companies want to be diagnosed. Having said that, this is often all they are willing to do. No changes are following the audit report and initial proposals. This attitude can be linked to the following reason.
Fear of change
Some people simply do not like changes both in professional and private life. Others are afraid of a new organizational structure, new job responsibilities, and losing power or their job. The agile transformation causes many changes in different aspects of the company. But they do not necessarily mean losing a job. Traditional empowerment may be converted into Agile leadership.
Agile Consultants are not heard, Scrum Masters are not empowered
In most organizations I have met, Scrum Masters are perceived as teams' secretary, facilitator, scribe, tool admin, etc. In fact, a Scrum Master provides service for teams, Product Owners and the entire organization. This last part is frequently ignored.
Agile Consultants are sometimes hired to "heal" the organization, although their (usually in cooperation with Scrum Masters) ideas, proposals, experiments are not considered worth trying.
Conclusion
These reasons mentioned above are the most common in my experience from consulting and teaching in several countries and continents. You may observe different ones. Diagnosing what the reason is for the failing Agile Transformation is the key. The remedy would be reversing these anti-patterns. Always remember to show the real data. Talking about facts has a huge impact and you may avoid discussion about emotions, feelings and beliefs.
Despite many attempts to engage the top management in the Agile implementation, when I still observe a low level of their commitment, I share this fact with them transparently and how this disengagement negatively impacts the transformation. Sometimes it helps and may move towards positive changes to Agility.
Agile leadership, removing impediments, empirical paradigm, taking care of anxiety, engaged top and middle management, empowered Scrum Masters and Product Owners, lively culture and motivated employees, striving for customer value, measuring value, and making decisions based on facts are ingredients of the successful Agile journey for the organization.
The article initially was published on my website: https://magdalenafirlit.com/why-sometimes-agile-transformations-fail/
[1] https://hbr.org/2015/11/how-company-culture-shapes-employee-motivation
[2] https://www.scrum.org/resources/blog/how-efficiency-mindset-leads-zombie-scrum