Is "scrummish" worth doing Scrum
Our team is trying to use Scrum in an environment that isn't really ready for it, so I'm skeptical about the benefits. Every Scrum team that I've been in have, in my opinion, really been "scrummish" at best, meaning that at least one of the cornerstones of scrum has been missing.
Given the following,
Client: Very large government agency. Somewhat rigid, process closer to waterfall in that requirements are pretty much laid out before any coding is done. The requirements include view mockups, (very) formal use cases, DB changes etc.
Team: 12-14 devs, 4 testers, 1 BA
New versions are delivered a couple of times per year. Most delivery dates are set in stone (ie. come from changes in the relevant laws, not negotiable). When one version is being developed, the previous version is usually in the testing phase and all bugs found in testing must be fixed immediately, so that the deadline is not jeopardized. Also, critical production bugs are fixed ASAP.
Process: BA talks to the client and gathers requirements. Dev and testing teams give requirements an effort estimate based on time, not story points. This is because most (not all) features are fixed-price. BA together with PM prioritizes requirements. Team selects requirements according to priority in sprint planning. Because production and testing bugs must be given priority, work always gets added mid-sprint. Sprint demo and retro don't involve the client, only BA and PM. The client isn't even in the picture, only the BA talks to the client.
My observations: It's very hard to calculate velocity because we estimate hours, not storypoints and because sprint work changes during the sprint. In fact we don't calculate velocity nor try to use it. We do follow sprint burndown, but because work always gets added mid-sprint, it's pretty useless. There is no real feedback loop to/from the client.
My conclusions: As I understand it, the power of Scrum comes from a) Ever-changing requirements get managed, b) Well-functioning feedback loop reduces risk and c) Improved collaboration reduces risk (as stakeholders are intimately involved). It seems we don't have any of those, so I honestly can't see any benefit in using Scrum in this environment.
I'm not an expert on Scrum and, as I said, have never worked in a project that does real Scrum. Any insight into how we could still benefit from Scrum would be greatly appreciated. Or, as I'm inclined to think, should we totally scrap Scrum and try something else (Kanban, Scrumban)?
The situation you describe is quite typical of a large government agency. In my experience the rate of agile transformation in such environments is about 8 - 10 times slower than in the private sector. Transitions that takes 2 -3 years elsewhere might, I suppose, end up taking 20 - 30 years if indeed they ever really happen at all.
The typical scenario is to have buy-in for agile ways of working at a very high level (vision or policy level), plus buy-in at an operational level (developers who want to leverage agile practices). The sticking point usually lies with middle management...actually I hesitate to call it management at all, it is more a series of layers of vested interests. This means that there is often no coherent strategy in place for agile rollout, i.e. there is nothing that really connects the political vision to the teams' operational needs. The result is the kind of situation you describe.
What you can do is to apply Scrum at a team level in order to provide transparency. All of that work coming in mid-sprint needs to be made visible, and it needs to be shown how it impacts burn. Dependencies and blockages need to be exposed. You might not be able to fix the problems, but you can highlight them. That's why Scrum is worth doing even in this situation.
The most important single thing you can do, in regard to all of that, is to try and keep each team together so that meaningful metrics can be elicited. The worst situation is when team members are pulled out of teams by external authorities to work on ad-hoc tasks in a reactive manner. It's much better to get unplanned work put on the backlog or even fast tracked by the same team, as scope change is easier to deal with than team uncertainty.
Rather than doing scrummish, I guess it would be good to stick with the waterfall model and apply the good practices like transparency, inspection, adaptation, and self organisation in your standard delivery elements.
Thanks Ian, good points. One thing that troubles me is the pre and post-sprint steps, sprint planning and demo/retro. They seem to offer little value compared to their cost (long meetings with full conference rooms). I would like to do away with sprints altogether and move to continuous development. We can still follow burn but on a release level, and have a daily (Scrum) meeting to reveal dependencies and blockages. Do you see any value in doing sprints in our case?
Kanban can be used in support of continuous development, roughly along the lines you suggest, but it makes little sense without continuous integration and deployment (CI/CD) as well. The need for good feedback loops is, if anything, accentuated further. Tracking burn at release level, and forsaking sprints, implies that feedback would be per release. That would be no good.
I know where you are coming from. It can be very tempting to say "the organization can't support effective sprint planning, therefore we'll move away from sprints altogether".
This can be OK for BAU work, where there are only small and repeatable changes to be done, CI/CD is indeed in place, and there is no project risk. It's not appropriate as a solution though when you have project work to do.
Sprints de-risk project work by putting the emphasis on clear sprint goals, and on planning, reviews, and retrospectives that support those goals...all within the context of building a potentially releasable increment.
When an organization can't support this, that's exactly what needs to be exposed. When sprint value can't be established, that needs to be made clear and the effectiveness of product ownership challenged.
Abandoning sprints will mask these problems, not solve them.
BTW a team of 12-14 devs is rather large in agile terms. That is probably at least part of the reason why the reviews & retrospectives are too long.
A case can probably be made for refactoring this into at least two smaller teams. Don't make the suggestion to management first though. You don't want to set a precedent of external authorities deciding team make-up. Instead, ask this rather large team to consider self-organizing into smaller and more efficient teams. Ask them for their ideas and if necessary make suggestions based on splitting the value stream into smaller, less sluggish channels that encourage faster and leaner throughput.
Thanks again, Ian. I neglected to mention that we have two separate teams on two locations (one slightly smaller than the other). Both have their own sprint backlog and burndown, but both follow the same sprint cycle. However, in sprint planning both teams divvy up the work that needs to be done (by priority) and demo/retro also covers both teams. The goal is to not have separate teams working on features that are dependent on each other, but that is not always possible. In any case, both teams share the same product backlog and PO (who is also a member of one team).
Like you mentioned, the bigger problem involves various social and/or organizational issues like management buy-in etc. However, my short term goal is to make the current process work, and, to paraphrase Einstein, make the process as lean as possible, but not leaner. I'm still trying to digest what you said about the benefit we should be getting from using sprints. If I could trouble you just a little bit more, could you give a few concrete examples, maybe in the form of "this is the problem" and "this is how we tackle the problem with sprints."
Imagine an ecommerce project with various components; a CMS, a registration facility, payment, stock query, order composition with associated business rules. Business want it all by March 31st. They aren't clued up about incremental delivery and don't currently value such a method.
That's poor risk management. Sprinting would give clear goals for delivering maximum value first. So the first few sprints might target training for the CMS and stock query, then order composition (for manual printout & mailing initially), then online payment, then registration. Each increment is potentially releasable into live and can provide at least some business value and market testing. That's better project risk management through sprinting.
Now, this ability to break things up into sprints is the difficulty. The organization just isn't culturally geared for it and people aren't used to working that way, or valuing the incremental delivery and de-risking strategies you can offer.
That's no reason to stop sprinting. You still need to sprint to show the accumulated risk at key inspection points, and to make that risk transparent to stakeholders. Your message should be "Another sprint has gone by, and you have nothing. No proven CMS, no payment mechanism, nothing. Nothing has been released and validated. There is no feedback. All the risk is piling up. Help us to help you avoid a train wreck on March 31st".