should simple technical tasks be backlogged and estimated?

Do you personally think that those tasks will have an influence on other tasks the team pulls from the product backlog (might they pull less due to the task?)? How would you create transparency if the tasks are not documented as part of the sprint? Does the technical task add value to the product?

Does the *team* consider it part of the sprint commitment / forecast even if the technical task is relevant to a single specialist's domain (and assumed to be worked by that specialist)? 

Would the Development Team cease to be accountable for that work and any consequences it may have for product quality?

Ideally tasks need to be supporting a user story, and hence get estimated as part of the story. However, if there are tasks that help put the 'plumbing' in place for multiple stories, then we have used 'enablers', which do get estimated during Sprint planning.

In my opinion, any item that appears in a Sprint Backlog is part of the Development Team's forecast.   

If the technical task can only be completed by a single "specialist" on the Development Team, that is indicative of an existing team constraint that should be addressed.   How can the item be included in a Sprint Backlog if only one team member is capable of working on it?   If that team member becomes unavailable, what happens to the item?

So...let's talk about this from two different perspectives:

1. The Scrum Position

User Stories are not a mandated part of Scrum; they are a best practice that many teams have adopted because they encourage the focus of software to be on delivering as much value to stakeholders as possible. So, Scrum isn't going to proscribe how these tasks get done as long as those tasks are considered and transparent to the team in some way (Definition of Done, as part of another Product Backlog Item, as their own Product Backlog Item, whatever.)

2. What About Actual Practice?

Now this is where we get into "advice" and practice discussion, which has many different options. Knowing absolutely nothing about these tasks (whether they're 1-offs, a actually a dependency for a story and not articulated as such, a recurring task, their priority compared to the other work in the Sprint, or the value in even doing them,) my recommendation would be to document them as Product Backlog Items, order them by priority, and plan them as part of any other work in the Sprint. If they're things that come up in the middle of a Sprint and interrupt work (like a production issue,) then those should still be Product Backlog items that either get pulled into the Sprint with the understanding that it puts other work in the Sprint at risk or get put to the top of the product backlog to be picked up immediately in the next Sprint.

If they're recurring tasks that need to be completed in order to ensure an integrated increment is delivered at the end of the Sprint, then they should probably part of the team's Definition of Done, and should be indicated as tasks belonging to the product backlog items selected for the sprint.

If it's a matter of doing refactoring to alleviate technical debt (e.g. using a newer version of an API,) then you should just include that as part of the implementation of a feature that uses that API, and is thus a task within a story, because you should ALWAYS be refactoring and optimizing within reason.

If they're more like ongoing maintenance tasks that happen on a regular schedule, then you can use the concept of "Routines" and add them to a Routine checklist, recognizing that it means that the team's velocity will probably decrease overall to accommodate.

At this point we're not talking about "the Scrum way," we're talking about many different options that all fit within the Scrum Framework and have their own pros and cons.

Thanks for the responses thus far. I understand there's no single right way to address this question, and am mostly curious to hear the different ways that folks are handling it. 

@Julien - I don't quite understand your first question. As for your last question, let's assume the task doesn't add business value in isolation. Regarding transparency, I know one scrum master that has his team keep a distinct list/backlog (separate from the sprint backlog) of these kind of items for individuals to pluck off when the timing seems right. Sometimes this happens when a specialist has completed his specialty work in the sprint backlog, and has looked for ways to help the team but the team has concluded at that particular time it will be counterproductive toward meeting the sprint goal for him to get involved in other areas  (i.e. as it will demand support and/or oversight from other folks on the team who are focused on sprint backlog work at risk). I understand this goes against promoting generalizing specialists and T-shaped individuals, but it is nevertheless a reality I've seen play out quite a bit. This dovetails a bit into what Timothy was asking about...

@Timothy- I used to take the same angle as you. However, I've read some literature indicating that specialists on a scrum team tend to be a reality (i.e. DBA as a common example). Moreover, while I agree we want to strive for cross-pollination across the team, it's unrealistic to think that everyone on the team is going to be well versed in every area (i.e. or at least enough to hit the ground running on any particular task that comes up). Let's say the technical task can be done by 3 folks on the team (out of 7) instead of a single specialist. Is that still considered a constraint? Can the item be included in a Sprint Backlog if less than half of the team is capable of working on it? Where is the line drawn before we can comfortably say the team is accountable? @Ian, since you raised the topic of accountability, what is your opinion here?

@Raj (and @Kevin) I used to take the approach of associating refactor tasks to user stories that might have some association / dependency, and estimating that story accordingly. One problem I found with this approach is that the refactor may be significantly larger than the work that would otherwise be needed to accomplish the story (and obtain business value). There have been times when the team is then resistant to pull in the story to the sprint backlog merely because there is not enough time projected to complete the refactor, even though there would be time to achieve the business value from the story without the refactor. How would you guys handle that scenario?

@Kevin - thanks for the various approaches you highlighted. I like your position on "routine" tasks. Do your teams accept these routine tasks into the Sprint Backlog, or are they tracked elsewhere?

I know these were directed at Timothy, but I'm going to throw my two cents in:

I've read some literature indicating that specialists on a scrum team tend to be a reality

I 100% agree with you on this. Scrum demands that the TEAM be cross-functional, not that each team member be cross-functional. I also agree that cross-pollination is a good thing (esp in the case that someone is out of the office,) and that's also why if you're doing Scrum well, you're also practicing XP and are pairing and/or mobbing.

Can the item be included in a Sprint Backlog if less than half of the team is capable of working on it?

Absolutely. I don't care how many people are working on a Product Backlog Item, it is the TEAM's work, not an individual's, and it WILL impact the TEAM somehow, whether it's because that individual is occupied by what they're working on, or because they need to get input or a review from another team member. One of the first things I did when I joined my current engagement was insist that we stop calling things "So-and-so's story" or "so-and-so's bug," and drastically reduce our number of work in progress items because people were bouncing around trying to do 10 different things at once; it was madness.

Now onto your comments to me:

One problem I found with this approach is that the refactor may be significantly larger than the work that would otherwise be needed to accomplish the story (and obtain business value).

Yup, but you've got to be ruthless about it, because if you keep putting off the refactor you're going to keep accruing technical debt. At my current engagement they have continually avoided making crucial updates to one of their core applications. The team is actively trying to modernize their database now and migrating it to the cloud -- they're struggling, and their stakeholders are getting more and more frustrated because they're not seeing the benefit of this work (which has already taken about 2 months,) but the team's hand has been forced because the on-prem servers are being shut down. If they had been ruthless about refactoring, they would have incrementally done the migration in as part of executing other work.

Putting off the refactoring doesn't make the need to refactor go away, it just continues to accrue technical debt. It's how you end up with legacy software that can never be turned off because it's been jammed so full of features that when it comes time to build the "new" version, you never have the time or budget to "completely replace it."

Don't get me wrong, you do need to weigh the pros and cons of refactoring, and know when to cut bait and move on because you're getting diminishing returns. One of the tools I use for helping the team manage this is having a set of Architecture Principles that the team develops and continually revisits the same way they do their Definition of Done. The assumption is that any new work will adhere to the Architecture Principles and be estimated as such -- once you get into the Sprint we can discuss the impediments preventing them from doing so or where we need to compromise, but that conversation happens in the context of what's necessary to accomplish the Sprint Goal, not a decision made before we even start working.

Do your teams accept these routine tasks into the Sprint Backlog, or are they tracked elsewhere?

As long as they are regularly recurring and relatively predictable (as in low complexity and low ambiguity with a relatively consistent amount of effort each time) I have the team handle these outside of the Sprint Backlog. I usually have the team schedule specific recurring time for them (I know, I know, meetings are bad, but it's really more about carving out a specific time to do those tasks at a consistent time to reduce complexity and ensure that they happen.) I have the team keep checklists, and these tasks would probably go onto one of those checklists and there'd be a link to the list from the scheduled meeting on the calendar.

Again, I use caution with this recommendation, as I don't know what these tasks are in your situation. I prefer to include as much as possible within a story, and it's easy for teams to start taking work that should REALLY be part of a Story and included on the Sprint Backlog and treat them as "routines." For example iOS 12 just came out, and our team needed to validate that one of our applications worked with iOS 12. The team's initial impulse would have been to do the validation outside of the Sprint Backlog, BUT that would have been the wrong way to go.

We need to include iOS 12 as one of our supported versions moving forward, so we updated our principles and Definition of Done to account for iOS 12, and that validation along with all of the refactoring and bug fixes necessary to support iOS 12 was included as Tasks and Issues for one of the Stories we were working on for a new feature. If we hadn't been working on anything for that app in that Sprint, we would have added a Story specifically for iOS 12 validation, as the initial update is a one-off activity, but we need to continue supporting that moving forward.

A good example of something that you might include in a routine is "Run a refresh of the QA environment database with production data," which you might do once a week (for *sigh* reasons,) BUT you should also start asking yourself and your team, if there is a way to start automating these activities so you don't need to do them manually anymore. And the effort to automate that WOULD be handled as a Product Backlog Item in a Sprint, because you should ALWAYS be looking for ways to eliminate waste.

@Kevin, thanks again for the detailed response. Lots of great, interesting points here. 

The Architecture Principles seem like a great concept. Are you able to elaborate on that (e.g. some examples of principles)?

While I understand where you're coming from with being diligent about paying down technical debt, I'm not sure I agree with the inflexibility of that approach where the refactor HAS to be done in a loosely associated user story. I can imagine scenarios where the payoff of a releasable story without refactoring would be worth the risk of the accrued interest on the technical debt. Do you disagree?

One other dilemma I find in bundling refactoring / tech debt along with loosely related user stories has to do with the estimation and subsequent prioritization. It's possible to associate technical debt with many different stories sometimes. So how do you choose just one? As stated before, taking the technical debt into account sometimes inflates the estimation of the user story significantly. And then when weighing cost/benefit it becomes hard to prioritize that story against others (i.e. that would have been candidates to associate the tech debt to). How do you handle this sort of situation?

A Development Team ought to have a technical debt policy which acknowledges the team's responsibility for managing debt and the urgency for remediation. For example, a team may decide to pay off debt over the next 3 sprints. The cost of doing so may then be amortized against the corresponding projected work during Product Backlog refinement. The estimates may be spread across that body of work rather than against one story. In effect, those Product Backlog items become more expensive because more technical work (e.g. refactoring) will be involved. If and when items are re-ordered during refinement, it may be necessary for the Development Team to revise their estimates.

@Ian, thanks for the reply. 

The estimates may be spread across that body of work rather than against one story. 

In practice, how do you achieve this? Are all related stories collectively estimated? Is an estimate given specifically on a technical debt backlog item? ...

If and when items are re-ordered during refinement, it may be necessary for the Development Team to revise their estimates.

Could you please elaborate on this? Are you suggesting the dev team should re-estimate stories any time a refactor effort is identified that may be associated with an already estimated story?

Let's say the technical task can be done by 3 folks on the team (out of 7) instead of a single specialist. Is that still considered a constraint? Can the item be included in a Sprint Backlog if less than half of the team is capable of working on it? Where is the line drawn before we can comfortably say the team is accountable? @Ian, since you raised the topic of accountability, what is your opinion here?

I'm also still interested to hear your opinion on this :)

In practice, how do you achieve this? Are all related stories collectively estimated? Is an estimate given specifically on a technical debt backlog item?

Either way can work. The latter is the simplest and most preferable, but it requires that the Product Owner be sufficiently technically literate to understand and value the technical debt item. Otherwise, the former way may be preferable if the Development Team is to ensure that the technical debt it has incurred is  well managed.

Are you suggesting the dev team should re-estimate stories any time a refactor effort is identified that may be associated with an already estimated story?

Yes, in so far as that refactor constitutes work that they must do. The Product Backlog should always show the amount of work which is believed to remain.

Let's say the technical task can be done by 3 folks on the team (out of 7) instead of a single specialist. Is that still considered a constraint? Can the item be included in a Sprint Backlog if less than half of the team is capable of working on it? Where is the line drawn before we can comfortably say the team is accountable? @Ian, since you raised the topic of accountability, what is your opinion here?

Is there any point - any point at all - where a Development Team ceases to be accountable for the development work performed by one or more of its members?

I’d suggest that, in Scrum, the line you refer to may not be drawable in the first place. There is after all a reason why the role is Development Team rather than Developer.

My  is that there is no task that can't be expressed as some value to the user.

As a user, I would like to use this application long into the future and I don't want to have to spend a lot of time looking for another one because the one I know and love is too slow, too jenky, too weird, and too slow to develop the newer features I see coming up in its competitors. - Acceptance Criteria - "<fix technical debt item a>"

or if it gets too hard - focus on stakeholder as a story benefactor

As a company, i want to make sure my development resources are used efficiently so i can increase velocity on future sprints. Acceptance Criteria: "fix <any thing that bugs the team the most>"

Anything can be expressed as an ROI benefit to long term and short term objectives and easily prioritized in your backlog - that is - if your product owner is technically competent and understands the work that needs to be done which in my opinion is crucial. The whole point of stories and the story format is to communicate the "Why" and to get the team focused on "why" so that the "what" is right. If the PO does not understand the "Why" you are in trouble.