What is backlog refinement, when and how is it performed and what advantages does it offer?
Backlog Refinement Definition
The continuous maintenance and further development of the product backlog is referred to as Backlog Refinement (or alternatively Backlog Estimation or Backlog Grooming*). It is a regular process in which the product owner and developer bring the backlog up to date.
Typical activities in backlog refinement are:
- Adding new user stories, features and epics to the backlog
- Refinement of existing epics, features and user stories and, if necessary, division into several small backlog items
- Summarising user stories
- Discussion of acceptance criteria, assumptions and limitations
- Identification of dependencies
- Correction of cost estimates based on new findings
- Change in prioritisation due to new findings
- Removing irrelevant user stories, features or epics from the backlog
- Elimination of ambiguities through joint discussion, which is about “what” and not “how”.
The Goal of Backlog Refinement
The goal of backlog refinement is to prepare the higher-priority items in the backlog in such a way that they can be used well for sprint planning. This leads to a shortening of planning meetings while at the same time taking current information into account.
According to the current Scrum Guide 2020 the Product Owner is responsible for the Product Backlog, as a consequence he is also responsible for the refinement, even if he delegates the task and/or developers actively participate in the refinement.
Backlog Refinement and Timeboxing
The purpose of timeboxing is to set and limit times for projects, processes and activities. It is important to determine the duration of a timebox in a meaningful way. With Backlog Refinement, however, the timeliness of the backlog items stands above everything else, because this has many effects on further development. It is hard to imagine that this important activity is terminated just because the end of a timebox is reached.
Backlog Refinement in Practice
Backlog Refinement and Scrum
The Backlog Refinement is not an officially defined part of Scrum like the Daily Scrum or the Sprint Planning, but it is recommended to do it as a regular activity or meeting. The Scrum Guide 2017 formulated it as follows: „Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog. This is an ongoing process in which the Product Owner and the Development Team collaborate on the details of Product Backlog items. During Product Backlog refinement, items are reviewed and revised. The Scrum Team decides how and when refinement is done. Refinement usually consumes no more than 10% of the capacity of the Development Team. However, Product Backlog items can be updated at any time by the Product Owner or at the Product Owner’s discretion”.
The Scrum Guide 2020 does not provide such a comprehensive description as well as a recommendation regarding maximum effort. In fact the refinement is mentioned only once: “Product Backlog refinement is the act of breaking down and further defining Product Backlog items into smaller more precise items. This is an ongoing activity to add details, such as a description, order, and size. Attributes often vary with the domain of work”.
Regardless of the differences between the different Scrum Guide versions, in practice the Backlog Refinement or Backlog Estimation is about the following aspects:
- Continuous process with product owner and development team
- Detailing of the product backlog, e.g. also the division of an item into several
- Estimation of backlog items and adjustment of existing estimates
- Sequence of backlog items and adjustment of existing priorities
- Delete items when they become obsolete
- Assessment and revision of the backlog items including the inclusion of new items
- Limiting effort to no more than 10% of the total capacity of the development team
- Product owner’s right to update information at any time
Best Practices in Backlog Refinement
There is a whole range of best practices:
- Even though the Scrum Guide sees the Product Owner as responsible and the developer as participating, it is recommended that the Scrum Master also participates in the Backlog Refinement. He can intervene if discussions last longer than intended – in this case, discussion with time limits per user story is a good idea.
- William Wake’s INVEST approach is very useful in defining a good user story.
- The refinement of backlog items should follow the DEEP approach of Roman Pichler.
- It is important for the team to be able to see the user stories well, e.g. as an expression on a wall. Working with user story maps has proven its worth here.
- Together, the participants discuss acceptance criteria and ensure the completeness of the stories, which are soon to be implemented.
- Also the joint estimation – preferably via story points as a measure for the complexity of a user story – is very important. If there are discrepancies in the effort estimate, this could be due to different understanding or different implementation options. This would lead to new estimates or the prior definition of technical stories or spike stories.
Advantages of Backlog Refinement
The backlog refinement offers several advantages:
- The product owner receives support in maintaining and further developing the backlog.
- The understanding of interrelationships and priorities in the team increases.
- The backlog reflects the current state of knowledge at the latest after the next refinement meeting.
- Ambiguities can be addressed and eliminated.
- Efforts in sprint planning are reduced, because there it is no longer necessary to discuss “what” instead of discussing the “how”.
- The atmosphere in the team is enhanced by the cooperation, especially as unnecessary work, e.g. the development of meanwhile irrelevant user stories, is avoided.
Challenges for Companies
Tips for the Backlog Refinement
Companies face the challenge of efficiently planning and executing backlog refinements. There are various tips for this:
The preparation is the task of the product owner. Ideally, the upcoming user stories are already periodised, because this saves a lot of time for everyone involved. The product owner must prepare the meeting well. The important upcoming backlog stories must be prioritised, clearly understandable and described in sufficient detail (ideally according to the INVEST approach and the DEEP approach).
- The development of acceptance criteria
It is advisable to define acceptance criteria together in the team, because this leads to a better, common understanding of the participants. Alternatively, the criteria could only be defined by the product owner; this could reduce the effort, but gaps in understanding could possibly remain undiscovered.
The focus in the Backlog Refinement is not on estimating the effort, it is just one component of several, such as adding new user stories, eliminating existing items or splitting individual user stories.
- The frequency
You often read about weekly backlog refinement, but this doesn’t make sense for every organisation. It would be better to focus on your own possibilities and ideas. For example, if a sprint lasts 4 weeks, it might make sense to hold a meeting every 2 weeks. Irrespective of the frequency, the backlog refinement must of course always be carried out before the next sprint planning.
- The duration of a refinement
Define a fixed duration and use your Scrum Master, because he can pay attention to the observance of the agreed rules and ensures, for example, that meetings and discussions in the meetings do not take up too much time. If the backlog refinement takes longer than originally planned, it is usually better to make a new appointment.
- Digital postprocessing
For many companies, traceability and revision security are important when working with user stories. There are various solutions from software vendors that can help you capture your results. It’s a matter of taste whether you want to digitise the entire process from the start or work with haptic user stories and story mapping.
* The “Backlog Grooming” or “Scrum Grooming”, which is frequently used in German-speaking countries, is rarely used in English-speaking countries. Until about 10 years ago grooming was understood as care with which e.g. horses or dogs were carefully combed. Since then, the meaning has shifted and grooming is linked to the idea of adults approaching children. We therefore consciously use Backlog Refinement.
Impuls to discuss:
Refinement does not mean constantly talking about things that are already “fine”.
A recommendation for the practical use of backlogs can be found in the article: Boost your backlog.
Here you will find additional information from our Smartpedia section: