To make things worse, some teams do not take the retrospective seriously. Maybe after the potential stress of the sprint review, the largely internal retrospective is a relief. A chance to chill out, maybe share some food, and pat each other on the back. However, innovation and learning take conscious effort, forward planning and accountability.

As I work with organizations, I often sit in on retrospectives. Of all the regular workshops/ceremonies, these sessions are typically the least prepared for and worst executed. I often see lazy retrospectives where a basic lessons-learned format is used, but timings are not managed and the recommendations for the next sprint get skimped as they run out of time.

The pie chart below shows a typical planned allocation of time—and the reality of how time is actually spent:

In these lazy retrospectives, people are slow to start, spend longer on recording what went well than what could be improved, and then try to cram the recommendations for experiments (the most important part) into the last few minutes. As a result, experimentation suffers. Few experiments are scheduled for the next sprint, and those that are run are not evaluated properly.

This is not how agile retrospectives are supposed to operate. An excellent guide to running effective retrospectives is Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen. In it they describe a five-step process:

1. Set the stage – Help people focus on the task at hand; check that people are ready and willing to contribute. Outline and gain consensus on the process we will use. Techniques we can use include: check-in, working agreements, focus-on/focus-off (see the book for full descriptions of these techniques).
2. Gather data – Create a shared view of what happened during the sprint/iteration. When completed, we should have a common understanding of the observations and facts. Team activities we can use include: timeline, mad-sad-glad, team radar.
3. Generate insights – This focuses on understanding the implications of our findings and discussions. We need to see the impacts of the problems we are faced with before trying to solve them. Techniques we can use include: five whys, fishbone analysis, dot voting.
4. Decide what to do – Now we move from thinking about the iteration that just ended to what we should try next to improve things. We identify the highest-priority action items, create plans for experiments and set measurable goals to achieve the desired results. Techniques we can use include: SMART goals, circle of questions, short subjects.
5. Close the retrospective – Here we reflect on the retrospective process and express our appreciations. We may summarize what we have decided to keep or change and what we are thankful for. Team-based activities we can use include: plus/delta, return of time invested (ROTI), appreciations.

This is a more useful format. However, despite people having access to good retrospective advice, poor implementations are still common. Teams continue to attend late, start slowly and run out of time or rush the agreement on what experiments to run.

The recurring theme is poor experimentation design and restricted learning. Gunther Verheyen summarized the problem nicely in his recent post entitled “Inspection Without Adaptation Is Pointless.” Gathering data and deciding what to do is pointless if it is not acted upon. We are doing most of the preparation work but not getting any of the benefits.

Experiment Design to the Rescue
Fortunately, there are some good models we can use. We need to manage time and effort more effectively and use retrospectives to plan and evaluate more experiments. We should spend only 50% of the available time on gathering information and the remainder reviewing the results of past experiments, making wins part of our process, designing new experiments and learning from inevitable failures.

We can help the time management problem by assigning work to be done in advance. People should be thinking about issues and potential solutions independently. There are benefits of group discussion and consensus gathering on agreed experiment design, but observation and idea generation is best done individually.

The New Yorker magazine [3] describes numerous studies that show how brainstorming groups think of fewer, lower-quality ideas than the same number of people who work alone and later pool their ideas. There have been numerous reports on the downsides of brainstorming ideas as a group. Groupthink and the halo effect stifle idea generation. So, ask for people to come with ideas, then use the group setting to vet and vote for them.

Visualizing the ideas and experiments is an effective way to bring collective attention to them. Trent Hone and Andrew Jarding developed the “Ideas and Experiments board” pictured below. It shows the progression of ideas through experiments and their success or abandonment:

Ideas and Experiments Board (Image Credit: Trent Hone and Andrew Jarding, MindSettlers)

As discussed in the last article, by design, 50% of our experiments should fail since we are trying to maximize our learning, not validate things we already know. So I would expect to see an equal number of abandoned experiments as successful ones.

However, this format (or a slightly modified version that represents an experiments Kanban board) is a useful tool to bring the focus for retrospectives to the experiments being run and considered. With some pre-work on idea generation and an increased focus on experiments, we can structure more effective retrospectives.

This retrospective format saves some time by assigning idea generation as pre-work; this also helps avoid the groupthink pitfalls. It furthermore places emphasis on the experiments—the inputs for learning and innovation.

I have experienced pushback from teams about the goal of 50% experiment failure. People understand it optimizes learning—but say it sets people up for too much failure. I understand the sentiment but counter with two perspectives.

First, these are experiments; they should be dispassionate explorations, not evaluations of the people undertaking the work. We need to be professional and try to overcome habits of internalizing results. I know this is easier said than done, so also offer a second reason: We need to kill bad ideas early to save time and money for better ones.

In the article “The Hard Truth About Innovative Cultures,” Gary Pisano describes how killing bad ideas is critical. He profiles Flagship Pioneering, a Massachusetts-based R&D company. It uses a disciplined exploration approach to run small experiments minimizing expenditure. Instead of running experiments to validate ideas, it designs “killer experiments” to maximize the probability of exposing an idea’s flaws. The goal is to learn what went wrong early and move in a more promising direction.

Other useful ideas from the paper include:

• Tolerance for failure, but no tolerance for incompetence – Hire the best people you can. Explain the goals clearly and let go of those that do not perform.
• Psychologically safe, but brutally candid – Encourage frank but respectful two-way dialog. It may feel uncomfortable, but it can prevent issues or concerns from going unreported.
• Collaboration—but with individual accountability – Encourage discussions, but avoid groupthink and hold people accountable for decisions and outcomes.

These are all great concepts and align with the frustrations I experience when I see teams not taking retrospectives seriously—or following through on conducting experiments. I realized I needed a better model for discussing the problem. Fortunately, I found the field of collaborative problem solving (CPS).

CPS is the study of how we work together in groups to solve new problems, innovate and build products. The innovation process and retrospective workshop fall squarely within its scope. CPS skills are quite separate from individual task-focused skills, meaning people can be great at working individually but poor at working together.

A good introduction to CPS frameworks can be found in the article “Advancing the Science of Collaborative Problem Solving.” One model they feature is the “PISA 2015 Collaborative Problem-Solving Assessment.” Unfortunately, like many academic models, the degree of difficulty goes downward, which may make sense as you read down through more advanced stages. However, I think graphically, so I have redrawn the model to show degrees of completeness and difficulty radiating up and out from a 0,0 origin, as shown below:

Along the X-axis, we see three categories of collaborative problem-solving competencies. These are:

1. Establishing a shared understanding
2. Taking action to solve the problem
3. Establishing and maintaining team organization

Up the Y-axis, we have four categories of problem-solving:

1. Understanding the problem
2. Representing the problem
3. Planning and executing
4. Monitoring and reflecting

Within the body of the model, each square is labeled with the column number and row letter, and describes the tasks that occur in that space.

The model provides a diagnostic tool for identifying broken and lazy retrospectives. The poor engagement and weak follow-through I see in many Scrum teams is characterized by an incomplete execution of column 1 and only half-completion of columns 2 and 3 (as shown by the red outline below):

Teams are not spending time in “(D1) – Monitor and repairing the shared understanding,” nor are they getting to the “(C2) Enacting plans,” (D2), (D3) and (C3) areas to follow through on plans and hold each other accountable for actions and results.

What we want is a complete execution of all the collaborative problem-solving competencies; only then is the framework complete (along with the feedback mechanisms to keep things in check and moving in the right direction):

Summary
Innovation involves combining the right mindset and philosophy with tools and practical steps to ensure its execution. Motivation and attitude are paramount; people have got to want to do this work, enjoy it and create a pull demand for the tools and process that enable it. Trying to foster innovation with demotivated teams is like trying to push a rope.

When motivated and happy people create a strong pull demand for innovation, we need to be ready with the right tools to support the process and keep the momentum going. This includes designing experiments to maximize learning and killing bad ideas quickly—all while demanding competence, accountability and candor.

It is not easy to master the combination of soft skills and techniques required for successful improvement and innovation. However, organizations that succeed can respond to market changes faster and are poised to exploit new technologies and opportunities. Ideas and inventions are spreading quicker than ever. Learning how to build collaborative, innovative teams has become a critical skill.

References

1. Book: Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen
2. Article and video: “Inspection Without Adaptation Is Pointless” by Gunther Verheyen
3. Article: “Groupthink: the Brainstorming Myth” by Jonah Lehrer
4. Article: “The Hard Truth About Innovative Cultures” by Gary Pisano
5. Article: “Advancing the Science of Collaborative Problem Solving” by Arthur Graesser, et al.

[Note: I first wrote this article for projectmanagement.com here]