With the Scrum Team Survey, teams can diagnose themselves with a scientifically validated approach, inspect the results together, and improve with evidence-based feedback. Every team member can complete the survey anonymously. The aggregated results are shown in one team report. This post isn’t intended as a promotion, our survey is one example of a diagnosis tool. Totally fine if you have your own, the challenge I describe in this article remains valid.
Since Christiaan Verwijs and I started the development of the Scrum Team Survey, almost 12.000 teams and 30.000 people diagnosed the state of their team. Sometimes the results completely matched the expectations of the participants, but the opposite also happened. In that case, team members or people supporting the team expected a totally different outcome. In their opinion, the results are too positive or negative and they can’t relate to it at all.
Imagine that you’re part of an Agile or Scrum Team and you frequently experience how your team struggles to navigate conflict effectively, or the Sprint Retrospective only results in meaningless improvements, or Sprints never have a predefined Sprint Goal. These are three (random) problems you experience within your team, and your hope was to make all these problems visible with the Scrum Team Survey. Yet somehow the results indicate the team is doing totally fine in all these areas! How is that possible?!
We’ve discussed this situation in different shapes & forms with participants of the survey. And we’ve noticed how it can lead to disappointment, frustration, and disbelief. Some people even start to doubt their own opinion and become skeptical or even cynical.
So what can you do when this happens? In this blog post, we’ll offer our thoughts and ideas on the question:
“What if my (Agile/Scrum) team continuously overrates its skills, capabilities, and performances?”
What if your team continuously overrates its skills, capabilities, and performances, while you have a totally different experience?
Tip 1: Explore the level of psychological safety
What can you tell about the degree of trust and psychological safety in your team? If people don’t feel safe, they might feel the need to hide problems and paint a picture that doesn’t match reality. Amy Edmondson defines psychological safety as “a shared belief about the consequences of interpersonal risk-taking”. You can encourage psychological safety by inviting people to connect on a personal level, rather than just as professionals. Psychological safety is not about always agreeing with each other, or about giving compliments all the time. It is much more about encouraging people to speak up when they have worries, uncertainties, or doubts.
Try the DIY workshop “Build Psychological Safety With Appreciative Interviews” to explore and improve the level of psychological safety in your team.
“If people don’t feel safe, they might feel the need to hide problems and paint a picture that doesn’t match reality.”
Tip 2: Inspect the results together
Before you share your feelings with the team, first inspect the results together without any upfront judgment. The first step is to give everyone time to look through the results and to do this in silence and individually to avoid social pressure and biases. Maybe, when inspecting the results, the team itself already raises questions about the results. Maybe they’re as surprised as you are. Once everyone studied the results individually, work together to identify patterns, and draw conclusions. This could be a good moment to share your thoughts, questions, and observations.
“Once everyone studied the results individually, work together to identify patterns, and draw conclusions.”
In this DIY workshop, we use the Liberating Structure “What, So What, Now What” to inspect the results of the survey. This structure helps by asking us to step back and consider what is going on. It structures our thinking by breaking our experience down into three steps: “What do we notice?”, “So, what does this mean?” and “Now, where do go from here?”.
“What, So What, Now What” takes inspiration from the Ladder of Inference by Chris Argyris.
Tip 3: Use (Kanban) metrics to bring in more data
Kanban is a strategy for optimizing the flow of value through a process that uses a visual, pull-based system. Kanban offers a great set of metrics to bring more data into the conversation. These metrics provide a more objective perspective on the process of teams and can reduce biases.
- Work in Progress (WIP): the number of Product Backlog items started but not finished. Teams get more done when they work on less at the same time.
- Cycle Time: the time that transpires between when work begins on an item and when the item ships.
- Work Item Age: the amount of time between when a work item started and the current time.
- Throughput: the number of work items finished per unit of time.
- Lead time: the time that transpires between when a stakeholder request enters the Product Backlog and when it’s fulfilled to that stakeholder through a release.
Use these metrics to gather more data on how responsive your team really is. How much focus on value do we actually have? And what does the data tell us about our release frequency?
An example of a Scrum board with work-in-progress limits and flow-based metrics. From our book, the Zombie Scrum Survival Guide.
Tip 4: Bring in different perspectives
A good strategy to reduce biases is to bring in a different perspective on your team's performance. In the end, Agile & Scrum Teams exist to deliver value to stakeholders. So why don’t you ask their opinion on how the team is doing? How satisfied are they with the value the team delivers? How happy are they with the release frequency? With the Scrum Team Survey, it’s possible to invite your stakeholders to a short, anonymous survey that we designed specifically for them. Your team report is automatically updated every time a stakeholder participates.
Do make sure to invite your *real* stakeholders. People with an actual stake in your product. A stakeholder is someone who has at least some “skin in the game”. When your product succeeds, they benefit from it. When it fails, they share the pain with you. It’s easy to invite people with an opinion about the work you do as a team, but the most valuable feedback is provided by people that actually use your product or benefit from it. Focus on those people first.
“It’s easy to invite people with an opinion about the work you do as a team, but the most valuable feedback is provided by people that actually use your product or benefit from it.”
Interested in learning more? Try the experiment “Discover Your Stakeholders With A Stakeholder Map”.
This is a snapshot from a demo team report of the Scrum Team Survey. The green markers are the average scores of the team, the yellow markers are the average scores of the four stakeholders. In this example, the stakeholders are even more satisfied than the team itself!
Tip 5: Rethink how you run the (Scrum) events
Does the Sprint Review make the state of the increment (painfully) transparent? Do you gather feedback from stakeholders? Are the real, meaningful problems discussed during the Sprint Retrospective? Is everyone’s voice included in the various Scrum events? Is there sufficient focus on the outcome and the value you want to deliver, or are most conversations focused on output? The way your team runs the Scrum events can already clarify how they rate their own performance. So take a critical look at your Scrum events, and ask yourself:
“How can we increase transparency with our Scrum events?
“How can we improve inspection?”
“How can we change adaptation?”
Interested in learning more? Check the DIY workshops “Invite Deep Learning Into Your Sprint Retrospectives” or “Involve Stakeholders In The Sprint Review”.
Tip 6: Assume the team has the best intentions
Maybe your team truly believes it has great skills, and capabilities, and is performing outstandingly! Maybe the team lacks the knowledge and experience to understand it could actually be way better. It’s important to assume the team had the best intentions while completing the survey. Not doing so can damage trust, increase tension, and damage team dynamics. The next time you ask them to complete an assessment, they will only be more skeptical or don’t want to participate anymore.
A good option to validate your assumption is to playfully assess the knowledge of your team. You can create your own Agile or Scrum exercise, or draw inspiration from an exercise like Scrum Mythbusters. We developed this exercise to create transparency around the purpose, principles, and values of the Scrum framework. When your team struggles with most of the statements, you can inspect by refreshing the purpose of the Scrum framework together.
Exercises like Scrum Mythbusters — or exercises with a similar purpose — can offer you a good opportunity to discuss your team's performance rating. If the knowledge proves to be shaky, what does this tell the team about how they assessed their performance recently? What topics are worth revisiting?
“Maybe your team truly believes it has great skills, and capabilities, and is performing outstandingly! Maybe the team lacks the knowledge and experience to understand it could actually be way better”
Tip 7: Use creative exercises to make the state of the team visible
In the past years, we created a wide variety of (digital) card exercises. Each exercise makes the state of your team and organization visible in a specific area, triggers inspection, and provokes adaptation. For example, with the Definition of Done exercise, you create transparency around the ability of your team to deliver completely “Done” software every Sprint. We designed the Management in Scrum exercise to create transparency around how the roles of the Scrum framework are fulfilled in your organization. And more importantly, what happens when the roles are fulfilled differently than intended by the Scrum framework? And with Panarchy, you help Scrum Teams, stakeholders, and supporters understand how their system works and find leverage points for improvement.
Exercises like this, are a great opportunity to start a conversation about the actual state of your team. Run the exercise, make the current situation visible, and let the team interpret the results and draw conclusions for themselves. Your role is to show the mirror, it’s up to the team to face the reality.
“Your role is to show the mirror, it’s up to the team to face the reality.”
Make the current situation visible, and let the team interpret the results and draw conclusions for themselves. Your role is to show the mirror, it’s up to the team to face the reality.
Tip 8: Let it be…
This final tip is something Francesco Bianchi suggested. “A controversial perspective that I’m exploring these days is “does it matter if they overrate their skills/capabilities/performances?”. Very often we measure to satisfy reporting requirements. That, to me, most of the time is a purely wasteful activity.
Less frequently (but maybe it’s better in the case of your survey) we measure to reflect on how to improve. One of the things that one can do to improve is to become more aware. That usually happens naturally if people keep reflecting on how they are doing. In addition, if they constantly overrate themselves, they will very very soon cap the scale of the measuring system and that will cause them to enter a plateau. At this point, they will have to necessarily reflect on why they not improving anymore. In other words, let it be and if it matters it will sort out itself.”
Closing
In this blog post, we suggested ideas on how to handle situations where your team continuously overrates its skills, capabilities, and performances. I’m super curious to learn from your ideas as well! How would you handle this situation? What other ideas or thoughts do you have?
Feel free to share them! Let’s learn and grow, together!