So how’s your agile transformation going?

If your organization is in the midst of an agile transformation, ideally this change was justified through a business case which articulated expected benefits and the means by which those benefits would be measured. But we rarely live in an ideal world.

So how could you assess whether the initiative is delivering value or not?

You could look at a metric like average time to deliver scope but this has limitations. Averages by themselves mean nothing. If there is an overall reduction in the distribution of release times and ideally a shrinkage in the variation for these release times, that might be cause for optimism if a sufficiently representative sample was taken before and after.

Just because we are delivering scope sooner doesn’t mean we are reaping the full rewards of an agile transformation. A team might miss the mark by prioritizing schedule over quality and we would end up producing a product which the customer doesn’t want.

And, this says nothing about how we delivered that scope. Over short timeframes using Theory X-type behavior it is possible to whip a team into delivering quickly but we would usually see a corresponding reduction in quality and in team satisfaction.

Perhaps we could look at velocity across teams. While we know that velocity should never be used to assess performance between teams or at an individual level, surely an ongoing, incremental increase in velocity across the majority of teams would be a positive indicator? Unfortunately, without introducing other measures to add perspective, it would be relatively easy for a team to claim such improvements at the expense of quality, or delivery of real value to their customers.

In place of these vanity metrics, consider these:

1. The distribution of lead time to deliver utilized features – by filtering out unutilized features, our time to market distribution will focus on true value add to our customers
2. Features utilized/features completed – this ratio will assess how effective teams are at meeting the true needs of your stakeholders
3. The distribution of defect severities and counts – this will assess whether quality is being sacrificed at the altar of speed
4. The total number of high impact organizational blockers – assuming teams are surfacing and escalating organizational impediments to full agility, a reduction in the number of these blockers should translate into improved delivery outcomes
5. Team satisfaction – this will keep a pulse on team morale to ensure that it is not suffering through the transition
6. Customer or key stakeholder satisfaction – this will be another balancing measure like #5 to ensure that the end is not justifying the wrong means

Developing a balanced, holistic approach to measuring outcomes should help to sustain leadership support and to focus continuous improvement efforts on the right things but just remember:

“…not everything that can be counted counts, and not everything that counts can be counted.” – William Bruce Cameron