How do I measure value for a major IT infrastructure piece of work? How do I measure value? Before I even get into, how do I measure value? What is value? A lot of people struggle with this. I wrote an article about this here, what is value really?
I think one of the reasons why the world is in the way it is, is because we over-indexed on a couple of meanings for that.
Organizational Value
One of them being organizational values where maybe you’re trying to protect the reputation of the company or trying to protect the revenue that you have or protect whatever value that you have.
Organizational, from the point of view of protecting the organization’s interests, maybe protecting the reputation of the company, maybe reducing costs, things like that. That’s quite a common interpretation of value.
We’re saving some money or something like that.
Market Value
Then there’s market value, which is where a customer, end-user interacts with our product or service and they get something there, we see a change in behavior. They can do something more easily. They complete a task more easily.
They can do it in less, with less steps and so on. There’s less friction. It’s easier to do all these kind of things.
Societal Value
And then there’s societal value, an example of which for me, would be sustainability. Have we reduced our carbon footprint? Have we reduced our nuclear footprint?Have we reduced our plastic footprint?
Plastics seem to be even more damaging than the climate crisis. What are we doing around that area? Or it could be around a risk reduction, reducing risk. Maybe we’ve got a technical mess left behind this. Maybe the product looks all nice, but it’s just hanging together by a string but a lot of people don’t know that. It could be failure demand. Maybe we delivered something and we didn’t do something right for the customer. Some piece of work went live and then the call center was just rammed with complaints and some person left the building with a major bonus having delivered on time and on budget, but we just messed up the call centers and maybe 80, 90% of our calls at the call centers are complaints.
Risk reduction could also be the cost of not doing something. If we don’t do this, then something bad might happen kind of thing. And a lot of the time we don’t know when that bad thing might happen. It’s intangible really. We can’t really say when it will happen.We know that at some point it might happen.
Learning Value
A kind of a more mature lens at value as well would be, say we have some learning, we learn something about one of these above and there’s value also in learning. And I would argue that learning is the first citizen. A lot of the time we don’t actually know for sure if we will deliver this value.
And so sometimes you can run some experiments to discover do we have the capability to harvest that value? And so that’s for me, what value is. I had a case where I could just put a monetary value on something and we didn’t use it. We didn’t use it because at the time we were using some app and I think we had a million downloads or something like that, of the app.
And we could really have zoned in on users of the app and get them to convert, redeem some offer, tailored or something like that. And then we’d see the money. We could literally see the sales going up as the offers were being consumed on the app, but we decided not to use sales as the measure of value because we wanted 50 million people downloading the app, 50 million people using the app.
And what’s the point in milking the 1 million people who downloaded the app already when we could actually have done something much, much bigger. Even then we didn’t use money so in fact, I never have used money. So really value is relative. It’s not exact. We don’t know what we’re gonna get.
And if you have a backlog, for example, you’ve got different items in your backlog and you think some things are more valuable than others. It’s relative. We have an idea that some things might be more available than others, but there was a famous case study by Maersk Cargo called Black Swan farming, where they discovered that some items were hundreds or thousands of times more valuable than others, but they only found out after they went live. So they talked about cost of delay and all that. But what I picked out of that paper was you actually don’t know how valuable something is until you actually go live. So maybe you need to do some small little bets.
It’s tiny little bets on each to see where the money is. So instead of going into a casino and putting all your money on the first four tables, which is what a lot of people do in MVP, minimum viable product, which would mean the crappiest product in the time. I don’t use that term anymore.
I prefer what’s the smallest thing we can do, the smallest thing we can do, and the next most important thing. So instead of putting all your money in the first four tables in the casino, maybe we could put money on 10 tables and maybe all the actions on table number nine, actually.
So different ways of understanding value, market value, organizational value, societal value, risk reduction, or learning, and it’s relative. But when do you reap that value?
Reaping the value
You only reap that value when you release, when you give whatever you have to customers to give feedback. And often people draw a graph over time and they might have, a value in the vertical axis and maybe time on the horizontal axis.
And when you start with an agile piece of work even if you’re using something like scrum, where you’re going in, there’s some kind of sprint cycle and you’re trying to release some product at the end of that cycle and so on. We call it the increment and that increment gets bigger and so on.
Even if you’re doing that, in my opinion, when you start doing scrum unless you’re like a bunch of Navy seals or something where you can just click together and everything’s just perfect as soon as you start, most of the time, we just suck in the first sprint. Maybe for the first two months, we suck and there’s a change curve that people are probably familiar with as well.
When you start, you try something different, a lot of the time things get worse. You can reduce how much worse it gets. But typically gets worse, in my experience, it can take two months for the performance of the team to come back up and then you really get into a nice trajectory and you leave the other people behind. So when I start with the value I would argue that value would go up a little bit in the early sprint, but it wouldn’t be stellar. It wouldn’t be going upright, cuz we’re doing lots of learning about each other. We’re learning more about the vision and the product and what we’re trying to achieve. And if we’re having goal orientation, hopefully there’s some kind of North Star that we’re striving for at the end of every sprint kind of thing.
And going for some overall kind of North Star that we’re striving for. Hopefully we’re doing that. And then we got some goal orientation, but even with all of that in the first couple of sprints, maybe we don’t deliver as much value as we would hope. You only get value when you release and you might find out that the value is negative.
You find out that you’ve actually made things worse. Don’t forget that. I really admired a guy once, who was in one of my leadership classes. I don’t recommend this by the way, but he said, John I take things out of production. I said, what do you mean you take things out of production?
He says, yeah, they’re just putting rubbish in out, I said what happens if they notice? He says they rarely notice. And if they do, I just say, oh, sorry, that was a mistake. And then they put it back in. So he understood actually that a lot of the time we make things worse, you add more bugs and you add more, just makes it more messy and so on.
But most of us, hopefully we deliver more useful stuff as we go on. Although we do know from lots of studies, that two thirds of what we build, unless we do discover to deliver, which I wrote about here.
If we just deliver stuff like robotically from the backlog, two thirds of those items are rarely or never used.
So the man actually is probably not in a bad track, not sure I recommend it though. Probably the better way would be to use discovery, to figure out the stuff you should never build, do experiments. But let’s say the team is they’re delivering value because they, they’re releasing something, and then things get better and then they deliver more value and you can release during the sprint as well remember? You don’t have to wait until the end of the sprint. Actually, I hope what will happen is you deliver a bit of value during the sprint and then you actually have some feedback as well at the sprint review. And so we can course correct. We’re not just like relying on the opinions of the people, the stakeholders inside the building. We’re actually, we’re talking to customers. How about that? And we’re looking at the analytics, if it’s software, for example, or maybe there’s no analytics as well. If it’s non-software you can still tell how the product is doing and yes, you can run more experiments and just find out what’s going on.
You can do interviews, you can just talk to them, pick up the phone. And so with approaches like Scrum for example, the delivery of real value increases over time. Do we know exactly what it is? No, it’s relative for me. There are techniques but I don’t make up money. It’s just relative really.
But if cash is coming back in, I guess that’s clear, you could count that. But will it be clear actually that the extra cash that’s come in is because of your team? This is one of the things I like with the logic model, you’ve got inputs, budget, people, customer needs, things like that.
You got activities, all the things we do to deliver the work and then outputs, we actually delivered something that you can release and then you release it and you get some feedback and you hopefully get some outcome out that, but impact is much bigger than us normally. And it’s profit of the company, for example.
So unless you’re a startup or something like that, how do you know actually that the improved cash position helped that. Sometimes the accounting people are very good and they can tell, particularly if you’re looking at organizational value or maybe the cost of not doing something you can see on the bottom line already the difference that we made particularly if you reduce costs but the question I was asked was how do I measure value for a major IT infrastructure piece of work? And the person who asked the question won’t like the answer unless you’re releasing something in chunks along the way, you don’t have any value, zero nada, nothing. And if that time period, is quite long, say 18 months, you’re in a pretty precarious situation because what’s probably going to happen now is people aren’t really going to measure value anymore. They’ll be looking for some proxies. And typically what I see is they’re looking at story points.
Some other kind of proxy measure and sometimes there is story point bingo. So it’s not even an output measure to done it’s like, activity, cuz they’re just playing games with claiming velocity at the end of a sprint. And so then people conflate effort considering complexity and risk with value, which is that we’ve made a difference to the organization, to the world to ourselves, or we’ve learned something more about all of these, if you don’t release, if you don’t have people and even if you do release, if you don’t have people giving feedback about whether we’ve made a difference or not, you don’t have any value. And you’re probably following more a Waterfall profile, which would be where there’s basically no value whatsoever. No value whatsoever. And then we get towards the end and then whew. And I’m actually being optimistic here because I’m assuming that with the Waterfall, that you deliver the same value as the agile guys. But if you think about it with a traditional approach, the best thing you can hope for is what the people asked for.
That’s the best you can hope for and it’s usually not that because the different levels of translation that go on in between and so on. The best you can hope for, with some kind of an agile approach, you could do this with Kanban as well. The best you can hope with an agile approach is something much better than you originally thought of.
And I love what Rich Hundhousen says . He’s one of my peers in scrum.org, he says the sprint review is where the customer gets to see what she asks for, but doesn’t want. You get it? So you go on for a long time. You can use whatever frameworks you want. There’s lots of nice frameworks out there.
Evidence based management for example is really nice. I think Chris Matts has some kind of value hierarchy as well. There’s lovely approaches out there, but the thing is that if you’re not releasing anything, if you are not putting it into hands of someone else where we can actually see if it’s making a difference.
You’re not getting anything.
I did do something in a bank, it was a major application and there was IT infrastructure under the application as well.
And a huge economy was based on it. And the risk was that if we went live that we could damage the economy, no pressure. People were talking about, oh, we need a fixed window. We need 10 hours or whatever it was and blah, blah, blah. I said, this is just too much risk.
No-one’s gonna pull that lever. I certainly wouldn’t pull that lever. So I recommended Martin Fowler’s strangler pattern and it’s a nice little pattern actually, I had a strangler fig in my back garden in Chiswick when I was living there and it was really sad. It was like this little twig.
Image courtesy of Rany ElHousieny
And what it does is it lands in the little crack in a branch, little kind of takes root into the branch and it kind of sucks the nutrients outta the branch. It starts growing, even starts growing its own fruit and everything. I was confused thinking, oh, what’s going on? Oh, I didn’t think that was a big tree.
It was like, it was so good at camouflaging itself on the tree. It looked like it was part of the tree. And then what it does is it drops the roots down to the ground. And then those roots, once the roots get into the ground, the fate of the tree is dead, it’s basically dead. Cause what happens then is it just takes all the nutrients out of the ground that’s even coming into the roots of the tree, grabs all those wraps the tree, and then eventually the tree dies.
And then there’s like a hollow thing inside. So you can actually see these fig plants, you can get them in some garden centers and you should never plant them outside, cause they’ll destroy all your garden. But if you put them in a little box, I guess they’re okay. Anyway the metaphor here is I said gals, guys, what we need to do here is we need to strangle the old system. They said what are you talking about? What we need to do is we need to put just a few transactions through the new system.
But we can’t do that. We gotta be. I said, no, what you need to do is you need to design this kind of reconciliation system. So if a transaction goes through the old system, you push it through the new system.
And if it goes through the new system, you push it through the old system. And then at the bottom you have some reconciliation as well and so the idea is that it gives you a kind of an incremental goal life strategy cuz what you can do is you can say you can just open the top a little bit and just put a little bit of your market into this new system and you’ll see some problems and you’ll fix those problems.
And then you say, oh how confident are we feeling now about adding more people to this and so you open the top a bit more and so more stuff is going into the new system and we find some discrepancies. We reconcile the old system. It’s a lot of extra work, don’t get me wrong. But it’s very good at reducing the risk of the goal life.
Anyway, in that situation, within four months, I actually left after that recommendation. But they followed my advice and within four months it was live across the whole country. They actually decommissioned the old system and it was unlike previous attempts at replacing that system before where they were left, you know, when you tried to put in a new system and you try to replace the old system and then it doesn’t really quite work. So you end up with two systems. And so they were worried, that’s why they had a big bang approach. They wanted to get rid of the old systems. But with this approach, they did get rid of the old systems it’s called the strangler pattern.
So maybe you should be thinking less about how do you measure value and maybe you need to be thinking more about how do you reduce the risk with the deployment and how can you actually deliver any value in the shorter terms, you can follow a trajectory, like the agile one, rather than the kind of Waterfall pattern here, which is where you get nothing for a long time.
And then it eventually goes up. So keep feeding through your questions to me. Thank you so much for that question.
So just a reminder please donate to agile with Ukraine at agilewithUkraine.com. It’s very good cause to raise funding for life saving equipment, humanitarian and medical help, and I wish you a good day.
Thank you.
https://linktr.ee/johncolemanxagility - social and podcast links
https://linkpop.com/orderlydisruption - order training from right here