I facilitated a version of the Product Owner Value Game at IBADD2016 . Dojo Breddels and Paul Kuijten created the Product Owner Value game to use during their coaching efforts to help people become more value driven. They presented the game at Agile2015 where they shared the rules of the game and provided some thoughts on how to facilitate discussions result from playing the game.
IBADD2016 is the second time I facilitated the game, and I primarily use it to try and get across two main learning objectives:
- How to become more value driven
- How to Maximize Outcome with Minimum Output
Since I had an hour and 45 minute workshop time slot, I decided to extend the retrospective that occurs after we finished playing the game. To aid the retrospective, I posted three questions and asked the participants to put their thoughts on sticky notes for these questions. The three questions were:
- What did you learn?
- What still puzzles you?
- How can you apply this at your job on Monday?
I committed to the attendees that I would tackle the items listed in What still puzzles you? in either blog posts, or with pointers to other materials. This post addresses several of the questions that all centered around the same topic – business value and measuring it.
If you would like to see a copy of the materials I used for the game, let me know and I will send you copies.
In order to make the game effectively simulate what happens when a team is working through a backlog, Dajo and Paul created a deck of cards that include Feature cards
and User Story Cards.
The mechanism used to track work (i.e. output) is very familiar – story points. The key idea for this post is what the game uses to track value – business value points. In fact, the goal of the game is to be the team that delivers the most business value points.
Dajo and Paul did not create the idea of business value points for purposes of the game. The idea has been around the agile community for almost 10 years and is based on the following assumptions:
- We want to know about business value to help us make decisions.
- It’s better to measure and track the business value we deliver rather than the work we do to produce it because that helps us to determine success based on delivering outcome rather than just output
- Business value is difficult to measure and track because it is subjective in many situations. This is especially the case for user stories that have been split so that they can be done within a sprint.
- We don’t want to spend too much time trying to estimate business value because it takes away from actually delivering it.
- We can use the analogy of story points as a way track and measure business value that doesn’t take too much time to estimate.
Here are several different posts describing the idea and advocating it’s use:
- Why Value Points? Using Agile to Optimize Value of Projects Elena Yatzeck
- Measuring Business Value in Agile Software Development by Kelly Waters
- Release Planning – Business Value Points by Joe Little
- How do you measure value presentation from ThoughtWorks:
- Determining Business Value by Jim Highsmith
Business value points sound great, don’t they? Several of the puzzles that came up as a result of playing the game surrounded that idea:
A) Calculating business value at the story level
B) What if the PO doesn’t want to estimate value?
C) How do you determine point value?
D) How to effectively determine business value points
E) How to accurately establish business value
I could answer questions A,C,D,E with the following guidance:
- Get your stakeholders together
- Give them each a hand of cards with each card representing values along the modified Fibonacci sequence (1,2,3,5,8,13,20,40,80,100 for example). You can even reuse your team’s planning poker cards for this!
- Have the stakeholders play planning poker with each story, but instead of determining the relative size required to deliver the story, have them determine the relative value of each story.
Except:
- that guidance doesn’t really indicate what value a point has (it’s a unit-less unit of measure like story points)
- I’m not sure how effective that approach really is
- I’m not sure how accurate it is either (though I’m not sure how much I care about that.)
Plus, the person who asked B in some respects was on to something.
Look back at the first assumption that resulted in the creation of business value points: “We want to know about business value to help us make decisions.” There are a couple of considerations with this:
1) Business value means you reach the outcome you want to get to.
What do I mean about outcome? It’s the need you’re trying to satisfy or the change you’re trying to make. In this sense, you’re looking to your understanding of business value to determine what you work on and what you don’t. Doing the things that bring business value means do those things that help you reach the outcome you seek.
2) How do you know when you’ve succeeded?
Ideally there is some measurement of the outcome you produce. Years of working on projects has conditioned us to want to track things in a continuous fashion (X dollars, Y weeks, Z story points). Outcomes may be a measure, such as an increase in the number of subscribers a website has, or the amount of revenue that a product produces. They may also be binary. You have either solved the problem, or you haven’t. Even when the outcome can be measured in terms of subscribers or revenue, the impact of software development work is generally not direct or continuous. You may have one particular change that will present a large stepwise movement toward the outcome you are looking to accomplish. In other cases, you have to make several changes that will work together to generate some movement toward the outcome. The changes your team makes do cause movement toward an outcome, but it’s not an additive thing that means you can say “this story is going to get us 5 points toward the outcome” and “this other story is going to get us 8 points toward the outcome.” You have to make some changes, put them into place, and then measure the result.
This is where the phrase falsely attributed to Yogi Berra comes into play: “In theory, there is no difference between practice and theory. In practice, there is.”
I’m betting that most of the people who advocate using Business Value Points have good intentions. They want to get away from making decisions based on the amount of work (cost) required to deliver a particular change. They know that tracking progress based on story points is not necessarily the way to go, that it focuses too much on output and it can lead to a team doing a lot of work to deliver something that really doesn’t drive much movement toward the desired outcome.
But here’s the thing: measuring business value points is still measuring output. You are measuring what you have done just based on a different perspective. You don’t really know for sure if you are doing the right things. Sure, items that have more business value points should correspond to the right things, but consider how those scores are derived. They are relative scores saying this backlog items is more valuable than this other one. The tricky bit is whether that value is determined based on getting closer to the desired outcome or whether they were determined in isolation.
Business value points could be helpful for making decisions by looking at multiple items and saying that the ones with higher business value points are most likely the better ones to do, but I don’t think I would use an aggregate number of business value points to tell me when I’m done. Sure, I can see how much I’ve delivered, but I don’t know the actual impact on the outcome I’m trying to reach until I measure my metric of choice.
If you use business value points to make decisions, that means they are still worth something, right? Perhaps. If all you are using them for is comparing items to each other, ask yourself “do I need to go to the extra effort of putting points to them, or can I just look at the items you are comparing and do the equivalent of the eye doctor test (better first or second)?” Doing direct comparison of value between items may not require numbers at all.
Perhaps you find numbers useful, but you don’t need to do the relative sizing. Instead use different criteria for the numbers you assign. Brandon Carlson shared an approach that he used which seemed to work well in situations where he had multiple stakeholders who were all involved in making decisions: Story prioritization at the “Blink” of an eye.
I don’t advocate sizing items in a backlog and then measuring success as the point where you’ve delivered the total number of business value points that were in the backlog. That leads to waste:
1) The time spent relative valuing all of the backlog items.
2) Defining success as delivering all of these business value points means that you will most likely end up delivering things that really aren’t needed.
Think about it this way. You initially create your backlog at the beginning of an effort, when you know the least about the project. Your knowledge about the problem and solution will inevitably grow as you move along. (If your knowledge is decreasing as you proceed, you’ve got bigger problems.) When you make your initial guess at what will drive the outcome you are seeking, you are doing so with incomplete information. You are going to learn along the way, and that refined understanding will cause you to identify some additional items, and also realize that you don’t need everything that you had originally identified. Sometimes because you didn’t understand the solution, sometimes because you misunderstood the problem, sometimes because your priorities changed.
Once you start changing things, and you’ve identified success as completing a certain number of points (either story or business value) you end up in a quandary. If you missed a lot of things, your “measurable success” number is too low and you have to figure out what you want to do. Keep in mind it’s still important to have the appropriate time/cost constraints (if they are present) because this provides additional information to help you make your decision. But the discussions you have should be:
we have 6 weeks available to us. What are the right things to work on during those six weeks to get us where we need to be?
not
How can we get 200 Business Value Points done within those 6 weeks?
If you go the later route, you could successfully deliver your 200 points of Business Value, but the combination of the changes you make is not the right combination that makes a viable solution, or they provide changes in an area that was not really the focus of the project because you chose to focus on “low hanging fruit.”
The other problem (and frankly a good problem to have) is that you identified more items than are necessary. If you define success based on delivering the complete list of scope items, or delivering 200 business value points, you could find yourself going down the road of delivering functionality in order to meet that goal, instead of making your decision based solely on whether that functionality is needed to achieve the desired outcome. Going that route results in delivering more functionality than is needed, leading to wasted time (which could have been spent refactoring existing functionality to make it easier to maintain and more robust) and added to the maintenance load down the road.
So how do I measure business value? By finding a business focused metric that describes the outcome I’m looking for and then testing how the output I produce impacts that metric. I’ll write about that in a future post. But in the meantime, I’d be interested in hearing about your experience with business value points, and also what you have used to measure business value.
The above ideas are heavily influenced by a couple of posts from Pascal Van Cauwenberghe: Estimating Business Value and How do you estimate the Business Value of User Stories? You don’t. and the insights of Chris Matts.
(Photo Credit: https://unsplash.com/@ugmonk)