When looking at how to measure a team’s predictability, it’s often common to start with their capacity. In the case of a Scrum team (or even some Kanban teams) story points are a good place to start. What I’ll usually look for is, what is a team’s average velocity. (Sum of story points accepted / number of sprints completed) Once I’ve got that calculated, I will take a look at the trend of percentage of planned work vs delivered work. What I mean by that is, coming out of Sprint planning or Queue replenishment, I’d expect the team to place a planned velocity on that sprint. Then at the end of the sprint, I’d calculate the percentage. For instance
Planned 20 points during sprint planning
Accepted 19 points by the end of the sprint
That then equals a 95% delivery rate.
But let’s say that a team seems not not be on track for the current release plan. There are always a million different reasons for this. But if we would look at things at a system level and not necessarily a team/person level, you might be surprised at what you find. So I started producing the following chart
One of the things I like to see teams do is categorize their work as it sits on the backlog. In the case of the above, Yellow work is “Production Support”, Green work is “Normal Development”, Orange work is “Maintenance” and the missing blue one is “Expedite”. What I really like about this is that when plotted as a column chart, you can easily see where the story points by sprint are being completed. This could easily be a throughput chart if you were doing Kanban, but I like to show the points if I have them. Where I see the biggest value is when plotting the “Planned” velocity as a line and overlaying on top of the columns. This gives you a nice visualization of how a team is doing sprint over sprint compared to their plan. As an example, Sprint 3, the team takes a bit of a hit on the number of “Normal” work they completed but as you see they still hit their planned velocity. Those points were divided up between “Production Support” and “Maintenance”. Really cool when you combine these two data points together.
Of course with any metric or visualization, there is always more to the story. I’ll always lean towards knowing my team and what’s going on with the dynamics vs blindly trusting some chart. However, if you know somethings not right and you aren’t quite sure how to diagnosis, data is your friend.