07 March 2015
Scrumdo customers frequently ask us about how they can to improve the way they plan at the start of a project. Put another way, they struggle with how to reliably scope their projects.
It’s neither new nor a surprise that most businesses require some sense of the time and expense associated with a new project or venture before making a decision to undertake it. Software projects are not different. Expecting super precise expectations before the start of a software development project is unrealistic, however, because software development (or knowledge work in general) tends to expand as improved understanding of the work is ‘discovered’ by the doers (or the customers) as work is undertaken. Does this mean we should just give up and not seek any predictability at all? Of course not. There are ways we can do better.
Because ScrumDo customers are typically employing Scrum as their development framework, they are usually familiar with sizing stories through a relative sizing method (and using ScrumDo’s built-in Planning Poker feature to facilitate team estimation). For project planning purposes, however, we need to evaluate the size of a project that will ultimately be made up of many user stories. So while story sizing can work really well in the context of a limited number of individual stories, it’s difficult to scale this up front to estimate an entire project.
Coming Soon – a New Project Estimation Tool
ScrumDo’s new project estimation tool will help reduce uncertainty in early project planning stages by providing teams with an easy mechanism for better understanding the size and scope of work to be undertaken. It does this by using a technique called Randomized Batch Sampling (an approach developed by fellow LKU Trainer Dimitar Bakardzhiev and based on work originally developed by Raymond Jessen in 1955 for application in an horticultural context. I’ve also summarized the technique in chapter 6 of my book. Let’s explore how this agricultural artifact has been adapted for application in a software development context.
Randomized Branch Sampling was developed so an individual only had to count the number of fruit that existed on a limited number of branches of a fruit tree to arrive at a highly reliable estimate of the actual number of fruit that could be harvested from the entire tree. Dimitar has shown how this same technique can be applied to forecasting user stories and tasks (or scenarios for those using ATDD or BDD practices) across a range of proposed epics.
In applying this technique to a Scrum context, it’s helpful to think of the release or product representing the trunk of a tree, epics representing major branches, user stories representing terminal shoots, and story points representing the actual fruit (See Figure ##). Rather than counting the number of fruit on a branch to establish our initial data, however, we elaborate on a given epic to calculate an initial set of user stories and story points.
Here are the actual steps we follow in practice:
- If not already done, break your project into epics.
- Randomly sample one of the epics.
- Record the number of stories in your chosen epic.
- Randomly sample one of the stories.
- Estimate the number of story points for your chosen story.
- Calculate the story points for the project using this formula:
'Estimated Number of Story Points =
(Story Points of Sampled User Story) /
(1 / # of Epics in Project) x (1 / # of User Stories in Sampled Epic)'
- Repeat steps 2 through 6 between 7 to 11 times.
- Calculate the distribution of total story points for the project
Figure. How to think of a project when applying Random Branch Sampling to estimate its scope.
Dimitar and I recently tested RBS on some real Scrumdo projects, and below are some of the amazing results from our experiments to determine the correlation of original sizes and actuals. In conducting this experiment, we randomly selected projects data from the vast number of real Scrumdo projects that met the following criteria:
1) Clear Epic-Story-Task breakdowns
2) A successful release history
3) Stable teams (systems)
4) Commercial projects
5) A minimum size of 12 epics / features.
Here’s our actual RBS calculation based on data from one project:
Calculating the mean, median, mode and distribution of values from the data provides a reliable set of information upon which to estimate the actual number of story points represented by the project.
Table: Scrumdo Project 13 RBS Estimates
At this point in the process, it can be very tempting to simply apply a team’s historical velocity against calculated story points to forecast the number of sprints needed to complete the project. But it’s important to remember that (1) story points are estimates of the effort required to complete a story, and are only loosely correlated to the size of the work at hand, and (2) this data is only representative of 1 project. We can enhance our estimation by applying correlation from other projects. This additional perspective provides a systems context sensitive understanding of specific correlation and project estimations.
Table: Showing comparison of actuals and estimations from RBS
As seen on the scatterplot below, there is a very strong correlation between project sizes estimated using the RBS technique and the actual number of story points associated with work undertaken in completed Scrumdo projects.
Fig 2. Visualizing the correlation
Though a Scrumban technique, RBS can be applied in either a pure Scrum or Scrumban context. You can find the ScrumDo data and results here.