How Estimates Improve with Scrum

A Team's first Sprint is the roughest and most imprecise. Often this is the first time the team members have worked together, and certainly this is the first time they have worked together on this problem. The problem described in the Product Backlog might be well known to the Team, but often it requires more understanding. The technology being employed by the Team has sometimes been used before, but often at least one new piece of technology or new release is thrown into the project. As the Team sits in this stew of imprecision and complexity, we ask the Team to commit to how much it can deliver in a 30-day Sprint. We ask the team members to tell us this within the eight-hour time-box of the Sprint planning meeting. Of course their estimate is going to be off!

We accept that the Team's estimate will be imprecise in the first Sprint. Team members delivering something approximating their commitment in the first Sprint is a testimony to human pride and determination—not the Team's estimating accuracy. I see this happen over and over. We accept the Team demonstrating more or less than that to which it committed because we know the complexities with which it is wrestling. The complexities usually stop anything from getting done. Scrum is often brought in when projects have failed, and the primary cause of failure is that the projects are floundering in the complexity. The failed projects are unable to get going. Scrum rewards action, rewarding a Team for just delivering something. Scrum asks the Team to tackle the complexity and deliver something. We limit the amount of complexity the Team is asked to tackle by time-boxing the work in a 30-day Sprint. And teams deliver. In my experience, when the imprecision and unpredictability of the effort are accepted, teams are willing to proceed and do their best. The job of the stakeholders is to accept the imprecision. The imprecision is worrisome, but it is inherent in the problem of software development.

How do we deliver releases on time that meet customer needs if the problem domain is this imprecise? Part of the answer is that estimates do get better. As the Team works together building the requirements into functionality on the selected technology, they get better. They unearth more of the unknowns. By the third or fourth Sprint, Teams are able to deliver pretty much what they commit to during the Sprint planning meeting. However, complexities still occur and disrupt this improved accuracy.

The rest of the answer is that the Product Owner and all stakeholders are responsible for figuring out what to do given how much functionality is delivered every Sprint. Given what the Team has delivered, what should the release consist of? Given how quickly or slowly the Team is turning Product Backlog into increments of functionality, when does it make sense to implement or release a set of the functionality? The Product Owner and stakeholders are driving the development cycle by trading off functionality and time. If they execute more Sprints, they can have more functionality. If they execute fewer Sprints, they will have less functionality. Or maybe they can add more Teams and determine how much this will increase the delivery of functionality. All of these decisions are adaptations that the Product Owner and stakeholders are making based on their inspection of what the Team actually does, not what it estimates it can do.

What Happens If Actuals Are Compared to Estimates

People are very complex, and often they don't do what we want them to do. I remember a situation at a large computer manufacturer that sold a very complicated high-speed printing system. Although the printing system could print reports very quickly, it kept breaking down. Customer engineers (CEs) worked many hours at customer sites keeping the printing systems working and the customers happy. But the computer manufacturer's management wasn't happy. The number of hours that the CEs were working was too costly, and the printing system division was losing money. To remedy the problem, management implemented new measurements: CEs were given bonuses based on how little time they spent repairing equipment. But to ensure that this didn't impact customer satisfaction, the CE bonuses also depended on customer satisfaction. After implementing this new bonus system, management was pleased that the cost of CEs working on problems dropped dramatically, and customer satisfaction stayed high. Several months went by before someone in management noticed that the cost of parts had skyrocketed during this time. Upon investigation, it turned out that the CEs were stocking entire subsystems at each customer site. Rather than fixing problems and repairing equipment, they would immediately replace anything that didn't work with a new subsystem.

People in software development teams are the same. When management tells them that it wants them to improve the accuracy of their estimates, what they hear is that management doesn't want any surprises. Some organizations try to improve estimates by first building databases of actual and estimated hours worked and then deriving statistics of the variances. For example, such statistics might show that a team worked 24 percent more hours than it estimated across four Sprints. Management naturally sees this as a problem. Management might then create a system of rewards if the team can reduce this imprecision. Management might tell the team that part of its performance review will depend on improving this variance to less than 20 percent. Once this target has been established, I guarantee that the team members will meet it because their salaries depend on improving this metric. Their success will cause management to view them favorably and perhaps promote them or give them more interesting work. Regardless, good things will come if the team members do what management wants. The typical way that team members then improve estimating accuracy is to drop quality or to implement the functionality with less quality. They might stop refactoring out duplicate code. They might not follow standards. They might implement a control that is less difficult but that isn't as user friendly. They might not rationalize the database. None of these actions are visible to management. All of these tricks are employed to meet the measurements and for the team members to do well in management's eyes.

Lessons Learned

The problem I've described here is called "suboptimal measurement." If you focus on improving only one part of a system, you might cause another part of the system to go haywire. The overall result is then worse than before. However, if you measure the right things, improvements can be made. In this case, increasing the accuracy of estimating by comparing actual hours worked to estimated hours worked is a suboptimal measurement. Comparing what the Team actually produces to a desired release date and release goals is a much more appropriate measurement.

For inspection and adaptation to work, we must know what we are inspecting. If we tell a Team that it can only demonstrate quality and actual working functionality, the Team will comply, and we will know real progress on delivering a release. If we tell a Team that we want it to improve the accuracy of its estimates, it will improve this metric regardless of the shortcuts it takes. Scrum asks management to focus on the overall delivery of functionality and eschew suboptimal measurements.

Peter is on track in wanting to improve estimates. To his surprise, they will improve naturally, Sprint by Sprint, as the Team becomes more competent in dealing with the technology, the business domain, and each other. What Peter needs to remember is the overall measurement—delivering the best system possible on the most appropriate date, and with excellent quality. All other measurements must be carefully implemented so that they support this overall measurement rather than undercut it. We must always factor into our measurement systems an awareness of the innate human desire to please, often regardless of the consequences.

I've mentioned many times already that Scrum is difficult. It requires frequent inspection and adaptation because these are the only known control mechanisms for complex problems. Management finally starts to understand and love Scrum when it comes to accept this and accept that this hard work is part and parcel of complex problem solving.

Was this article helpful?

0 0

Post a comment