Scaling Agile — Estimation as a Tool for Continuous Improvement

Most software developers would agree that planning ahead is difficult. Especially in fast-moving environments, where high levels of uncertainty stand in the way of accurate estimation, it may be tempting to give it up as a bad job.

When done right, however, estimation can offer a source of continuous feedback, which, in turn, allows individual engineers, development teams and entire product development organizations to work smarter. Much like a CI/CD system delivers feedback that enables developers to ship good quality code at speed, a well-oiled planning loop provides developers with feedback that allows them to optimize the critical path of their projects from start to finish. What’s more, accurate estimation will give developers more power to weigh in on strategic decisions.

The benefits of a planning loop

Estimates are used by Product Development leadership for a variety of purposes like making cost/benefit assessments, budgeting and staffing decisions and tracking the progress of software development projects. But estimation is a useful tool for development teams, too.

By engaging in a continuous cycle of breakdown → estimate → ship → learn → repeat, development teams get data for improving, not just the accuracy of their estimates, but their overall performance and ways of working. Identifying patterns for blockers and bottlenecks helps solve them for the future. Spotting interdependencies helps take into account how certain actions in one project may influence other projects in the big picture. Noticing repetitive or wasteful practices enables teams to streamline their operations.

Just like the feedback a CI/CD system produces is only as good as the tests in the pipeline, the feedback from a continuous planning loop is only useful if the original breakdown and estimates were appropriately detailed. As a rule of thumb, we maintain a backlog of breakdowns and estimates that add up to about two to three months’ work. In our rapidly changing environment, beyond that point the level of uncertainty increases and makes any plans too blurry to be of any real use. Development teams’ confidence in the accuracy of their estimates increases when they routinely reflect on completed projects and learn from missed estimates.

Accurate estimations provide development teams the credibility they need to have a stronger voice in strategic decision-making. When they can more confidently predict how much time and effort goes into turning product managers’ visions into reality, it’s easier to take a stand on how projects should be prioritized and where resources are put into best use.

The ability to estimate timelines precisely makes collaboration with other teams easier, too. Product Management teams can make better directional decisions when they know both sides of the equation, what customers need and what developers can deliver and when. Moreover, being able to predict lead times helps both Product Development and Product Management teams communicate with Marketing, Sales, and Customer Success teams on when features can be expected to go live, giving them more time to devise go-to-market plans.

Strengthening our planning muscle at Smartly.io

While we have always done planning, we haven’t been highly systematic about it before now. As an agile organization, we constantly develop our ways of working, and we are now in a situation where more structured planning and estimation would serve the needs of the organization. In our case, the most important reason is the promise of improved collaboration and coordination with the customer-facing teams like Sales and Marketing that comes from increasing the length of the planning horizon and being able to more confidently estimate when new features will be implemented.

Take one of our development teams, Midas, as an example. Team Midas uses historical data on how long on average it has taken to implement a story in the past three months to project how much time it will take to complete future stories and epics. At the planning stage, multiply the default length of a story, 14 days, by the number of stories to estimate the time it takes to complete the entire epic. Then they can actively retrospect completed stories and epics to detect slowdowns or blockers and their root causes. The goal is to get better at planning and breaking unwieldy stories into multiple smaller stories.

“This is a very lightweight approach to estimation that doesn’t put too much pressure on developers, who can concentrate on planning and implementing stories instead of delving deeply into relative estimations of the complexity”, says Iiro Saukkonen, Engineering Manager for Midas. “We are in the early stages of practicing this kind of projection, but already we can see a drop in the average cycle time of a story by doing more rigorous planning beforehand”.

Product Development teams Pioneer and AweSoMe collaborate to expand Smartly.io’s product to a new advertising platform. As two teams work on the same project, good planning and accurate estimation is a must to keep things running smoothly. “When the teams plan their stories, they follow a rule that, in general, it should take two developers a maximum of ten working days to complete one story”, Pioneer Engineering Manager Adam Nagy says.

“We don’t have separate indicators for story complexity, like story points, but by keeping stories equally sized and splitting them when necessary, we are able to accurately estimate how long it will take to complete the work, commit to a certain timeline, and communicate that timeline to the rest of the organization with confidence” adds Tuukka Koivisto, Engineering Manager for AweSoMe. Using this method, the teams not only reached their shared goal, but were also able to make incremental improvements in the plans throughout the process.

Getting useful feedback is the ultimate purpose

Smartly.io operates in a highly dynamic and fast-paced industry, which makes velocity a key requirement for product development. As both our business and organization keep growing, and the complexity and ambition level of our projects increase, also the universe of potential projects to engage in expands. In such an environment, it is easy to spread ourselves too thin, and risk slowing ourselves down. Building a continuous planning loop can help us make smart strategic decisions on which paths to pursue and when.

Our product development teams are highly autonomous and individual developers have a lot of ownership of their work. As we move forward with estimation, we want to make sure the way we do it aligns with our company culture. A very centralized model of planning wouldn’t fit our style, so teams need to explore and experiment to find what works best for them. Feedback, in general, is fundamental to the Smartly.io culture, so in that sense, the idea of a planning loop as a source of fuel for constant improvement sits in quite well with the way we like to do things.

At the end of the day, all planning methodologies are flawed — especially when we are dealing with high levels of complexity and creating novel products. But that doesn’t mean we should discard them. We evaluate different approaches to planning by how much they provide development teams with useful feedback that allows them to continuously improve.

‍