Scaling Agile Teams: Using Predictive Metrics to Drive Change

Agile methodologies come from the digital world where they have been transformative. But applying Agile principles outside of software teams is much more difficult. If done wisely, implementing Agile at scale promises to bring speed, responsiveness, and continuous improvement across the enterprise. 

In many companies, however, Agile at scale faces a high cultural hurdle. As change begins to seed itself, managers and change agents face the need to measure and assess the progress of Agile principles throughout the culture. The best leading indicators measure the key behaviors that drive cultural change.

Change Management for Agile at Scale

Implementing Agile across an increasingly large portion of an enterprise takes time and persistence. Since what gets measured gets done, metrics are a key portion of any large-scale cultural change.  

When the timeframe of your Agile transformation is two or three years, you could wait six months to a year to measure your success, even if you determine the best metrics to track. But by definition, metrics you wait to collect are lagging indicators. Some of them may be necessary to track, but you also need leading indicators that will help you track your progress. 

Avoid driving by looking through the rear view mirror

What you want to measure are the real levers of change that drive behavior. We call these predictive metrics. They’re numbers you track because they measure the levers that create behavioral change that, through repetition, becomes an organizational habit, and then seeds, spreading itself throughout the organization.

For example, during your Agile at Scale transformation you will probably introduce certain terminology such as Sprints, Scrum, Scrum-Master, Burn-down charts, Stories, Epics, etc. The adoption of this terminology correlates with the spread of Agile. What if you measured the number of new project plans that used the new Agile terminology? The growth in the number of such plans is a good leading indicator of Agile adoption. Thus an increase in the number of plans with the new terminology predicts the adoption of Agile, a decrease predicts the opposite.

Generally, these predictive metrics are few in number. There are a small number of levers that drive the key behaviors that foster cultural change, as they become ingrained over time through habit. The key to good predictive metrics is to understand the levers that drive behavior. 

Reactive vs. Predictive Metrics

There are many uses for predictive metrics such as milestone hit rate, Agile burn down charts, and weekly variance to schedule.  In managing change, we extend this concept of predictive metrics to allow their users to track the behaviors that drive change. 

They differ from reactive metrics, i.e. metrics that are tracked infrequently, and that measure a result they may not appear for years. These lagging indicators, like Time to Market (TTM) for example, are important to measure, but they allow you only to react after the fact. 

Reactive metrics are like using a scale to reduce weight, rather than tracking calories and exercise time.

Measuring time to market does not, in itself, allow you to predict either fast or slow TTM. Counting the number of product development projects using a standard, consistent process, might be a better predictor of faster TTM. 

Consider the following measures. 

• R&D spending as a percentage of sales. [Reactive]
• Percentage of current year sales from products released in last N-years. [Reactive] 
• Number of project plans using new Agile vocabulary. [Predictive] 

Each of these metrics are useful and necessary in their context. But their origin and purpose are entirely different. For instance, R&D spending is determined through a budgeting process; it is a fixed number that is static. Comparing the R&D spend with current year revenue might influence next year’s budget but it is not likely to influence the behaviors that determine your company culture day-to-day. R&D spending as a percentage of sales is an essential metric but, in itself, it is not predictive. 

The second metric above, percentage of current year sales from products released in the last N-years, is a measure of the health of your new product development capability. But it is a number that by definition you can track only after N-years have passed. A metric like this allows you to look back on the past, and to measure past successes. It indicates the value of something done in the past, rather than predict what’s likely to occur in the future. It indicates a long-term trajectory but it is not useful for fast course-corrections. 

Now consider the third metric: Number of project plans using new Agile vocabulary. Such projects may be in product development, Marketing, Sales or another function or business area. By measuring the number of project plans that adopt the new vocabulary, change agents can track Agile adoption in real time. If the number decreases, they know this quickly and can take corrective action to improve the situation. 

You would track this metric as you go; you do not wait a long period of time and then count the number of projects using the new vocabulary. This would constitute a lagging indicator that is not amenable to course corrections. 

When you track predictive metrics in real time, you create a clear link between your targets and your current status. Tracking and measuring in real time allows you to be reasonably sure that as the metric increases, Agile adoption also increases. In other words, the metric you have chosen is a truly predictive metric, targeting the behavior you wish to change.

Predictive Metric: Example

A large, well-established tech company drove its change management initiative with predictive metrics. They developed and piloted a brand new product development process. They used it with a small number of teams and worked out the kinks. 

They then looked at indicators that would signal adoption of the new process, i.e. that would drive the desired behavior: compliance with a new release planning process. Since the best evidence of release planning was to see these plans in the backlog and associated with a sprint, they measured the number of teams that had estimated sizing before prioritizing stories for a sprint. 

The teams that placed the release plan into their backlog and earmarked a sprint dedicated to the next release plan were adopting the new Agile process. They set a target which was to have 95% of the teams on board with the new process. Scheduling the next release plan proved to predict the adoption of the Agile process, more reliably than attendance at a training. It showed that Agile was in action with these teams.

Although this example comes from product development, the same principles and similar metrics could apply just as well to many other types of products, in other functions and business areas. 

Successful Predictive Metrics

The power of predictive metrics is that they provide visibility to your goals for Agile adoption and provide indicators that enable course corrections. They keep your change management initiative on track while they also provide a dashboard that functions as an early warning system if your implementation begins to go awry. 

Five tips for success with predictive metrics:

1. Think about the key drivers of behavior: you will get what you measure so think hard!
2. Create a quantitative metric to measure the behaviors in question.
3. Keep the metrics few (say three-to-five) and simple to calculate and collect. 
4. Ensure the metric is not ambiguous or subjective and easy to capture.
5. Create a target for your metrics and measure your performance against it. 

Predictive Metrics As Tools for Change

Change happens when people behave in different ways and not a minute before. Metrics are one of the most powerful ways to influence change. Predictive metrics are especially powerful because they are based on tracking only the critical few indicators that support the cultural changes you are targeting.

Agile at scale usually takes time measured in years. The best implementations start small and slowly seed improvements throughout the organization. Many companies take on too much, and measure too much. In most companies, Agile starts in the software area, and then needs a great deal of tailoring, coaching, and training to port anywhere outside that domain.

The predictive metrics approach is to start small. Start with realizable goals, a small set of behaviors you want to see shift in your organization, or a small number of Agile principles you want to seed throughout your organization. Begin with a small number of teams and build out, measuring the key behaviors that drive your goals. 

Keep the measurement lightweight and easy to collect. Remain consistent and committed to the long haul. Measure in real time and keep tracking. Cultural change is difficult. The key is to instill new habits. Predictive metrics are tools to help you slowly shift a culture.