Agile is here to stay. Once the radical alternative to Waterfall development methods, these legacy methodologies are being disrupted and replaced by Agile practices that improve time-to-market, reduce development costs, and produce higher quality software that better meets customer expectations. As the world demands more software, development teams – from scrappy startups to big corporations – are meeting the challenge with Agile.
But while Agile software development projects scale across the enterprise, management is still searching for the best way to gain deeper visibility into these projects. Large organizations cannot rely on the subjective anecdotes of those closest to the work; they require quantitative insight upon which to base business decisions.
Here are some quick tips to quantify the impact of the choices you make during and after your Agile transformation.
1. Start with desired outcomes, not what’s easy to measure
Better measurement leads to better insights, which in turn leads to better decisions and eventually better outcomes. Most people start by measuring what’s easy. But measuring what’s easy can drive the wrong behavior. Let’s take a look at this story about two NBA players.
In 2010, Monta Ellis, with the Golden State Warriors, was the 9th highest scorer in the NBA. Carmelo Anthony, with the Denver Nuggets, was the 8th highest scorer. Measuring individual scoring totals is easy. You would assume that because they were prolific scorers, their teams would win. However, when they were in the game, their teams won less. Scoring is a function of two other measures: 1) the number of shots and 2) the percentage of those shots that go in the basket. It turns out, these two “stars” have low measures for #2, their shooting percentage.
The only reason they are high scorers is because they take more shots. They are literally stealing shots from their teammates who might have a better chance of scoring.
So, while the flow of learning goes from measures to outcomes, the way we think about it should start with outcomes. That’s why we call this ODIM:
better OUTCOMES ← better DECISIONS ← better INSIGHTS ← better MEASURES
The NBA players should focus on the outcome of winning more games rather than being a high scorer. If they used the overall likelihood of the team scoring under various conditions as feedback, it would help them make better game-time decisions to achieve the ultimate outcome of winning. This brings us to our second tip.
2. Think of measurement as feedback, not levers
Frequent feedback is the primary difference between Waterfall and Agile development. Successful Agile projects incorporate short iterations with fast feedback from customers. The key to effective Agile measurement is to think of measurement in terms of feedback, not as the traditional lever to motivate behavior. This often devolves into keeping score, which is where the dark side of measurement starts — avoid it.
There is a subtle, but important, distinction between “feedback” and “lever.” Feedback is something you seek to improve your own performance. Levers are used to influence others. The difference is more in how you use the measure than the measure itself.
For example, healthy use of a burndown chart tells the team if they are on track with their commitment so they can make adjustments in time. The counterexample is a manager using burndown charts to red-flag projects in trouble. While it may drive improvement, nobody wants the red flag thrown at them, so the tendency is to keep the metric in the green regardless of the reality of the situation.
You can’t make better informed decisions if the metrics you are using to gain insight don’t accurately represent reality (see tip #1). Rather, the manager could provide coaching on tools that the team can use to improve their own performance — a subtle but critical difference.
3. A balanced measurement regime or none at all
Balance in Agile measurement (Figure A) includes four cornerstones:
- Do it fast.
- Do it right.
- Do it on time.
- Keep doing it.
Figure A
Without balance of these four elements, it’s easy to focus on just one. For example, if we focus only on increasing productivity this will likely drive down quality and customer satisfaction.
4. Measure specific outcomes for software
- Productivity
- Responsiveness
- QualityCustomer
- Satisfaction
- Predictability
- Employee Engagement
These six outcomes are the elements of the Software Development Performance Index (SDPI), used to quantify insights about development work and provide feedback on how process and technology decisions impact the development team’s performance. Know what to measure and focus on each individual element.
5. Listen to Experts
Agile has caught the attention of leading industry analysts, asserting itself as a key part of application lifecycle management (ALM) evaluations. These evaluations encompass more than just functionality for developers; they assess commitment to the ALM market, ALM product strategy, corporate strategy, and market presence.
In fact, independent research firm Forrester Research, Inc. recently evaluated the most significant ALM software providers, treating Agile and Lean as critical tests of an ALM vendor’s offering. The report also found that businesses “can no longer accept historical gulfs between business and application development and delivery teams as, increasingly, firms now expect to manage application development and delivery as a business and treat it as a competency.”
6. The Agile perspective for software development metrics
In the move to Agile, overall goals are largely the same as before: to delight users with a quality product delivered in a predictable and efficient manner. Even after your Agile transformation, you will largely do the same “types” of things: analyze, design, code, test, release, maintain, and, yes, measure.
It’s the perspective you take when doing these that is different with Agile.
Prior to starting work full-time on his PhD, Larry served as the Manager of Software Assurance Initiatives for the CyLab at Carnegie Mellon where he promoted the development and widespread adoption of best practices, tools, and methods.
His interests include measurement/analytics, Agile methodologies, software engineering, software craftsmanship, and software assurance. He also have a strong interest in information visualization and has a passion for coding in general which includes programming language technology and design patterns.
