Quantify the Impact of Agile App Development

Agile is here to stay. Once the radical alternative to Waterfall development methods, these legacy methodologies are being disrupted and replaced by Agile practices that improve time-to-market, reduce development costs, and produce higher quality software that better meets customer expectations. As the world demands more software, development teams – from scrappy startups to big corporations – are meeting the challenge with Agile.

But while Agile software development projects scale across the enterprise, management is still searching for the best way to gain deeper visibility into these projects. Large organizations cannot rely on the subjective anecdotes of those closest to the work; they require quantitative insight upon which to base business decisions.

Here are some quick tips to quantify the impact of the choices you make during and after your Agile transformation.

1. Start with desired outcomes, not what’s easy to measure

Better measurement leads to better insights, which in turn leads to better decisions and eventually better outcomes. Most people start by measuring what’s easy. But measuring what’s easy can drive the wrong behavior. Let’s take a look at this story about two NBA players.

In 2010, Monta Ellis, with the Golden State Warriors, was the 9th highest scorer in the NBA. Carmelo Anthony, with the Denver Nuggets, was the 8th highest scorer. Measuring individual scoring totals is easy. You would assume that because they were prolific scorers, their teams would win. However, when they were in the game, their teams won less. Scoring is a function of two other measures: 1) the number of shots and 2) the percentage of those shots that go in the basket. It turns out, these two “stars” have low measures for #2, their shooting percentage.

The only reason they are high scorers is because they take more shots. They are literally stealing shots from their teammates who might have a better chance of scoring.

So, while the flow of learning goes from measures to outcomes, the way we think about it should start with outcomes. That’s why we call this ODIM:

better OUTCOMES better DECISIONS better INSIGHTS better MEASURES

The NBA players should focus on the outcome of winning more games rather than being a high scorer. If they used the overall likelihood of the team scoring under various conditions as feedback, it would help them make better game-time decisions to achieve the ultimate outcome of winning. This brings us to our second tip.

2. Think of measurement as feedback, not levers

Frequent feedback is the primary difference between Waterfall and Agile development. Successful Agile projects incorporate short iterations with fast feedback from customers. The key to effective Agile measurement is to think of measurement in terms of feedback, not as the traditional lever to motivate behavior. This often devolves into keeping score, which is where the dark side of measurement starts — avoid it.

There is a subtle, but important, distinction between “feedback” and “lever.” Feedback is something you seek to improve your own performance. Levers are used to influence others. The difference is more in how you use the measure than the measure itself.
For example, healthy use of a burndown chart tells the team if they are on track with their commitment so they can make adjustments in time. The counterexample is a manager using burndown charts to red-flag projects in trouble. While it may drive improvement, nobody wants the red flag thrown at them, so the tendency is to keep the metric in the green regardless of the reality of the situation.

You can’t make better informed decisions if the metrics you are using to gain insight don’t accurately represent reality (see tip #1). Rather, the manager could provide coaching on tools that the team can use to improve their own performance — a subtle but critical difference.

3. A balanced measurement regime or none at all

Balance in Agile measurement (Figure A) includes four cornerstones:

  • Do it fast.
  • Do it right.
  • Do it on time.
  • Keep doing it.


Figure A

Without balance of these four elements, it’s easy to focus on just one. For example, if we focus only on increasing productivity this will likely drive down quality and customer satisfaction.

4. Measure specific outcomes for software

  • Productivity
  • Responsiveness
  • QualityCustomer
  • Satisfaction
  • Predictability
  • Employee Engagement

These six outcomes are the elements of the Software Development Performance Index (SDPI), used to quantify insights about development work and provide feedback on how process and technology decisions impact the development team’s performance. Know what to measure and focus on each individual element.

5. Listen to Experts

Agile has caught the attention of leading industry analysts, asserting itself as a key part of application lifecycle management (ALM) evaluations. These evaluations encompass more than just functionality for developers; they assess commitment to the ALM market, ALM product strategy, corporate strategy, and market presence.

In fact, independent research firm Forrester Research, Inc. recently evaluated the most significant ALM software providers, treating Agile and Lean as critical tests of an ALM vendor’s offering. The report also found that businesses “can no longer accept historical gulfs between business and application development and delivery teams as, increasingly, firms now expect to manage application development and delivery as a business and treat it as a competency.”

6. The Agile perspective for software development metrics

In the move to Agile, overall goals are largely the same as before: to delight users with a quality product delivered in a predictable and efficient manner. Even after your Agile transformation, you will largely do the same “types” of things: analyze, design, code, test, release, maintain, and, yes, measure.

It’s the perspective you take when doing these that is different with Agile.

Larry
Larry started working at Rally Software in 2009 and is currently Rally’s Director of Analytics. He is currently working on his PhD in Software Engineering at Carnegie Mellon.

Prior to starting work full-time on his PhD, Larry served as the Manager of Software Assurance Initiatives for the CyLab at Carnegie Mellon where he promoted the development and widespread adoption of best practices, tools, and methods.

His interests include measurement/analytics, Agile methodologies, software engineering, software craftsmanship, and software assurance. He also have a strong interest in information visualization and has a passion for coding in general which includes programming language technology and design patterns.

Larry Maccherone
Larry started working at Rally Software in 2009 and is currently Rally’s Director of Analytics. He is currently working on his PhD in Software Engineering at Carnegie Mellon.Prior to starting work full-time on his PhD, Larry served as the Manager of Software Assurance Initiatives for the CyLab at Carnegie Mellon where he promoted the development and widespread adoption of best practices, tools, and methods.His interests include measurement/analytics, Agile methodologies, software engineering, software craftsmanship, and software assurance. He also have a strong interest in information visualization and has a passion for coding in general which includes programming language technology and design patterns.

The Related Post

Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Three of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
Testing is often looked upon by many as an unmanageable, unpredictable, unorganized practice with little structure. It is common to hear questions or complaints from development including: What are test teams doing? Testing takes too long Testers have negative attitudes
How to fit automated testing into scrum, and keep testers in sync with other teams One of the benefits of the approaches of agile projects is their friendliness towards testing. The testing activities, and the testers with it, are integrated into the teams, and testing and quality are redefined as team responsibilities. Automation nowadays is a must-have ...
Mark Levison has over twenty years experience in the IT industry, working as a developer, manager, technical lead, architect, and consultant. He discovered Agile in 2001 and is now a Certified Scrum Trainer and Agile Coach with Agile Pain Relief Consulting. Levison has introduced Scrum, Lean and other Agile methods to a number of organizations and coaches from ...
The No-Nonsense Guide for How to Write Smarter and Low Maintenance Test Cases Test design is a phrase that is often used when planning testing and test efforts, but I do not believe it is well understood. Also, opinions vary widely about the importance of test design ranging from irrelevant to the crucial ingredient for ...
When quality assurance teams and management who have adopted Agile practices first put the ideas to work, they face a significant impediment in unlearning the traditional mind-set and practices that experience in traditional practices has instilled in them. “He who knows to unlearn, learns best.” — Anonymous The following are some of the key aspects ...
To begin this article, it would be a good idea to, remember this key point: Agile Manifesto Value #1 Individuals and interactions over processes and tools Tools work at the service of people. People, particularly intelligent people, can never be slaves to tools. People talking to each other, working together and solving problems is much ...
Writing code that is easy to read and easy to test is difficult to achieve. The fact that poorly written code can function often leads to coding practices that are effective but not necessarily efficient. Too often, many programmers fresh out of school write code in the manner that was effective for passing their courses, but contains ...
“Agile is to software development what poetry is to literature. It is very difficult to do well, very easy to do poorly, and most people don’t understand why a good poem is good and a bad poem isn’t…” – from the web
There is a multitude of Agile testing techniques that are quite sophisticated. The DAD process can help guide your process of tailoring decisions. Agile developers are said to be quality infected, and disciplined agilists strive to validate their work to the best of their ability. As a result they are finding ways to bring testing and ...
SKILLS Agile teams need training! One of the missing links in the implementation of Agile development methods is the lack of training for teams. I noticed in our recent survey on Agile that only 47% of the respondents answered “Yes” that they had been trained in the Agile development process, with over half responding “No.” ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part One of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile” LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe