Letter from the Editor – March 2021

A while ago, I helped start a Software Quality Certificate Program as a part of the Software Engineering Program at the University of California, Santa Cruz Extension in Silicon Valley. I was on the Board of Advisors. While putting the curriculum together, a few people suggested a Measurement and Metrics course. Since I was teaching a few classes in the program, they asked me to write and lead the class. When they did, I did not realize how quickly––or how loudly––I replied “NO!”

The whole room froze and looked at me. I laughed. I told them that there was no way I would go near that class––not even with a 10-foot pole.

The problem with measurement is that every organization is unique, every product is at a different level of maturity and has its own meaningful measurement needs, team members have personal preferences to what informs their decisions, different tools have predesigned reports or dashboards of what they think you may want to know… what I’m getting at here is that it’s all context-driven. Reporting from QA and Test Teams serves many purposes, but reporting must also support the business goals. Thus, measurements and metrics must be relevant to these goals. The reporting needs to be actionable, meaning that the data informs decision-makers and arms them with what they need for:

  • Assessing product readiness (Go, no-go, not yet, etc.),
  • Ascertaining process efficiency (that testing and development are getting done as efficiently as needed), and
  • Accurately scoping project sizing, staffing, resourcing, tool needs, devices, etc.

Measurement and metrics reporting have a rocky history because measurements are easy to generate, so some people over-report, making it the reader’s job to see through and make sense of all of the data. Eliminating waste, being Lean, and cutting overhead is better management. “Just the Facts” and “Less is More” are better mantras for reporting for today.

Better than inventing some acronym, from my experience, the reporting you do must be:

  1. Correct. This may go without saying, but I could tell you horror stories of the incorrect or missing information or measurement leading to problems.
  2. Easy to generate or capture. Once you have defined the measurements to report to the team, they need to be easy to capture. If you regularly have to manually grab a piece of data from one tool and manually grab another piece of data from another tool, then put them into an Excel spreadsheet and calculate something to send the decision-makers or team members… think again. Capturing and calculating reporting needs to be automated.
  3. Understood. The team members that you are reporting this information to need to understand what it does and does not mean––and perhaps even what good numbers might look like and what bad numbers might look like. It is super common that measurements or metrics are reported and misjudged by various members of the team because they have never had them fully explained nor did they get an agreement on what is actually being measured and why.
  4. Used! The reporting is used for action. The right people do things, make decisions, make changes, get staff, get time, bless a release, or hold a release based on the numbers we are giving them. If it turns out that people are not making decisions based on these numbers…
  5. Reevaluate. Change them. If people are not taking action on what you report, if there is too much misunderstanding, or you are not getting the result you intended, get a different measure that’s easy to understand and that the team will be able to use for decision-making. Remember, there are no “best practices,” only continuous improvement.

Reporting is complex because it is always context-driven and often political. My biggest advice for reporting is to be Lean and be careful.

This issue is packed full of content aimed at helping QA leaders ensure their team’s efforts are actively (and accurately) communicated to business decision makers. Our cover story, How to Stop “Flying Blind” When Accounting for QA was written by LogiGear’s SVP of Sales Clayton Simmons and offers actionable insight for QA leaders who may be struggling to find the “perfect mix” of metrics when reporting to decision-makers. The Midpoint Crisis: How Automation Can Make More Manual Testing Work was written by Michael Larsen and explores how to prevent having to manually execute tasks as our Automation program matures. How to Be Seen was written by Kristin Jackvony and is aimed at entry-level QA professionals who are trying to find ways to make their work known and climb the corporate ladder. This issue’s Blogger of the Month, Wes Silverstein, explores the important role that QA has when it comes to planning Automation sprints in The Role of QA in Sprint Planning. Our infographic, 5 Incredibly Useful KPIs for Test Automation, explores 5 of our top-recommended testing metrics to use in your QA reports, including formulas and reasons for why they’re pertinent. Finally, check out the new features and functionality of TestArchitect Version 9.0 in TestArchitect Corner.

We hope you enjoy this issue––happy testing!

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

This is our first Trends issue in our 10- year history. Trends are important to help foresee what is on the horizon and coming next.
For everyone still celebrating holidays: Happy Lunar New Year! At this time of the year many teams and companies are starting new projects, new initiatives, and hiring new staff. LogiGear Magazine will continue to be the resource for you for better testing with much less stress! We are excited about the focus of this month’s ...
There has been a tectonic shift in software development tools in just the past few years. Agile practices and increasingly distributed teams have been significant factors but, in my opinion, the main reason is a new and more intense focus on tools for testing driven by more complex software and shorter development cycles. There have ...
There is a growing software development dynamic of teams without Testers. When I first went into Software Quality, I learned one thing right away: My role was user advocate. My main job was to find bugs. This is the Lean principle called Amplified Learning. We learn about behavior by testing. Even then, validation was not ...
Hi everyone and welcome to our fourth edition of LogiGear Magazine. This month we finish Michael Hackett’s piece on “Agile in Testing” with part five, Tools.
Integrated teams Something we’ve learned in the Covid-19 pandemic is that we have to work together-whatever together means. Very few teams stayed co-located; even teams in the same town worked at home. We’re all working remote. Hopefully all the thinking, tools, work and effort we put into having offshore teams work together benefited us here. ...
Software development projects are multifaceted. There is staffing and budget work. There are communication and team dynamics. There are project and process issues from what the customer wants, when they want it, revenue projections, and production dates. As part of my work in helping people deliver software, I get involved in all aspects mentioned above. ...
I once consulted for a company to give a week-long course on testing and QA. It was a survey course covering a wide range of topics. I was setting up and chatting with students in the room. One man came over to me and said: “I have been testing for 6 months and I am completely ...
Automation is a mantra in testing. Anyone associated with software development wants more test automation, but it’s often misunderstood. People who do test automation know how difficult it can be. But some people do not understand that automation is code, and that it needs to have architecture and design just like production code. They do ...
A lot has changed since I began staffing test projects. From hiring college students and interns for summer testing programs, to building networks of offshore teams around the world, and from having 24-hour work schedules to having instant crowdsourced public beta or bug bounty testing—things have changed.
In the November 2011 issue: Mobile Application Testing, I began my column with the statement, “Everything is mobile.” One year later the statement is even more true. More devices, more platforms, more diversity, more apps. It boggles the mind how fast the landscape changes. Blackberry has been kicked to the curb by cooler and slicker ...
Testing the Software Car. As usual with the LogiGear Magazine, we are tackling a big subject. With our goal of having single-topic issues, we have the ability to grab and disseminate as much information as we can related to a current topic that is interesting and also on the frontier of Software Testing.   Some ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe