Bonus Bugs

One of the most dreaded kinds of bugs are the ones caused by fixes of other bugs or by code changes due to feature requests. I like to call these the ‘bonus bugs,’ since they come on top on the bug load you already have to deal with.

Bonus bugs are the major rationale for regression testing in general and Test Automation in particular, since Test Automation is the best way to quickly retest an entire application after each round of code changes.

Since this is probably a ‘bonus’ you want to avoid, how do we prevent the bonus bugs from occurring, and how do we detect them when they have been introduced? I will give some notes here from the perspective of the developer, the tester and the manager respectively.

Let’s first talk about the developer. A developer can do quite a lot to reduce the chances of bonus bugs. Today’s systems are becoming more and more complex, and this complexity only increases over time as changes to the system are made. Any change can easily trigger a problem somewhere else, thus producing a bonus bug.

There is a lot written about commenting and documenting code, which I will not go into here, but whatever standard you adhere to (or are told to adhere to), make sure that somebody can easily “inherit” your code. It should take minimal energy for somebody to “decipher” and maintain the code you have written. Code should be written in small blocks each, of which starts with a meaningful comment. For example, if there is something that you want the next person to know about the code (e.g. some technical pitfall that you had to work around), state it explicitly in the code comments.

Another good policy is to have code changes reviewed and approved by either a peer programmer, or even better by a supervising “architect” who understands how the system is built up and what consequences of system changes could be.

From the point of view of the tester, there are two main items to worry about: test design and level of Automation.

Test design is one of the most underestimated topics in IT. Most tests that I encounter in companies and other organizations are “lame”; they simply follow the system requirements one-by-one and don’t even attempt to combine several different parts of the system functionalities with each other in creative ways that could reveal unexpected problems––like bonus bugs. Even though requirement based tests are useful, they have a low “ambition level,” and it can pay out to allocate time and resources to make more aggressive tests.

A high level of Test Automation will greatly enhance your capability to catch the bonus bugs before they reach the release. To get to such a high level, simply buying a test tool will not be enough. A well thought-out method of Test Automation, such as Keyword-Driven Testing, is essential, combined with training and coaching by experienced Test Automation experts.

Finally, a few words from the perspective of the manager: Here the recommendation is in fact quite simple: Make a determination on what bonus bugs can cost and what it is worth to prevent them. This is a business estimate and decision: having bonus bugs can cost money; efforts to prevent them cost money too. Effects of bonus bugs (or any other kind of bugs) can typically be loss of time before or after system release, and/or decreased appreciation by end-users of you and your company. Preventing bonus bugs takes extra time and money to follow policies and procedures for development and testing, which can include reviews of code and setting up a high level of Test Automation.

By understanding how and why bonus bugs get introduced into applications, we can both prevent them from being introduced, and find them when they are. This takes a combined effort of the developers, testers, and managers, and it’s a very important step in ensuring that your end-product satisfies your customers and other stakeholders.

Hans Buwalda

Hans leads LogiGear’s research and development of test automation solutions, and the delivery of advanced test automation consulting and engineering services. He is a pioneer of the keyword approach for software testing organizations, and he assists clients in strategic implementation of the Action Based Testing™ method throughout their testing organizations.

Hans is also the original architect of LogiGear’s TestArchitect™, the modular keyword-driven toolset for software test design, automation and management. Hans is an internationally recognized expert on test automation, test development and testing technology management. He is coauthor of Integrated Test Design and Automation (Addison Wesley, 2001), and speaks frequently at international testing conferences.

Hans holds a Master of Science in Computer Science from Free University, Amsterdam.

Hans Buwalda
Hans Buwalda, CTO of LogiGear, is a pioneer of the Action Based and Soap Opera methodologies of testing and automation, and lead developer of TestArchitect, LogiGear’s keyword-based toolset for software test design, automation and management. He is co-author of Integrated Test Design and Automation, and a frequent speaker at test conferences.

The Related Post

Regardless of the method you choose, simply spending some time thinking about good test design before writing the first test case will have a very high payback down the line, both in the quality and the efficiency of the tests. Test design is the single biggest contributor to success in software testing and its also ...
I’ve been reviewing a lot of test plans recently. As I review them, I’ve compiled this list of things I look for in a well written test plan document. Here’s a brain dump of things I check for, in no particular order, of course, and it is by no means a complete list. That said, if you ...
With complex software systems, you can never test all of the functionality in all of the conditions that your customers will see. Start with this as a fact: You will never test enough! Step 2 in getting started is to read and re-read The Art of Software Testing by Glenford Myers. This classic will set the ...
Test design is the single biggest contributor to success in software testing. Not only can good test design result in good coverage, it is also a major contributor to efficiency. The principle of test design should be “lean and mean.” The tests should be of a manageable size and at the same time complete and ...
LogiGear_Magazine–March_2015–Testing_Strategies_and_Methods-Fast_Forward_To_Better_Testing
At VISTACON 2011, Harry sat down with LogiGear Sr. VP, Michael Hackett, to discuss various training methodologies. Harry Robinson Harry Robinson is a Principal Software Design Engineer in Test (SDET) for Microsoft’s Bing team, with over twenty years of software development and testing experience at AT&T Bell Labs, HP, Microsoft, and Google, as well as ...
VISTACON 2010 – Keynote: The future of testing THE FUTURE OF TESTING BJ Rollison – Test Architect at Microsoft VISTACON 2010 – Keynote   BJ Rollison, Software Test Architect for Microsoft. Mr. Rollison started working for Microsoft in 1994, becoming one of the leading experts of test architecture and execution at Microsoft. He also teaches ...
In today’s mobile-first world, a good app is important, meaning an effective Mobile Testing strategy is  essential.  
This article first appeared in BETTER SOFTWARE, May/June 2005. Executives and managers, get your performance testing teams out of the pit and ahead of the pack Introduction As an activity, performance testing is widely misunderstood, particularly by executives and managers. This misunderstanding can cause a variety of difficulties-including outright project failure. This article details the ...
March Issue 2019: Leading the Charge with Better Test Methods
Let’s look at a few distinctions between the two process improvement practices that make all the difference in their usefulness for making projects and job situations better! An extreme way to look at the goals of these practices is: what makes your work easier (retrospective) versus what did someone else decide is best practice (post-mortem)? ...
There are many ways to approach test design. These approaches range from checklists to very precise algorithms in which test conditions are combined to achieve the most efficiency in testing. There are situations, such as in testing mobile applications, complex systems and cyber security, where tests need to be creative, cover a lot of functionality, ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe