Key Principles of Test Design

Test design is the single biggest contributor to success in software testing. Not only can good test design result in good coverage, it is also a major contributor to efficiency. The principle of test design should be “lean and mean.” The tests should be of a manageable size and at the same time complete and aggressive enough to find bugs before a system or system update is released.

Test design is also a major factor for success in test automation. This is not that intuitive. Like many others, I initially also thought that successful automation is an issue of good programming or even “buying the right tool.” Finding that test design is the main driving force for automation success is something that I had to learn over the years-often the hard way.

What I have found is that there are three main goals that need to be achieved in test design. I like to characterize them as the “Three Holy Grails of Test Design” – a metaphor based on the stories of King Arthur and the Round Table as the three goals as the three goals are difficult to reach mimicking thestruggle King Arthur’s knights experienced in search of the Holy Grail. This article will introduce the three “grails” to look for in test design. In subsequent articles of this series I go into more detail about each of the goals.

The terminology in this article and the three subsequent articles are based on Action Based Testing (ABT), LogiGear’s method for testing and test automation. You can read more about the ABT methodology on the LogiGear web site. In ABT test cases are organized into spreadsheets which are called “test modules.” Within the test modules the tests are described as a sequences of “test lines,” each starting in the A column with an “action,” while the other columns contain arguments. The automation in ABT does not focus on automating test cases, but on automating individual actions, which can be re-used as often as necessary.

The Three Goals for Test Design

The three most important goals for test design are:

1. Effective breakdown of the tests

The first step is to breakdown the tests into manageable pieces, which in ABT we call “test modules.” At this point in the process we are not yet describing test cases; we simply place test cases its its appropriate “chapters.” A break down is good if each of the resulting test modules has a clearly defined and well-focused scope which is differentiated from the other modules. The scope of a test module subsequently determines what its test cases should look like.

2. Right approach per test module

Once the break down is done each individual test module becomes a mini-project. Based on the scope of a test module we need to determine what approach to take to develop the test module. By approach I mean the choice of testing techniques used to build the test cases (like boundary analysis, decision tables, etc.), and who should get involved to create and/or assess the tests. For example, a test module aimed at testing the premium calculation of insurance policies might need the involvement of an actuarial department.

3. Right level of test specification

This third goal is deciding where you can win or lose most of the maintainability of automated tests. When creating a test case try to specify only the high-level details that are relevant for the test. For example, from the end-user perspective “login” or “change customer phone number” is one action; it is not necessary to specify any low-level details such as clicks and inputs. These low-level details should be “hidden” at this time in separate, reusable automation functions common to all tests. This makes a test more concise and readable, but most of all it helps maintain the test since low-level details left out will not have to be changed one-by-one in every single test if the underlying system undergoes changes. The low-level details can then be re-specified (or have their automation revised) only once and reused many times in all tests.

In ABT this third principle is visible in the level of the actions to be used in a test module. For example, in an insurance company database, we would write tests using only “high-level” actions like “create policy” and”check premium,” while in a test of a dialog you could use a “low level” action like “click” to see if you can click the OK button.

Conclusion

Regardless of the method you choose, simply spending some time thinking about good test design before writing the first test case will have a very high payback down the line, both in the quality and the efficiency of the tests. If your test automation project is currently applying the Keyword-Driven Testing method and you want to improve your Test Design, check out this article: Keyword-Driven Testing: The Best Practices You Can’t Afford to Miss

Hans Buwalda

Hans leads LogiGear’s research and development of test automation solutions, and the delivery of advanced test automation consulting and engineering services. He is a pioneer of the keyword approach for software testing organizations, and he assists clients in strategic implementation of the Action Based Testing™ method throughout their testing organizations.

Hans is also the original architect of LogiGear’s TestArchitect™, the modular keyword-driven toolset for software test design, automation and management. Hans is an internationally recognized expert on test automation, test development and testing technology management. He is coauthor of Integrated Test Design and Automation (Addison Wesley, 2001), and speaks frequently at international testing conferences.

Hans holds a Master of Science in Computer Science from Free University, Amsterdam.

Hans Buwalda
Hans Buwalda, CTO of LogiGear, is a pioneer of the Action Based and Soap Opera methodologies of testing and automation, and lead developer of TestArchitect, LogiGear’s keyword-based toolset for software test design, automation and management. He is co-author of Integrated Test Design and Automation, and a frequent speaker at test conferences.

The Related Post

Introduction This 2 article series describes activities that are central to successfully integrating application performance testing into an Agile process. The activities described here specifically target performance specialists who are new to the practice of fully integrating performance testing into an Agile or other iteratively-based process, though many of the concepts and considerations can be ...
Internet-based per-use service models are turning things upside down in the software development industry, prompting rapid expansion in the development of some products and measurable reduction in others. (Gartner, August 2008) This global transition toward computing “in the Cloud” introduces a whole new level of challenge when it comes to software testing.
Back from more training, I was up at a client in Bellevue and really enjoyed teaching a performance class to a world class testing organization. I found that the students were very receptive to many of the concepts and ideas that the class offers.
When it is out of the question to delay delivery, the solution is a prioritization strategy in order to do the best possible job within the time constraints. The scenario is as follows: You are the test manager. You made a plan and a budget for testing. Your plans were, as far as you know, ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Article Synopsis There are many misconceptions about Software Testing. This article deals with the 5 most common misconceptions about how Software Testing differs from other testing. Five Common Misconceptions Some of the most common misconceptions about ...
PWAs have the ability to transform the way people experience the web. There are a few things we can agree we have seen happen. The first being that we figured out the digital market from an application type perspective. Secondly, we have seen the rise of mobile, and lastly, the incredible transformation of web to ...
One of the most common challenges faced by business leaders is the lack of visibility into QA activities. QA leaders have a tough time communicating the impact, value, and ROI of testing to the executives in a way that they can understand. Traditional reporting practices often fail to paint the full picture and do not ...
I’ve been reviewing a lot of test plans recently. As I review them, I’ve compiled this list of things I look for in a well written test plan document. Here’s a brain dump of things I check for, in no particular order, of course, and it is by no means a complete list. That said, if you ...
Reducing the pester of duplications in bug reporting. Both software Developers and Testers need to be able to clearly identify any ‘Bug’, via the ‘Title’ used for the ‘Bug Report’.
This article was originally featured in the May/June 2009 issue of Better Software magazine. Read the entire issue or become a subscriber. In my travels, I’ve worked with a number of companies that have attempted to assess the quality of their testing — or worse, their testers — using poorly considered metrics. Sometimes the measurement ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Introduction Many look upon Software Testing as a cost. While it is true that Software Testing does cost money, in many cases significant amounts of money, it is also an activity that helps an organization to ...
At VISTACON 2011, Harry sat down with LogiGear Sr. VP, Michael Hackett, to discuss various training methodologies. Harry Robinson Harry Robinson is a Principal Software Design Engineer in Test (SDET) for Microsoft’s Bing team, with over twenty years of software development and testing experience at AT&T Bell Labs, HP, Microsoft, and Google, as well as ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe