The best part of Action-Basted Testing is that it is for thinking people. It is intelligent and creative. It is a much saner way to evolve a testing project.
All testers and quality engineers hear about Action-based testing (ABT) or keyword-driven testing somewhere. There are automation tools focused on keywords and actions. Maybe people have read an article about it. But do you actually do ABT as a test method? Probably not. What’s the big deal with ABT? Why don’t more teams do it?
ABT sounds good to use as a method. I often wonder: “what do people really know about Action-based testing?” I hear: “Isn’t ABT about making a bunch of small, modular automated test cases— and calling them actions, then reuse and string them together into bigger scenarios? Isn’t that ABT?” Actually not. That would be using action words, or just keywords for that matter for your automation without using ABT as the test method to design and develop your tests.
Here are some more examples of missing the point of ABT. To me, some of the most important parts of Action-based testing are:
- It all starts with test design. Stop starting a test project by writing mindless test steps and more steps and more steps- the maintenance cost is too high.
- ABT is iterations of greater specificity—start high and build into lower levels of detail.
- ABT is separation of test design and test execution. Even though it may wind up being done by one person, the test designer/subject matter expert hat is worn first. Automaton can be specified later, by the same person or someone with different skills.
- Focus on intelligent test design and not mindless test execution. It is more important to focus on what is going to get tested rather than how those tests will get executed.
- ABT easily enables greater sharing of test cases, faster test documentation, better division of work, and helps other team members by not repeating the same actions over and over. Its more human- easier sharing, easier and better communication.
So why don’t more teams use the ABT method? I think it is difficult for many teams to break away from the “what we have always done” mindset. It’s difficult to get team or management buy-in on testing not focused uniquely on requirements. Plus, too many people think ABT is only about automation.
This article focuses on the infrequently discussed aspects of ABT— the thinking and human side of the method. Action-based, or keyword-based testing has become so popular with automation tools that many people think it is just a method of automation. It’s true that ABT will give you more, better and easier test automation and you can find that written about in many places. I want to focus on the areas of ABT I like that are independent of test automation.
ABT in a Nutshell
ABT is much more than creating action words! It’s a method to organize and design testing by focusing on test modules and test objectives rather than how the tests will be executed. ABT emphasizes thoughtful design and focuses on what you are testing rather than on how to script tests. Trying to accomplish too much with detailed scripting is what usually results in high maintenance. ABT creates easier to understand documentation, better test coverage, and easier, more scalable test automation with the full participation of both testers and automation (coding) engineers.
Too often ABT is reduced to merely making a bunch of reusable actions and modular testing. That is not ABT, the actions are definitely a byproduct and huge benefit of ABT, but if it stops there, you are missing the point and biggest benefits of ABT.
Test Design—Think, Think, Think
Some automation tools lend themselves to mindless scripting that produces high maintenance, low re-use, non-sharable test scripts. Bad automation. ABT, to me, is about test design. Test Design is the process of analyzing test requirements and arriving at the test modules, test objectives and then test cases. It starts with an analysis of what areas will be tested and the creation of modules. A test module is a way to organize test cases into logical, related functions. A test module is a much larger unit than a test case, and contains multiple test cases.
The test design process is thinking about what you are testing and examining the product you are about to develop in order to define what has to be tested.
In ABT, it is crucial to have a good high level test design in order to effectively define test modules. The modules and test objectives are where we see the test design, not in the test steps or cases. Each test module should have a clear scope that is different from the others to avoid redundancy. Each module should be self-contained and to avoid interdependency to reduce maintenance and provide for easier distribution of work.
A test module consists of test objectives and action lines, and will resemble a spreadsheet. The test objectives outline the scope of the test module into individual verbal statements defining what needs to be tested in the module. Test objectives are my favorite part of ABT. They are the ideas, risks, objectives—things I want to test. Anyone can easily read these sentences and get the complete understanding of what I am testing rather than wading through endless steps and steps and steps.
The tests within the modules are defined by a series of action lines, often further organized in one or more test cases. Every action line consists of an action word that defines the test step, and arguments that define the data for the action. Arguments can include input values and expected results.
Not only will good test design result in good coverage, it is also a major contributor to efficiency. The principle of test design should be “lean and mean.” The tests should be of a manageable size, and at the same time complete and aggressive enough to find bugs before a system or system update is released.
Some people define test design as specifying test cases. I disagree. Test design is the process of thinking, using your brain, organizing the project, identifying high level ideas on what has to be tested, without the specifics. A test case is too small and too isolated of a unit to give good direction to test development. Rather than having a predefined list of test cases to be developed, I like to make a list of test modules, and let the test cases in them be the result of test design, not the input of it.
After the test modules are defined and taking a first pass on the test objectives, you can begin assembling test cases and actions. You create test cases to execute the objectives, but create them by using action words as opposed to long sets of steps that are hard to read and very soon become cumbersome and then require a great time investment to create and maintain.
Focus on What, Not How
From management practices to self-improvement to software programming best practices, Focus on what, not how is the effective mantra, and its the core of ABT.
Focusing on what you test is the beauty of ABT, how you test it comes later. What you are testing dictates test design—it’s creative, imaginative, thoughtful and intelligent. How you will test is a different issue—it’s engineering, or simple, step-by-step robotics. Yes, the how may require creative intelligent solutions, but with ABT, this is less common.
Don’t focus too much on details in the beginning. Start with big picture, how you test and lower level details get resolved in later iterations of planning. Immediately scripting step by step is not a sophisticated approach. Though it is common, its a bad idea!
What is now. How comes later. What — the goals and success factors and coverage; How — low level, detailed later in the process. Letting go of the how until later also unblocks your mind and allows ideas and solutions to flow to you.
ABT on Agile Teams
Starting with what has to be tested and building increasing levels of detail fits perfectly into the agile mindset. For estimating the time for testing in a sprint planning session, it is more important to focus on what has to be tested than how it will be tested. Are there cases where how something is tested greatly impacts the test estimate? Yes, but rarely. Too often the extent of testing, the variety of testing and the ranking of tests is ignored in favor of how long an individual test takes to run in giving estimations.
During release planning, at the beginning of a set of sprints and at sprint planning meetings, the features, UI, and implementation is not yet planned.
How can you plan the execution of the test? You can’t without a high maintenance cost. What you absolutely can do is plan out and design what has to be tested the areas, modules and ideas, objectives to be tested. You know that features have to be tested in a specific way, on already defined platforms, with known data, using certain interfaces—its test design. This can be a big asset in conveying risk, coverage, estimates, your thinking and needs for the testing project in language the scrum team can understand—without focusing on how the test will be executed.
The most common practice is, and will remain, to develop and execute tests as part of sprints. In a sprint, functionality is progressively understood from user stories and conversations to become clear enough for testers to test it. This can be done in developed tests similar to ABT test modules, as well as exploratory and interactive testing. It can also be good practice to capture at least some of the “interesting” interactive tests in test modules for future use.
Have as many business level tests as possible, as they add great value to overall depth and quality, as well as being resilient against system changes that do not pertain to them. Use the high level test design step in ABT (where the test modules are identified) to determine what can be done early on in business level tests, and what needs to be completed in detail tests as part of development sprints.
Benefits of ABT
ABT is more human. It’s easier to communicate. Easier to share actions and test cases. It enables better test understanding and team collaboration. When the automation finally gets specified as actions, the individual actions can easily be shared and reused, making writing test cases easier, faster and, lower maintenance.
ABT Improves the organization of tests. Instead of randomly grouping tests together, each module has a clear scope that tests must fall under which gives better visibility into testing. It also enhances business functionality test coverage. For example, each module can cover a major functional requirement (e.g. “Make Reservation” or “Cancel Reservation”) or a non-functional requirement (e.g. Scalability Test that registers 1 million users).
Organizationally, the big advantage with ABT is scalability. Unlike with coding, using actions eliminate redundant steps and automation. This makes it faster to create new tests, so there is less than 1:1 effort required for testing as the complexity of the software increase, so you can quickly create a large number of tests.
Is ABT perfect? No. Is it better than what the vast majority of teams do? Absolutely. There is a ramp-up curve to learning the details and implementation nuances of ABT. Defining modules, high-level and low-level action, are skills to be built. While starting a test project with step-by-step test cases is certainly faster, it has less intelligence, higher maintenance, more work, with low visibility.
Also—you use your brain for ABT. You get to think, envision what needs to be tested early in the project. I see this as very positive as opposed to the boredom of creating test steps.
In summary, to me, the big payoff with ABT is the speed at which I can document a test project. I can‘t stand the step-by-step, brainless mess. I do not like working on mindless projects.
Yes, there is a huge payoff when it comes to automation—clearly. But the best part of ABT is that it is for thinking people. It is intelligent and creative. It is a much saner way to evolve a testing project.
| Michael Hackett Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University. |
