TestArchitect for Visual Studio is a keyword authoring platform extension designed
specifically to enhance coded UI test automation in Visual Studio 2012.
Microsoft’s Visual Studio’s ALM solution helps organizations manage the entire lifespan of application development and reduce cycle times. With the introduction of coded UI as an integrated component of Visual Studio® in 2010, Microsoft offered a built-in alternative to third party testing tools.
The coded UI framework in Visual Studio has record and playback functionality for test case creation as well as the flexibility to write tests in code. Incorporating a coded UI test platform in Visual Studio helped expand the ability for team collaboration but the burden of maintaining coded UI tests creates limitations for continuous builds and large-scale testing needs.
Modern Keyword-Based Testing
The TestArchitect test authoring platform gets its roots from the Action Based Testing (ABT) approach pioneered by Hans Buwalda over a decade ago to address the challenges of testing increasingly complex software.
ABT uses actions (defined by keywords) that serve as building blocks for test cases. Actions can be used to create test cases, and combined to create more complex actions. Both actions and test cases are reusable so test development becomes faster with every test.
While there are similarities in the technical implementation of keywords, ABT embodies a test design methodology and a production process that focus on the three essential pillars:
- Test system design: Effective break down of the tests.
- Test module design: Right approach per test module.
- Test case design: Right level of test abstraction.
This framework makes it possible to create tests that are far less fragile when applications change. The benefits are long-term maintainability and high reusability of tests.
A significant advantage of TestArchitect’s modular building block approach to test design is that actions, interfaces and test lines are all separate. When requirements change or a new platform is added, only minimal modifications are needed to maintain the tests. This also makes it possible to design and write tests before functionality is complete. For teams that have embraced Agile practices or operate in rapid development environments, the ability to create tests early adds efficiency.
Test cases are created from a series of keywords with arguments. The automation focuses not on automating test cases, but on automating the keywords. Since there will be fewer keywords than test cases, and keyword implementations tend to be shorter than scripted test case implementations, the automation effort is faster and much more manageable.
In a well-designed keyword based test suite, only a limited number of keywords have to be maintained when an application changes. Apart from the more manageable and maintainable automation, a keyword test is more readable and becomes self-documenting compared to a verbose format where test steps are described in natural language.
All test development and execution in TestArchitect is based on test modules. Test modules are work files—very specialized spreadsheets—where tests are created. Test modules contain the following elements:
- Objectives – A statement of what must be tested in the software under test.
- Test cases – One or more actions followed by one or more checks, usually simulating end-user interaction with the software under test.
- Actions- A series of operations and checks.
Test modules act as containers for a group of test cases with a similar, narrowly defined scope. Proper organization of test modules is an essential factor in realizing efficiency gains.
Test modules should be well defined and differentiated from each other. Test cases within a test module can have dependencies among themselves, but test modules should be designed to run independently so that all actions, checks, etc. directly relate to the scope of the test module. This makes it easy to keep abreast of large tests and assure functional changes of a system under test only have a localized effect on the test set.
Each test module contains a list of test objectives – one for each test case. Test objectives serve to further refine the scope of the test module. These allow a reader to understand why test cases are designed the way they are, and give a quick insight in correctness and, more importantly, completeness of a test.
A key benefit of well-designed test modules is readability. Stakeholders with relatively little knowledge of technology or of testing details can easily understand a test. The details needed to execute the test (like which menu item to choose to open a dialog) are not visible in the test module unless they are important to the scope of that test module. Test details are handled by actions, which themselves are maintained in an action library.
The other essential elements are actions. Actions are re-usable containers of a series of operations and checks. Instead of specifying operations and checks in detail in the test module, the tester only has to enter the action and its arguments on a line, Arguments define input values and expected outcomes.
Actions are best regarded as a byproduct of the test design. Testers essentially define the actions and their arguments. While this makes it possible for testers to create both the test and automation, it’s best to collaborate with the automation engineer when developing tests for larger scale automation.
Actions need to be organized and managed well to avoid having more than one action for more or less the same operation. Actions should be clear and unambiguous and should be well documented (meaning, arguments and their default values, etc.).
Definition of “Action” in TestArchitect
Actions are characterized in the terms of levels:
- Low-level: UI related – “click” or “select” (many of these are predefined in TestArchitect).
- High-level: business related – “enter customer” or “check balance”.
- Intermediate levels: such as “enter name and address” or “enter car details”.
Many low-level actions, like “click”, “enter” and “check” come pre-programmed and are located in the action library in TestArchitect for Visual Studio. Low-level actions are application agnostic and apply to a wide range of UI objects. For example the action “click” can activate a button in a window, or it can also activate a hyperlink in a web page.
Intermediate actions typically address one screen in the application under test, and are the building blocks for higher level actions like “enter customer”.
There are a couple of ways to create higher level actions. Actions can be created by programming in C#, but the need to program actions has been minimized. The “action definition” feature allows defining how the new action will work by using existing actions. The action definition feature is the preferred method to create actions that don’t contain complicated technical functionality since it’s much simpler than coding. Replacing automation scripts with keyword-driven actions makes it possible to develop the majority of tests without programming so only the few highly technical test routines need to be coded.
In addition to simplifying the development, execution, and maintenance of tests, TestArchitect for Visual Studio enhances the testing and test-automation development lifecycle. It seamlessly integrates with Microsoft Team Foundation Server (TFS) to define testing based on the same team projects that other areas of the organization use. Test cases created in TestArchitect can be run as part of a test plan in Microsoft Test Manager (MTM) by associating automation with the test cases.
Test Automation for Today’s Software
The sophisticated functionality of today’s software combined with fast-paced agile development environments, compressed release cycles, and platform proliferation makes large-scale software testing extremely complex. TestArchitect for Visual Studio simplifies coded UI testing by replacing coded UI scripts with keyword-driven preprogrammed actions. By speeding the development, execution, and updating of coded UI, programmers and automation engineers have more time to focus on critical tasks making the team as a whole more productive. Increased test coverage, decreased testing time, and the ability to release software that will deliver the expected customer experience are the benefits that TestArchitect for Visual Studio delivers.
About TestArchitect
TestArchitect for Visual Studio is now available for anyone using the Ultimate or Premium versions of Visual Studio 2012. You can request a trial of TestArchitect for Visual Studio by going to www.testarchitect.com.
