To adapt with the fast software delivery pace in today’s world, the same automation test suites should be able to run on a range of various platforms: different Operation System versions, different Application under test versions, different localization versions, etc. All while keeping the test development and maintenance effort at the minimum.
However, in the testing field, we cannot simply evolve the tests to adapt to changing AUTs and environments, then leave the past behind. That is to say, every time we adapt a test to a change in a system or platform, we do not necessarily have the luxury of discarding the previous test. Multiple systems and platforms tend to remain valid as testing configurations. And taken together, these factors can combine to form a complex matrix resulting in a very large number of valid testing scenarios.
Take for instance, a commercial application that has gone through several upgrades (each of which remains on the market), has been localized for four different language environments, and has been designed to operate with the last four generations of the Windows operating system. A graphical representation of all the resultant possible testing scenarios might look like the following, in which each intersection of lines represents a different configuration.
TestArchitect, however, uses two types of variations which address the problem of requiring multiple test configurations. TestArchitect allows different versions of any given actions, interfaces, or data sets to coexist with each other and they are needed by your test for the particular combination of application version, operation system, environment, localization, and more, that your project may be called on to test.
The implementation of variations in TestArchitect takes two forms: linked variations and keyword variations.
Linked variations are the preferred method for addressing what might be termed progressive variability. This is the kind of variability that comes from continuous development or improvement of some aspect of the system/platform mix, such as progressive versions of the AUT itself, or progressive versions of an operating system in which the AUT runs. Linked variations are appropriate when the various versions of a system or platform can be depicted graphically in either a linear form or tree structure. For example, imagine a Car Rental application that has gone through several cycles of development, so that the version history takes this tree form, as is typical in software development:
System and Platform tree
Linked variation of action ‘back to home’
Keyword variations, by contrast, are perhaps most appropriate when we can define certain categories of distinctions between different system/platform mixes, where the differences are not due to any progressive development or refinement of any aspect of the mix. Different “flavors” of an application, such as different language versions, are a good candidate for keyword variations, as are other localization-based differences, such as differences in date formats, currencies, physical units, etc. Another possible keyword group might involve different market editions of the application, such as a student edition, personal edition, small business edition, or enterprise edition.
Variation of action ‘login’ to work on edition ‘enterprise’ and language ‘spanish’
Thanks to variations, beside the default environment, the same test suites can now run on Application under test version 1.1 and 1.2, and/or enterprise edition, and/or Spanish language. The test modules just contain keyword test steps and leave the environment factors for the settings at execution time.
