Every year, LogiGear Magazine devotes one full issue to Test Automation. We could do more than one, and perhaps even that would not be enough.
The problems around automation have become increasingly complex. And now, automation is much more integrated into the software development process. For over a decade teams have been faced with “do more with less”: do more testing, do more automation, and do it all with less staff. Then Agile/Scrum came along and we had to automate it faster. As the XP practice of continuous integration (CI) caught fire, our automation suites – smoke tests and full regression suites – got integrated into the autobuild development process, which in most cases was out of our control. Other people and tools are now running our automation and reporting back results – not by us kicking off automation when we choose to, but whenever a build takes place.
Today this process is moving at an even more extreme pace and further away from us. We see CI moving onto virtual machines and DevOps running our automation all the time (continuous testing), on all kinds of environments.
Many teams are still struggling with getting automated test into their current sprints, or Sprint +1 (getting new functionality automated, but only in the sprint following that function’s development). Some teams struggle just to get more tests automated in their development cycle at all, and end up settling for adding new automation after a release, because they just do not have the time. This is not OK. If this is your situation, you need to fix it. It may not be an easy fix, but not fixing it has a negative impact on development.
What do we have to do?
- First, automate more and automate faster. With shorter cycles, you need automated tests, or you will never reach levels of coverage acceptable enough to have confidence in your product. Yes, automate faster.
- You need a framework with reusable and low maintenance functions.
- Finally, choose effective methods. We all know the idea that tests need to be low maintenance. But how do you do that? When you have a big suite of tests and some break – and not because of application bugs – how do you unbreak the test suite to run again? Simply automating step-by-step test scripts is a surefire formula for failure. Instead, choose a more sophisticated method for developing tests, like Action Based Testing.
Our tests have to be effective at validating functionality and finding bugs or breaks. And they must be efficient – suites should do this in the minimum number of tests possible.
We know that our tests are going to be run, in most cases these days, across a large matrix of configurations, browsers, devices, and appliances. In addition, now the tests will more than likely be run on a variety of build environments. It is becoming increasingly common to run the same suite of tests on a dev environment, testing environment, user acceptance or staging environment, and sometimes live/production environments. For some tools and suites, the performance demands are too great: the tool itself becomes an issue, not just the suites it runs. I myself have used some tools that develop huge problems running tests as the number of virtual machines increases. And that is only the start.
Our automation has to get better. But more automation is not always the answer. Today, the answer must be: better and faster automation. I hope this issue of our magazine gives you valuable guidance to achieve this.
We’ve also just published our 2016 editorial calendar, to give you an idea of what’s ahead for next year. As always, if you’d like to submit an article, just let us know.
All of us at LogiGear wish you a joyful and healthy holiday season and a happy new year. We look forward to continuing to provide you with great software test information in 2016!
