With the new year just around the corner, here’s a look at the Test Automation trends that have the potential to dominate.
DevOps is being relied upon more than ever. With there being strong Market Drivers for the adoption of DevOps, the need for Test Automation has also never been greater. But what’s next after DevOps ?
Well, new trends come year after year and some of them fade away pretty quickly. But we’re here because we don’t want to miss the ones that stay and reshape the industry.
In this article, I’ll lay out the driving factors, trends related to Test Automation, as well as the changes in technology that are impacting us as we head into the new year, and beyond. In addition, I also give my observations and predictions for many questions that you might have such as:
- What AI can and cannot do to solve Test Automation problems?
- Progress Web Apps is coming and what does it mean for Test Automation?
- Is Selenium-less Automation the right solution?
- How will the new trends and technologies in Test Automation affect your organization?
However, before we jump into the exciting things for 2019, let’s take a look at what’s happened in 2018.
CURRENT STATE OF TEST AUTOMATION IN 2018
Automation is Still The Main Driver Towards Achieving DevOps
The increase of Continuous Delivery adoption continues to be the main driver for Automation today, so much that it surpasses the main benefit—the perceived ROI of automation itself. Many companies adopt automation because they need to do DevOps, which is part of their CTO’s agenda.
These processes rely so heavily upon Test Automation that they would not exist or function if they were to become broken or disappear. Without them, there is no Automation, no Continuous Delivery, no DevOps, none of it. Broken Automation means broken Continuous Delivery. Automation has become the top barrier to adopting DevOps. This has led new software to improve Automation overall and reduce the chances of any problems occurring.
Selenium 4 Is Coming Out As a W3C Standard
If you haven’t heard already, Selenium WebDriver is the leading Test Automation Tool.
The shift to open source has instilled the idea that companies are increasingly looking for freemium options to meet better customization for their needs, as well as the obvious benefit of saving costs.
Finally, Selenium 4 with updated protocol that meets and becomes W3C Standard is coming out this December. In addition to protocol changes in WebDriver, there will be improvements for Selenium Grid and Selenium IDE. From now on, the actual Automation interactions (find element, click, etc.) are officially responsible by the browser vendors.
Increasing Coding Skill Expectation for Testers
Because of the popularity of Selenium—like it or not, QA testers are expected to know coding. This isn’t necessarily a bad thing, but it does leave current testers without this skill set scrambling. Ultimately, it’s better for the future of testers and their employers that they can code. And testers know that this will be better because this can help them step up their skill and their career to be a SDET (Software Development Engineer in Test).
However, this is something easier said than done. In Agile environments, testing teams never have enough time to test, let alone time to absorb new Automation and coding knowledge. This gap could be bridged by the right tools and methods that support different levels of technical skill within a testing team.
In addition to coding, testers need to have a good understanding about how underlying systems work as well as good Test Automation design to avoid test flakiness.
AI-Powered Test Automation Tools
We are in the midst of a paradigm shift, where businesses across all industries and sizes on a global scale are trying to adopt AI for its competitive advantages. The Test Automation industry is no exception. There is a wave of new Automation Tools that have identified themselves as “AI Automation” or “Machine Learning Automation”. If you are unfamiliar with these terms, you can peruse our glossary at the back of this issue.
At this time, most of these new tools are focusing on using AI to solve issues around locating web elements. They tout features such as tests record & playback under industry jargons such as “codeless Automation” or “self-healing Automation”. Some tools also have the capability to use NLP (Natural Language Processing) to “translate” Manual Test cases written in plain text to automated tests.
However, when it comes to modifying generated Automation scripts for future reuse/parameterization, things get less exciting very quickly. Some of these tools won’t let you do it. Some offer this capability through UIs which is very tedious and unproductive compared to a real IDE.
It’s very interesting to watch how these tools will mature and create breakthroughs in Test Automation.
TRENDS TO WATCH IN 2019
AI-Powered Test Automation Tools in 2019: Make or Break?
Since these tools have been in the market for a while now, 2019 is the year for them to make or break. Let me explain why.
Historically, using AI in Test Automation is not new. Computer vision technologies, which are part of the AI umbrella such as OCR or KeyPoint, have been used for a long time for UI element identifying purposes. These technologies are recommended as alternative/workaround solutions when the Test Automation tool is unable to recognize those UI elements at the object level. There are two reasons for this:
- Firstly, image-based element recognition is super slow compared to object-based recognition.
- Secondly, test flows that depend on visual recognition are very easy to break when the representation of the application changes-which happens all the time.
Think and decide which method is more reliable and the most effective way to identify a button: using its unique ‘id’ assigned by a developer or using its visible text. You know, accuracy, speed, and reliability are what makes Automation, Automation.
As AI technologies get better every day, the potential of AI shouldn’t be stopped at merely locating the web elements. In my opinion, AI, especially Machine Learning and Natural Language Processing, has the potential to solve data-related problems for Test Automation. Such results would include the following:
- Reduce test failures due to UI changes in an automatic way by proactively detecting UI mapping changes at both UI and code level
- Select and execute impacted tests for new code commits within a CD pipeline
- Save testers time for automatic aggregating and analyzing test results, especially for projects with thousands of test runs every night
- Generate automated tests based on different sources: plain text test cases, application log files, test results, etc.
Prediction: In general, AI can help us do the actual work or it can provide relevant information to help us make better decisions. To me, the latter would be the case for Automation: AI will play a key role in drastically improving an Automation engineer’s productivity by improving data-driven recommendations, but not doing the actual UI interaction works.
Bonus Prediction: Unfortunately, the question “Will AI replace Test Automation engineers?” will be asked by no Test Automation engineers, ever.
The Powerful Web Just Got Better
With new technologies such as Progressive Web Apps or Web Bluetooth, web applications can deliver great user experience just like mobile native applications.
According to Wikipedia, “Progressive Web Applications (PWAs) are web applications that load like regular web pages or websites but can offer the user functionality such as working offline, push notifications, and device hardware access, traditionally available only to native mobile applications. PWAs are an emerging technology that combine the open standards of the web offered by modern browsers to provide benefits of a rich mobile experience”.
Open source projects like Web Bluetooth and Physical Web allow web applications to interact directly with mobile devices through communication standards like Bluetooth Low Energy (BLE).
Prediction: PWA will gradually take over mobile native apps because it’s much easier for developers to deliver great and consistent user experience across devices. It’s good news for mobile Test Automation as the web is so standard that there’s much less effort needed to worry about device coverage. On the other hand, test plan for web Test Automation will need to extend to cover Bluetooth and hardware communication testing.
JavaScript Automation Can Only Be More Popular
JavaScript is the most downloaded binding language of WebDriver in the last few years. It’s a logical result of the popularity of JavaScript and NodeJS.
New application development projects today use JavaScript frameworks. The language choice for Test Automation simply follows that trend.
Now that the new WebDriver 3.8 is on the way out for public release, it will make perfect sense to use JavaScript to automate the browser. This is due largely to the fact that it is already shipped with the browser, and like we mentioned earlier, it also meets W3C standard.
Prediction: JavaScript (or its super set TypeScript) will be the definitive scripting language for new Automation projects.
Selenium-Less Automation
As always happens in business, people are already talking about Selenium-less Automation. One example is Cypress. It has developed its own JavaScript-based Automation library which runs inside the browser instead of using WebDriver. Its main advantage is the ability to access internal objects/models of the application under test, which means more flexibility to set up an application state for the tests. It also runs faster than WebDriver because it does not have a “proxy” layer like Selenium Server to talk to.
Prediction: We will see increasing adoption for these non-Selenium WebDriver tools by developers. What we don’t know yet and hence, will be its viability. This is dependent on if it gets an eco-system of third party tools to support them from behind—a must have for large projects, and large-scale adoption.
Blockchain Testing
If you’ve been reading headlines, you may have heard about Bitcoin or Ethereum. These cryptocurrencies are typical applications of blockchain technology and get the most press, but that doesn’t mean you should rule out blockchain. Blockchain has a much wider range of universal applications across industries such as financial, logistics, health care, etc.
According to Wikipedia, blockchain is “an open, distributed ledger that can record transactions between two parties efficiently and in a verifiable and permanent way”. And by design, “a blockchain is resistant to modification of data”. The main use case for blockchain is when there are needs to remove the single point of failure a.k.a. “Decentralization”.
Companies such as IBM, Oracle, and Amazon have launched their products for the blockchain market. Even though this technology is currently in “Early Adopters” phase in the hype cycle, we expect to see more and more blockchain applications being developed.
Testing for blockchain applications is very different. Setting up a test environment is very challenging. It also requires extensive understanding about security, law, exchange of value, distributed programming, and more.
Prediction: Blockchain adoption in businesses/enterprises will increase and Test Automation can fit very well in this environment.
Failure Injection Testing (Chaos Engineering) is Going to be A New Wave of Testing
According to http://principlesofchaos.org, a community initiated by Netflix, “Chaos Engineering” is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production, hence the need for Failure Injection Testing.
Chaos Engineering practitioners use tools like Simian Army to simulate failures to test the reliability, security, and resiliency of their production systems. Simian Army is able to perform tasks from health check for the servers, to totally simulate a service unavailability situation of certain Amazon zones.
Prediction: Currently, this type of disaster simulation has been done mainly by SysAdmin or DevOps Engineers. In the future, I expect to see this special kind of testing to be done by testers that are supported with a tool out of the box.
Test Results Management with Elastic Stack
For larger Automation projects that produce thousands of test results every day, you probably have some reporting and dashboard solution in place to help you analyze and manage those results. This capability is typically provided by traditional test management software which is not designed for Test Automation. These tools work well with standard and primitive data such as test case status but they can’t help you identify Automation errors, whether they are script issue or because of application changes.
If it sounds familiar, you need something like Elastic Stack, formerly ELK Stack. It is a collection of three open source projects: Elasticsearch, Logstash, and Kobana, that allows you to collect, search, analyze, and visualize the logs that are consolidated from different machines.
Using Elastic Stack you can analyze test failures much quicker because test results have been consolidated and are ready to be analyzed using powerful search and analytic capabilities. Elastic Stack can be very useful when you know what you are looking, from analysis to trends, but it is perhaps even more useful when you don’t. It can detect the uncommonly common or statistically unusual test result data. These statistical anomalies are usually indicative of something interesting that needs your attention.
Automatic root cause analysis is possible by combining elastic search with AI technologies such as Machine Learning and Natural Language Processing. When done right, it can save you time, by a factor of X.
Prediction: Elastic Stack or similar log management platforms will be widely adopted by the Test Automation industry.
Intelligent Automation Utilities will be on The Rise
While there are many tools in the market, more often than not, they are silo-ed. Open source tools and frameworks are getting more powerful and easy to extend. However, many don’t have a lot of out-of-the-box support for teams that have multiple levels of technical skills. Meanwhile, commercial tools tend to have more ready-to-use features for test design and test management. Nevertheless, they don’t work with each other or even with WebDriver in some cases.
Let’s say I like the recording feature of tool A, the model based testing feature of tool B, the Action-Based Testing feature of tool T and Selenium Automation. I want all those things to work together, along with the DevOps techs that my developer folks selected. How is that possible?
Prediction: Given that WebDriver is becoming a W3C standard, and with advancements in open source software and AI, I think that there will be a host of small utilities that are built around WebDriver to improve Selenium (and the like) experience across the Test Automation cycle. Areas include, but are not limited to: test management, test design, page objects repository, autonomous test execution, test result management, etc.
Conclusion
Expect to see more new and exciting methods and tools, perhaps, delivered in smaller packages. SDET (Software Development Engineer in Test), full-stack Test Automation, or technical test engineers will continue to be in demand, more than ever. These key trends including Test Automation for DevOps, API testing, AI, JavaScript Automation, et al all collectively require new technical expertise that only a modern SDET type of tester can fulfill. QA organizations today will need to reconfigure the existing teams while trying to bring in more SDET to be well-prepared into the future.
New trends come year after year and I hope you find them helpful and inspiring.
Some trends stay to reshape the industry and some of them fade away pretty quickly. Take the time and ask yourselves which Test Automation trends will be most important to you and your organization in the year ahead?
If you find these trends helpful and are ready to level up your Automation effort, talk to us.
Request More Information
Do is passionate about understanding and solving complex test automation projects to help organizations continuously deliver higher quality software faster.
