As our world continues its digital transformation with excitement in the advancement and convergence of so many technologies- from AI, machine learning, big data and analytics, to device mesh connectivity, nor should we forget VR and AR- 2017 promises to be a year that further transforms the way we work, play and take care of ourselves through product innovation.
We have already seen the application of these technologies in robotics, digital assistants, self-driving/autonomous vehicles, wearables, IoT’s, drones, etc. The product innovation based on these technologies will help enhance how we learn and improve ourselves from basic skills to playing sports and musical instruments. The applications are everywhere, from family, kids, fitness, to gaming and education, and a lot more. What does it mean for software testing? I believe testing will continue to help fuel new and disruptive product innovation for years to come. The rising user demands and adoptions for accessing data and decision-support analytics anywhere, anytime, on any device, coupled with smart things, powered by artificial intelligence (AI) are stimulating fresh changes to the testing world.
On the business end, software technology companies continue the journey into a leaner and faster delivery organization. Delivering new product features and updates to customers rapidly without compromising quality will continue to drive the adoption of Agile development plus DevOps’ Continuous Delivery (CD) and Continuous Testing (CT) practices. With these forces of change, having the right mix of software testing competency and QA capabilities, aligned with technology and practice revolutions, will help transform companies’ competitive advantage as well as their readiness to respond to customer demands in the new, intelligent world order.
It’s time to plan for the future, and look at how technology trends and practice transformations will impact the trends in software testing and QA in 2017. Be ready for some huge adjustments and great changes.
Last year, I saw an increase in CD/CI adoption as part on DevOps transformation in many organizations because of its big benefits. DevOps is still a very hot practice that all Organizations want to achieve. In order to maximize the concept of Continuous integration, Continuous Delivery & Continuous Testing, each of the steps within DevOps must be fulfilled. So, I think this adoption continue to accelerate, along with other trends.
- Extreme Test Automation
With the advent of continuous integration, continuous testing and continuous deployment, the need for increasing test automation coverage will be at the forefront of this movement. Test execution turnarounds must be faster and bugs must be identified quicker. More importantly, automation of integration testing in development, during iterations sprints will have to be smoothly executed in each cycle of continuous integration.
Test automation will evolve into a product that can be reused in the continuous integration pipeline. In fact, it will be the most laborious, on-going production effort of continuous delivery—this is because it will directly tie to product functionalities and changes. Having a robust strategy to maintain a test automation asset will be essential. Expect to see more innovation in test automation. With a demand for these skill sets, I predict there will be a shortage in the test automation workforce, including engineers, coders, and test engineers who can automate to handle the high demand.
- Automated Test Design Gains an Edge
There were many discussions back and forth about this topic in 2016: Will we continue to see progress in automation of test design and test authoring? Much of our test design and authoring today is still a manual production. The main question was if the mindset of an automation engineer is shifting from how to automate execution to how to design intelligent programs to smartly automating tests. For me, I can see automated test case design/ automated test generation continue to advance with the application of model-based, fuzzy logic, artificial intelligence, machine learning, etc.; which will all contribute the efficiency of test authoring production.
- Dumb Monkeys are Getting Smarter
On one hand, I expect to see introductions of testing support tools to help test AI/ML applications and systems. On the other hand, a development of some applications using AI/ML algorithms to improve test automation including the use of bots is probable. While using bots for testing is not new (i.e. monkey testing), and AI/ML application to testing is still in its infancy, I predict there will be more introductions of AI/ML to smart bots that emulate user-generated events and data inputs in software testing automation. Another area of application as discussed earlier is in automated test case design and generation based on existing test data.
- Containers and Microservices Will Have a Huge Impact on Testing
Containers will help reduce configuration and compatibility testing demand, and microservices will help break testing down into smaller units as test automation continues to Shift-Left* from the UI toward the services/API layer. With the benefits of shifting the testing to the services level, they also come with some new changes. For example, despite the benefits of containers and microservices, configuration/compatibility continues to be a laborious task due to the fact that we have to deal with the permutation of microservices versions currently deployed. When we do integration testing with other services, would we test against the versions of the other services that are currently in production, or against the latest versions of the other services that are not yet in production, or both? The challenges also come with figuring out how to improve reusability of test automation across the different language, technology used in the microservices.
- Security Testing needs a solution for IoT progress
The biggest issue in IoT testing continues to be in security testing. IoT testing requires testers to possess additional technical knowledge beyond domain expertise. Testing the interactions between layers—application, middleware, data, etc. – will require some grey box testing approaches. IoT services/APIs are most likely to be virtualized and, in turn, it will help testing to be done effectively. Grow knowledge and skill testing on virtualized environments.
- Basic Application Security Testing Becomes Mainstream
Lightweight security tools, particularly dynamic security testing tools, and security testing frameworks (while static analysis tools are used by developers) will gain more popularity (e.g., Nmap for port scanning, sqlmap for SQL injection; Zapr and Arachni for security scanning; ClamAV; bundler-audit, Node, or SafeNuGet for checking external dependencies for known vulnerabilities). Security testing frameworks (e.g., BDD security testing, ATF, OWASP, etc.) will provide a common way to specify and validate security scenarios.
- Tackling Testing Applications in the Cloud
Putting Testing applications in the cloud continues to be problematic when dealing with multiple interfaces. Additionally, services mix with applications that run on a combination of private, public, hybrid and on premise clouds. The upcoming trend of serverless apps for mobile with cloud computing platform such as AWS Lambda, as well as adoption of app stream will produce more new explorations and approaches to testing cloud-based mobile apps.
- Testing Mobile Applications Continues to Advance
Similar to Application testing in cloud service, Mobile testing is also moving to the cloud but it’s having its own hurdles. While mobile emulator and device clouds (though they can be expensive) help elevate the challenges of having the right mix of devices and configurations available in-house, having the right test environments continue to be problematic. For one, the rapid pace of technology changes and updates force the applications and testing to update even when there is no new feature introduced. Your test automation will also have to be updated accordingly. Furthermore, having the right tools and expertise to test the mix of mobile and variety IoT devices proves to be an on-going difficulty for testing. Functional testing and security testing will continue to be a challenge in testing multi-channel devices (mobile, wearable, social and traditional), and mesh and conversational systems. As mobile apps can be easily constructed using reusable APIs as well as apps can be easily created by citizen developers, there will be more testing demand.
- Demand for Full Stack Testing
One will need to fill the knowledge/skill deficiency to be productively contributing in CD/DevOps initiatives. Those skills include test environment and virtualization, provisioning automation, test data setting up and management, application quality monitoring, programming and software engineer (in CI engineering particularly), test practices such as TDD, BDD and ABT, and basic non-functional testing skills (performance and security); expanding test automation knowledge and skills toward CI, CT and CD automation.
- Manual Testing is Not Dead
While test automation coverage continues to be critical to rapid product releases, hence time to market, with the continuation of digital transformation user experience, an area that is more challenging to automate, manual testing continues to be the key to success. Hence, human/manual testing will continue to be in demand. IoT testing requires testers to possess additional technical knowledge beyond domain expertise. Intelligent things, as Gartner has predicted to be part of 2017 technology trends, such as intelligent IoT’s, robots or drones will need testing by human testing beyond automation. Furthermore, with complex testing, humans must test to give it a green light before bots take over. In-sprint testing, agile development will continue to be manual. Gartner also predicts that Augmented Reality (AR) rather than Virtual Reality (VR) will begin to gain traction. AR applications will need manual testing for visual validation until automation advances and takes over.
At the end of the day, 2017 is the year that we will see a whole new world of product innovation that accentuates the digital revolution. As a data point, I just came back from CES 2017. This year it ended with more than 3,800 exhibiting companies, of which more than 600 are start-ups, and more than 175,000 participants; it’s the biggest ever. I can only predict what is to come based on my observations and experience. I hope that these top testing trends may help you strategize and navigate where your testing should be heading in the New Year. The important thing to note is that we must be ready to adapt and grow. The world is changing rapidly. Our software testing world will have to change and we must be ready to do our part in this journey.
