Book Review: Continuous Testing for DevOps Professionals

For this month’s book review, I read Continuous Testing for DevOps Professionals: A Practical Guide from Industry Experts, by various authors and edited by Eran Kinsbruner. The book is divided into 4 sections: Fundamentals of Continuous Testing, Continuous Testing for Web Apps, Continuous Testing for Mobile Apps, and The Future of Continuous Testing.

The Fundamentals of Continuous Testing section was my favorite because it focused the most on developing a good Continuous Testing strategy and the elements required. In Continuous Testing for Web Apps, strategies for testing Responsive Web Applications (RWAs) and Progressive Web Applications (PWAs) were discussed, along with cross-browser testing strategies. In Continuous Testing for Mobile Apps, chapters included strategies for testing React Native apps and chatbots, as well as tips for using tools like Appium, Espresso, and XCUITest. Finally, The Future of Continuous Testing took a look at the uses of AI for Continuous Testing, as well as strategies for testing IoT-enabled devices and Over-the-Top devices.

Since this book obviously covered a lot of ground, I’ll focus on my favorite section:
“Fundamentals of Continuous Testing.” Contributor Yoram Mizrachi says there are 3 types of Automated Testing failures: test code issues; test lab problems, such as an unstable test environment; and execution problems, such as not enough platforms available to run the tests. There has been much written about solving test code issues, but not enough about solving environment and execution problems, so I was happy to see the suggestions in this book. To solve environmental problems, Brad Johnson suggests using containers such as Docker and Kubernetes to spin up environments for testing. Because these environments are temporary, they can be completely controlled in terms of data and application state, so there’s less chance of test failures due to environmental problems. And Genady Rashkovan offers a solution for execution problems through setting up an automatic detection system for system failures. After gathering initial data, this detection system can be programmed to predict when failures are about to happen, and execute an automatic reboot or spin up a new VM to mitigate a failure before it happens. 

I also found Tzvika Shahaf’s chapter on using smart reporting very insightful. He notes that test data reporting is often siloed: reports on UI tests use a different format from the reports on API tests, which are in turn different from the reports on performance tests, and so on. This makes it very difficult for managers to get a sense of the health of the application. Shahaf recommends creating a unified report for all tests using this process: tag events so they can be easily identified, normalize the test data so it can be used by a single report, correlate events so similar tests are grouped together, and finally display the events with relevant artifacts. He advises reducing the noise of defects by determining what the most common causes are for test failures and removing the failures that are false negatives. For example, a test failure that was caused by the test environment going down does not actually indicate that something has gone wrong with the software, so a test report designed to show whether a new code is working correctly doesn’t need to display those failures.  I recommend Continuous Testing for DevOps Professionals for anyone who is working on creating a Continuous Testing system for their application. There are suggestions for Test Automation strategies, solving common Mobile Automation problems, testing connected devices, creating reliable test data, and much more. My one complaint about the book was that the Kindle version was formatted poorly: the chapter divisions were unclear, there were often footnotes in the middle of the page, and diagrams were broken into pieces over 2 or more pages. For that reason, you may want to purchase a paper copy of the book. But in spite of these problems, I found the book to be very valuable.

Kristin Jackvony
Kristin Jackvony discovered her passion for software testing after working as a music educator for nearly 2 decades. She has been a QA engineer, manager, and lead for the last eleven years and is currently the Principal Engineer for Quality at Paylocity. Her weekly blog, Think Like a Tester, helps software testers focus on the fundamentals of testing.

The Related Post

LogiGear University announces the launch of a new, free video series on Testing in DevOps and Continuous Testing which is available today.
DevOps may be the next big buzzword, but Test teams really need to focus on its little sister, Continuous Delivery If you pay attention to trends in software development—from the perspective of what some sophisticated teams are doing, what articles and books are being written, to conference topics, you may have noticed the tools being ...
  LogiGear_Magazine_June 2016_Testing in the New World of DevOps  
Times have changed, the tools have improved, and with books like this available you have no reason to not give CI a go. I still remember the first time I was on a project that used NAnt and CruiseControl.NET. It was years ago and both were new tools with plenty of bugs. The project manager ...
Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
LogiGear Magazine June Issue 2018: TESTING in DEVOPS
The book is an incredibly effective and valuable guide that details the risks that arise when deploying cloud solutions. More importantly, it provides details on how to test cloud services, to ensure that the proposed cloud service will work as described. It is a great start to the topic. The 6 chapters detail a paradigm ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
In this article, I share some of my experiences and observations on training teams, mainly in corporate settings. The operative word here is “team”, not “individual”. When training teams or groups in an organization, many of the considerations and benefits are different than those for the individual. We’ll examine those differences and I will share successful solutions. ...
…On what you need to know before making the transition to EaaS 1. What are the main differences between cloud-based environments and cloud infrastructure? An environment is a collection of infrastructure elements working in conjunction to enable an application stack to work. For example, a simple 3-tier application, with a web front-end component, a business logic ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe