Book Review: Continuous Testing for DevOps Professionals

For this month’s book review, I read Continuous Testing for DevOps Professionals: A Practical Guide from Industry Experts, by various authors and edited by Eran Kinsbruner. The book is divided into 4 sections: Fundamentals of Continuous Testing, Continuous Testing for Web Apps, Continuous Testing for Mobile Apps, and The Future of Continuous Testing.

The Fundamentals of Continuous Testing section was my favorite because it focused the most on developing a good Continuous Testing strategy and the elements required. In Continuous Testing for Web Apps, strategies for testing Responsive Web Applications (RWAs) and Progressive Web Applications (PWAs) were discussed, along with cross-browser testing strategies. In Continuous Testing for Mobile Apps, chapters included strategies for testing React Native apps and chatbots, as well as tips for using tools like Appium, Espresso, and XCUITest. Finally, The Future of Continuous Testing took a look at the uses of AI for Continuous Testing, as well as strategies for testing IoT-enabled devices and Over-the-Top devices.

Since this book obviously covered a lot of ground, I’ll focus on my favorite section:
“Fundamentals of Continuous Testing.” Contributor Yoram Mizrachi says there are 3 types of Automated Testing failures: test code issues; test lab problems, such as an unstable test environment; and execution problems, such as not enough platforms available to run the tests. There has been much written about solving test code issues, but not enough about solving environment and execution problems, so I was happy to see the suggestions in this book. To solve environmental problems, Brad Johnson suggests using containers such as Docker and Kubernetes to spin up environments for testing. Because these environments are temporary, they can be completely controlled in terms of data and application state, so there’s less chance of test failures due to environmental problems. And Genady Rashkovan offers a solution for execution problems through setting up an automatic detection system for system failures. After gathering initial data, this detection system can be programmed to predict when failures are about to happen, and execute an automatic reboot or spin up a new VM to mitigate a failure before it happens. 

I also found Tzvika Shahaf’s chapter on using smart reporting very insightful. He notes that test data reporting is often siloed: reports on UI tests use a different format from the reports on API tests, which are in turn different from the reports on performance tests, and so on. This makes it very difficult for managers to get a sense of the health of the application. Shahaf recommends creating a unified report for all tests using this process: tag events so they can be easily identified, normalize the test data so it can be used by a single report, correlate events so similar tests are grouped together, and finally display the events with relevant artifacts. He advises reducing the noise of defects by determining what the most common causes are for test failures and removing the failures that are false negatives. For example, a test failure that was caused by the test environment going down does not actually indicate that something has gone wrong with the software, so a test report designed to show whether a new code is working correctly doesn’t need to display those failures.  I recommend Continuous Testing for DevOps Professionals for anyone who is working on creating a Continuous Testing system for their application. There are suggestions for Test Automation strategies, solving common Mobile Automation problems, testing connected devices, creating reliable test data, and much more. My one complaint about the book was that the Kindle version was formatted poorly: the chapter divisions were unclear, there were often footnotes in the middle of the page, and diagrams were broken into pieces over 2 or more pages. For that reason, you may want to purchase a paper copy of the book. But in spite of these problems, I found the book to be very valuable.

Kristin Jackvony
Kristin Jackvony discovered her passion for software testing after working as a music educator for nearly 2 decades. She has been a QA engineer, manager, and lead for the last eleven years and is currently the Principal Engineer for Quality at Paylocity. Her weekly blog, Think Like a Tester, helps software testers focus on the fundamentals of testing.

The Related Post

Over the years we’ve provided an extensive number of articles, videos, and infographics that provide a wealth of knowledge about Continuous Delivery.
Making the leap to CT is easier than you think— follow this guide to transform your testing process No pain, no gain! Achieving Continuous Testing shouldn’t take a “Hans and Franz” attitude. It should be painless, more like a natural progression from implementing certain practices over time.
Throw away clunky hyper-visors, and stop thinking about computer hardware and software license during your development projects. The first thing you think about when you hear “The Cloud” may not be development and testing. The Cloudy market is filled with SaaS applications, hosting, and cloud-based file systems. All are very useful, and offer a clear ...
Sauce Lab’s Perspective on Integration with Continuous Testing The term ‘DevOps’ implies that implementing an effective development and production workflow is as easy as developing the collaborative interaction between developers and IT operations. On the surface, DevOps may appear as a simple and straightforward idea but, as you will find out, there is more than meets ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
Fitting QA into a modern DevOps group In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities. However, for folks outside the engineering team – say in Operations ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
DevOps has been described as Agile on Steroids; DevOps has also been described as Agile for Operations/IT. I like both of those descriptions. Many organizations want Development, Test, and Operations teams to move to DevOps now. DevOps is a big topic, but DevOps is not the focus of this article. We will not be talking ...
Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments. Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be ...
How to Adopt the “Third Way” in the Dojo’s Method to Master CD In The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations the authors describe “The Three Ways” – the underlying principles forming the basis for all DevOps practices. 
Times have changed, the tools have improved, and with books like this available you have no reason to not give CI a go. I still remember the first time I was on a project that used NAnt and CruiseControl.NET. It was years ago and both were new tools with plenty of bugs. The project manager ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe