Book Review: Experiences of Test Automation

Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including both successes and failures.

Many books simply provide you the success path. This book also provides you with the steps you could possibly be taking that could lead to failure, helping you to change your path before failing.

The book starts with a nice overview of the case studies and an introduction to the key issues addressed by the case studies. Besides each case study being summarized, it also includes introducing the topics and pointing out the chapter they can be found in. They are broken down into management and technical issues. The management issues include Objectives for Automation, Management Support, Return on Investment and Metrics, Automation in Agile Development, Skills, Planning, Scope, and Expectations, Relationships with Developers, Triggers for Change and Getting Started, Tools and Training, and Political factors.

The technical issues covered include Testware, Abstraction, Architecture, Test Execution Tool, Automation Standards, Reusability, Documentation, Flexibility, Results and Reporting, Testing the Tests, What to Automate, Failure Analysis, and Finding Bugs.

The book includes a really nice table of Case Study Characteristics. Some of the characteristics include location, lifecycle (process used), number of team members, time span, tool types, pilot done, ROI measured, was it successful, and is the project still going on. This table really helps you hunt down topics you are interested in reading about first. The index of this book is really nice also. I mention that because I have seen some books lately where the publisher didn’t want to foot the bill for a nice one. That can be very aggravating.

The book’s last chapter is titled Test Automation Anecdotes. It is filled with experiences from the field that the authors felt were worth repeating, but did not constitute an entire chapter.

The book also has a nice table in the appendix that lists all the tools mentioned in the book. It includes which chapter they are in, where or not they are open source, and a link to the tool owner’s website.

I have repeatedly seen attempts at test automation fail for a verity of reasons. This book included them all: lack of management support, believing the tool is all you need, trying to automate tests without documenting them, and trying to automate every test. The level of difficulty and effort is almost always underestimated. This book definitely puts the level of effort into perspective.

Almost every story’s environment is unique. I really like the way the stories provide solutions to problems that could not be solved by simply purchasing a tool. These solutions are not the industry’s best practice solution, but rather home grown solutions to problems unique to their environment. Now that this book is out they may become best practice solutions. The primary thing they do is make you think out of the box. It is really refreshing to read such a real world book.

Every story is well written and well edited. This is one of the best resources available for helping expand your experience level without having to make the mistakes to learn from along the way. I wish this same format would be done with software projects in general; that is, with the same level of honesty. I see failing projects constantly being touted as successes. Buggy, over budget, late projects, are not a success. Kudos to those authors who stepped up to write about the failed projects!

The authors come from a wide range of technologists. You can check out the 28 case study summaries on the author’s web site.

I only have one word of warning. These stories suck you in. You may find yourself saying “Just one more”, and then suddenly looking at the clock and realizing it is 3 am.

All in all I highly recommend this book to everyone in the business of building software. Before you attempt to automate your testing, read this book! 

Tad Anderson
Tad lives in Mount Joy, PA with his wonderful wife Wendy and his faithful companion Artimus (4 year old Maltese). He has been programming since 1992 and specializes in Software Architecture. He is currently serving in the role of Enterprise Architect at Penn National Insurance.

The Related Post

Introduction Many executives have some very basic questions about Software Testing. These questions address the elements of quality (customer satisfaction) and money (spending the least amount of money to prevent future loss). The basic questions that executive have about Software Testing include: Why care about and spend money on testing? Why should testing be treated ...
One of my current responsibilities is to find ways to automate, as much as practical, the ‘testing’ of the user experience (UX) for complex web-based applications. In my view, full test automation of UX is impractical and probably unwise; however, we can use automation to find potential UX problems, or undesirable effects, even in rich, ...
One of the basic challenges with test automation is adoption. I can’t tell you how many times I’ve cataloged licenses for a company and found out they already have many different automation software packages, none of which is being used. Traditionally I’ve been told that is because the tools don’t work and that the teams ...
Regardless of your current state of tools, building an effective Continuous Integration suite of significant automated regression tests is the key to moving to a higher level of confidence in today’s development world. In the evolution timeline of software development tools, new tools have recently proliferated. We have all been sold on collaboration, transparency and ...
As our world continues its digital transformation with excitement in the advancement and convergence of so many technologies- from AI, machine learning, big data and analytics, to device mesh connectivity, nor should we forget VR and AR- 2017 promises to be a year that further transforms the way we work, play and take care of ...
All too often, software development organizations look at automating software testing as a means of executing existing test cases faster. Very frequently there is no strategic or methodological underpinning to such an effort. The approach is one of running test cases faster is better, which will help to deliver software faster. Even in organizations that ...
I feel like I’ve spent most of my career learning how to write good automated tests in an Agile environment. When I downloaded JUnit in the year 2000 it didn’t take long before I was hooked – unit tests for everything in sight. That gratifying green bar is near-instant feedback that everything is going as ...
The success of Automation is often correlated to its ROI. Here are 5 KPIs that we find universally applicable when it comes to quanitfying your Test Automation.
Test automation can provide great benefits to the software testing process and improve the quality of the results…. but its use must be justified and its methods effective. The reasons to automate software testing lie in the pitfalls of manual software testing… As we all know too well, the average manual software testing program:
“Testing Applications on the web” – 2nd EditionAuthors: Hung Q. Nguyen, Bob Johnson, Michael HackettPublisher: Wiley; edition (May 16, 2003) This is good book. If you test web apps, you should buy it!, April 20, 2001By Dr. Cem Kaner – Director of Florida Institute of Technology’s Center for Software Testing Education & Research Book Reviews ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Introduction There are many potential pitfalls to Manual Software Testing, including: Manual Testing is slow and costly. Manual tests do not scale well. Manual Testing is not consistent or repeatable. Lack of training. Testing is difficult ...
There are few topics in quality assurance testing that cause as much confusion as smoke testing versus sanity testing. The two names would seem to describe very different practices— and they do! But people still get them confused, since the distinction is somewhat subtle.

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe