Book Review: Experiences of Test Automation

Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including both successes and failures.

Many books simply provide you the success path. This book also provides you with the steps you could possibly be taking that could lead to failure, helping you to change your path before failing.

The book starts with a nice overview of the case studies and an introduction to the key issues addressed by the case studies. Besides each case study being summarized, it also includes introducing the topics and pointing out the chapter they can be found in. They are broken down into management and technical issues. The management issues include Objectives for Automation, Management Support, Return on Investment and Metrics, Automation in Agile Development, Skills, Planning, Scope, and Expectations, Relationships with Developers, Triggers for Change and Getting Started, Tools and Training, and Political factors.

The technical issues covered include Testware, Abstraction, Architecture, Test Execution Tool, Automation Standards, Reusability, Documentation, Flexibility, Results and Reporting, Testing the Tests, What to Automate, Failure Analysis, and Finding Bugs.

The book includes a really nice table of Case Study Characteristics. Some of the characteristics include location, lifecycle (process used), number of team members, time span, tool types, pilot done, ROI measured, was it successful, and is the project still going on. This table really helps you hunt down topics you are interested in reading about first. The index of this book is really nice also. I mention that because I have seen some books lately where the publisher didn’t want to foot the bill for a nice one. That can be very aggravating.

The book’s last chapter is titled Test Automation Anecdotes. It is filled with experiences from the field that the authors felt were worth repeating, but did not constitute an entire chapter.

The book also has a nice table in the appendix that lists all the tools mentioned in the book. It includes which chapter they are in, where or not they are open source, and a link to the tool owner’s website.

I have repeatedly seen attempts at test automation fail for a verity of reasons. This book included them all: lack of management support, believing the tool is all you need, trying to automate tests without documenting them, and trying to automate every test. The level of difficulty and effort is almost always underestimated. This book definitely puts the level of effort into perspective.

Almost every story’s environment is unique. I really like the way the stories provide solutions to problems that could not be solved by simply purchasing a tool. These solutions are not the industry’s best practice solution, but rather home grown solutions to problems unique to their environment. Now that this book is out they may become best practice solutions. The primary thing they do is make you think out of the box. It is really refreshing to read such a real world book.

Every story is well written and well edited. This is one of the best resources available for helping expand your experience level without having to make the mistakes to learn from along the way. I wish this same format would be done with software projects in general; that is, with the same level of honesty. I see failing projects constantly being touted as successes. Buggy, over budget, late projects, are not a success. Kudos to those authors who stepped up to write about the failed projects!

The authors come from a wide range of technologists. You can check out the 28 case study summaries on the author’s web site.

I only have one word of warning. These stories suck you in. You may find yourself saying “Just one more”, and then suddenly looking at the clock and realizing it is 3 am.

All in all I highly recommend this book to everyone in the business of building software. Before you attempt to automate your testing, read this book! 

Tad Anderson
Tad lives in Mount Joy, PA with his wonderful wife Wendy and his faithful companion Artimus (4 year old Maltese). He has been programming since 1992 and specializes in Software Architecture. He is currently serving in the role of Enterprise Architect at Penn National Insurance.

The Related Post

Divide and conquer was a strategy successfully employed by ancient Persian kings against their Greek enemies. It is a strategy that can still be used successfully today. Fundamentally, by dividing something into smaller more manageable pieces (in the case of the ancient Persians, they divided the Greek city states), it becomes much more manageable.
One of the basic challenges with test automation is adoption. I can’t tell you how many times I’ve cataloged licenses for a company and found out they already have many different automation software packages, none of which is being used. Traditionally I’ve been told that is because the tools don’t work and that the teams ...
Learn how to leverage TestArchitect and Selenium for turnkey, Automated Web testing. TestArchitect lets you create, manage, and run web-based automated tests on different types of browsers—using either a WebDriver or non-WebDriver technique. In this article, we will explore employing WebDriver for testing a web-based application with TestArchitect. TestArchitect with WebDriver is a tool for automating ...
For this interview, we talked to Greg Wester, Senior Member Technical Staff, Craig Jennings, Senior Director, Quality Engineering and Ritu Ganguly, QE Director at Salesforce. Salesforce.com is a cloud-based enterprise software company specializing in software as a service (SaaS). Best known for its Customer Relationship Management (CRM) product, it was ranked number 27 in Fortune’s 100 ...
Introduction Many executives have some very basic questions about Software Testing. These questions address the elements of quality (customer satisfaction) and money (spending the least amount of money to prevent future loss). The basic questions that executive have about Software Testing include: Why care about and spend money on testing? Why should testing be treated ...
LogiGear Magazine – The Big Testing Issue – April 2012
The 12 Do’s and Don’ts of Test Automation When I started my career as a Software Tester a decade ago, Test Automation was viewed with some skepticism.
*You can check the answer key here
Introduction As a consultant and trainer, I am often asked by my clients and students how to deal with automated acceptance tests. One common question is whether automated acceptance tests should use the graphical user interface (GUI) provided by the application.
Recently while teaching a workshop on Testing Dirty Systems, I uttered this “Randyism” off the top of my head, “Test automation is not automatic.” I realized immediately that I had just concisely stated the problem in making test automation a reality in many organizations. Most testers know that test automation is not automatic. (Wouldn’t it be great?) However, ...
For those that are new to test automation, it can look like a daunting task to undertake For those who are new to Automation, it can look like a daunting task to undertake, but it only seems that way. If we unpack it and pinpoint the fundamentals, we can have the formula for the desired ...
This book isn’t for everyone, but everyone can get some value out of it. What I mean by that rather confusing statement is that folks working in Agile environments will likely want to throw the book across the room while folks in more bureaucratic environments like CMMI or other waterfall environments will likely get a ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe