Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including both successes and failures.
Many books simply provide you the success path. This book also provides you with the steps you could possibly be taking that could lead to failure, helping you to change your path before failing.
The book starts with a nice overview of the case studies and an introduction to the key issues addressed by the case studies. Besides each case study being summarized, it also includes introducing the topics and pointing out the chapter they can be found in. They are broken down into management and technical issues. The management issues include Objectives for Automation, Management Support, Return on Investment and Metrics, Automation in Agile Development, Skills, Planning, Scope, and Expectations, Relationships with Developers, Triggers for Change and Getting Started, Tools and Training, and Political factors.
The technical issues covered include Testware, Abstraction, Architecture, Test Execution Tool, Automation Standards, Reusability, Documentation, Flexibility, Results and Reporting, Testing the Tests, What to Automate, Failure Analysis, and Finding Bugs.
The book includes a really nice table of Case Study Characteristics. Some of the characteristics include location, lifecycle (process used), number of team members, time span, tool types, pilot done, ROI measured, was it successful, and is the project still going on. This table really helps you hunt down topics you are interested in reading about first. The index of this book is really nice also. I mention that because I have seen some books lately where the publisher didn’t want to foot the bill for a nice one. That can be very aggravating.
The book’s last chapter is titled Test Automation Anecdotes. It is filled with experiences from the field that the authors felt were worth repeating, but did not constitute an entire chapter.
The book also has a nice table in the appendix that lists all the tools mentioned in the book. It includes which chapter they are in, where or not they are open source, and a link to the tool owner’s website.
I have repeatedly seen attempts at test automation fail for a verity of reasons. This book included them all: lack of management support, believing the tool is all you need, trying to automate tests without documenting them, and trying to automate every test. The level of difficulty and effort is almost always underestimated. This book definitely puts the level of effort into perspective.
Almost every story’s environment is unique. I really like the way the stories provide solutions to problems that could not be solved by simply purchasing a tool. These solutions are not the industry’s best practice solution, but rather home grown solutions to problems unique to their environment. Now that this book is out they may become best practice solutions. The primary thing they do is make you think out of the box. It is really refreshing to read such a real world book.
Every story is well written and well edited. This is one of the best resources available for helping expand your experience level without having to make the mistakes to learn from along the way. I wish this same format would be done with software projects in general; that is, with the same level of honesty. I see failing projects constantly being touted as successes. Buggy, over budget, late projects, are not a success. Kudos to those authors who stepped up to write about the failed projects!
The authors come from a wide range of technologists. You can check out the 28 case study summaries on the author’s web site.
I only have one word of warning. These stories suck you in. You may find yourself saying “Just one more”, and then suddenly looking at the clock and realizing it is 3 am.
All in all I highly recommend this book to everyone in the business of building software. Before you attempt to automate your testing, read this book!
