Testing tools – very important, very often overlooked, and very often where mistakes are made.
First, the most common mistake people make about tools is thinking tools are only about test automation! False. Automation tools are merely one type testing tool. We will try to balance this issue between test automation tools and other test tools.
I heard a story about test automation, very recently in fact, of a company that paid a large licensing fee to a major tool vendor, and invested a lot of training and time to develop a phase 1 automation project. The intent was to scale up the initial phase into a large product suite automated regression tool. After just 6 months the project was dropped due to high test maintenance costs, over-idealized expectations and time demands. A whole lot of money, time, and tester goodwill went down the drain! I have heard too, too many of these stories with test tools, and specifically automation tools.
This story is not new. I heard nearly the same thing in 1994. This really highlights the need to address issues like designing code for testability, high reusability automation frameworks, lower maintenance test design, unrealistic tool vendor marketing, unrealistic staff expectation, etc.
Even after twenty years, too many teams still suffer from the problem of shelfware. Shelfware is slang for owning or licensing software that you don’t use (e.g. it sits on a shelf). It was a story I first early in my software career. And it’s particularly problematic with test automation tools.
Clearly, do whatever it takes to avoid shelfware at all costs! Be part of the tool evaluation process; Demand training; Get direct vendor support; Make sure teams are aware of possible long ramp-up efforts; Make sure there is significant set-aside automation time, separate from testing time. And, always, treat automation development as its own development project!
A great suggestion these days is to get professional services help to jumpstart your automation program. There are experts who can build a framework specific to your application of environment or work with you side-by-side and who can give 1-on-1 coaching to get the project started right or repair a broken automation effort. Yes, shelfware is a preventable problem.
Tools are meant to be the solution not the problem. But evaluation and selection is a unique process. We provide a few tips and there are many great articles on how to go through a tool evaluation process that will help you and your team get the process right! Don’t just rely on a tool comparison article. For starters there are some past LogiGear Magazine issues devoted to test tools. In 2012 (logigear.com/magazine/2012/09), we focused on Integrated Test Platforms such as ALM tool suites like TFS/Visual Studio, Jira, Rally, Rational/IBM, Thoughtworks. It’s worth a read. When carefully selected, carefully implemented, and you allocate the time and effort to maintaining your automated tests, automation tool are a huge bonus!
In this issue, Joe Luthy and I recommend that utilizing professionals can help you avoid epic testing fails; Robert Galen of Velocity Partners explains that picking the right automation tool can drive down test time and impress business folks; LogiGear staff present a set of criteria to pick the automation tool that’s right for you; Jim Holmes reviews Implemented Software Testing by Elfriede Dustin, Thom Garrett and Bernie Gauf; I provide some insight into the varieties of test execution tools and utilities for your toolbelt and Randall Rice of Rice Consulting provides a list of free and cheap software test tools that can help you across all phases of testing.
Tools help you do things. They can help you test better, faster, more informed, and help you isolate issues. Good luck! I hope you add some useful tools to your test effort.
