For those that are new to test automation, it can look like a daunting task to undertake
For those who are new to Automation, it can look like a daunting task to undertake, but it only seems that way. If we unpack it and pinpoint the fundamentals, we can have the formula for the desired test results. You would be surprised how much easier Automation is when you have a few simple, effective yet essential skills that can easily be learned.
As Test Automation consultants, we’re frequently meeting people from various companies and across several industries, each with varying skillsets, resource pools, decision making processes, and of course, budgets. No matter what their differences, we tend to find a common theme in our conversations—functional automation is really hard!
In fact, functional automation is so difficult that these decision makers often find themselves asking: “Is it really worth it?” Isn’t it cheaper and easier to just hire more testers? Can’t I just outsource this to foreign testers?” These are all valid questions and ones that anyone looking to test their application or software should ask themselves. However, most ultimately arrive at the same conclusion: even if the testers were free, the effort to manage them and the time needed to thoroughly test their products/services is simply not worth it.
Why is test automation difficult for practically every organization?
In short, automation is hard because of permutations.
I remember the first time I implemented an automation suite in the early 90s. Back then, most people were using WinRunner/XRunner built by a great new company called Mercury Interactive. Pretty much everything back then was client server. Remember ActiveX? If you do, you are probably trying hard to forget about it. Thick client meant you had control over pretty much everything but the OS it was installed on, and even then you barely broke a sweat since there were fewer OS versions with less frequent releases to worry about. To be more specific, a typical set of tests would involve executing the regression suite against 4 – 5 different OS versions, with perhaps 2 or 3 different back end systems. Again, a lot of testing, but not necessarily an overwhelming challenge.
When testing software in the 90’s, you could easily see the summit of the mountain and that had a huge impact on attitude and ambition. If you’ve ever gone hiking in Colorado, you’ll traverse some huge mountains, but whenever the top is visible, your legs become full of energy and might. However, there are moments when every rise you crest seems to reveal another massive hill and your strength immediately wanes.
I think that this diminishing strength is what happens to anyone considering an automation project on modern websites or applications. The complexity of projects today resemble mountain peaks shrouded in clouds with dangerous cliffs, recent avalanches, and massive boulders standing between you and your goal. Many QA professionals give up before they start, reasoning that it is better to not waste time and money trying to accomplish something they can never conquer anyway.
How do you overcome this automation induced paralysis?
By convincing yourself that you have already conquered the mountain.
The first time I hiked, it was to a little-known, high mountain glacier-fed lake called Ice Lake, (aptly named as it is frozen over about 10 months of the year) I was apprehensive. I had not done a hike of that magnitude before and quite honestly, I wasn’t in the best shape for it. We started at an elevation of approximately 10,000 feet, going to just over 14,000 and dropped down the backside of the mountain to the lake, which sits at about 12,500 feet. I turned to my hiking partner and asked, ‘Aren’t there any other lakes on this side of the mountain? Can’t we just hike to those?’ My partner assured me it was worth the trip.
He explained the route we would take, alternate paths we could explore if the water was too high in some of the rivers, and what we would see along the way. So off we went, with 60-pound backpacks into the rocky mountain thin air.
Today, I like to think of myself as an experienced guide to Ice Lake. I’ve taken the trip many times, stumbling once or twice along the way, learning from my mistakes, and finding a few shortcuts. Although, the hike is a challenge, the reward is just as great.
Tackling Mount Automation’s Challenges Today
As a test automation consultant, I like to think of myself as a guide that can help you conquer the path that you’ve set yourself on whilst warning you of some of the common pitfalls you’ll encounter and unveiling a few shortcuts along the way.
Many of our customers each face similar challenges. They demand that their websites and apps run across multiple browsers and device types whilst their customers demand minimal bugs, great performance, and new features all the time. If it sounds daunting, that’s because it truly is.
The best way to overcome a challenge is to fully understand it, pick your battles carefully, and not do it alone.
Understanding how your user base interacts with your product is critical. This helps to create applications that better meet their needs and helps to focus your testing efforts. What browser or mobile device is most popular for your client? If Chrome is used by 65% of your users and Safari by 30%, focus on Chrome first! Also, which versions of those browsers are most used by your customers? This helps to clear the clouds away from the top of the mountain.
Understanding your own self-doubts
When I started that hike to the aforementioned Ice Lake, I was very excited by the promise of beautiful landscapes and adventure that laid before me. But after about 30 minutes of trudging through wet underbrush, climbing over boulders the size of school buses, and feeling the effects of less air at higher altitude, my excitement dwindled and my 60-pound backpack began to feel 10 times heavier. I was ready to lay down and die, but not before pushing my hiking partner off a cliff for convincing me to come along on this crazy jaunt.
We finally stopped and rested for a bit. I drank water, had a bite to eat, and listened while my hiking partner explained that this feeling was normal and that the altitude and physical exertion was causing a shock to my untrained body which would eventually subside and we would get our ‘second wind’.
He made a few adjustments to my pack and off we went again.
When an automation project starts, the same thing will happen. You will inevitably hit a wall and it will feel like you are not making the progress you want to make. A 6 month project can instantly feel as though it is about to turn into a 60 month project overnight. This is more likely to happen when you first start to peel back the layers of the application and face some unexpected challenges. This is where having someone who has traversed these challenges can help to allay your fears.
An Automation professional can help you take a step back and realize the true progress you are making up Mount Automation no matter how slow your pace may seem. They can show you specific test assets that have been created, framework development that is in progress and can be demoed to you, and training that is taking place so that everyone can be involved. It is important that both parties — the guide and the guided — communicate and make sure everyone is clear on progress, and next steps.
Automation made easy
At times we hear that the demand for constant software releases makes it impossible to implement good test automation. This simply is not true. If anything, frequent releases makes it easier to implement good test automation!
How is that possible? By following the same principles to implement automation as you would for developing your application. Build the base, add to it, release often, and repeat.
This is not an unfamiliar concept. As a kid I’d often travel with my family to Amarillo, Texas and visit a restaurant called the Big Texan. For decades the Big Texan has offered up a free 72-ounce steak, with all the sides for free, if you can finish it in an hour. So, for those daring enough to accept the challenge, they’d be designated to a special table and would dig in while the clock ticked away on the wall behind them.
Of course, most failed, but there was one man who had successfully won the challenge several times. I once asked him, “How do you do it?” He looked at me and said, ‘Son, don’t try to swallow the whole cow.” What he meant was that you shouldn’t look at the whole steak—instead stay focused on the next bite.
What does this have to do with automation?
If you look at having to automate 3000 test cases, it can seem overwhelming. You can feel defeated before you even start. Instead, start with your first script and then move on to the next one and then the next. Whether you’re a cowboy tackling the steak challenge, a hiker on a 14,000 foot mountain, or a QA team member looking to implement automation, you must keep looking forward and focus on the next step.
Track your progress and celebrate success
Let’s also adopt the core principles of Continuous Integration and Continuous Delivery. Build often, build small, and release when ready. Break up the task into discrete, manageable bite-sized pieces, and dig in. Keep in mind that running just one script per day provides valuable data that can save time and increase productivity. Yes, these can often be mundane and simple tests but this will provide reliability and repeatability that cannot be achieved by a human tester.
We highly recommend starting with a set of smoke tests that give immediate feedback on the health of a build. Then add in your regression tests. As you add more scripts and see these incremental gains, your confidence will grow, and you’ll begin to see a return on your investment.
Sometimes organizations are so eager to get started, they just want to start scripting on day 1. That is an almost guaranteed plan to fail. I often tell organizations when we start, that they may not see any scripts for a few weeks. Some shockingly ask, “What do you mean no scripts for a few weeks?!? What are you going to be doing?!”
Let’s go back to our hike to illustrate why that is the case.
My friend didn’t call me one day and say, “Hey! Let’s hike to Ice Lake. We leave in an hour’. We had to plan considerably in advance. First, we had to pick a time that worked for both of us and then we had to get our equipment—I had no idea how much was needed for a hike! I had been camping many times before but the equipment needed for backpacking is totally different. For hiking, your equipment is all about space and weight as everything you take will be carried on your back. That tent that weighs 12 lbs? Forget it. You need to get a 2lb tent. You aren’t hauling 15 gallons of water on your back, you need water filtration. Very light, very thin and, very warm clothes is vital. Only take the food that you need and be sure it requires little preparation. The list goes on and on but ultimately, before we even set a foot on the mountain, we have spent hours planning and preparing in advance.
The same is true for any automation project. Considerable planning is necessary. Creating scripts is actually the easy part—any automation engineer will tell you that! The hard part is building out a solid framework that supports implementation of business logic and maintainability of the scripts. If scripts are hard to maintain, they will be quickly abandoned.
To get started, you’ll want to analyze your manual test cases and identify the best candidates for automation. Some aspects of your application will be difficult to apply test automation, so those could actually be de-prioritized until everything else has been completed.
You’ll also want to spend a considerable amount of time evaluating automation and testing tools to determine which fits your needs best (I won’t go into tools further in this article because the possibilities are truly endless).
In some instances, manual testing may still be required but this is a good opportunity to identify redundancies in your methodologies, and merge or remove some manual tests completely to gain efficiency.
Now it’s time to build your smoke test. Determining which areas of functionality you will test and to what extent, will help you define a well-functioning and responsive smoke test. There are a few key questions to ask yourself as you build your first tests:
- Will it need to be integrated into a CI/CD process?
- What build and deployment system is being used?
- How can we integrate into it?
Furthermore, clearly defining automation objectives and aligning stakeholders is also crucial. Again, there are key questions to ask yourself so as to stay on track:
- Who will be looking at the results?
- What do they care about?
- How often do they need the results?
- Who will build the script?
- Who will maintain it?
- How will changes be handled?
- What browsers/OS/devices need to be included?
The consideration that goes into building out the framework and automation plan is extensive which is why it takes time before scripting begins but ultimately saves time and ensures a successful project.
In Conclusion
Automation isn’t easy but, it is not as tough as many people think. It is within the grasp of most organizations, if they have the right people helping them along the way. There are few greater feelings than standing on the top of a mountain and seeing the world below you—it’s in this moment that the pain you experienced throughout your journey fades to nothing. If you approach test automation in a similar fashion, with considerable planning and clear objectives, you’ll feel similar when you sit down with your cup of coffee in the morning, open your email, and review that morning’s test results.
