Letter from the Editor – September 2018

As part of my work, I spend a lot of time at client’s sites and talk to various software development organizations. I am beginning to see a problem arise regarding Test Automation. There is too much automation! Surprised?

While there are still many teams struggling to make progress with Test Automation, many teams have been doing very significant Test Automation for a long, long time. Last year, we did a survey on Testing Essentials with our readership to figure out the current trends happening in our field. We discovered that 20% of respondents had no automation at all.

There has always been a lot of focus on organizations who are just starting on their Test Automation journey. There is good reason for this. They need the most help.

But, often times I think too little focus is given to companies that have been automating for a while who have big automation suites. As these automation programs continue to grow and mature, they need a different kind of help. In light of businesses trying to deliver faster than ever through Continuous Delivery and, even Continuous Deployment, long time automating businesses with overly large regression suites are possibly hindering their organization’s progress.

Continuous Delivery demands immediate feedback. When a team has been automating for a while, has a decent framework and some smart people; thousands of tests are easily attainable. When you run them against multiple VMs, configurations, devices or browsers, it can take days to get feedback. But, when the pipeline is automated, you think can run the full regression suite and let the team know what happens in 5 days? No.

This is a very common issue today that clearly can not continue. So what is a team supposed to do? The solutions are difficult, if not painful. We need to cut tests, clean up and clean out tests; remove duplicates in order to reduce the number of tests for faster runs. Cleaning out old, failing tests and duplicates is an easy idea but may take a long time to do. There’s a catch, cutting seemingly important tests to get faster runs but cutting coverage may be a political battle. We need to trace and tag tests to code and functions to streamline runs. But, cutting coverage increases risk. Team members and managers do not want to hear this.

The most important thing to remember at this point is that CD’s goal is for smaller changes with smaller releases. The idea is to roll out one release at a time, to keep it isolated, thereby limiting risk. Our automated suites must reflect this. It’s easy to visualize this idea using containers. If you swap out one container of code, do you have to run your 5 day full regression suite? Also remember, CD is based on significant, repeatable, formalized re-running of developer unit tests. The typically larger increase in unit level tests can offset the increased risk of cutting API or UI level tests.

Any company moving to CD knows there are new risks with speedier deployments. Risk should be well-known and communicated. Risk should be managed. Risk should be localized and small in scope, with easy roll-backs. There is a give and take here. It’s not only faster and faster, smaller deploys mean less risk. When I see an organization’s automation goal of “More, more, more!”, I tell them a better goal is to stay Lean and Mean.

Our Test Automation issue is one of the biggest and best read every year. That is a good thing. I hope there is useful information about Test Automation for every team- teams new to automation as well as long time automators. New ideas and help is a good thing for all of us.

In this issue we look in-depth at automation across a wide variety of spectrums. Our own Tu Nguyen and Christine Paras discuss How to Automate Oracle ERP Testing in the cover story. James Willett discusses Alan Richardson’s book, Automating & Testing a REST API, and Daniel Knott returns to discuss the mobile test pyramid. This issue of TestArchitect Corner discusses how TestArchitect integrates with HP Quality Center. Rounding up the lineup, Eran Kinsbruner discusses how to connect Test Automation to specific app features and devices. This issue is also filled with a resource section on automation videos for you to watch, and is great for newbies and advanced practitioners alike. We also have an infographic on Anti-Patterns and how to avoid them. Don’t forget to peruse our events Calendar. LogiGear will be at both STARWEST and STPCon-Fall. If you’re planning on attending these shows then stop by and say hello!

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

There is a growing software development dynamic of teams without Testers. When I first went into Software Quality, I learned one thing right away: My role was user advocate. My main job was to find bugs. This is the Lean principle called Amplified Learning. We learn about behavior by testing. Even then, validation was not ...
I was just recently at a company that had a beautiful test architecture, framework, and Cucumber with tons of well-automated tests. But there was no good test management on top of the Cucumber tests, and they did not do a good job tagging the tests. Although almost everybody on the team could write and maintain ...
Big and complex testing. What do these terms conjure up in your mind? When we added this topic to the editorial calendar, I had the notion that we might illustrate some large or complex systems and explore some of the test and quality challenges they present. We might have an article on: building and testing ...
This is our first Trends issue in our 10- year history. Trends are important to help foresee what is on the horizon and coming next.
Change is constant. What’s different today is the rate of change. Moore’s law resulted from the observation that that the rate of change in computing power is exponential. The products, services and software landscape appears just as dynamic. At the same time, we pretty much take for granted the ubiquitous presence of software running our ...
Hello everyone – I’m hoping each one of us is having a great October. This time of the year is always my favorite, with the changing of the seasons, Fall was always my favorite time of year; it signified change and renewal – but I don’t want to digress to much from what’s going on ...
Testing the Software Car. As usual with the LogiGear Magazine, we are tackling a big subject. With our goal of having single-topic issues, we have the ability to grab and disseminate as much information as we can related to a current topic that is interesting and also on the frontier of Software Testing.   Some ...
Testing Embedded systems and testing the Internet of Things could each have their own issue of LogiGear magazine. But these days they are referred to presupposing knowledge of the other, so we thought it would be a good idea to tackle the two together in this issue to give a broad understanding of the landscape ...
For everyone still celebrating holidays: Happy Lunar New Year! At this time of the year many teams and companies are starting new projects, new initiatives, and hiring new staff. LogiGear Magazine will continue to be the resource for you for better testing with much less stress! We are excited about the focus of this month’s ...
Everything is mobile. What else can we say? Everything. If your product or service is currently not, it will be very soon. As Apple says: “There’s an app for that.” There is an app for everything. The race for mobile apps has consumed the software development world. I did a few projects at Palm Computing in the ...
As we settle into autumn, we’re taking the time to start some new traditions. This is LogiGear magazine’s first issue on SMAC. SMAC—social, mobile, analytics and cloud. We will be doing more issues in the next few years on these topics since so much of the product world is moving to this development stack.
This is LogiGear magazine’s first issue on the big world of DevOps. DevOps is a very large topic. Just when you thought you were safe from more process improvement for a while—not so fast. There’s DevOps, Continuous Testing, Continuous Delivery and Continuous Deployment. In this issue, we are focusing on Continuous Testing, the part most ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe