Where Does QA Fit in DevOps?

Fitting QA into a modern DevOps group

In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities.

However, for folks outside the engineering team – say in Operations – they generally consider Development and QA to be in the same group. From this perspective those teams are working together to do a single job, with a single responsibility: deliver a product that works.

So what happens with QA in a DevOps organization? When Development and Operations merge together, where does that leave QA? How does the testing team fit into a modern DevOps group? This article will take a look at exactly that question.

The Reason Behind DevOps: Automated Deployment

When you look at the trend towards DevOps, it’s pretty clear that companies are adopting this organizational model in order to facilitate a practice of automated software deployment. DevOps provides the structure that enables teams to push software out as a service on a weekly, or daily, or even hourly basis. The traditional concept of a “software release” melts away into a continuous cycle of service improvement.

DevOps is Agile taken through its logical evolution: removing all the obstacles to getting high-quality software in the hands of customers. Once you have a smooth process for agile development and continuous integration, automating the deployment process makes total sense because it’s fulfilling the objectives that business managers crave:

  1. Faster time to market
  2. Higher quality

Increased organizational effectiveness

But before we move on, let’s ponder that for a moment: the purpose of DevOps is to get high-quality product out to the market faster – even automatically. The notion of “Quality” is built into the fabric of DevOps. If you couldn’t reliably push high-quality software out the door, DevOps would fail as a function.

Kongress_Feature image

Clearly, there is a critical role for QA in a DevOps group. So how are people fitting it in?

The Product IS The Infrastructure

We asked Carl Schmidt, CTO of Unbounce, what he thinks about QA and DevOps. Unbounce runs a SaaS solution for online marketers, making it really easy to build, publish, and A/B test landing pages without IT resources. Unbounce has three development teams, each with a resident QA team member, and practices DevOps throughout the organization.

Carl states, “I’m of the mindset that any change at all (software or systems configuration) can flow through one unified pipeline that ends with QA verification. In a more traditional organization, QA is often seen as being gatekeepers between environments. However, in a DevOps-infused culture, QA can now verify the environments themselves, because now infrastructure is code.”

The infrastructure is code. It’s a game-changing claim to any traditional development organization. Historically there was a clear separation between what constituted the product and what constituted the operation. You built a product, deployed it into a test environment where it could go through some quality control, and then eventually deployed it onto a live production system where real users could get at it. If there was a problem in product, the operations team had to fix it.

But as the lines blur between product and operation – as the very name DevOps implies – it’s no great leap to recognize that the environment itself is a part of the product.

Carl continues, “It’s QA’s responsibility to actually push new code out to production, so the DevOps team has been providing them with tools that make blue-green deployments push-button easy. Our QA team can then initiate deploys, verify that the intended change functions as expected, cut over to the newly deployed code, and also roll back if there is any reported issue.“

DevOps QA Is About Preventing Defects, Not Finding Them

QA takes a critical role in this organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not. This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs. Today QA groups are charged with preventing defects from reaching the public site.

This has several implications:

  • QA owns continuous improvement and quality tracking across the entire development cycle. They are the ones who are primarily responsible for identifying problems not just in the product but also in the process, and recommending changes wherever they can.
  • Tests are code, as any test automation expert will tell you. It’s a necessity, of course. If your process is designed to publish a new release every day (or every hour) there is no room for manual testing. You must develop automation systems, through code, that can ensure quality standards are maintained.
  • Automation rules. Anything that can be automated, should be automated. When Carl describes Unbounce’s deployment process as “push-button easy,” this is what he’s talking about.

Testers are the quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability.

Beyond Functional Testing: Automation for Load Testing, Stress Testing, and Performance Testing

Now we are at a very exciting time in the transformation of QA, because while many organizations have mature processes for automating functional testing, they are only just beginning to apply these practices to other areas of testing like security and stress testing.

In particular, load testing and stress testing are critical disciplines for DevOps organizations that are moving at high velocity. A bottleneck introduced to a critical transaction process on an eCommerce website can bring a business to its knees. You want to do everything you can to identify scaling problems before a product is pushed out to a production environment, and you also want to keep a close eye on performance after it’s been released.

If you are curious to understand how your process holds up, check out this infographic: How Automated Your Performance Testing Is.

Creating a DevOps Culture

Unbounce CTO Carl Schmidt has some wise advice about his DevOps group: “we dislike using that term in favour of saying that we have a DevOps ‘culture.’” DevOps isn’t an individual, it’s a core value of a development organization. DevOps is more about trust, people, and teamwork than it is about process. It’s about the creating of software as an ongoing service, not a static product.

Although it’s not in the name (DevQuops, anyone?), the only reason that DevOps works is because quality is built into the entire system. You can’t get much more important than that.

Tim Hinds – Product Marketing Manager

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices. Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

Tim Hinds
Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices. Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.
Tim Hinds on Linkedin

The Related Post

How to Adopt the “Third Way” in the Dojo’s Method to Master CD In The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations the authors describe “The Three Ways” – the underlying principles forming the basis for all DevOps practices. 
From the culture shift, to differences in Agile, Dave Farley and Michael Hackett discuss the nitty gritty of Testing in DevOps. For this issue of LogiGear Magazine, our very own Michael Hackett sat down with one of the godfathers of Continuous Delivery, David Farley. In this exclusive interview, David discusses how test teams and automation ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
    Eric Minick is internationally recognized as a leading authority on continuous delivery and DevOps. Eric joined IBM four years ago with the acquisition of UrbanCode where he had worked as a developer, technical seller, and evangelist for a decade. Today, he has responsibility for leading the product management team overseeing continuous delivery solutions ...
From adopting the culture, to implementing Continuous Delivery With the relative newness of DevOps, there are not yet a ton of DevOps books. That’s why we’ve assembled a list of the 7 best DevOps books based on four criteria: the number of ratings from Amazon, the average Amazon rating, number of ratings from GoodReads and the ...
As a software development company, what is your goal? What is the one thing you feel you need to do to ensure you have a job at the beginning of each wonderful work week? The answer is actually quite simple; You need to deliver a quality product. Like how I used the word simple? Although the answer I ...
By Jez Humble and David Farley Continuous Delivery from Jez Humble and David Farley is an important contribution to the field of software development. It takes continuous integration to the logical conclusion and covers how to set up a continuous integration system, delving into everything from check-in to delivery to production. It doesn’t state you ...
Making the leap to CT is easier than you think— follow this guide to transform your testing process No pain, no gain! Achieving Continuous Testing shouldn’t take a “Hans and Franz” attitude. It should be painless, more like a natural progression from implementing certain practices over time.
DevOps may be the next big buzzword, but Test teams really need to focus on its little sister, Continuous Delivery If you pay attention to trends in software development—from the perspective of what some sophisticated teams are doing, what articles and books are being written, to conference topics, you may have noticed the tools being ...
How Halliburton leveraged outsourcing to achieve their goals. Organizations are focusing on speed, both in Continuous Integration and rapid deployment as a competitive advantage. Many software development organizations can significantly shorten development cycles by implementing one or a combination of Agile practices, continuous integration & deployment methods, and feature branches. While these frameworks and techniques ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe