Fitting QA into a modern DevOps group
In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities.
However, for folks outside the engineering team – say in Operations – they generally consider Development and QA to be in the same group. From this perspective those teams are working together to do a single job, with a single responsibility: deliver a product that works.
So what happens with QA in a DevOps organization? When Development and Operations merge together, where does that leave QA? How does the testing team fit into a modern DevOps group? This article will take a look at exactly that question.
The Reason Behind DevOps: Automated Deployment
When you look at the trend towards DevOps, it’s pretty clear that companies are adopting this organizational model in order to facilitate a practice of automated software deployment. DevOps provides the structure that enables teams to push software out as a service on a weekly, or daily, or even hourly basis. The traditional concept of a “software release” melts away into a continuous cycle of service improvement.
DevOps is Agile taken through its logical evolution: removing all the obstacles to getting high-quality software in the hands of customers. Once you have a smooth process for agile development and continuous integration, automating the deployment process makes total sense because it’s fulfilling the objectives that business managers crave:
- Faster time to market
- Higher quality
Increased organizational effectiveness
But before we move on, let’s ponder that for a moment: the purpose of DevOps is to get high-quality product out to the market faster – even automatically. The notion of “Quality” is built into the fabric of DevOps. If you couldn’t reliably push high-quality software out the door, DevOps would fail as a function.
Clearly, there is a critical role for QA in a DevOps group. So how are people fitting it in?
The Product IS The Infrastructure
We asked Carl Schmidt, CTO of Unbounce, what he thinks about QA and DevOps. Unbounce runs a SaaS solution for online marketers, making it really easy to build, publish, and A/B test landing pages without IT resources. Unbounce has three development teams, each with a resident QA team member, and practices DevOps throughout the organization.
Carl states, “I’m of the mindset that any change at all (software or systems configuration) can flow through one unified pipeline that ends with QA verification. In a more traditional organization, QA is often seen as being gatekeepers between environments. However, in a DevOps-infused culture, QA can now verify the environments themselves, because now infrastructure is code.”
The infrastructure is code. It’s a game-changing claim to any traditional development organization. Historically there was a clear separation between what constituted the product and what constituted the operation. You built a product, deployed it into a test environment where it could go through some quality control, and then eventually deployed it onto a live production system where real users could get at it. If there was a problem in product, the operations team had to fix it.
But as the lines blur between product and operation – as the very name DevOps implies – it’s no great leap to recognize that the environment itself is a part of the product.
Carl continues, “It’s QA’s responsibility to actually push new code out to production, so the DevOps team has been providing them with tools that make blue-green deployments push-button easy. Our QA team can then initiate deploys, verify that the intended change functions as expected, cut over to the newly deployed code, and also roll back if there is any reported issue.“
DevOps QA Is About Preventing Defects, Not Finding Them
QA takes a critical role in this organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not. This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs. Today QA groups are charged with preventing defects from reaching the public site.
This has several implications:
- QA owns continuous improvement and quality tracking across the entire development cycle. They are the ones who are primarily responsible for identifying problems not just in the product but also in the process, and recommending changes wherever they can.
- Tests are code, as any test automation expert will tell you. It’s a necessity, of course. If your process is designed to publish a new release every day (or every hour) there is no room for manual testing. You must develop automation systems, through code, that can ensure quality standards are maintained.
- Automation rules. Anything that can be automated, should be automated. When Carl describes Unbounce’s deployment process as “push-button easy,” this is what he’s talking about.
Testers are the quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability.
Beyond Functional Testing: Automation for Load Testing, Stress Testing, and Performance Testing
Now we are at a very exciting time in the transformation of QA, because while many organizations have mature processes for automating functional testing, they are only just beginning to apply these practices to other areas of testing like security and stress testing.
In particular, load testing and stress testing are critical disciplines for DevOps organizations that are moving at high velocity. A bottleneck introduced to a critical transaction process on an eCommerce website can bring a business to its knees. You want to do everything you can to identify scaling problems before a product is pushed out to a production environment, and you also want to keep a close eye on performance after it’s been released.
If you are curious to understand how your process holds up, check out this infographic: How Automated Your Performance Testing Is.
Creating a DevOps Culture
Unbounce CTO Carl Schmidt has some wise advice about his DevOps group: “we dislike using that term in favour of saying that we have a DevOps ‘culture.’” DevOps isn’t an individual, it’s a core value of a development organization. DevOps is more about trust, people, and teamwork than it is about process. It’s about the creating of software as an ongoing service, not a static product.
Although it’s not in the name (DevQuops, anyone?), the only reason that DevOps works is because quality is built into the entire system. You can’t get much more important than that.
