DevOps and Continuous Testing: Better Together

Sauce Lab’s Perspective on Integration with Continuous Testing

The term ‘DevOps’ implies that implementing an effective development and production workflow is as easy as developing the collaborative interaction between developers and IT operations. On the surface, DevOps may appear as a simple and straightforward idea but, as you will find out, there is more than meets the eye.

Yet adopting DevOps effectively is more complex than combining Dev and Ops. Quality Assurance Engineers who oversee software tests are an equally crucial component to successfully implementing DevOps. In addition, organizations that attempt to implement DevOps without building Continuous Testing into their delivery pipelines will fall short of achieving the full benefits of DevOps.

Continuous Testing is the key to successful integration of quality assurance into DevOps workflows. It is the only way to ensure that quality assurance is as continuous, Agile, and reliable as the rest of the DevOps operation. Considering this point, Continuous Testing serves as the glue that binds together all of the processes that comprise the Continuous Delivery pipeline. Without automated and Continuous Testing, DevOps is inefficient.

The Benefits of DevOps

To understand why automated testing is critical for effective DevOps, it is necessary to identify the benefits that organizations seek to achieve by following DevOps principles. The primary benefits of a well-designed and maintained DevOps environment include the following:

  • Seamless communication across all parts of the organization. DevOps does this by eliminating the silos that have traditionally separated different teams from one another.
  • Rapid, continuous, and reliable software changes delivered on a continuous basis. This requires software updates to be broken into small parts that can be designed, written, tested, and released into production continuously.
  • Maximize agility. When software delivery is agile, applications in your tool chain can easily scale in response to fluctuations that are in demand. In addition, software delivery teams have the ability to switch easily between development frameworks and tools according to shifting needs or preferences.
  • The elimination of unforeseen delays in software production. These delays typically result from having to fix problems with the code after it is in production, at this point rollbacks are costly and time-consuming. Continuous Testing can help us avoid this risk by ensuring that code is tested automatically, as a component of the delivery pipeline, before it goes into production.

Why DevOps Needs Continuous Testing

The term DevOps entails collaboration between development teams and IT operations teams can optimize software delivery. However, this is not completely true. Software testers also need to be seamlessly integrated into the Continuous Delivery chain to work alongside development and IT Ops teams.

To be precise, integrating development with IT Ops is one important aspect of achieving DevOps. Software is delivered faster and more reliably when programmers are in constant correspondence with the system admins, who deploy and manage the software in production. Furthermore, speed and reliability increase when both of these groups enjoy maximum visibility into the state of application development.

Yet, this type of collaboration is not enough on its own to make DevOps work well. If software testers remain in a silo and do not participate in the Continuous Delivery chain, a number of problems arise. These problems include:

  • The inability to test software updates at the pace at which they are produced and deployed. If automated software testing is not part of the Continuous Delivery pipeline, changes made by developers to the code cannot be tested continuously. They will instead have to be tested irregularly; whenever the testing team is able to address them.
  • Agility is undercut. Even if the rest of the software delivery chain is Agile, failure to integrate automated testing into the Continuous Delivery pipeline will undercut the organization’s ability to derive value from that agility.
  • Inadmissible quality. Part of the value of DevOps is its ability to standardize and streamline the software delivery processes. When automated testing is not part of the Continuous Delivery pipeline, tests remain irregular and ad hoc.
  • Continuity rollbacks. To make the most of DevOps, testing should be automated and continuous.
  • The introduction of bugs that force developers to roll back code once it has been written is a serious hamper to continuity.

Organizations that leave software testing in a silo rather than integrating it into the Continuous Delivery pipeline fail to achieve the objective of DevOps, even if the rest of their DevOps schema is well designed.

Continuous Testing: The Glue between Dev and Ops

Continuous Testing is crucial for DevOps because tests are the tie that binds development to IT Ops, and ensures that updates flow smoothly from the beginning to the end of the Continuous Delivery pipeline.

A typical DevOps delivery chain starts with developers, who design application changes and then write the code to implement them then, increasingly, go on to design the corresponding tests. The delivery chain ends with IT Ops, which is responsible for pushing the updates into production and maintaining them.

Software tests should be executed at every step along the way. Integration tests ensure that changes or new features written by developers can be added to the application without breaking it. End-to-end tests identify flaws that developers might not have foreseen when designing code. They also prevent those problems from reaching end users. Device compatibility tests ensure that code written and built in development environments will function as intended in real-world settings, which are much more complex, involving many more hardware and software variables than development environments.

A Continuous Delivery pipeline that lacks automated and Continuous Testing will not enable developers and IT Ops teams to interact effectively with one another.

How to Enable Continuous Testing

In order to integrate software testing effectively into a Continuous Delivery pipeline, DevOps teams should implement testing solutions that enable and reinforce DevOps goals. When choosing a testing platform, look for the following essential features:

The ability to test quickly. In order to avoid delays to your Continuous Delivery chain, you need to be able to perform tests quickly. Performing parallel tests on a large scale is one way to achieve this. Another option is, to first, run compatibility tests on simulated devices first (since these tests take less time) and then performing the more time-consuming tests on real devices later in the pipeline, prior to the code entering production.

The ability to scale. Your testing platform should be able to perform tests as quickly as needed and it should support as many parallel tests as you require. On-premises testing solutions are unlikely to offer the necessary scalability because they will be constrained by limited hardware resources. In contrast, a cloud-based testing platform can scale as seamlessly as the rest of your Continuous Delivery pipeline.

High automation. DevOps teams achieve their speed and agility in part by automating as much of the software delivery process as possible. Your testing solution should be as automated as the rest of your DevOps toolset. You should be able to trigger tests, analyze results, and share testing information across the organization in a completely automated fashion.

On-demand testing. To avoid kinks in the Continuous Delivery pipeline, you need to be able to perform tests whenever they are necessary. There are two ways to do this. One way is to maintain a massive on-premises testing environment with enough resources to perform tests whenever you need them, as well as maintain the test infrastructure. The solution is a cloud-based testing platform that can complete tests quickly upon request. A cloud-based platform is much more cost-efficient because it obviates the need to implement and manage an expensive on-premises test grid that is underutilized most of the time.

A Continuous Testing solution that includes the qualities that enable your developers and IT Ops teams to work together in the most efficient manner. This ensures that the code you put into production is reliable and stable across diverse environments. It also empowers your organization to derive full value from its migration to a DevOps-based workflow by maximizing the agility, visibility, scalability, and continuity of your software delivery pipeline.

Lubos Parobek
At Sauce Labs, Lubos Parobek leads strategy and development of Sauce Labs’ web and mobile application testing platform as VP of product. His previous experience includes leadership positions at organizations including Dell KACE, Sybase iAnywhere, AvantGo and 3Com. Parobek holds a Masters in Business Administration from the University of California, Berkeley.

The Related Post

It’s no secret that the cloud is growing at an exponential rate. By 2016, two-thirds of the world’s server workloads will exist in the cloud. But according to Cisco’s 2012 Cloud Index, less than half of server workloads currently run in the cloud. Closing the gap between current capabilities and future requirements is a mission-critical ...
The book is an incredibly effective and valuable guide that details the risks that arise when deploying cloud solutions. More importantly, it provides details on how to test cloud services, to ensure that the proposed cloud service will work as described. It is a great start to the topic. The 6 chapters detail a paradigm ...
Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
How Halliburton leveraged outsourcing to achieve their goals. Organizations are focusing on speed, both in Continuous Integration and rapid deployment as a competitive advantage. Many software development organizations can significantly shorten development cycles by implementing one or a combination of Agile practices, continuous integration & deployment methods, and feature branches. While these frameworks and techniques ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments. Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be ...
Making the leap to CT is easier than you think— follow this guide to transform your testing process No pain, no gain! Achieving Continuous Testing shouldn’t take a “Hans and Franz” attitude. It should be painless, more like a natural progression from implementing certain practices over time.
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...
Throw away clunky hyper-visors, and stop thinking about computer hardware and software license during your development projects. The first thing you think about when you hear “The Cloud” may not be development and testing. The Cloudy market is filled with SaaS applications, hosting, and cloud-based file systems. All are very useful, and offer a clear ...
LogiGear Magazine June Testing in Continuous Delivery Issue 2017
DevOps has been described as Agile on Steroids; DevOps has also been described as Agile for Operations/IT. I like both of those descriptions. Many organizations want Development, Test, and Operations teams to move to DevOps now. DevOps is a big topic, but DevOps is not the focus of this article. We will not be talking ...
For this month’s book review, I read Continuous Testing for DevOps Professionals: A Practical Guide from Industry Experts, by various authors and edited by Eran Kinsbruner. The book is divided into 4 sections: Fundamentals of Continuous Testing, Continuous Testing for Web Apps, Continuous Testing for Mobile Apps, and The Future of Continuous Testing. The Fundamentals ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news