This is our third issue concerning topics of Continuous Delivery (CD) and DevOps with the inclusion of Continuous Testing. DevOps has been around for a while and I hope the period of buzz is over and companies moving towards building a development pipeline have begun their process, including changing their test strategies.
Many organizations recognize that they have to hit the brakes and likely do an engineering project to re-architect the system before commencing.
Very different from organizations jumping on the Agile and Scrum bandwagons in name only, DevOps is more about business change, Operations/IT/deployment change, and investing in tools more so than Dev and Testing practice change. Although there is a big impact on the Dev team since many Dev teams take the lead on configuring the new pipeline tools. Dev and Testing got turned on its head with Agile. It’s more commonly the product architecture, tools, and deployment process getting turned upside down now.
Encouraging Operations/IT to get involved early in development, shifting Dev testing tasks left earlier in the cycle, and automating as many Ops tasks as possible are all foundation ideas in DevOps. Commonly talked about tools for this include Puppet, Chef, and Docker.
As much as tool and deployment changes are important, successful teams need DevOps Culture changes such as building trust, building cross-functional knowledge, and cutting out blame. This is a culture where everyone is working towards deployment.
You ask, what part of CD/DevOps do test teams need to focus on? I can say: first, understanding Continuous Testing. The distributed nature of quality ownership in CT includes much more Dev white box testing, specifically unit testing than many organizations are used to. This is part of the shift-left practice. The theoretical result of this is that more unit testing leads to more robust functionality earlier that includes a different need for higher level testing and Test Automation. You can refer to many published works on the “Test Automation Pyramid,” where more testing is happening at the unit level than at the UI level.
Another profound development change that some organizations have a difficulty grasping—culturally— is the notion of a much smaller release than what we are accustomed to. DevOps is based on much smaller changes streamed into the production pipeline deployed in small increments to lower the risk of traditionally larger releases that most organizations do. This “small change” mindset has profound impacts, such as easy deployment, easy rollbacks when necessary, small focused Dev and Test periods, faster automation, and smaller regression suites with a minuscule focus. All this for another foundation idea in DevOps: immediate feedback.
The much smaller, and faster regression test suites are a stumbling block for many test teams. I sympathize with this! The demand to continuously automate more persists in software development teams. In many cases, little thought was given to significant tagging, traceability, and most
importantly—retiring tests. This led to bloated and slow regression suites. Continuous Testing demands lean and mean automated suites. This is a shift in mindset that may take some time for test teams to catch up to, as it is a different style of automation. It’s lean. But still remember to automate, automate and automate, but make it smarter and leaner! Making choices to not run all the automation, at all times is risky and difficult. Be sure to communicate and collaborate. Explain the test coverage for various regression runs.
If you are doing DevOps/Continuous Delivery/Continuous Testing, pay attention to this issue! In this issue, we gear our focus towards Testing in DevOps. There’s an intriguing interview with Dave Farley, one of the pioneers of Continuous Delivery. Also included is an article from our partner, Sauce Labs that explores the complexity of combining Continuous Testing and DevOps. Whether you are just beginning your Continuous Delivery journey, or have already implemented a pipeline, this issue has you covered!
