Make Design a Priority for Cloud Testing

Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be easily ramped up by access to additional computing resources. How this happens, and the configuration of those resources, can vary. Designing tests that can simulate these real-world scenarios and that are maintainable takes a significant amount of up-front planning.

When Cisco’s Telepresence division migrated the initial PC based applications to the cloud one of the primary goals was to automate as many of the test routines as possible, and to create the automation in a way that would allow it to scale as the application itself grew and changed. Rather than try to build the automation around a particular automation software, the focus was on test design first and tool second. In the end this proved to be the right approach.

Cloud Testing Considerations

Cloud deployments require testing the application, platform and infrastructure. Depending on whether it’s a private or public cloud there may be access issues that need to be addressed, and these will need to be accommodated in the test design. Below are several of the primary considerations the test teams for this project had to address.

Application changes and on premise interfaces

  • Would moving the application to the cloud require changes to the application to fit the environment available on the cloud platform?
  • How should the test cases be designed and test automation implemented to provide the required test coverage for these changes?

Data migration

  • Would new tests be needed for the storage services available on the cloud platform?
  • Data moved from managed servers to cloud storage may have to be encrypted for security and compliance needs. How would the additional security requirements be tested?

Performance

  • Was a plan needed to load and stress test the application on the cloud platform to ensure that system response is as per SLAs?

Scalability

  • Would the test scenarios be robust enough to simulate on-demand needs of the application?

Priorities for testing

Testing the applications being migrated to the cloud required a careful assessment from the perspective of what changes were required for cloud deployment. This assessment helped the test team understand application areas that needed specific focus during testing on the cloud platform. In addition, knowledge of the cloud platform capabilities for scalability, availability and disaster recovery had to be disseminated to the teams to assure successful test planning.

The testing plan was based on a model that included testing across the application, platform and infrastructure.

The Process

The project consisted of separate teams for portal testing and testing the supported devices. The Portal team was responsible for testing web applications that included a user portal and an administration portal that controlled all account information and updates, subscriptions information and modifications. Proper setup of the test environment required the team to become familiar with the target cloud platform.

Both LogiGear and Cisco teams collaborated to develop the ideal test plan for the existing web applications to be deployed in the cloud. Test engineers designed and developed python-based automated test cases utilizing the Action-Based-Testing (ABT) method. The big advantage of using ABT for this project was the ability to create well-defined test cases that were able to run independently of each other. This approach localized the impact of functional changes under test allowing the automation to be easily maintained. Maintainability of automated tests is essential to any automation project, but critical for testing of cloud applications where the application owner may not be involved in changes to the infrastructure the application runs on. It is precisely these types of projects where module-based test design proves its value.

This project required additional resources for computations and data storage to provide a true representation of the production data and expected load. Using virtual machines (VMs) proved to be the most effective way to accomplish this, plus VMs made it possible to scale quickly. After the initial testing, the VMs were saved as images in cloud storage, and all the cloud resources were released to keep the overall testing costs low. After the initial tests the VMs were restored as needed to recreate the test environment for subsequent releases.

Key learning

Test design planning for the cloud migration project was ultimately the primary component in achieving success for this project. Automation was the only way to handle the number of tests, and the amount of data to be tested, and in turn, the module-based approach made it possible to maintain tests easily even in a very fluid development environment.

When moving to the cloud there are two important test considerations. The first is the need to test all aspects of the cloud environment, starting with the cloud client through the application (SaaS), platform (PaaS) and Infrastructure (IaaS) layers. The second consideration is the capability of the team to plan and design the tests. Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Joe Luthy
Joe has over 15 years product marketing experience in technology and Internet services for major companies including AT&T and Verisign.TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012.

The Related Post

Times have changed, the tools have improved, and with books like this available you have no reason to not give CI a go. I still remember the first time I was on a project that used NAnt and CruiseControl.NET. It was years ago and both were new tools with plenty of bugs. The project manager ...
Cloud computing has been the buzzword in the world of Information Technology for quite some time and it is likely to retain that status in the coming years. Cloud computing has been helping business enterprises deliver services faster and cheaper compared to all other existing delivery models. Small and medium business enterprises have changed their ...
How to ensure a successful test-driven environment In order to ensure a higher quality product is released in the end, many teams have turned to test-driven development. Under this scenario, quality assurance metrics professionals first create various QA tests, and then software engineers code based on these tests, typically while using a robust enterprise test management ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
From adopting the culture, to implementing Continuous Delivery With the relative newness of DevOps, there are not yet a ton of DevOps books. That’s why we’ve assembled a list of the 7 best DevOps books based on four criteria: the number of ratings from Amazon, the average Amazon rating, number of ratings from GoodReads and the ...
LogiGear Magazine – December 2013 – Cloud Testing
As a software development company, what is your goal? What is the one thing you feel you need to do to ensure you have a job at the beginning of each wonderful work week? The answer is actually quite simple; You need to deliver a quality product. Like how I used the word simple? Although the answer I ...
Do you want to speed up your automated tests by a factor of 10 and deploy your application continuously? In this article we share how the JIRA development team at Atlassian has accomplished this using Stages in Bamboo. Stages have allowed the JIRA Development team to take a week’s worth of testing and condense it ...
LogiGear Magazine June Testing in Continuous Delivery Issue 2017
Disclaimer: This article was originally published in the LogiGear Magazine, but has been updated for the June 2020 issue. Leading an organization into Continuous Delivery is a daunting task that usually takes software development teams months if not years. This guide covers the main points that organizations need to consider to increase their chance of ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe