Make Design a Priority for Cloud Testing

Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be easily ramped up by access to additional computing resources. How this happens, and the configuration of those resources, can vary. Designing tests that can simulate these real-world scenarios and that are maintainable takes a significant amount of up-front planning.

When Cisco’s Telepresence division migrated the initial PC based applications to the cloud one of the primary goals was to automate as many of the test routines as possible, and to create the automation in a way that would allow it to scale as the application itself grew and changed. Rather than try to build the automation around a particular automation software, the focus was on test design first and tool second. In the end this proved to be the right approach.

Cloud Testing Considerations

Cloud deployments require testing the application, platform and infrastructure. Depending on whether it’s a private or public cloud there may be access issues that need to be addressed, and these will need to be accommodated in the test design. Below are several of the primary considerations the test teams for this project had to address.

Application changes and on premise interfaces

  • Would moving the application to the cloud require changes to the application to fit the environment available on the cloud platform?
  • How should the test cases be designed and test automation implemented to provide the required test coverage for these changes?

Data migration

  • Would new tests be needed for the storage services available on the cloud platform?
  • Data moved from managed servers to cloud storage may have to be encrypted for security and compliance needs. How would the additional security requirements be tested?

Performance

  • Was a plan needed to load and stress test the application on the cloud platform to ensure that system response is as per SLAs?

Scalability

  • Would the test scenarios be robust enough to simulate on-demand needs of the application?

Priorities for testing

Testing the applications being migrated to the cloud required a careful assessment from the perspective of what changes were required for cloud deployment. This assessment helped the test team understand application areas that needed specific focus during testing on the cloud platform. In addition, knowledge of the cloud platform capabilities for scalability, availability and disaster recovery had to be disseminated to the teams to assure successful test planning.

The testing plan was based on a model that included testing across the application, platform and infrastructure.

The Process

The project consisted of separate teams for portal testing and testing the supported devices. The Portal team was responsible for testing web applications that included a user portal and an administration portal that controlled all account information and updates, subscriptions information and modifications. Proper setup of the test environment required the team to become familiar with the target cloud platform.

Both LogiGear and Cisco teams collaborated to develop the ideal test plan for the existing web applications to be deployed in the cloud. Test engineers designed and developed python-based automated test cases utilizing the Action-Based-Testing (ABT) method. The big advantage of using ABT for this project was the ability to create well-defined test cases that were able to run independently of each other. This approach localized the impact of functional changes under test allowing the automation to be easily maintained. Maintainability of automated tests is essential to any automation project, but critical for testing of cloud applications where the application owner may not be involved in changes to the infrastructure the application runs on. It is precisely these types of projects where module-based test design proves its value.

This project required additional resources for computations and data storage to provide a true representation of the production data and expected load. Using virtual machines (VMs) proved to be the most effective way to accomplish this, plus VMs made it possible to scale quickly. After the initial testing, the VMs were saved as images in cloud storage, and all the cloud resources were released to keep the overall testing costs low. After the initial tests the VMs were restored as needed to recreate the test environment for subsequent releases.

Key learning

Test design planning for the cloud migration project was ultimately the primary component in achieving success for this project. Automation was the only way to handle the number of tests, and the amount of data to be tested, and in turn, the module-based approach made it possible to maintain tests easily even in a very fluid development environment.

When moving to the cloud there are two important test considerations. The first is the need to test all aspects of the cloud environment, starting with the cloud client through the application (SaaS), platform (PaaS) and Infrastructure (IaaS) layers. The second consideration is the capability of the team to plan and design the tests. Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Joe Luthy
Joe has over 15 years product marketing experience in technology and Internet services for major companies including AT&T and Verisign.TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012.

The Related Post

Disclaimer: This article was originally published in the LogiGear Magazine, but has been updated for the June 2020 issue. Leading an organization into Continuous Delivery is a daunting task that usually takes software development teams months if not years. This guide covers the main points that organizations need to consider to increase their chance of ...
LogiGear Magazine June Testing in Continuous Delivery Issue 2017
DevOps may be the next big buzzword, but Test teams really need to focus on its little sister, Continuous Delivery If you pay attention to trends in software development—from the perspective of what some sophisticated teams are doing, what articles and books are being written, to conference topics, you may have noticed the tools being ...
Do you want to speed up your automated tests by a factor of 10 and deploy your application continuously? In this article we share how the JIRA development team at Atlassian has accomplished this using Stages in Bamboo. Stages have allowed the JIRA Development team to take a week’s worth of testing and condense it ...
LogiGear University announces the launch of a new, free video series on Testing in DevOps and Continuous Testing which is available today.
LogiGear Magazine – February 2013 – The Rapidly Changing Software Testing Landscape
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
The book is an incredibly effective and valuable guide that details the risks that arise when deploying cloud solutions. More importantly, it provides details on how to test cloud services, to ensure that the proposed cloud service will work as described. It is a great start to the topic. The 6 chapters detail a paradigm ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
A test team’s job is to report test results, not set or guarantee that you will meet the SLAs. In the rush to cloud services, with everything-as-a-service, you will hear people talking about SLAs. What is this about and what does it have to do with testing? A Service Level Agreement, or SLA, is a ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing
Fitting QA into a modern DevOps group In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities. However, for folks outside the engineering team – say in Operations ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe