An Interview with Skytap’s Sumit Mehrotra

…On what you need to know before making the transition to EaaS

LogiGear_Skytap's Sumit

1. What are the main differences between cloud-based environments and cloud infrastructure?

An environment is a collection of infrastructure elements working in conjunction to enable an application stack to work.

For example, a simple 3-tier application, with a web front-end component, a business logic component and a database component requires, at a minimum, the following infrastructure components: a few VMs, a few storage disks, and a network. All of these components are required, and they need to work together as an environment for the application to be functional. Note, that as the complexity of the application stack grows, the definition of an environment grows bigger than just infrastructure components. It can include VPNs, public IP addresses, an object store, and other service end-points.

From an application development and testing perspective, the application environment is the starting point of productive work. However, we see teams spending a large amount of time just getting the infrastructure components to a working state and hence losing out on a good portion of their productive work. Working with environments as a unit of work can enable these teams to be more productive and hence service customer and business requirements at a faster pace with higher quality.

2. What is “environment or infrastructure as code?” How can we use this idea in testing?

As the name suggests, infrastructure-as-Code is practice of representing infrastructure as code blocks so that they can be executed whenever needed to build an application stack (environment) in a repeatable and consistent manner. Depending on the actual tool used, it can allow you a variety of things including:

  • Deploy basic infrastructure elements, VMs, disks, networks, etc.
  • Install and configure operating systems
  • Deploy and configure application components
  • Populate application data

With a repeatable and consistent process, bugs can be reproduced, fixed, and validated efficiently. Developers can recreate the same environment that a tester is using to find bugs by using the same Infrastructure-as-Code scripts as the tester is using. This practice goes a long way in solving the problem of ‘it-works-on-my-machine’ syndrome.

There are various paradigms available to create consistent environments in a repeatable manner. Two such popular paradigms are “patterns” and “clones.” With patterns, you can create the environment from scratch each time—allowing you to progress the same exact code at each point of your software delivery pipeline. But it takes time for the environment to be provisioned and can be error-prone especially, for very complex environments. With clones, you can create another copy of the environment from a pre-existing copy. The process can be very fast, even for complex environments. However, moving the same clone through your SDLC can be a heavyweight process.

There is no right or wrong paradigm here. Successful teams have chosen a combination of paradigms and have made trade-offs between time and tooling consistency, to achieve the velocity they need in their software delivery lifecycle.

3. How do companies use cloud-based environments to facilitate testing in DevOps?

Testing especially in the DevOps world has the following key requirements:

  • Instant self-service access to application stacks for testing
  • Support the complexity and scale of the application stack needed at various stages of testing
  • Consistent and reproducible ways to create test environments
  • Ability to collaborate and share environments with developers and testers as part of a feedback loop. Constant feedback is a key aspect of Agile development
  • Ability to ensure that the right resources are available at any time and are being utilized judiciously

All of these requirements are hard to meet without a cloud-based platform. So, cloud-based environments are essential to facilitate testing workflows.

4We know that getting the right environments for testing can be difficult, and sometimes it’s out of a Test team’s control. How are innovative organizations solving this challenge?


Organizations are increasingly turning to the cloud for on-demand and self-service access test environments, but not every cloud platform is suited for the needs of the applications in an organization’s portfolio. Innovative organizations evaluate the environment needs of their applications, and some of the criteria that we have seen customers use are as follows:

  • Configurability – Can the platform deliver test environments that capture the complexity of my application at each stage of testing?
  • Consistency – Can the platform deliver test environments that are in the exact state that we need them to be, whenever I want?
  • Collaboration – Can the platform make it easier for my team (Devs + QA + Ops) to work together more productively?
  • Control – Does the platform have controls to ensure the right people have the appropriate resources to do their jobs?

5. How long does it take to build these environments? Some Test teams do not have dedicated IT members—does the Test team often do this?

The time to get a fully configured environment to test can vary in range from a few minutes to a few weeks. The availability of cloud platforms, and especially IaaS has reduced the far-end of the range to a few days primarily by taking the acquisition of hardware out of the equation. This is the lowest layer of the environment and typically, the one that used to take the longest.

Reducing the time to build these environments to minutes requires more than just infrastructure. It requires a platform that can support the deployment of fully configured environments. Paradigms like patterns and clones, discussed earlier, are essential to achieve that speed of deployment.

Additionally, these paradigms help with reuse of expertise within the organization as well. Environments capture the expertise needed to deploy various aspects of the application, be it infrastructure (IT team), presentation layer (UI team), business logic (app server team), database layer (DBAs). All the tester has to do is click a button or call an API to deploy the environment. The individual components are automatically deployed based on the definition of each component captured in the environment definition.

6. How does this impact running an automated test suite?

Automated testing at each stage can be made more efficient and predictable by using consistent, fully configured environments that can be deployed using API calls. Continuous Integration and Continuous Delivery can actually be realized if the environments of desired size and configuration can be deployed within seconds and minutes versus hours and days.

7. Getting the right data to test with, or setting up the right data is also a problem area for many teams. What strategies or tools help with this?

A few strategies can help with test data management:

  • Populating data in base environments which can then be cloned into new copies of the environment when needed. Once testing is done, the environment along and data can be thrown away. It is cheap to recreate a new environment with the desired data set.
  • Creating a database as part of environment creation. In this model, the data is created by using some automation code. This slows the set-up of the environments but ensures that the data creation is based on the latest desired state of the test data set.
  • Using data virtualization and data-masking tools to continuously keep a close-to-production copy of test data for testing. This is especially important in the later stages of testing like pre-prod certification testing.

8. Are cloud-based environments also a popular choice for those doing continuous integration and or continuous testing?

Absolutely. The key requirement for CI/CD—speed of execution and feedback—cannot be achieved until test environments can be deployed on-demand, in an automated fashion, in a consistent state, and with desired complexity.

9. We know containers are a stripped down wrapper around software components or services. How does Skytap use containers?

Skytap views containers as another component of the application stack just like all other components: infrastructure, middleware and applications. As customers modernize parts of their application that are a good fit for container technology, they are able to use them in Skytap environments, along with the other parts of their application that are not containerized. This is a very common scenario for traditional enterprise applications that tend to incorporate new technologies and are modernized over time.

With containers in Skytap, not only are customers able to use the container toolchain with container hosts in Skytap, but they are also able to still achieve the same efficiencies in their SDLC that they have traditionally done with environments. Additionally, Skytap customers have full control over their container hosts. This is not available in many container services, but is often desired by our enterprise customers. Customers can run containers on host VMs in their Skytap environments. They can also use the Skytap driver for Docker Machine to manage containers running on hosts in Skytap. If you are interested in learning more about Skytap’s support for containers, please contact

10. How do you see the use of containers impacting testing?

Container technology can affect testing in many ways. Some of these are:

  • Deploying test environments: Application test environments that are entirely container-based or those that use containers for some application components require new ways for deploying the test environments while still meeting the requirements of consistency.
  • Testing both code and deployment: Containers encapsulate both the code changes and the deployment process by design. Containers deploy fully configured application components across the testing pipeline. Testing that traditionally only used to focus on code changes and deployment changes in isolation will need to accommodate both.
  • Integration with tools across the testing toolchain: Consideration must be given to the choice of build, testing, and automation tools that can work with traditional application components side-by-side with container-based components.
Sumit Mehrotra
Sumit Mehrotra is Sr. Director of Product Management at Skytap, a role in which he is responsible for product strategy and roadmaps. Prior to Skytap, Sumit worked at Microsoft in different roles and has shipped a number of products, including Windows Azure and Windows operating system. Sumit holds an MBA from University of Chicago Booth School of Business and a Masters in Computer Science from Boston University.

The Related Post

LogiGear University announces the launch of a new, free video series on Testing in DevOps and Continuous Testing which is available today.
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
By Jez Humble and David Farley Continuous Delivery from Jez Humble and David Farley is an important contribution to the field of software development. It takes continuous integration to the logical conclusion and covers how to set up a continuous integration system, delving into everything from check-in to delivery to production. It doesn’t state you ...
As a software development company, what is your goal? What is the one thing you feel you need to do to ensure you have a job at the beginning of each wonderful work week? The answer is actually quite simple; You need to deliver a quality product. Like how I used the word simple? Although the answer I ...
Over the years we’ve provided an extensive number of articles, videos, and infographics that provide a wealth of knowledge about Continuous Delivery.
The book is an incredibly effective and valuable guide that details the risks that arise when deploying cloud solutions. More importantly, it provides details on how to test cloud services, to ensure that the proposed cloud service will work as described. It is a great start to the topic. The 6 chapters detail a paradigm ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
DevOps has been described as Agile on Steroids; DevOps has also been described as Agile for Operations/IT. I like both of those descriptions. Many organizations want Development, Test, and Operations teams to move to DevOps now. DevOps is a big topic, but DevOps is not the focus of this article. We will not be talking ...
From adopting the culture, to implementing Continuous Delivery With the relative newness of DevOps, there are not yet a ton of DevOps books. That’s why we’ve assembled a list of the 7 best DevOps books based on four criteria: the number of ratings from Amazon, the average Amazon rating, number of ratings from GoodReads and the ...
  LogiGear_Magazine_June 2016_Testing in the New World of DevOps  
Throw away clunky hyper-visors, and stop thinking about computer hardware and software license during your development projects. The first thing you think about when you hear “The Cloud” may not be development and testing. The Cloudy market is filled with SaaS applications, hosting, and cloud-based file systems. All are very useful, and offer a clear ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news