SLAs and Cloud Testing

A test team’s job is to report test results, not set or guarantee that you will meet the SLAs.

In the rush to cloud services, with everything-as-a-service, you will hear people talking about SLAs. What is this about and what does it have to do with testing?

A Service Level Agreement, or SLA, is a contract a service provider promises for a defined level of service, such as response time, throughput or capacity.

When a customer signs up for service, the provider promises, in contract, certain levels of service. The most important aspect is usually availability. 

Availability is the ability to access the system. Everyone wants their service available all the time. This is an impossibility for both good and bad reasons. Good reasons – downtime, patches, new build migration and system upgrades. Bad reasons – system crashes, security problems – denial of service, network/infrastructure problems.

Downtime happens and SLAs are meant to provide a promise from the provider, of how available the system will be.

This is an important part of marketing, sales and contracts for any cloud service provider, from HP and Amazon to consumer products in the cloud like Netflix and Foursquare.

Gartner analyst Lyida Leong blogged that Amazon Web Services, which Gartner named a market-leader in infrastructure-as-a-service cloud computing, has the “dubious status of ‘worst SLA (service level agreement) of any major cloud provider.” She also wrote, “HP’s newly available public cloud service could be even worse.”

What are reasonable SLAs for availability? What is common? The answer differs based on the service. For example, many people use “4 9s” which represents 99.99% uptime:

Think about this: for four nines availability allows 1 minute downtime per week. Wow. How safe do you think it is for a company to guarantee this? In one full year, that means down time of less than an hour.

So what does this mean for testing? Testing SLAs is all about system performance testing; load testing, stress testing. It is measurement of the various attributes of the product; capacity, response time, against agreed upon standards. What you have to remember is a test team’s job is to report test results, not set or guarantee you will meet the SLAs.

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

Special considerations that should be applied to an application running in the cloud. Over the last weeks, I have found myself in several rather intense discussions about “cloud testing”: what it is, what it isn’t, and what it means for testing and QA professionals. The major source of confusion in these discussions usually revolves around ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing
If you haven’t already caught a glimpse, we announced in January the Testing in DevOps and Continuous Testing videos series, available on YouTube!
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
In this article, I share some of my experiences and observations on training teams, mainly in corporate settings. The operative word here is “team”, not “individual”. When training teams or groups in an organization, many of the considerations and benefits are different than those for the individual. We’ll examine those differences and I will share successful solutions. ...
Cloud computing has been the buzzword in the world of Information Technology for quite some time and it is likely to retain that status in the coming years. Cloud computing has been helping business enterprises deliver services faster and cheaper compared to all other existing delivery models. Small and medium business enterprises have changed their ...
It’s no secret that the cloud is growing at an exponential rate. By 2016, two-thirds of the world’s server workloads will exist in the cloud. But according to Cisco’s 2012 Cloud Index, less than half of server workloads currently run in the cloud. Closing the gap between current capabilities and future requirements is a mission-critical ...
From adopting the culture, to implementing Continuous Delivery With the relative newness of DevOps, there are not yet a ton of DevOps books. That’s why we’ve assembled a list of the 7 best DevOps books based on four criteria: the number of ratings from Amazon, the average Amazon rating, number of ratings from GoodReads and the ...
Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
Times have changed, the tools have improved, and with books like this available you have no reason to not give CI a go. I still remember the first time I was on a project that used NAnt and CruiseControl.NET. It was years ago and both were new tools with plenty of bugs. The project manager ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe