A test team’s job is to report test results, not set or guarantee that you will meet the SLAs.
In the rush to cloud services, with everything-as-a-service, you will hear people talking about SLAs. What is this about and what does it have to do with testing?
A Service Level Agreement, or SLA, is a contract a service provider promises for a defined level of service, such as response time, throughput or capacity.
When a customer signs up for service, the provider promises, in contract, certain levels of service. The most important aspect is usually availability.
Availability is the ability to access the system. Everyone wants their service available all the time. This is an impossibility for both good and bad reasons. Good reasons – downtime, patches, new build migration and system upgrades. Bad reasons – system crashes, security problems – denial of service, network/infrastructure problems.
Downtime happens and SLAs are meant to provide a promise from the provider, of how available the system will be.
This is an important part of marketing, sales and contracts for any cloud service provider, from HP and Amazon to consumer products in the cloud like Netflix and Foursquare.
Gartner analyst Lyida Leong blogged that Amazon Web Services, which Gartner named a market-leader in infrastructure-as-a-service cloud computing, has the “dubious status of ‘worst SLA (service level agreement) of any major cloud provider.” She also wrote, “HP’s newly available public cloud service could be even worse.”
What are reasonable SLAs for availability? What is common? The answer differs based on the service. For example, many people use “4 9s” which represents 99.99% uptime:
Think about this: for four nines availability allows 1 minute downtime per week. Wow. How safe do you think it is for a company to guarantee this? In one full year, that means down time of less than an hour.
So what does this mean for testing? Testing SLAs is all about system performance testing; load testing, stress testing. It is measurement of the various attributes of the product; capacity, response time, against agreed upon standards. What you have to remember is a test team’s job is to report test results, not set or guarantee you will meet the SLAs.
