Free Delivery: 5 Things Amazon Prime Taught Me About Deployment Automation


Eric Minick brilliantly demonstrates how automation works in Continuous Delivery using this real-world “case study”.

As a product manager at IBM working on our UrbanCode products, I think a lot about Continuous Delivery, how to do it and what it does for teams. Continuous Delivery techniques enable teams to deliver software faster and with less risk. The ideal in continuous delivery is for a developer to push a code change, and have that change rapidly built and tested. Minutes later, the code should either be rejected as having a rejection, or be available to be released into production. Needless to say, most teams are still working towards that ideal.

The Build, Deploy and Release tools that I work on form the skeleton of a continuous delivery pipeline by automating many of the activities that transform code into software, and deploy that software into test and production environments. The “muscle” that attaches to that “skeleton” is automated testing. To determine fitness for production rapidly, teams must be able to get code into test environments quickly and correctly, then execute their tests equally quickly.

Continuous Delivery is a team sport. It requires collaboration between development, testers and operations.

A recent experience with Amazon Prime impacted how I thought about Continuous Delivery. I had noticed a light bulb at home went out and we were out of spares. Instead of adding a trip to the store to my to-do list, my first instinct was to order a light bulb off Amazon with two day shipping. That would be a little crazy if I didn’t have Amazon Prime.

Prime has an annual fee, after which any purchase I make for the year has free two-day shipping. So instead of a $10 pack of light bulbs that I pay $5 to have shipped in a week, I just pay $10. The goods show up before I would bother to go to the store, for less effort, and without paying for gas.

So, what does this have to do with automated software deployments? When you make it cheap and easy to do something, you change behavior in radical ways.

1. Frequency Increases

One of the ways Prime is a winner for Amazon, is that it encourages me to buy more frequently. It is easy and without shipping price penalties I feel like I’m getting a bargain. I also buy tons more music off iTunes than I ever did from music stores.

Likewise, automation makes deployments cheap enough that you do more deployments. We’ve seen customers increase their number of deployments to test environments by an order of magnitude once automated.

For some clients, this has resulted in manual testers being overwhelmed by the flow of change coming towards them. More typically though, more frequent delivery reduces the frequency of bugs being reported which have already been detected and fixed in a more recent version.

Small Things Count: There’s a surprising difference between having a web page where someone can easily request a deployment and automatically deploying to test as the result of a new build or a nightly schedule. On event/schedule gets has a bigger impact over self-service. In Amazon terms, this is “subscribe and save“. New parents, take note, this was made for diapers. Developers, this helps turn CI into CD.

2. It’s About Effort, Not Speed

If I’d needed drain cleaner, speed might have been paramount in deciding how I shopped. Likewise, for some deployment teams, automation’s main benefit is shorter outage windows across development, test, and production environments.

But for most of us, the attractive part of shopping online is just how easy it is. Online, I do my part of the purchasing in 5 minutes, rather than a half hour in the real world. Most teams who are implementing automation are doing so because the deployment guys are stretched beyond capacity and need time back. Shorter outages are a side benefit.

3. Batch Sizes Shrink

I used to avoid buying just one at a time when shopping online. I would wait until I had a few things to get and save on shipping by ordering them together. Looking at my orders before and after purchasing Prime, I’ve seen my orders get smaller and smaller.

The same thing happens with deployment automation. When deployments are a big, scary ordeal teams set up release weekends and release trains where numerous projects go out in a big effort. With automation comes confidence, comfort and eventually more independence project to project. When something is ready to ship, it goes out. It doesn’t wait for a bunch of unrelated other projects to make a release weekend worthwhile.

Similarly, in test environments, because the frequency of deployment has increased the number of changes represented by each update is smaller. This means that when something breaks, the culprit is much clearer. More effort is expended fixing problems and less is spent assigning blame.

This is a nice little side effect. You see faster time to market, and those practicing Lean are thrilled. They see less “inventory” waste, and smaller batch sizes tend to appeal to both the Lean and Agile camps for a host of reasons.

4. Free Returns

Our family now buys many of our clothes through Amazon Prime and shoes from Zappos (an Amazon acquisition). In both cases, the ability to order, try on, and return any rejects with free shipping both ways has been a huge factor. My wife no longer goes to six stores to try things on and can simply order with confidence that if she’s wrong about the fit, reconsidering is easy.

This is the real key to deployment rollback automation. Because rollbacks allow us to reconsider more easily, they enable us to be more aggressive in releasing (even to just test environments). Partial (canary) production deployments build on this pattern.

5. Do Things the “Right” Way

All these little benefits add up to one key thing for Amazon— we do more of our shopping there than we otherwise would have. We buy more stuff how they want us to— on their website.

If you are trying to wrangle people and get them to deploy following the proper approvals, and deployment plans, learn this lesson. Make it easier for people to deploy following protocol than to work around the system. Push button deployment automation is so much easier than manual deployments, that any process you build into that automation system has much higher odds of being followed than something enforced by sternly worded memos.

People just want to get their jobs done. They want to write code. They want to test. They want to deliver value to the customer. While there will always be defenders of the status quo, if you provide an easier way to get things done, most people will follow that path. If the easy path includes all the right checks and controls, the checks and controls will be followed.

*This is me talking about a service I consume and like. Amazon hasn’t endorsed this, and all their trademarks belong to them. Their services, pricing, etc can change. And seriously, think about the gas, jet fuel, and people’s time you waste when you order a light bulb two day shipping. At least get an eight-pack or one of those nifty new LED bulbs. On the other hand, don’t have any guilt about deploying more.

Eric Minick
Eric Minick is internationally recognized as a leading authority on continuous delivery and DevOps. Eric joined IBM four years ago with the acquisition of UrbanCode where he had worked as a developer, technical seller, and evangelist for a decade. Today, he has responsibility for leading the product management team overseeing continuous delivery solutions in IBM Hybrid Cloud software including the UrbanCode suite.

The Related Post

It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
From adopting the culture, to implementing Continuous Delivery With the relative newness of DevOps, there are not yet a ton of DevOps books. That’s why we’ve assembled a list of the 7 best DevOps books based on four criteria: the number of ratings from Amazon, the average Amazon rating, number of ratings from GoodReads and the ...
It’s no secret that the cloud is growing at an exponential rate. By 2016, two-thirds of the world’s server workloads will exist in the cloud. But according to Cisco’s 2012 Cloud Index, less than half of server workloads currently run in the cloud. Closing the gap between current capabilities and future requirements is a mission-critical ...
How to Adopt the “Third Way” in the Dojo’s Method to Master CD In The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations the authors describe “The Three Ways” – the underlying principles forming the basis for all DevOps practices. 
Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments. Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be ...
Fitting QA into a modern DevOps group In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities. However, for folks outside the engineering team – say in Operations ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
LogiGear Magazine – December 2013 – Cloud Testing
Sauce Lab’s Perspective on Integration with Continuous Testing The term ‘DevOps’ implies that implementing an effective development and production workflow is as easy as developing the collaborative interaction between developers and IT operations. On the surface, DevOps may appear as a simple and straightforward idea but, as you will find out, there is more than meets ...
From the culture shift, to differences in Agile, Dave Farley and Michael Hackett discuss the nitty gritty of Testing in DevOps. For this issue of LogiGear Magazine, our very own Michael Hackett sat down with one of the godfathers of Continuous Delivery, David Farley. In this exclusive interview, David discusses how test teams and automation ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news