Free Delivery: 5 Things Amazon Prime Taught Me About Deployment Automation


Eric Minick brilliantly demonstrates how automation works in Continuous Delivery using this real-world “case study”.

As a product manager at IBM working on our UrbanCode products, I think a lot about Continuous Delivery, how to do it and what it does for teams. Continuous Delivery techniques enable teams to deliver software faster and with less risk. The ideal in continuous delivery is for a developer to push a code change, and have that change rapidly built and tested. Minutes later, the code should either be rejected as having a rejection, or be available to be released into production. Needless to say, most teams are still working towards that ideal.

The Build, Deploy and Release tools that I work on form the skeleton of a continuous delivery pipeline by automating many of the activities that transform code into software, and deploy that software into test and production environments. The “muscle” that attaches to that “skeleton” is automated testing. To determine fitness for production rapidly, teams must be able to get code into test environments quickly and correctly, then execute their tests equally quickly.

Continuous Delivery is a team sport. It requires collaboration between development, testers and operations.

A recent experience with Amazon Prime impacted how I thought about Continuous Delivery. I had noticed a light bulb at home went out and we were out of spares. Instead of adding a trip to the store to my to-do list, my first instinct was to order a light bulb off Amazon with two day shipping. That would be a little crazy if I didn’t have Amazon Prime.

Prime has an annual fee, after which any purchase I make for the year has free two-day shipping. So instead of a $10 pack of light bulbs that I pay $5 to have shipped in a week, I just pay $10. The goods show up before I would bother to go to the store, for less effort, and without paying for gas.

So, what does this have to do with automated software deployments? When you make it cheap and easy to do something, you change behavior in radical ways.

1. Frequency Increases

One of the ways Prime is a winner for Amazon, is that it encourages me to buy more frequently. It is easy and without shipping price penalties I feel like I’m getting a bargain. I also buy tons more music off iTunes than I ever did from music stores.

Likewise, automation makes deployments cheap enough that you do more deployments. We’ve seen customers increase their number of deployments to test environments by an order of magnitude once automated.

For some clients, this has resulted in manual testers being overwhelmed by the flow of change coming towards them. More typically though, more frequent delivery reduces the frequency of bugs being reported which have already been detected and fixed in a more recent version.

Small Things Count: There’s a surprising difference between having a web page where someone can easily request a deployment and automatically deploying to test as the result of a new build or a nightly schedule. On event/schedule gets has a bigger impact over self-service. In Amazon terms, this is “subscribe and save“. New parents, take note, this was made for diapers. Developers, this helps turn CI into CD.

2. It’s About Effort, Not Speed

If I’d needed drain cleaner, speed might have been paramount in deciding how I shopped. Likewise, for some deployment teams, automation’s main benefit is shorter outage windows across development, test, and production environments.

But for most of us, the attractive part of shopping online is just how easy it is. Online, I do my part of the purchasing in 5 minutes, rather than a half hour in the real world. Most teams who are implementing automation are doing so because the deployment guys are stretched beyond capacity and need time back. Shorter outages are a side benefit.

3. Batch Sizes Shrink

I used to avoid buying just one at a time when shopping online. I would wait until I had a few things to get and save on shipping by ordering them together. Looking at my orders before and after purchasing Prime, I’ve seen my orders get smaller and smaller.

The same thing happens with deployment automation. When deployments are a big, scary ordeal teams set up release weekends and release trains where numerous projects go out in a big effort. With automation comes confidence, comfort and eventually more independence project to project. When something is ready to ship, it goes out. It doesn’t wait for a bunch of unrelated other projects to make a release weekend worthwhile.

Similarly, in test environments, because the frequency of deployment has increased the number of changes represented by each update is smaller. This means that when something breaks, the culprit is much clearer. More effort is expended fixing problems and less is spent assigning blame.

This is a nice little side effect. You see faster time to market, and those practicing Lean are thrilled. They see less “inventory” waste, and smaller batch sizes tend to appeal to both the Lean and Agile camps for a host of reasons.

4. Free Returns

Our family now buys many of our clothes through Amazon Prime and shoes from Zappos (an Amazon acquisition). In both cases, the ability to order, try on, and return any rejects with free shipping both ways has been a huge factor. My wife no longer goes to six stores to try things on and can simply order with confidence that if she’s wrong about the fit, reconsidering is easy.

This is the real key to deployment rollback automation. Because rollbacks allow us to reconsider more easily, they enable us to be more aggressive in releasing (even to just test environments). Partial (canary) production deployments build on this pattern.

5. Do Things the “Right” Way

All these little benefits add up to one key thing for Amazon— we do more of our shopping there than we otherwise would have. We buy more stuff how they want us to— on their website.

If you are trying to wrangle people and get them to deploy following the proper approvals, and deployment plans, learn this lesson. Make it easier for people to deploy following protocol than to work around the system. Push button deployment automation is so much easier than manual deployments, that any process you build into that automation system has much higher odds of being followed than something enforced by sternly worded memos.

People just want to get their jobs done. They want to write code. They want to test. They want to deliver value to the customer. While there will always be defenders of the status quo, if you provide an easier way to get things done, most people will follow that path. If the easy path includes all the right checks and controls, the checks and controls will be followed.

*This is me talking about a service I consume and like. Amazon hasn’t endorsed this, and all their trademarks belong to them. Their services, pricing, etc can change. And seriously, think about the gas, jet fuel, and people’s time you waste when you order a light bulb two day shipping. At least get an eight-pack or one of those nifty new LED bulbs. On the other hand, don’t have any guilt about deploying more.

Eric Minick
Eric Minick is internationally recognized as a leading authority on continuous delivery and DevOps. Eric joined IBM four years ago with the acquisition of UrbanCode where he had worked as a developer, technical seller, and evangelist for a decade. Today, he has responsibility for leading the product management team overseeing continuous delivery solutions in IBM Hybrid Cloud software including the UrbanCode suite.

The Related Post

By Jez Humble and David Farley Continuous Delivery from Jez Humble and David Farley is an important contribution to the field of software development. It takes continuous integration to the logical conclusion and covers how to set up a continuous integration system, delving into everything from check-in to delivery to production. It doesn’t state you ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
DevOps has been described as Agile on Steroids; DevOps has also been described as Agile for Operations/IT. I like both of those descriptions. Many organizations want Development, Test, and Operations teams to move to DevOps now. DevOps is a big topic, but DevOps is not the focus of this article. We will not be talking ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
DevOps may be the next big buzzword, but Test teams really need to focus on its little sister, Continuous Delivery If you pay attention to trends in software development—from the perspective of what some sophisticated teams are doing, what articles and books are being written, to conference topics, you may have noticed the tools being ...
Fitting QA into a modern DevOps group In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities. However, for folks outside the engineering team – say in Operations ...
Over the years we’ve provided an extensive number of articles, videos, and infographics that provide a wealth of knowledge about Continuous Delivery.
A test team’s job is to report test results, not set or guarantee that you will meet the SLAs. In the rush to cloud services, with everything-as-a-service, you will hear people talking about SLAs. What is this about and what does it have to do with testing? A Service Level Agreement, or SLA, is a ...
Sauce Lab’s Perspective on Integration with Continuous Testing The term ‘DevOps’ implies that implementing an effective development and production workflow is as easy as developing the collaborative interaction between developers and IT operations. On the surface, DevOps may appear as a simple and straightforward idea but, as you will find out, there is more than meets ...
…On what you need to know before making the transition to EaaS 1. What are the main differences between cloud-based environments and cloud infrastructure? An environment is a collection of infrastructure elements working in conjunction to enable an application stack to work. For example, a simple 3-tier application, with a web front-end component, a business logic ...
Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news