Free Delivery: 5 Things Amazon Prime Taught Me About Deployment Automation


Eric Minick brilliantly demonstrates how automation works in Continuous Delivery using this real-world “case study”.

As a product manager at IBM working on our UrbanCode products, I think a lot about Continuous Delivery, how to do it and what it does for teams. Continuous Delivery techniques enable teams to deliver software faster and with less risk. The ideal in continuous delivery is for a developer to push a code change, and have that change rapidly built and tested. Minutes later, the code should either be rejected as having a rejection, or be available to be released into production. Needless to say, most teams are still working towards that ideal.

The Build, Deploy and Release tools that I work on form the skeleton of a continuous delivery pipeline by automating many of the activities that transform code into software, and deploy that software into test and production environments. The “muscle” that attaches to that “skeleton” is automated testing. To determine fitness for production rapidly, teams must be able to get code into test environments quickly and correctly, then execute their tests equally quickly.

Continuous Delivery is a team sport. It requires collaboration between development, testers and operations.

A recent experience with Amazon Prime impacted how I thought about Continuous Delivery. I had noticed a light bulb at home went out and we were out of spares. Instead of adding a trip to the store to my to-do list, my first instinct was to order a light bulb off Amazon with two day shipping. That would be a little crazy if I didn’t have Amazon Prime.

Prime has an annual fee, after which any purchase I make for the year has free two-day shipping. So instead of a $10 pack of light bulbs that I pay $5 to have shipped in a week, I just pay $10. The goods show up before I would bother to go to the store, for less effort, and without paying for gas.

So, what does this have to do with automated software deployments? When you make it cheap and easy to do something, you change behavior in radical ways.

1. Frequency Increases

One of the ways Prime is a winner for Amazon, is that it encourages me to buy more frequently. It is easy and without shipping price penalties I feel like I’m getting a bargain. I also buy tons more music off iTunes than I ever did from music stores.

Likewise, automation makes deployments cheap enough that you do more deployments. We’ve seen customers increase their number of deployments to test environments by an order of magnitude once automated.

For some clients, this has resulted in manual testers being overwhelmed by the flow of change coming towards them. More typically though, more frequent delivery reduces the frequency of bugs being reported which have already been detected and fixed in a more recent version.

Small Things Count: There’s a surprising difference between having a web page where someone can easily request a deployment and automatically deploying to test as the result of a new build or a nightly schedule. On event/schedule gets has a bigger impact over self-service. In Amazon terms, this is “subscribe and save“. New parents, take note, this was made for diapers. Developers, this helps turn CI into CD.

2. It’s About Effort, Not Speed

If I’d needed drain cleaner, speed might have been paramount in deciding how I shopped. Likewise, for some deployment teams, automation’s main benefit is shorter outage windows across development, test, and production environments.

But for most of us, the attractive part of shopping online is just how easy it is. Online, I do my part of the purchasing in 5 minutes, rather than a half hour in the real world. Most teams who are implementing automation are doing so because the deployment guys are stretched beyond capacity and need time back. Shorter outages are a side benefit.

3. Batch Sizes Shrink

I used to avoid buying just one at a time when shopping online. I would wait until I had a few things to get and save on shipping by ordering them together. Looking at my orders before and after purchasing Prime, I’ve seen my orders get smaller and smaller.

The same thing happens with deployment automation. When deployments are a big, scary ordeal teams set up release weekends and release trains where numerous projects go out in a big effort. With automation comes confidence, comfort and eventually more independence project to project. When something is ready to ship, it goes out. It doesn’t wait for a bunch of unrelated other projects to make a release weekend worthwhile.

Similarly, in test environments, because the frequency of deployment has increased the number of changes represented by each update is smaller. This means that when something breaks, the culprit is much clearer. More effort is expended fixing problems and less is spent assigning blame.

This is a nice little side effect. You see faster time to market, and those practicing Lean are thrilled. They see less “inventory” waste, and smaller batch sizes tend to appeal to both the Lean and Agile camps for a host of reasons.

4. Free Returns

Our family now buys many of our clothes through Amazon Prime and shoes from Zappos (an Amazon acquisition). In both cases, the ability to order, try on, and return any rejects with free shipping both ways has been a huge factor. My wife no longer goes to six stores to try things on and can simply order with confidence that if she’s wrong about the fit, reconsidering is easy.

This is the real key to deployment rollback automation. Because rollbacks allow us to reconsider more easily, they enable us to be more aggressive in releasing (even to just test environments). Partial (canary) production deployments build on this pattern.

5. Do Things the “Right” Way

All these little benefits add up to one key thing for Amazon— we do more of our shopping there than we otherwise would have. We buy more stuff how they want us to— on their website.

If you are trying to wrangle people and get them to deploy following the proper approvals, and deployment plans, learn this lesson. Make it easier for people to deploy following protocol than to work around the system. Push button deployment automation is so much easier than manual deployments, that any process you build into that automation system has much higher odds of being followed than something enforced by sternly worded memos.

People just want to get their jobs done. They want to write code. They want to test. They want to deliver value to the customer. While there will always be defenders of the status quo, if you provide an easier way to get things done, most people will follow that path. If the easy path includes all the right checks and controls, the checks and controls will be followed.

*This is me talking about a service I consume and like. Amazon hasn’t endorsed this, and all their trademarks belong to them. Their services, pricing, etc can change. And seriously, think about the gas, jet fuel, and people’s time you waste when you order a light bulb two day shipping. At least get an eight-pack or one of those nifty new LED bulbs. On the other hand, don’t have any guilt about deploying more.

Eric Minick
Eric Minick is internationally recognized as a leading authority on continuous delivery and DevOps. Eric joined IBM four years ago with the acquisition of UrbanCode where he had worked as a developer, technical seller, and evangelist for a decade. Today, he has responsibility for leading the product management team overseeing continuous delivery solutions in IBM Hybrid Cloud software including the UrbanCode suite.

The Related Post

It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments. Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be ...
LogiGear Magazine June Issue 2018: TESTING in DEVOPS
By Jez Humble and David Farley Continuous Delivery from Jez Humble and David Farley is an important contribution to the field of software development. It takes continuous integration to the logical conclusion and covers how to set up a continuous integration system, delving into everything from check-in to delivery to production. It doesn’t state you ...
Sauce Lab’s Perspective on Integration with Continuous Testing The term ‘DevOps’ implies that implementing an effective development and production workflow is as easy as developing the collaborative interaction between developers and IT operations. On the surface, DevOps may appear as a simple and straightforward idea but, as you will find out, there is more than meets ...
For this month’s book review, I read Continuous Testing for DevOps Professionals: A Practical Guide from Industry Experts, by various authors and edited by Eran Kinsbruner. The book is divided into 4 sections: Fundamentals of Continuous Testing, Continuous Testing for Web Apps, Continuous Testing for Mobile Apps, and The Future of Continuous Testing. The Fundamentals ...
As a software development company, what is your goal? What is the one thing you feel you need to do to ensure you have a job at the beginning of each wonderful work week? The answer is actually quite simple; You need to deliver a quality product. Like how I used the word simple? Although the answer I ...
Disclaimer: This article was originally published in the LogiGear Magazine, but has been updated for the June 2020 issue. Leading an organization into Continuous Delivery is a daunting task that usually takes software development teams months if not years. This guide covers the main points that organizations need to consider to increase their chance of ...
How Halliburton leveraged outsourcing to achieve their goals. Organizations are focusing on speed, both in Continuous Integration and rapid deployment as a competitive advantage. Many software development organizations can significantly shorten development cycles by implementing one or a combination of Agile practices, continuous integration & deployment methods, and feature branches. While these frameworks and techniques ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing
DevOps for Test Teams By Michael Hackett Now that Dev Teams have had a little time to settle into the Agile, the new wave of process optimization has arrived. DevOps. DevOps has been described as Agile on Steroids. DevOps has also been described as Agile for Operations/IT. I like both of those descriptions as well ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news