Amazon Prime Day, which celebrated the 20^th anniversary of the Prime members’ program on July 15, 2015, has taken a lot of flack as a bad customer experience, based on negative tweets and members’ frustration about the quality, availability, and accessibility of the items featured in the sale.

Prime Day Spoof posted on Reddit by Blue Mosquito

However, we suspect that the first annual “Prime Day” yielded the two major results that Amazon intended. It attracted many new Prime members. It gave Amazon and its retail partners an opportunity to stress test its infrastructure before the 2015 holiday season.

Boosted the number of Prime members

The mid-summer Prime members-only sale served to boost trial Prime memberships during an otherwise quiet retail period. Amazon reported that it added “hundreds of thousands” of new Prime Trial members to its current base of Prime users worldwide. We assume that at least 70% of these trial users will probably convert to becoming loyal, repeat Prime members, based on previous analysis. (Consumer Intelligence Research Partners stated in early 2015, commenting on the estimated 3 million new Prime users Amazon gained over the 2014 holiday season, that “the average number of members who pay after the 30-day trial stands at roughly 70 percent.”)

How many Prime Users are there?

The estimates for the number of Prime members globally range from 50 to 60 million. Amazon does not divulge the number of its Amazon Prime members; that official number remains elusive.

Business Insider's estimates of the number of Amazon Prime members as of December 2014 is about 53 million.

Just before Prime Day, according to Tricia Duryee at Geekwire, Consumer Intelligence Research Partners released another report in which they asserted that Amazon had 44 million Prime members in the U.S. alone:

“Consumer Intelligence Research Partners estimates that Amazon now has 44 million Prime members in the U.S., who spend an average of $1,200 a year, compared to about $700 a year for non-members.”…

CIRP put the total number of Amazon Prime users at 43 million before Prime Day.  Note the second quarter increase in year over year domestic growth. These estimates do not include the "several hundred thousand" new Prime Trial members acquired on Prime Day.

Jeff Bezos doesn’t confirm or deny any of these speculations. Amazon’s official press releases continue to say that Amazon currently has “tens of millions” of Prime members. It’s also hard to discern these numbers from Amazon’s published financials because of the accounting treatment that Amazon uses:

“Amazon Prime membership fees are allocated between product sales and service sales and amortized over the life of the membership, according to the estimated delivery of services."

The mid-July boost in $99/year Prime memberships by featuring a “members-only” sale and encouraging trial membership is a good strategy. However, due to the way it was structured--around Flash Deals--Prime Day attracted primarily bargain hunters rather than long-time, loyal customers.

Stress Testing

We believe that one of the main reasons for Prime Day (aside from celebrating the 20th anniversary of the Prime program), was e-commerce stress testing. “Prime Day” created a large sales event in mid-summer which gave the Amazon infrastructure teams a chance to test out many of their processes early enough to fix glitches before the big year-end holiday spike. Amazon promoted it as a day to “find more deals than Black Friday, exclusively for Prime members.” The volumes were huge:

“Customers ordered 34.4 million items across Prime-eligible countries, breaking all Black Friday records. Greg Greeley, Vice President, Amazon Prime, said: ‘Customers worldwide ordered an astonishing 398 items per second …. Worldwide order growth increased 266% over the same day last year and 18% more than Black Friday 2014.”

Fulfillment by Amazon

The other infrastructure Amazon was stress testing was its “Fullfillment by Amazon (FBA)” service.

Prime Day gave retailers an ability to test out Amazon’s fulfillment chops before the make or break holiday season. Retailers have to nail down their e-commerce and fulfillment infrastructure by the end of July in order to be ready for the November/December retail holiday shopping season. Amazon reported:

“Prime Day was also a success for sellers on Amazon that use the Fulfillment by Amazon service – they enjoyed the biggest day ever with record-breaking unit sales growing nearly 300% worldwide. These sellers included local retailers and businesses of all sizes. Some feedback from sellers include:”

• “'Every one of our deals was 100% sold! The FBA service has been great and on #PrimeDay was huge for our business. Holiday sales volume in July. What a day!'” – Mike Mitchell, COO for MMP Living
• “'Since we’re new to FBA, we crossed our fingers. I sold all 100 luggage sets in an hour, it was very surprising! I was floored. Needless to say we were impressed and excited. I wish I sent in more inventory. This reinforced what we thought FBA could do for our sales. This is a really good outcome for us, wish I could make $16K an hour every day.'” – Sumit Bhanote, Co-Founder of Luggage Point
• “'Loved participating in Prime Day. We had great results today and sold out within hours. More importantly our overall business did 4 times more sales on Prime Day than it did last Wednesday. FBA continues to be a growth lever for our business and Prime Day is one of many reasons to join and grow with FBA. Thanks!'” – Avrum Elmakis, CEO for Best Bully Sticks

Failed in User Experience

However, there were some unintended consequences, which apparently had the most negative impact on existing Prime users.

Few Good Items for Sale at Bargain Prices

Most of the deal were “flash deals” and were staggered throughout the day. So bargain hunters had to be glued to their computers all day and move fast. There were many more undesirable, than desirable items for sale.

Lousy User Interface for Deal Shopping

As Ki Arnould commented, in her User Testing blog post entitled Testing the Experience of Amazon Prime Day:

“Users were irritated by the random mashup of deals into big lists, and their inability to filter by categories that were personally relevant to them. Members—who theoretically use the site more often and cultivate wishlists, etc—tended to be more irritated than non-members.”

The user interface for navigating special deals was terrible. It did not meet an acceptable user experience threshold. You had to scroll sideways through a random list of items, which included items that were no longer available.

As a Prime member, If you found something you wanted, and you were logged in, you could click through and place it in your shopping cart, but you typically only had a few seconds to complete the transaction before the item would be removed and made available to someone else (which is only fair, but created a lot of stress and anxiety.) Essentially, all the "good" items were snapped up in the first few minutes of each sale and all the overstock/liquidated items lingered and generated a lot of negative tweets.

 I liked George Jacob's review of the UI of the Amazon "Deals Carousel" on Customer Think:

"The carousel is serviceable for users casually browsing for items, but not ideal for users actively tracking flash sales.

Let’s say you saw a cheap HDTV in the 'Upcoming' sales section, then clicked into the “All Available” section when it became available. Would you be able to scroll through the carousel fast enough to get one?

Probably not, because you’d only have a few seconds before they’d be sold out.

And so you’d sit, faced with a smoldering realization that there are vast numbers of people better than you at buying cheap televisions on the Internet."

Amazon Prime Took a Brand Experience Hit on Prime Day

The moral of the story: Amazon reached its objectives but damaged its Prime brand. In particular, this first annual Prime Day will go down in the annals of ecommerce history as a giant #Fail from a customer experience point of view.

However, those of us who are long-time Amazon shoppers remember the many times that Amazon screwed up on holiday shopping (unavailable merchandise) and fulfillment (gifts that didn't arrive on time). Amazon learned from those mistakes and corrected them.

Silver Lining: Look for a Breakthrough in "Internet Deals" Design from Amazon

In order for next year's Prime Day to redeem itself, Amazon will need to come up with a new user interface for Flash Deals. I'm going to ask our "Deals Maven" Ronni Marshak to describe the "must haves" in such an interface. (Stay Tuned!)

A new and improved User Experience for shopping for ephemeral deals on Amazon will be a big boon not only to customers, but also to all the small and medium-sized suppliers who rely on Fulfillment by Amazon to move their inventory quickly.