The Limits Of Amazon's Unlimited Storage

I have found the limits of Amazon’s Unlimited Storage, the hard way. Here’s what had happened to me.

My perfect backup plan

Late 2016 I have perfected my backup strategy. I use the deduplicating backup program borg to backup all my data on a daily basis. This backup is then uploaded to Amazon Unlimited Storage for offsite storage.
But how do I sync my data to Amazon from a Linux client? After some searching I found a program called rclone which can synchronize or copy to many different cloud storage providers, including Amazon, Google Drive and Dropbox. And the storage capacity on Amazon is unlimited (at least so they claim) for $60 a year, excluding VAT. That is quite affordable. There was even a 3 months free trial period.

So I signed up and somewhere in April 2017 I had some 7.5 TB stored there. There had been one little technical incident, which was resolved within about a day. And I must say the customer service was very friendly. Sometimes even a bit over the top, with dozens of apologies from their side. They even called me “a valued customer”.
At first it appeared that Unlimited really meant unlimited. I’ve never had any complaints from Amazon for using too much disk space.

There are limits

On 19 May my backup synchronisation failed. The error message showed that the rclone client was an invalid client. That can’t be right, so I contacted support. They confirmed that as of today the rclone client was no longer accepted, and it will not be re-enabled.
Excuse me! Pull the plug, without prior warning, just like that? And why?

After some more questions to Amazon’s support I still got no real good answer. They suggested me to use the Windows or OSx client to download my files. How?! I don’t have Windows or OSx. And if I did download my files, they were encrypted, by rclone.
Fortunately I don’t need my files, because they are just a copy of my backups. So I don’t bother downloading anything.

I’ve contacted the support team for a third time telling them that pulling the plug without prior warning was unacceptable. And that my Amazon drive space had become useless to me because I don’t run Windows or OSx. I demanded a total refund and cancellation of my account if they didn’t fix it before the end of May. I never received another reply from the previously friendly support team.

The (un)official reason for the ban of rclone

On the rclone forum I read the official reason why the rclone client has been banned. It’s a bit technical, but it boils down to the way in which rclone obtains the oauth access token for the user. It was secure for the user, but other programs could pretend to be rclone and could thus obtain an access token illegally.
That could be a valid reason of course. But why ban rclone users without prior warning? A much more user friendly approach would have been to stop accepting new oauth token requests, while giving existing users the chance to get their data back.

What could be the real reason for the ban?

I can think of another very plausible reason why rclone was banned though. Amazon will probably never admit to this reason, or they might even deny it.

The rclone client can encrypt the data before it sends it to the cloud, with the user holding the private key. So there’s no way Amazon can look into your files.
On the other hand Amazon’s own clients can encrypt your files too, but they are the ones holding the private keys. So they can always decrypt your files if they want to.
I guess this could well be the real reason why Amazon doesn’t like 3rd party sync clients.

Anyway, it’s another sad occasion for Open Source software, where the user wants to be in control of his/her own data.

Deadline

The end of May passed, without a reply from Amazon’s support team. That’s a great way to restore customer satisfaction, don’t you think? Simply ignore angry customers. They will run away by themselves. And another problem is solved by the wonderful customer service department!

So I’ve requested cancellation of my now useless drive space and demanded a total refund. Surprisingly I’ve got quite a quick reply, and they promised me that I’ll have my money back within 3 business days.

The last line of their reply was quite hilarious, if I may say so. "We appreciate your business and hope to see you again soon." As if I’ll ever consider using another service from Amazon in the future. NEVER. I don’t trust them anymore, after showing such disregard for their customers.

So think about it twice before you build a system, or even a business around Amazon services. They may pull the plug on your application, without prior warning or proper reasoning why.

Alternative

I haven’t found an affordable cloud alternative for offsite storage of about 7.5 TB. Hubic came closest, with a 10 TB solution for an affordable price. However I couldn’t get it to work. And I’ve heard that it's rather slow.

I’m now using 3 cheap NAS machines from Friendlyarm for offsite storage, placed in three different places at friends and relatives.

And if you’re looking for affordable VPS machines as an alternative to Amazon's EC2 I highly recommend Digital Ocean or Scaleway. They both offer VPS machines for quite some bang per buck. And you’ll know in advance what it is going to cost you. I have looked into Amazon’s Elastic Compute Cloud offers in the past, but couldn’t make head nor tails of their pricing scheme.