Recovering from and avoiding "cloud service" lock-in

We all love our shiny cloud services — until they break, die, or otherwise go away, turning all those unicorns and butterflies into what can only be described as a bark salad.  Case in point: sometime this week, I'll be wasting time migrating data out of DabbleDB.

DabbleDB, if you don't know, is was this great interactive "relational" database service: think of it as a massive spreadsheet where anything could be related to anything else — schema-free, BTW — with hooks into web forms for surveys and such, an excellent reporting and query engine, and all sorts of goodies like mapping geographically-related data, charting, etc. Similar services include WuFoo, Intuit's Quickbase, and ZoHo Creator.

I say was because DabbleDB was acquired by Twitter last year; as a result, the service is shutting down next week, and I need to yank the data we were storing there and reconstitute it into some corresponding in-house applications.  Mind you, there's nothing difficult about this, but I'm slightly irked at myself for being in this position.  While building the replacement apps will be far more costly over the long term than the $8/month we were paying DabbleDB, the real cost is the dislocation associated with relying upon a "cloud" service provider to provide a particular set of features and then being forced to roll back that reliance.

In this case, the precipitating event is a happy one for DabbleDB; they're good guys (go Smalltalkers!), and I got my data out just fine, but the scenario isn't so different than if they went out of business, or had a massive technical failure.

This perspective prompted me to think about what I would have done differently, what questions I should ask myself before again committing to use a particular "cloud" service, and what I should focus on as a vendor of such services to minimize the chances that my customers will be faced with the same grim drudgery that I'm facing now.  Obviously not a comprehensive treatment, but off the top of my head:

Things that (prospective) users of cloud services need to think about

  1. How likely is it that the providers of this service will be around in a year? Five years?  Do they have a reputation for retiring new services if they don't take over the world? (viz. Google Wave)
  2. Do you have a suitable backup plan? Just because data is in the cloud doesn't mean it can't be lost; service providers go *poof*, data centers burn.  Get and keep snapshots of your data just like you do for data on your local workstations and in-house applications. If that's not practical (i.e. you've too much data in the cloud to store locally), then at least push snapshots into another provider.
  3. Don't be (too) swayed by the chrome and glitter.  Many online services put design at the center of their offerings, and it's true that quality, functional design can be compelling — just make sure that you're not taking on a ton of risk just to get a shinier dashboard.
  4. For each service you use, ensure that you can either reasonably recreate it in-house, or source a comparable service from another provider.

Like all rules, you have to know when to break them.  If a service is amazing enough, the risks of using it may be dwarfed by its benefits, and maybe your data is transient or otherwise not worth bothering with backups.  In any case, the key is to choose who to do business with wisely, and after properly considering alternatives and attendant risks.

Things that builders of cloud services need to think about

If you're building (or already providing) a "cloud" service, you need to think about all of the issues your customers should be thinking about — to minimize the perceived risks associated with using your service at the very least, and ideally to maximize the actual trustworthiness of your service.

  1. Ensure that data loss is asymptotically impossible.  I could say "don't ever lose data", but that's sort of like saying "don't get into a car wreck".
  2. If data is compromised (not lost but obtained by someone unauthorized to have the data), make it so that that data is unusable.  This isn't always possible, but when it is, encrypting bits before shuffling them off to persistent storage is ideal.
  3. Always provide an obvious way for customers to get their data out, and in the most useful form(s) possible.  A silly counterexample are these CSV files I got out of DabbleDB — a dreadful format for schemaless yet relational data.  It's easy to view data export as an unnecessary early cost, but many potential early customers will rightfully view robust export capabilities as a necessary condition before they will trust your shiny new service.
  4. Assuming yours is not a commodity service, consider what it would take to reimplement or reprovision it if something went badly wrong.  Play out scenarios that include fatal design flaws in your software as well as failures of your upstream vendors, from a couple days' outage, to the-CEO-was-indicted-and-their-servers-are-already-liquidated.  Is it possible to fail over to other providers? Is it possible to replace your service with another as a temporary bridge until you can restore service properly? The result may not be perfect, but customers will always prefer a degraded service to a disappeared service.
  5. If you do find yourself in the happy situation of being acquired, but your service is not relevant to your acquirer, do right by the customers that got you there.  The DabbleDB guys did pretty well on this count, providing sane data exports and nearly a year of live service allowing their customers to calmly migrate (certainly far better than many other cloud services that get shut down within days or weeks after being hoovered up into Google and Twitter and Facebook).  Going beyond this would, in DabbleDB's case for example, mean partnering with a competitor or two to make migrations absolutely painless.  (It looks like Zoho Creator has done this to some extent on their own using DabbleDB's APIs.)

Ironically, enticing your customers to commit (née be locked in) to your service requires that you give them reason enough to believe that they can leave any time, and sanely survive your passing.