Eventual consistency
With the announcement of SimpleDB, Amazon has put together the last major piece needed for a fully cloud-based development approach. Looking at it holistically, I see a few things that are both game-changing and shifting people’s perspective on architecture.
First, you no longer do you have to buy any infrastructure. You can rent it, by the second or byte. This reduces the barrier of entry to effectively zero. Need an extra dozen machines for a few hours to deal with the Slashdot effect? No problem, and toss them away when you’re done. While Amazon isn’t necessarily the cheapest solution when examined from the traditional dedicated server perspective, that isn’t the right perspective to use. Instead, it’s all about right-sizing everything in an on-demand fashion. No more, no less. You might pay more for the base—but maybe not—but in the end, you pay less for your real operations.
Second, Amazon is pushing developers to no longer assume that everything is dedicated, and that everything is reliable. It never was in the first place, but many developers sloppily presumed that drives never die, DBs never go away, and that machines never crash. By telling people that your machine could disappear without warning, you have to deal with failure up-front. You have to deal with back-up, and you have to deal with restart and automation. They’re not trivial, but they’re not insurmountable, and they will serve you well long-term.
Lastly, with SimpleDB, Amazon is talking about eventual consistency, which means:
Amazon SimpleDB keeps multiple copies of each domain. When data is written or updated (using PutAttributes, DeleteAttributes, CreateDomain or DeleteDomain) and “Success” is returned, all copies of the data are updated. However, it takes time for the data to propogate to all storage locations. The data will eventually be consistent, but an immediate read might not show the change.
Consistency is usually reached within seconds, but a high system load or network partition might increase this time. Performing a read after a short period of time should return the updated data.
That might sound scary, and it can be if you’re presumption is that there’s always one authority for everything. But as with Amazon’s Dynamo, I suspect it just is the acknowledgement of the issues with large distributed systems. In some ways, to me, it is in the same spirit as Erlang’s “die on error” mentality. SimpleDB is built on Erlang.
This entry was posted at 12:16 pm on 14 December 2007 and is filed under Technology. You can follow any responses to this entry through the post-specific RSS 2.0 feed.
Without a doubt Amazon services is a great leap forward in publicly available computation-on-demand. However they are not ready for prime time.
Their Service Level guarantees look like a joke.
But the main problem for Amazon based deployments is absence of appropriate tools and frameworks. Yes, it is acceptable for a big company like Amazon to build apps ground up from very low database level. But the small/mid sized companies the Amazon obviously targets this might be an overkill.
I’ve written before about why SLAs are useless. I’ve yet to see an SLA, issued by any company, that would actually compensate the organization for anything resembling their true losses. It simply isn’t possible to do that without jacking up the costs to an unbearable level. SLA/SLGs are really only about rebating the lost services, not the lost revenue. You get 1 day of your expenses back, but not 1 day of profit.
As for building from the ground up, I think you’ve been asleep for the past few years, and I find that most smaller companies are more willing and able to build from the ground up. Big companies go buy it from Oracle, etc., small companies build it themselves.
My experience is that in the time it’d take to make a big commercial package do what I need, I can build it from scratch. This is true in many areas, but most especially so in web applications. There’ll be PHP, Python, Ruby and other bindings (Java for the Enterprisey), and it’ll work just great.
You can leave a response, or trackback from your own site.
[...] Eventual consistency at Pensieri di un lunatico minore “With the announcement of SimpleDB, Amazon has put together the last major piece needed for a fully cloud-based development approach…I see a few things that are both game-changing…First, you no longer do you have to buy any infrastructure.” – indeed (tags: amazon simpledb chrispetrilli haas saas) [...]