Socializing Eventual Consistency

I've followed Werner's blog for a while now and his latest on eventual consistency is one of my favorite posts. I recently advocated for an eventual consistency approach for a service offering we're putting together and lost. Part of that I think was that the technology to build an eventually-consistent client could really use some improvement and I'll talk to that in a later post. More at play though was the general issue of pushing people's comfort zones with new concepts.

I was somewhat conflicted after the discussion because the Agile part of my brain wanted to believe that the simplest thing we could do was flow with out skill set and just write to an SQL database. The systems experience in me was screaming the whole way at building a tightly-coupled, highly intertwined system when staring at a green field that didn't really have highly-structured relational data.

As always, I could have done a better job in making my case, focusing more on the technical and less on the emotional, but I sure wish I would have had read Werner's blog before the discussion. Two things really stood out:

"Eventually consistency is not some esoteric property of extreme distributed systems. Many modern RDBMS systems that provide primary-backup reliability implement their replication techniques in both synchronous and asynchronous modes."

When I talk to people about things like eventual consistency, I generally get a response along the lines of "that's cool, but we're not Amazon". No, most companies aren't, but I'm not sure that we shouldn't leverage all the brain power and learning from Amazon to our benefit. You can only make buggy whips for so long while thought leaders speed ahead.

The other piece that stood out was:

"Inconsistency can be tolerated for two reasons: for improving read and write performance under highly concurrent conditions and for handling partition cases where a majority model would render part of the system unavailable even though the nodes are up and running."

Customers are always going to want the C, A and P from commercial software, but experience tells us that they are not all possible. If you buy Werner's arguments, the P is impossible for distributed systems. Most business-focused users I've encountered, when really pressed, almost always choose the A over the C.

When trying to socialize concepts like these, there is truly an art to helping people out of their comfort zones. I'm not as good at it as many people, posts like Werner's go a long way towards helping with that.

Now, if they can just fix that SimpleDB API...

Scale in an Ontology Vaccum

I had the pleasure today of reading Clay Shirky's Ontology is Overrated. Being in the collaboration software business, understanding how people classify artifacts is not just a science, it is vital to the evolution of the software we build.

For our purposes, I think it's a dual-edged sword. Make things too wide open, and users won't evolve their own classifications and people will stray away from the true strengths of the tools. Make things too ontology-driven and users will naturally migrate away from the structure, save everything locally and keep emailing and searching with Google Desktop. We have to build a better mouse trap to survive.

For me, a big take-away from Clay's piece was that any attempt to define classifications will fail. The Dewey 200 category is a fantastic example and one that really drives home our inability as simple humans to step out of context.

So, to paraphrase Clay's hypothesis a bit, any attempt I make at trying to define an ontology will fail, mostly because there is no way I can out-think the masses. Ok, fair enough, so how to build software that better helps the masses? Not too difficult at face value, things like tag clouds work great. Unfortunately, the implementation problems grow exponentially under scale. As the volumes of information available electronically continue to grow, and as information compounds on top of itself through links, blogs, etc, the notion of aggregating tag clouds, and more importantly, doing things like suggesting similar clouds shifts from a usability problem into an architectural one.

For collaboration software to innovate, it is faced with several interesting dilemmas. First, how do we expand beyond the bounds of our own control and pull in relevant data outside our domain? It's scary, but if you believe that collaboration software must go down that path, then we're faced with more integration concerns and less of the simplified workflow and text processing which is how most people think of their collaboration tools today.

Assuming you can go that far, how do you actually make sense of all that data? You start to run into scale problems real fast. Customers want one node to run everything and at the same time roll-up data from five internal sources, and ten external sources with partial federated identity all while being responsive. That's a tall order, and one that starts to make the "C" in "CAP" increasingly difficult to archive.

My gut tells me that Werner Vogels' and Amazon's approach to eventual consistency described in Amazon's Dynamo will win out. When you're aggregating and analysing terabytes of data looking for relevance, you have to make sacrifices.