REST Sprinkled into my XMPP

Been doing some low-level IQ packet handling in XMPP lately and interestingly, RESTful philosophy seems to be influencing me from afar. We started out the packet design to send a list (roster, but not an XMPP roster) of users associated with an artifact to the remote replica. The initial protocol design had elaborate Add Document, Add User, Remove User, Delete Document packets with corresponding XMPP IQ responses and errors to each. It quickly became apparent that there wasn't really a need to actually have different packets for Add User, Remove User and even Delete Document. When we started envisioning the resource as a document, and simply hammering the list of users along with the document, much the way we would expect an idempotent PUT to work in REST, the entire protocol got much more simple - just send the list every time. When there are no users, send an empty document packet. Otherwise, just send the authoritative list. That's much easier than the book-keeping needed to track if I've delivered a notice for one user and if it's been acked - just send the whole thing every time.

Even better, if the destination misses a packet, the sender doesn't care, it can just send the current one because the new packet is authoritative at the time it's sent.

All said and done, this eliminated a ton of state maintenance on both ends of the protocol. The sender didn't have to remember what it had fired off to the recipient and if it was adding or removing someone, nor did it need to manage the corresponding ack/error packets in detail - if a packet failed, just resend current. Likewise, the receiver was able to simply take the incoming update and make it fact - the reconcile process would ultimately be far easier than the alternative which amounted to checking if a user existed on the document, removing if they did and erroring if they didn't. That level of detail was immaterial to the sender, the sender simply wanted to publish *the* authoritative list of users and nothing more.

Too bad it took a practical implementation to realize most of the early protocol design wasn't needed :) Kind of reminds me of the recent SOAP vs. REST debates.

My Bow-Legged Master - Comment

I tried to comment directly on Bill's Blog but comments were broken again. What I wanted to say was:

I think the real tragedy of this legacy is that we currently have installed, in nearly every networked node in existance, the potential for a simple, iterative, resource-driven client that can't really speak the protocol it was designed to handle. I repeatedly find myself grudgingly writing a curl wrapper to test a RESTful API in a pinch while my browser looks at me and laughs.

Oh, and the irony that your comments button says "Post" rather than "Put" is not lost.

Hopefully comments will be fixed soon and I'll cross post, er cross PUT.

Have a Seat on the Couch

No idea why it took me so long to find CouchDB, but I will be watching its development closely and will start probing the source once I have a better handle on Erlang. Highlights include a RESTful API, a bevy of clients to talk with that RESTful API (Ruby, Python, Java, JavaScript), data replication for reliability, and off-line capabilities. I can't think of a much better use of Erlang on the server side and I love the use of SpiderMonkey for a Query language , that's plain brilliant and something people should use more of.

Flexy Flex

I will be speaking at one of the Portland Flex User's groups on Sep. 20. The topic will be integrating Flex with RESTful applications, something that really should be easier than it is.

Apache Is My Service Hub

Ok, so I nicked the idea of this post from mnot's Squid is My Service Bus and tried to extend the concept based on ways I've used Apache HTTPD in the past.

In general, I'm a big fan of Apahce HTTPD. It's immensely powerful and has been my "Swiss Army Knife" of the web for some time now, proxying this, caching that, dynamically serving things all the while never once becoming the primary constraint in any distributed system. That's saying a lot as a perimeter service responsible for all ingress traffic.

It occurred to me in discussions with Mike and Patrick that Apache's utility makes it an ideal Service Bus of sorts. Service Hub might be a better term. Most of Apache's default functionality is geared towards moving resource requests around and not protocol mediation (a key criteria of a bus). However, nothing stops the Apache developer from making the server more bus like if the requirements so dictate.

The idea goes like this

• Service consumers, RESTful preferably (although SOAP will work too), desire to perform operations on a resource. The consumer has a client that is smart - it knows that it can rely on finding one or more Service Hubs in the local vicinity, but doesn't care exactly where that hub is.
• The client performs some measure of discovery, a DNS lookup backed by BIND zones, a Zeroconf search for the Service Hub, a broadcast search, something that lets the client discover, at runtime, it's friendly local Service Hub. The important part is that all clients search for their local Service Hub in the same way and in a manner that lets us have one or twenty hubs depending on our needs. We can scale up or down and the client behavior doesn't change.
• When the service client finds a hub, it makes a request of that hub. Next time around, it might use a different hub, or it might hold a keep-alive for the HTTP connection to that hub.

So, what makes Apache particularly well suited for this role?

1. Apache HTTPD is ubiquitous - Apache is capable as acting as an origin, a proxy or a gateway for resources all in one set of configuration while being 99% HTTP 1.1 compliant. It runs on all Unixes I know of, Windows and other platforms. I generally don't think of Apache as middleware, but if you abstract Apache up a layer and treat it as a RESTful gateway, it does fill a role traditionally reserved for middleware.
2. Apache HTTPD can reverse proxy - Apache has an added advantage that it can perform crude load balancing in a reverse proxy configuration. This means, you can have service clients consume HTTP resources without knowing that they are talking with one or twenty back-end origin servers. Sure, F5s have been doing this for some time now, but try installing thirty F5s in a large integration, one for each broadcast domain and see what it does for your budget. Additionally, Apache can reverse proxy for loads of protocols out of the box, HTTP(S), FTP and AJP open worlds of opportunities for the systems integrator.
3. Performance - Apache HTTPD builds on Apache's portable runtime (APR) which is a fairly low-level abstraction on top of most OS system calls. This gives it tremendous performance benefits (like using scalable non-blocking I/O ala epoll on Linux) while still remaining relatively platform-agnostic.
   
4. Extensible - Not long ago, writing Apache modules was a poorly-documented black art. That is improving substantially with books and better reference documentation. But, writing a module isn't necessary at all. With dynamic language alternatives like Python and Ruby that can run in-process with Apache workers, you can extend Apache as a Service Hub however you like and without writing C.

I don't expect to see wide adoption of this type of approach any time soon. If RESTful architectures catch on and people start to see the web as a series of resources, Apache HTTPD's role will continue to grow. It's high time we stopped looking at Apache HTTPD as just a web server and started viewing it as an HTTP Gateway.