(not quite) Time for Comet

Over the last few weeks I've been working in spare moments on a fun little project with real-time updates of a shared web page. Fun stuff, but not the point of this post. What I'm interested in here is that in order to get this working, I wanted to use comet.

Now comet is something I've been following since Alex Russell coined the term and my interest was re-ignited by Simon Willison's, Time for Comet presentation. Essentially comet is a way of using Javascript to stream data to the browser. It takes advantage of the fact that you can keep an http connection open and periodically pump some javascript down it, this gets parsed as soon as it loads so you can send periodic update messages. You can read more at Comet Daily.

The theory behind comet is easy enough, but in practice once you get load, normal HTTP servers (which tend to be designed to respond quickly and get things pushed out and done with) don't work too well. So off I went in search of a comet capable server. I could write something, but I want to focus energy on the actual project (have I been spoiled by mature Frameworks and Libraries? Maybe). So began by trawl through the various options available.

I had fairly basic needs, as far as I can tell. Playing nicely with jQuery (or some other library) via JSON was first key, being stable and not hogging the limited memory on my VPS second and having decent documentation was the third. Points two and three are actually the sticky ones, it turns out.

I first of all I had a look at Liberator. Its free version is still quite robust. Pros are definitely easy installation. Documentation is, however, very skewed towards financial services and Java developer-ese. English it ain't. Hunting around for how to plug other data didn't elicit much of use, so I moved on.

Next up was Orbited, which needs Python and Twisted. Some time upgrading later, via the wonders of Yum, my server has Python 2.5 and Twisted running. Mostly my fault for not upgrading earlier, but still irksome time spent. Orbited runs pretty quickly with a low memory footprint, but seems focused on providing an interface to other protocols. Probably great for hooking up to IRC, not ideal for my project. Again, documentation not great or at least very brief.

Since I'm playing with Python now, I go and look at the cometd python option. This should integrate with Dojo and use the JSON-based Bayeux protocol. Sadly, documentation appears not to exist yet. Installation requires more Python wrangling, and ends up giving internal server errors on handshake requests. Not good. Other people have this problem too, and their emails to the developer's list haven't seen much response. Scratch that option then, still too beta.

So then it's Jetty, a Java-based option. I'd avoided Java for bloat issues, and the fact that it's a langauge I have only passing familiarity with. Turns out Jetty's an easy install, but seems to be a bit of a memory hog on my VPS. Likely because it's a fully-featured server not just a comet black-box. However I can connect to demos and it handles Bayeux well enough. I imagine a bit of digging in documentation will turn up tuning information to stop it eating resources, but by this point I'm more than a little jaded with the whole escapade.

So what did I learn from this? It may be time for comet, but you'll need to do a fair bit of hunting around in mailing lists, documentation and irc even to get things working. Implementing something in top of it is another story...