I’ve also seen the term “real-time web” (RTW), which I like as well, but here I want to dive a bit into the underlying bytes that go back and forth to make the real-time web happen.

The binary “message” at the heart of the real-time web is a sequence of bytes controlled by the application: JMS-style messages, XMPP (Jabber) frames, a JSON object, serialized Java in Hessian, the packets for a Quake game, stock ticker updates, iPhone app messages, toll booth status control panel, on-demand music streaming, auto-manufacturing overview consoles.

Because the messages can vary in length from tiny, fast Quake messages where response time is critical, to larger packets like the music and video streaming, the underlying protocol must handle that range, but still be memory-efficient. It would be absurd to force a server to buffer an entire video before sending it, or even fully serialize an XML message just to find the length.

So a sane protocol needs binary-length encoded chunks (called “frames”) combined into messages. “Messages” are understood by the application, “frames” are invisible to the application but are used by clients, servers, and intermediaries to manage the messages.

Bringing those requirements together, the minimal protocol looks like the following:

stream ::= message*
message ::= (non-final-frame)* final-frame
final-frame ::= final-flag length <bytes>
non-final-frame ::= non-final-flag length <bytes>

At the moment, that’s a bit abstract since I haven’t defined the encodings for the length or the final-flag or allowed any kind of control messages, but it’s the heart of the protocol.

The key is sending chunks of binary data, where servers can use their own fixed buffering (like 8k buffers) to send arbitrary-length binary data.

Any text data is easily encoded as UTF-8 over the binary payload.

As part of the Resin 4.0 work, I’ve refactored and renamed the key BAM classes to better describe the architecture and roles. The key classes are SimpleActor and ActorStream.

BAM is a lightweight serializing messaging API, designed for small, bidirectional, interactive messages for applications like SAAS (software as a service) or online gaming. Resin 4.0 uses BAM/HMTP for its clustering protocols, including our distributed caching, JMS, and distributed deployment. BAM is designed to work with multiple wire protocols including HMTP (Hessian Message Transport Protocol) and XMPP (Jabber, eXtensible Messaging and Presence Protocol.)

(more…)

When you deploy an application using eclipse or ant, Resin propagates the .war updates to the cluster using the triad as the reliable store. The deployment is both incremental and transactional. Only changed files need to be sent, and the updates are only visible when they are complete and visible. Partial or incomplete deployment does not update the site.

1. eclipse/ant sends the .war to triad server A
2. Triad server A replicates the .war to triad servers B and C
3. The triad updates the rest of the cluster with the new .war file
4. Once all servers have the new version complete and verified, they can restart the web-app with the new data

(more…)

With the rise of instant events like twitter and RSS, publish/subscribe technologies are finally getting the respect they deserve. With HMTP, a client can subscribe to a message stream and wait for messages to arrive, avoiding the need for polling. In fact, since HMTP is an addressable architecture, the client can subscribe to multiple publishing services at once, with a single connection and receive notifications as they happen, not waiting every 15 minutes.

To support publish/subscribe (and following the Jabber/XMPP specification), HMTP has 8 packets devoted to subscriptions and presence notifications:

• presence - notify a service that a client has logged in
• presenceProbe - query for client capabilities (text, html, mp3, etc.)
• presenceUnavailable - notify that a client has logged out
• presenceSubscribe - request for a new subscription
• presenceSubscribed - successful subscription notification
• presenceUnsubscribe - unsubscribe from a service
• presenceUnsubscribe - notice of unsubscription
• presenceError - error occurring during presence/subscription

(more…)

Thanks to Frank Greco and everyone who came out to see my presentation at the NY JavaSIG. I got some great questions and it looked like a lot of people were starting to understand the problems with interactive applications using HTTP. BAM is the way to go! Anyway, here are the slides from my presentation: NY JavaSIG slides on BAM/HMTP

And, a correction: During my presentation, I stated that there is no standard intereface for Comet on the server. While technically true, the new Servlet 3.0 draft spec has an API. Someone mentioned this during the Q&A and I said that while I had looked at the spec, I didn’t actually see the Comet references in there. Well, they are there in Section 2.3.3.3. I haven’t had a chance to review the spec thoroughly, but it looks like a nice compromise. Webtide’s Greg Wilkins has a nice intro here that features Comet use cases in Servlet 3.0. There were a lot of attempts to create Comet APIs from us, Jetty, Tomcat, and Glassfish, so if we can standardize on something reasonable, that will help everyone. It still doesn’t change the fact that HTTP was not designed for this kind of interaction though.

Just a quick note: I’m sitting in the Brooklyn Bagel & Coffee Company on 8th Ave (Free Wifi!) with about 2.5 hours until my presentation at the NY JUG on BAM/HMTP. There’s a demo up here that I’ll be presenting during the meeting. It’s good enough to see what’s happening at the moment, but it will get even better in the coming weeks and might even be posted as a Facebook application.

See you at the JUG!

Just a quick note: I’ll be in New York next week (Wednesday evening) giving a presentation on HMTP at the NY JUG. Attendance is free, but you must register.

Last week I was working on our demo for JavaOne. It’s pretty much done, except for one small feature and I’m pretty excited about it. It uses our new HMTP (Hessian Messaging Transfer Protocol) to send messages back and forth between client and server. The demo itself is a Sudoku game where you play against another player on the same board. It requires both RPC style communication and messaging, both of which are natively supported by HMTP. Anyway, I’ll give a much more coherent explanation soon as I’m writing up a fact sheet and animation to show message flow today. We might be able to bring up the game live this week, so stay tuned for a link…

The demo is implemented in Flex 3 (open source :-)) using the AS3 implementation of Hessian and HMTP. I want to send a shout out to Riccardo Cohen who has been working with Hessian Flash. He was nice enough to write up a few tutorials to show how to use it with entity beans. Check it out here: Riccardo Cohen’s tutorials. Thanks, Riccardo!