HMTP - Hessian Message Transport Protocol
I’ve put together a brief description of HMTP, including a definition of the low-level wire protocol (skip to the end). This post includes the core messages, and tomorrow I’ll add the pub/sub presence and subscription messages.
Resin 4.0 uses HMTP for all of its internal clustering and deployment communication. We’ve organized Resin 4.0 clustering capabilities as services which communicate using HMTP/BAM. Because HMTP/BAM provides routing and addressing for the services, and supports unidirectional messaging as well as RPC, our services have the flexibility they need.
Because HMTP is based on Hessian, it keeps Hessian’s advantages:
- Object oriented - serializes and deserializes Java objects
- Language independent - e.g. Flash, Objective-C, etc.
- Binary - supports binary data efficiently like sending files (very important for our caching and deployment)
- Compact and fast
HMTP/BAM adds routing/addressing to Hessian, simplifying our services by providing a layer on top of TCP/HTTP routing. Our services don’t need to worry about connecting to other services, because HMTP/BAM handles the links itself.
All Resin services in a cluster have a unique name like “cache@baa.app-tier.admin.resin”. The “cache@” is the service name. “admin.resin” is a virtual host suffix for the current Resin domain. “app-tier” is the cluster, and “baa” is the server within the cluster. So “baa.app-tier.admin.resin” is the 2nd server in the app-tier cluster.
jids: addressing
HMTP uses a jid for addressing and routing. Because the jid looks like a mail address, it’s familiar (”cache@aaa.app-tier.resin”), and because it includes the virtual host, service name, and resource, it’s flexible enough to handle all our addressing needs. The resource id identifies a specific connection, like an iPhone login “ferg@hmtp.caucho.com/my-iPhone-1″.
Hessian payloads: object-oriented messaging
Because all HMTP payloads are objects serialized by Hessian 2.0, they inherit the Hessian 2.0 advantages: type-safe, language-independent, compact, fast and binary.
The type-safety of the Hessian payload is critical for HMTP/BAM since service methods are dispatched based on the Java class of the payload.
@Message
void startup(String to, String from, StartupMessage msg)
{
...
}
Core Packets: Messaging and RPC
Six packets form the core of HMTP, covering both unidirectional messaging and RPC-style method calls:
- message - unidirectional message
- messageError - unidirectional error message
- queryGet - RPC query for information (no side effects)
- querySet - RPC action (produces side effects)
- queryResult - RPC result for queryGet and querySet
- queryError - RPC error for queryGet and querySet
Message: unidirectional message
message ::=
int # type: code = 0
string # to: jid, e.g. "cluster@aaa.app-tier.resin"
string # from: jid, e.g. "cluster@baa.app-tier.resin"
object # value: Serializable payload, e.g. StartMessage()
MessageError: unidirectional message error
Services may optionally use MessageError to return error information for a failed message. Unlike QueryError, MessageError is optional, defined by the service protocol layered on HMTP. A service protocol might never send a MessageError, e.g. for a high-performance pub/sub service where dropped message might be expected for heavy loads.
messageError ::=
int # type: code = 1
string # to: jid, e.g. "cluster@aaa.app-tier.resin"
string # from: jid, e.g. "cluster@baa.app-tier.resin"
object # value: Serializable payload, e.g. StartMessage()
object # error: com.caucho.bam.BamError
QueryGet: query RPC
QueryGet is an RPC query for information, similar to a HTTP GET. Like HTTP GET, it may not perform any action or change persistent state, it is used for information only.
A service MUST reply to a QueryGet packet with either a QueryResult or QueryError. It must never ignore a QueryGet packet, because the caller might be blocking waiting for a result. If the server doesn’t recognize the jid address or if the service doesn’t recognize the payload, it much send a QueryError back.
The RPC packets are coordinated with a correlation id, a 64-bit integer which must be unique for any outstanding requests. The service must send the QueryResult or QueryError with the same id and swap the to and from arguments.
queryGet ::=
int # type: code = 2
string # to: jid, e.g. "cache@aaa.app-tier.resin"
string # from: jid, e.g. "cache@baa.app-tier.resin"
int # id: RPC correlation id to match queryResult
object # value: Hessian payload, e.g. GetCacheEntry()
QuerySet: action RPC
QuerySet is an RPC action, similar to a HTTP POST. Like HTTP POST, it may perform an action or change persistent state.
A service MUST reply to a QuerySet packet with either a QueryResult or QueryError. Like QueryGet, each call must have a matching reply. And like QueryGet, tThe RPC packets are coordinated with a correlation id, a 64-bit integer which must be unique for any outstanding requests. The service must send the QueryResult or QueryError with the same id and swap the to and from arguments.
querySet ::=
int # type: code = 3
string # to: jid, e.g. "cache@aaa.app-tier.resin"
string # from: jid, e.g. "cache@baa.app-tier.resin"
int # id: RPC correlation id to match queryResult
object # value: Hessian payload, e.g. PutCacheEntry()
QueryResult: RPC result
QueryResult is a reply to a QueryGet or QuerySet call, using the same correlation id as the call, and swapping the to and from addresses. The QueryResult (or a QueryError) MUST be sent as a response, even if the payload is null.
queryResult ::=
int # type: code = 4
string # to: jid, e.g. "cache@baa.app-tier.resin"
string # from: jid, e.g. "cache@aaa.app-tier.resin"
int # id: RPC correlation id to match queryGet
object # value: Hessian payload, e.g. CacheEntryResult()
QueryError: RPC error
QueryError is an error reply to a QueryGet or QuerySet call, using the same correlation id as the call, and swapping the to and from addresses. The QueryError or a QueryResult MUST be sent as a response, even if the payload is null.
The BamError contains information about the error. For example, the jid to might not exist, or the service might not understand the payload type, or it may reject processing, or it may simply fail.
queryError ::=
int # type: code = 5
string # to: jid, e.g. "cache@baa.app-tier.resin"
string # from: jid, e.g. "cache@aaa.app-tier.resin"
int # id: RPC correlation id to match queryGet
object # value: the Get or Set payload, e.g. GetCacheEntry
object # error: the BamError with error information
Conclusions
The key concepts underlying HMTP/BAM are the following:
- symmetrical: all peer agents serve as services, including logged in clients. So clients are services which may receive and process messages and RPC calls. With the exception of connection establishment and login, there are no true “clients” in HMTP, all are peer services.
- jid (mail) style addressing: HMTP/BAM is responsible for routing
- typed payloads: services dispatch methods based on the payload types, not method names like traditional RPC
- unidirectional and RPC style packets are core to the protocol
Although messaging and RPC calls are the core packets, HMTP includes packet types for managing publish/subscribe lists and handling instant messaging presence announcements. Tomorrow, I’ll describe the pub/sub and presence packets.