Mimeographs from the Future

If Berke­ley sock­ets didn’t exist, some­one would have to invent them. If that some­one then focused more on the abstrac­tion of what you want to accom­plish, rather than how, what you would get is ZeroMQ. If you then stole a char­ac­ter from Nor­we­gian, you’d get ØMQ. The cre­ators of ZeroMQ define it as:

ØMQ looks like an embed­d­a­ble net­work­ing library but acts like a con­cur­rency frame­work. It gives you sock­ets that carry whole mes­sages across var­i­ous trans­ports like inproc, IPC, TCP, and mul­ti­cast. You can con­nect sock­ets N-to-N with pat­terns like fanout, pub­sub, task dis­tri­b­u­tion, and request-reply. It’s fast and small enough to be the fab­ric for clus­tered prod­ucts. Its asyn­chro­nous I/O model gives you scal­able mul­ti­core appli­ca­tions, built as asyn­chro­nous message-processing tasks. It has over twenty lan­guage APIs and runs on most oper­at­ing sys­tems. ØMQ is open source and fully sup­ported by iMatix.

That’s a lot of ter­mi­nol­ogy in one def­i­n­i­tion, so let’s look at it in turn.

Embed­d­a­ble

This is key. ØMQ is not an appli­ca­tion. It’s a library, and it’s designed to be embed­ded into your application.

Con­cur­rency

The library is designed to deal with a lot of the issues around thread­ing and asyn­chro­nous responses. This helps enor­mously in build­ing more scal­able appli­ca­tions. In addi­tion, it allows the devel­oper to ignore a lot of that and write more famil­iar code.

Message-oriented

From the zmq_socket ref­er­ence: “Where con­ven­tional sock­ets trans­fer streams of bytes or dis­crete data­grams, ØMQ sock­ets trans­fer dis­crete mes­sages.” You either get the entire mes­sage, or you get nothing.

Trans­port

ØMQ is not designed just for net­work appli­ca­tions. While it obvi­ously can run over TCP/IP, and even mul­ti­cast PGM, it is just as happy run­ning inside a sin­gle machine (IPC), or even a sin­gle process space (inproc). The only dif­fer­ence is how you bind things together.

Pat­terns

Pat­terns can often be a dirty word. Some would argue that pat­terns are often com­pen­sa­tion for the fail­ure of a lan­guage, but in this case, pat­terns refer to com­mu­ni­ca­tion pat­terns. ØMQ sup­ports 4 core com­mu­ni­ca­tion pat­terns: publish/subscribe, request/response, pipeline and exclu­sive pair. The last is the most sim­i­lar to what peo­ple are famil­iar with when it comes to socket pro­gram­ming. It also sup­ports the tools nec­es­sary to cre­ate routers and inter­me­di­ary sys­tems with­out major effort.

Asyn­chro­nous

This is key. Effec­tively, you queue a mes­sage and then for­get about it. You don’t have to worry as much about how things work under-the-hood.

So, for me, the best way to think of ØMQ is as a robust toolkit for build­ing dis­trib­uted appli­ca­tions, in both large and small scales. I was con­fused at first by the sim­i­lar­ity between the name and tools like Rab­bitMQ, ActiveMQ, Web­sphere MQ. What ØMQ is is the tools you can build things like that with, but also a sim­pler approach. There’s a few things I’ve learned about it that I like.

While with The Matrix there was no spoon, with ØMQ, there is no bro­ker. Or, more truth­fully, the bro­ker is every­where. It’s also very easy to use. For exam­ple, here’s an exam­ple from the man­ual that imple­ments a net­work “hello world”:

import zmq
import time
 
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")
 
while True:
    # Wait for next request from client
    message = socket.recv()
    print "Received request: ", message
 
    # Do some 'work'
    time.sleep (1) # Do some 'work'
 
    # Send reply back to client
    socket.send("World")

In it, you can see a cou­ple things. First, we cre­ate a Context, which is a han­dle for all the back­end behav­ior of ØMQ. Then, we cre­ate a socket, where we spec­ify what kind of socket we want. This is the pat­tern that the socket is imple­ment­ing. Do not think of this as a socket in the Bek­er­ley sock­ets world, it’s not. It’s an “end point” for com­mu­ni­ca­tion. We then “bind” the socket to a spe­cific end point loca­tion, namely port 5555 for a TCP socket. The “*” means it will bind to all inter­faces. If you wanted to lis­ten to another port, let’s say port 6666, we could sim­ply add another line after it:

socket.bind("tcp://*:5555")
socket.bind("tcp://*:6666")

From the developer’s per­spec­tive, we only have to deal with that one socket end point, and not mul­ti­ple ones. This is a big dif­fer­ence between ØMQ and tra­di­tional socket pro­gram­ming. You could even bind it to mul­ti­cast and IPC mech­a­nisms and not have to worry about it. To me, this is a huge gain over tra­di­tional socket pro­gram­ming. Once we’ve bound things, we sit and wait for mes­sages to show up, using socket.recv(). Pre­tend to do some work, and then send some­thing back. It’s that easy. In fact, ØMQ even takes care of things like LRU load bal­anc­ing and fair-queuing for you. It might not be a 100% solu­tion for every­one, but it is for 99% of applications.

So what else is fun and excit­ing in ØMQland? Well, first, you can con­nect to a “remote” socket before the server actu­ally starts the socket up for lis­ten­ing. In tra­di­tional pro­gram­ming, this would go boom and you’d have to catch it. With ØMQ, it just sits wait­ing. Obvi­ously, this may not be what you want, but it often is. It “just works”.

So, every­thing is happy rain­bows and ponies? No, there’s some big imped­i­ments. First, until recently there was spotty and inad­e­quate doc­u­men­ta­tion for some­thing as intri­cate as a mes­sag­ing solu­tion. This has recently been rec­ti­fied with a new man­ual, which brings a lot of clar­ity to the sit­u­a­tion. There’s many things that now make sense that didn’t before. I highly rec­om­mend it as a good place to start.

In addi­tion, unlike a lot of tools, Java is a def­i­nite second-class cit­i­zen. It requires the baroque cre­ation known as JNI, and, at least on the Mac, there’s def­i­nite issues with 32 v 64-bit libraries. The libraries that are built if you use Mac Ports are 32-bit and you’ll have a very hard time get­ting them to work — if you even can — with the JNI-base Java libraries. I had to build every­thing from scratch, but Anto­nio Gar­rote has a great blog post on it. Once I did that, the Clo­jure and Java inter­faces worked fine, but read­ing online it seems like they’re def­i­nitely not a focus. Python inter­faces, how­ever, work pretty cleanly from the start

.

Next, while a bunch of trans­ports are sup­ported and bind­ing to them is triv­ial, they do not all behave exactly the same. There’s a few edge cases touched on in the man­ual about inproc espe­cially. This makes life slightly less double-rainbow. There’s also a gen­eral over­load of terms like “socket” with mean­ings that aren’t what they have been tra­di­tion­ally. This is a pet peeve of mine, and I think cre­ates some con­fu­sion in devel­op­ers try­ing to learn.

All told, though, it’s a great tool, and likely what I’ll be using to build network-based solu­tions. While it’s also designed to deal with multi-threaded prob­lems, most of my multi-threaded code already uses actors or STM to solve a lot of issues, and I find that works bet­ter for my needs. Your mileage may vary, espe­cially if you’re in a lan­guage that doesn’t have either.

Tagged: messaging, networking, zeromq, ØMQ