pyomq: pure-Rust ZMQ library for Python (no C compiler, built-in CURVE, io_uring)

General Show and Tell
1

pyomq is a Python ZMQ library built on omq.rs, a from-scratch Rust implementation of ZMTP 3.1. It ships as a single pip-installable wheel with no C compiler, no CMake, no libzmq, and no libsodium required.

pip install pyomq

The API is a drop-in for pyzmq on the common path:

import pyomq as zmq

ctx = zmq.Context()
push = ctx.socket(zmq.PUSH)
push.connect("tcp://127.0.0.1:5555")
push.send(b"hello")

Both sync and asyncio APIs are included. All 11 standard socket types (PAIR, PUB, SUB, REQ, REP, DEALER, ROUTER, PULL, PUSH, XPUB, XSUB) and 8 draft types (SERVER, CLIENT, RADIO, DISH, GATHER, SCATTER, PEER, CHANNEL) are supported, plus tcp, ipc, inproc, and udp transports.

Why another ZMQ library?

No native build step. The wheel is a single .abi3.so (stable ABI, Python 3.9+). No compilation on install, no platform-specific libzmq bundling. Currently publishing Linux x86_64 and aarch64 wheels. Requires Linux >= 6.0 (io_uring).

Built-in CURVE. CurveZMQ (RFC 26) is compiled into the wheel. pyomq.has("curve") returns True out of the box. No libsodium, no tweetnacl symbol conflicts. Key generation works the same way:

import pyomq as zmq
public, secret = zmq.curve_keypair()

Compression transports. Two transparent compression layers on top of TCP: lz4+tcp:// (fast, low latency) and zstd+tcp:// (higher ratio, auto-trained dictionary after 1000 messages). Change the scheme in the endpoint string; everything else stays the same. Useful for structured payloads like JSON on bandwidth-constrained links.

io_uring backend. The Rust core uses io_uring (via compio) on Linux. The runtime runs on a dedicated background thread. Blocking operations release the GIL; async operations run entirely on the background thread where the GIL is never held.

Performance

PUSH/PULL throughput vs pyzmq (Linux 6.12 VM, i7-8700B):

Size inproc pyomq inproc pyzmq ratio tcp pyomq tcp pyzmq ratio
8 B 1.30 M/s 627 k/s 2.08x 1.36 M/s 565 k/s 2.41x
128 B 1.31 M/s 516 k/s 2.54x 1.29 M/s 496 k/s 2.61x
8 KiB 1.04 M/s 368 k/s 2.83x 349 k/s 102 k/s 3.41x

Full benchmark tables: BENCHMARKS.md

Relevance to Jupyter

Jupyter’s kernel protocol runs entirely over ZMQ (ROUTER/DEALER for shell/control/stdin, XPUB/SUB for iopub, REQ/REP for heartbeat). pyomq supports all of these socket types and transports.

I’m aware of the ongoing discussion around adding CurveZMQ encryption to the kernel protocol (jupyter_client#808, JEP 75). pyomq could simplify that path: CURVE works out of the box with no additional dependencies or build complexity.

pyomq is not yet tested as a jupyter_client backend. That’s on the roadmap. If anyone is interested in experimenting with it, I’d welcome the collaboration.

Links:

2

This is really cool! I love to see a new, healthy project working on this, and happy to help anyone who wants to try it out, or if you have any zmq questions.

FWIW, these points don’t really motivate a new package:

No native build step.

pyzmq has native stable ABI wheels targeting many more architectures (windows 32/64/arm, musl linux, 32/64b manylinux, intel/arm mac, etc.) and PyPy. So pyomq requires native compilation in strictly more scenarios than pyzmq. But this should be surmountable by adding cibuildwheel to pyomq. I can’t recommend cibuildwheel highly enough!

Built-in CURVE

Also true of every pyzmq install, both wheels and from source.

A point in pyomq’s favor that you missed, I think, is that I believe OMQ.rs supports named pipes on Windows, which pyzmq (really libzmq) does not consistently. That’s something folks have definitely wanted, but been blocked on libzmq support.

3

Sorry for the wrong claims. I didn’t know pyzmq comes with that many prebuilt wheels. Tbh I’m mostly a Ruby guy (my job) and binary distributions aren’t very common there AFAIK. It’s either compiled during installation or FFI, meaning libzmq has to be installed.

To come clean: OMQ started as an experiment to see what AI can and cannot do. I’m personally super late to the AI party. I’m nuts for ZMQ, have been for at least 13 years, but some pain points never quite went away: Mostly installation. And the missing compression, since it’s almost free these days.

I had this super simple design (2 queues per socket) in mind for a pure Ruby ZMQ implementation on Ruby Async (which is amazing) and it blew through every expectation I had. It’s faster than any other libzmq/czmq binding, including the binidng that I’ve maintained for 11 years.

The experiment kept going with the pure Rust implementation. Many nights spent deliberating different designs, minimizing compromises, and my mind was blown again when it actually outperformed libzmq, first for medium and large messages, then for tiny ones (that one was tricky).

During benchmarking of Ruby OMQ and OMQ.rs I got to know pyzmq a little bit (impressive perf!) and thought it would be interesting to see how a Python binding would perform. Boom, pyomq born.

It’s all brand new. Not using it in production yet. But the extensive test suite, interop tests, protocol fuzzing tests, and long-running memory soaking tests tell me it’s not complete garbage.

In case anyone DOES need the perf, or the (hopefully) more stable CURVE implementation, or the compression transports (another mind blower imho), then I hope pyomq delivers. Missing wheels can be added on demand.

Windows named pipes: OMQ.rs comes with two different Async backends: Compio (single-threaded) and Tokio (multi-threaded). Pyomq uses Compio. Compio is supposed to support Windows, but I personally have no idea whether it works on Windows. Personally I’m Linux-first.