Potential problem with homebrew, python 3.12, libsodium 23/26?

General
1

I am occasional user of jupyter notebook, doing some data analysis from time to time.

Some application (I don’t know which TBH) updated my brew stuff, and all things python related. On the machine restart, jupyter notebook worked fine, but my kernel was missing numpy.

OK, I decided to go “full new” and said, give me the new jupyter.

So before running brew install jupyter I had this:
→ % jupyter --version
Selected Jupyter core packages…
IPython : 8.16.1
ipykernel : 6.25.2
ipywidgets : not installed
jupyter_client : 8.4.0
jupyter_core : 5.4.0
jupyter_server : 2.8.0
jupyterlab : 4.0.7
nbclient : 0.8.0
nbconvert : 7.9.2
nbformat : 5.9.2
notebook : 7.0.6
qtconsole : not installed
traitlets : 5.10.1

Then I’ve run brew command, and to my surprise… I have nothing:
% jupyter --version
Selected Jupyter core packages…
IPython : not installed
ipykernel : not installed
ipywidgets : not installed
jupyter_client : not installed
jupyter_core : 5.5.0
jupyter_server : 2.9.1
jupyterlab : not installed
nbclient : not installed
nbconvert : 7.10.0
nbformat : 5.9.2
notebook : 7.0.6
qtconsole : not installed
traitlets : 5.13.0

OK I thought - maybe it’s changed and everything works magically in the core itself, or something. I tried to run jupyter notebook, and I’m getting an error:

ImportError: dlopen(/usr/local/Cellar/jupyterlab/4.0.8_1/libexec/lib/python3.12/site-packages/zmq/backend/cython/_device.cpython-312-darwin.so, 0x0002): Library not loaded: /usr/local/opt/libsodium/lib/libsodium.23.dylib

OK, I went to check if I have this libsodium, and to my surprise - it is there, but filename/version is libsodium.26, not 23.

I’m beyond confused now, since brew installed both. I assume it’s not a jupyter problem per se, rather brew - but not sure how to solve it.

Any ideas?
Thanks,
I

2

The short answer is you need to reinstall pyzmq, either with one of the published wheels from PyPI, which bundle their own libsodium, or recompile with the libsodium/libzmq on your system:

pip uninstall pyzmq
pip cache remove pyzmq
pip install pyzmq

To force recompile, which should link again your updated libsodium, you can:

pip install --no-binary pyzmq pyzmq

Or to just ignore your cache:

pip install --no-cache pyzmq pyzmq

This should specifically remove pyzmq files from pip’s cache:

pip cache remove pyzmq

More info on what’s going on, below.

This error:

is because something (specifically pyzmq’s compiled extensions) linked a specific version of libsodium. Dynamic libraries put their ABI version in their filename to avoid loading incompatible files and allow having multiple versions of the same library present on the system without conflict. When there was a libsodium update in homebrew, the old version was removed and replaced with a new one, so pyzmq can no longer find the libsodium it needs at runtime. When this happens, you need to recompile whatever it was that linked the removed libsodium to point to the new one (in your case, pyzmq). This kind of breakage happens when you have two separate package managers maintaining dependencies in a single dynamic linking environment (pip for pyzmq, brew for libzmq/libsodium).

There is a tool, delocate that solves this by analyzing the dynamic libraries of a wheel and copying the used libraries into the wheel itself, so it’s not sensitive to changes in the surrounding system (this is how most redistributable wheels with external dependencies, including pyzmq’s, are made). You can run it on your on wheels, if you want.