Shared-data installed in wrong directory (Jupyter, pip, homebrew)?

This question is about the various recent issues getting notebook 7 (and other things) working with homebrew installs of python 3.x (e.g., here, here, this Notebook 7 github issue, this jupyterlab issue, and this nbclassic issue. Happens on macOS — don’t know about linux homebrew.

It seems to be due to an interaction between the homebrew python installation, and the pip install of the packages which uses the hatch backend (and the hatch-jupyter-builder). So I don’t really know where this question belongs…

After some exploration, I am pretty sure that all of these issues distill down to the following problem:

All of these projects install ancillary files into directories with names defined by tool.hatch.build.targets.wheel.shared-data in their own pyproject.toml below

    /opt/homebrew/

However, jupyter is looking for the files below

    /opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/

(or whatever current minor version) which is symlinked to /opt/homebrew/opt/python@3.11/ and /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/.

The hatch docs for [tool.hatch.build.targets.wheel] say that the shared-data files are installed “usually under sys.prefix”, which for my environment is:

$ python3 -c "import sys; print(sys.prefix)"
/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11

as per the search by jupyter, but not the pip installation. (I’m not sure what “usually” means in the sentence from the docs — where is the actual location set?)

The problem can be (mostly but not completely) ameliorated by explicitly setting JUPYTER_PATH=/opt/homebrew/share/jupyter JUPYTER_CONFIG_PATH=/opt/homebrew/etc/jupyter.

My understanding of the combination of homebrew, pip, hatch, jupyter, etc., is minimal and so it isn’t at all clear to me where the problem actually resides (or if it’s user-error, which is possible, but I’m clearly not the only person experiencing this).

This is being looked under the issues cited above, but I thought I’d try to get a little more understanding of the underlying mechanisms here…

2 Likes