I am a co-maintainer of the Spack package manager (https://spack.io) and we would like to ship more Jupyter extensions, such as ipympl in Spack.
From looking at the structure in other package managers such as Conda, we see that Jupyter extensions usually need to be installed into a common prefix, e.g., ${PREFIX}/share/jupyter/nbextensions/jupyter-matplotlib/.
Spack splits packages by default into their own prefixes, which allows to mix-and-match compatible packages for virtual environments via environment variables. Thus, is it possible to hint Jupyter to more than one prefix when looking for extensions? We would ideally like to set an environment variable similar to PATH and PYTHONPATH, e.g., NB_EXTENSION_PATH, which hints a list of (additional) prefix roots to check for additional packages.
Is that already possible or something to consider as an extended configuration option?
Most of the applications delegate this to the already complicated jupyter_core path techniques. There is minimal awareness of XDG paths, but not the extensible, :-separated variants. That repo would likely be the right place to raise this issue.
A reason we might push back: we have done some experimentation with other python techniques (e.g. entry_points, namespace packages) and they really start to drag as tools like tornado, jinja, and even some of our home-grown stuff isn’t really set up to create virtual directories from tens or hundreds of paths… it would take a substantial amount of caching (and cache invalidation) to handle this.
For JupyterLab, you can launch it with an extra list of directories to look for extensions (you can see this by doing jupyter lab --help-all and looking for the LabServerApp.extra_labextensions_path option). In fact, I think you can override the default path by setting LabServerApp.labextensions_path too.
For the classic Notebook, you can likewise launch it with the --NotebookApp.extra_nbextensions_path option to give some more paths to look for extensions.
Each of these options should be configurable via a config file as well.
In addition to giving specific extra paths to Jupyter Notebook or JupyterLab, you can also just give extra paths for Jupyter as a whole to consider by setting the JUPYTER_PATH environment variable.
You can run jupyter --paths --debug to get a listing of paths Jupyter will consider for data, config, etc., along with what environment variables did or did not contribute to those directories.
You probably want to set JUPYTER_CONFIG_PATH as well, as that will prepend paths for the config directories. Config directories would be used to show that certain extensions are enabled, etc., and are often also populated by extension packages.
Yes, fully understand. For that case, Spack also has a virtual environment feature where we can link compatible, active dependencies into a single directory, which would then save walking through a lot of paths.
