A request for help
I hope to get some insights from you regarding a situation common for JupyterHub’s utilizing Docker images.
A JupyterHub starts docker containers with an image having conda packages preinstalled in
/opt/conda, owned by the user. The JupyterHub mounts user storage in
/home/jovyan in this docker container. Everything outside this user storage will be reset on container restarts, so packages installed to
/opt/conda needs to be installed again. The same scenario arise for
pip but can be resolved quite neatly with
pip install --user <package> that allows the user to only store the additional package in the users personal storage.
End user installation experience
- (Awesome) Make the end user not worry about this, making installed packages persist without any additional flags to
- (Nice) Make the end user need to know they if need to install packages that will persist in between container restarts, they simply need to use a single flag or similar in the install statement.
- (Unacceptable) Require the user to create new environments etc, or duplicated installed packages.
- (Unacceptable) User storage is filled up with lots of duplicated installations of what is already available in
/opt/conda, perhaps because it ignored it.
- If I can create a new environment that augments
/opt/condathen I’ll be fine, but how to do that? I’ve heard about nested aka stacked environments, but I never got that to work as expected, but perhaps it is the way to go… Hmm…
Perhaps relevant stuff
The paths inspected by python currently
$ python -c "import sys; print('\n'.join(sys.path))" /opt/conda/lib/python36.zip /opt/conda/lib/python3.6 /opt/conda/lib/python3.6/lib-dynload /home/jovyan/.local/lib/python3.6/site-packages /opt/conda/lib/python3.6/site-packages