How can I specify the python version to use/install ? (system and hub is 3.10, while user is 3.9)

JupyterHub The Littlest JupyterHub
1

After installing tljh via:

curl -L https://tljh.jupyter.org/bootstrap.py \
  | sudo python3 - \
    --admin admin:PASSWORD

my jupyter notebooks all use python 3.9, even though my system python was 3.10 and it looks like 3.10 is also used by the hub:

$ python3 --version
Python 3.10.4
$ ls /opt/tljh/hub/lib/
python3.10
$ source /opt/tljh/user/bin/activate
(base) $ python3 --version
Python 3.9.7

How can I make the jupyter notebooks use 3.10 or even my system python directly?
Using my system python would make things easier as I managed to install all the packages I need successfully there.

I tried following the steps in the official documentation, as well as the fixes suggested here, but both give me errors.

Background:

More specifically, I am trying to set up a jupyterhub for meep and mpb with:

conda install -c conda-forge pymeep pymeep-extras
# or for tljh:
# sudo env PATH=${PATH} conda install -c conda-forge pymeep pymeep-extras

This command works for me in a miniconda install, but not in the tljh jupyter notebooks.
Or rather, it seems to work, but the following essential imports then fail in python scripts:

import h5py
from meep import mpb

They fail with:

>>> import h5py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/tljh/user/lib/python3.9/site-packages/h5py/__init__.py", line 33, in <module>
    from . import version
  File "/opt/tljh/user/lib/python3.9/site-packages/h5py/version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py/h5.pyx", line 1, in init h5py.h5
ImportError: /opt/tljh/user/lib/python3.9/site-packages/h5py/defs.cpython-39-x86_64-linux-gnu.so: undefined symbol: H5Pget_fapl_direct
>>> from meep import mpb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/tljh/user/lib/python3.9/site-packages/meep/mpb/__init__.py", line 355, in <module>
    from .solver import (
  File "/opt/tljh/user/lib/python3.9/site-packages/meep/mpb/solver.py", line 10, in <module>
    import h5py
  File "/opt/tljh/user/lib/python3.9/site-packages/h5py/__init__.py", line 33, in <module>
    from . import version
  File "/opt/tljh/user/lib/python3.9/site-packages/h5py/version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py/h5.pyx", line 1, in init h5py.h5
ImportError: /opt/tljh/user/lib/python3.9/site-packages/h5py/defs.cpython-39-x86_64-linux-gnu.so: undefined symbol: H5Pget_fapl_direct

In my miniconda install, it works fine.

2

I decided to focus on creating a new conda environment with a newer python version, which turned out to be easier.

Doing it system-wide as admin was a bit trickier, but here is a script that sets up a new “photonics” environment with meep and mpb installed:

#!/bin/bash
set -eu
# shows up in conda env list
MYENV_CONDANAME="photonics"
# shows up in the jupyter kernel list
MYENV_JUPYTERNAME="photonics"

# activate environment
source /opt/tljh/user/bin/activate
# list current environments
conda env list

set +e
# remove any previous installation
sudo env PATH=${PATH} jupyter kernelspec uninstall "${MYENV_JUPYTERNAME}" -y
sudo env PATH=${PATH} conda env remove -n "${MYENV_CONDANAME}"
set -e

# create new environment
sudo env PATH=${PATH} conda create -n "${MYENV_CONDANAME}" python=3.10 --yes
# activate it
conda activate "${MYENV_CONDANAME}"
# install ipykernel
sudo env PATH=${PATH} CONDA_DEFAULT_ENV="${MYENV_CONDANAME}" conda install ipykernel --yes
# make available in jupyter
sudo env PATH=${PATH} CONDA_DEFAULT_ENV="${MYENV_CONDANAME}" ipython kernel install --name="${MYENV_JUPYTERNAME}"
# install meep/mpb
sudo env PATH=${PATH} CONDA_DEFAULT_ENV="${MYENV_CONDANAME}" conda install -c conda-forge pymeep pymeep-extras --yes

# list current environments
conda env list

# check it worked
jupyter kernelspec list

# # deactivate conda env
# source /opt/tljh/user/bin/activate
# conda activate adminpython
# source /opt/tljh/user/bin/activate
# conda deactivate

echo "Environment successfully added."
1 Like