HI, I’m a total noob here so please go easy on me. My goal is to have JupyterHub/JupyterLab running on Docker inside an Azure Container Instance, and have a persistent Azure File Share mounted to the container instance.
Some background. I’m using Docker Desktop (windows) to create the image. Then I push the image to Azure Container Registry, then I build the Azure Container Instance from there. The volume mounts successfully and I can see a user folder being created in the file share.
But for the life of me, I cannot get the working directory of JupyterLab to change from /home/{username} to /home/user/data/{username}. I cannot navigate the tree and therefore, I cannot save a new notebook to the file share. Using the terminal I have verified the /home/user/data/{username} directory is there.
If someone with could please look at my configs and help me fix this, that would be amazing. I have tried many variations both with and without the jupyter_server_config file. I’m not sure what is right and what is wrong at this point.
Thanks,
----------------------------
**Dockerfile:**
FROM jupyterhub/jupyterhub
# Sets user as root
USER root
# Install JupyterHub dependencies
RUN pip3 install --upgrade pip && \
pip3 install \
jupyterhub-firstuseauthenticator \
jupyterhub-ldapauthenticator \
jupyterlab
# Install Python libraries and other dependencies
RUN pip3 install --upgrade pip && \
pip3 install \
pandas \
numpy \
matplotlib \
pulp \
azure-storage-file-share
# Installs various packages
RUN apt-get update && apt-get install -y \
r-base \
r-base-dev \
libzmq3-dev \
libcurl4-openssl-dev \
libssl-dev \
libfontconfig1-dev \
libharfbuzz-dev \
libfribidi-dev \
libfreetype6-dev \
libpng-dev \
libtiff5-dev \
libjpeg-dev \
git \
python3.10-venv \
python3 \
cifs-utils \
nano
# Install R kernel and other dependencies
RUN R -e "install.packages(c('repr', 'IRdisplay', 'IRkernel'), repos='http://cran.rstudio.com/')"
RUN R -e "IRkernel::installspec(user = FALSE)"
# Enables widgets
RUN pip3 install ipywidgets && \
jupyter nbextension enable --py widgetsnbextension --sys-prefix
# Disables JupyterLab Announcements
RUN jupyter labextension disable "@jupyterlab/apputils-extension:announcements"
# Creates 'jupyteradmin' user and grants administrator permissions to this account
RUN useradd -m jupyteradmin && echo "jupyteradmin:password" | chpasswd && adduser jupyteradmin sudo
RUN usermod -aG sudo jupyteradmin
# Installs Conda
RUN apt-get update && \
apt-get install -y wget && \
wget --no-check-certificate -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh && \
bash miniconda.sh -b -p /opt/conda && \
rm miniconda.sh && \
/opt/conda/bin/conda clean --all && \
ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
echo "conda activate base" >> ~/.bashrc
# Add Conda to system PATH
ENV PATH /opt/conda/bin:$PATH
# Disable SSL verification in Conda
RUN conda config --set ssl_verify false
# create a new conda environment
RUN conda create --name py38azml
# activate the new environment
SHELL ["conda", "run", "-n", "py38azml", "/bin/bash", "-c"]
# install ipykernel inside the new environment
RUN pip install ipykernel && \
pip install azureml-core && \
python -m ipykernel install --user --name py38azml --display-name "Python 3.8 - Azure ML" # install the kernel into jupyter kernels directory
# copy the kernel directory from root to shared directory
RUN cp -r /root/.local/share/jupyter/kernels/py38azml /usr/local/share/jupyter/kernels/
# create a new conda environment
RUN conda create --name py310azml-sdkv2
# activate the new environment
SHELL ["conda", "run", "-n", "py310azml-sdkv2", "/bin/bash", "-c"]
# install ipykernel inside the new environment
RUN pip install ipykernel && \
pip install azure-ai-ml && \
python -m ipykernel install --user --name py310azml-sdkv2 --display-name "Python 3.10 - SDK v2" # install the kernel into jupyter kernels directory
# copy the kernel directory from root to shared directory
RUN cp -r /root/.local/share/jupyter/kernels/py310azml-sdkv2 /usr/local/share/jupyter/kernels/
# create a new conda environment
RUN conda create --name py38azml_pttf
# activate the new environment
SHELL ["conda", "run", "-n", "py38azml_pttf", "/bin/bash", "-c"]
# install ipykernel inside the new environment
RUN pip install ipykernel && \
pip install tensorflow && \
python -m ipykernel install --user --name py38azml_pttf --display-name "Python 3.8 - Pytorch and Tensorflow" # install the kernel into jupyter kernels directory
RUN conda install pytorch torchvision -c pytorch
# copy the kernel directory from root to shared directory
RUN cp -r /root/.local/share/jupyter/kernels/py38azml_pttf /usr/local/share/jupyter/kernels/
# Sets environment variables
ENV AZURE_STORAGE_ACCOUNT=accountname
ENV AZURE_STORAGE_ACCESS_KEY=key
ENV NB_UID=1000
ENV NB_GID=1000
# Installs Azure File Share client
RUN pip install azure-storage-file-share
# User permissions
RUN mkdir /mnt/azure && \
chmod 777 /mnt/azure && \
mkdir -p /home/user/data && \
chown -R $NB_UID:$NB_GID /home/user/data && \
chmod -R 777 /home/user/data
# Copies the jupyterhub_config.py file from your local machine to the /srv/jupyterhub/ directory in the Docker image.
COPY jupyterhub_config.py /srv/jupyterhub/jupyterhub_config.py
COPY jupyter_server_config.py /srv/jupyter/jupyter_server_config.py
# Expose the JupyterHub port
EXPOSE 8000
# Start the JupyterHub server
CMD ["jupyterhub", "--ip=0.0.0.0", "--port=8000", "--config=/srv/jupyterhub/jupyterhub_config.py"]
---------------------------------------------------
**jupyterhub_config.py File:**
import subprocess
import os
# Jupyterhub settings
c.JupyterHub.ip = '0.0.0.0'
c.JupyterHub.port = 8000
c.JupyterHub.base_url = '/'
# Authentication Settings
c.JupyterHub.authenticator_class = 'firstuseauthenticator.FirstUseAuthenticator'
c.Authenticator.admin_users = {'jupyteradmin'}
# Spawner settings
c.JupyterHub.spawner_class = 'jupyterhub.spawner.LocalProcessSpawner'
c.LocalProcessSpawner.default_shell_cmd = ['/bin/bash']
c.LocalProcessSpawner.env_keep.append('HOME')
c.LocalProcessSpawner.cmd = ['jupyterhub-singleuser', "--config=/srv/jupyter/jupyter_server_config.py"]
from subprocess import check_call
def pre_spawn_hook(spawner):
setup_azure_mount(spawner) # call setup_azure_mount() with argument
username = spawner.user.name
try:
check_call(['useradd', '-ms', '/bin/bash', username])
except Exception as e:
print(f'{e}')
azure_user_path = os.path.join('/mnt/azure', username)
os.makedirs(azure_user_path, exist_ok=True)
local_user_path = os.path.join('/home/user/data', username)
if not os.path.exists(local_user_path):
os.symlink(azure_user_path, local_user_path)
spawner.environment['HOME'] = local_user_path # Set user's HOME environment variable
spawner.user_options = {'notebook_dir': local_user_path}
spawner.args = [arg.format(notebook_dir=local_user_path) for arg in spawner.args]
c.Spawner.pre_spawn_hook = pre_spawn_hook
def setup_azure_mount(spawner):
subprocess.run(
[
"/bin/bash",
"-c",
'echo "//containername.file.core.windows.net/test-jupyter /mnt/azure cifs vers=3.0,username=containername,password=password,dir_mode=0777,file_mode=0777" >> /etc/fstab && mkdir -p /mnt/azure && mount -a && chown -R 1000:100 /mnt/azure'
]
)
c.Spawner.mem_limit = '2G'
c.Spawner.cpu_limit = 1
# Needed for additional environments
c.Spawner.env_keep.append('VIRTUAL_ENV')
---------------------------------------------------
**jupyter_server_config.py File:**
# Configuration file for lab.
c = get_config() #noqa
## The directory to use for notebooks and kernels.
# Default: ''
import os
c.ServerApp.root_dir = f"/home/user/data/{os.environ.get('JUPYTERHUB_USER')}"