Cuda and images problem on JupyterHub(k8s)!

JupyterHub Zero to JupyterHub on Kubernetes
,
1

My JupyterHub had been ran on k8s well. However, there some problems were blocked to me. I would post my problems firstly, and then post my the config of JupyterHub.

1) Problems:

  1. Image was lost when the pods restart.
    When I first select my initial jupyter image, it running very well, and I could use pip to install any packages. However, after a while, I need to choice jupyter image again when the user’s pod restart. At the same time, I need to re-install my packages which had been installed at my first time.

  2. Cuda not available on notebook.
    I could see CUDA information in my local machine when I run nvidia-smi. But I could not see it in my notebook when I run !nvidia-smi. My docker has been installed CUDA:10.2-base, but how should I include it into jupyterhub with setting hub config?

  3. How can I integrate the gpu server into an image? Such as datascients image.
    I provided many images to user via profileList, but I just want to a gpu server as a default image. Users just directly start, they do not need to select which kind of images they need.

2) My config(local part)

singleuser:
  defaultUrl: "/lab"
  image:
    name: jupyter/minimal-notebook
    tag: 2343e33dec46
  profileList:
    - display_name: "Minimal environment"
      description: "To avoid too much bells and whistles: Python."
      default: true
    - display_name: "GPU Server"
      description: "Spawns a notebook server with access to a GPU"
      kubespawner_override:
        extra_resource_limits:
          nvidia.com/gpu: "1"
    - display_name: "Datascience environment"
       description: "If you want the additional bells and whistles: Python, R, and Julia."
       kubespawner_override:
         image: jupyter/datascience-notebook:2343e33dec46
    - display_name: "Spark environment"
      description: "The Jupyter Stacks spark image!"
      kubespawner_override:
        image: jupyter/all-spark-notebook:2343e33dec46
    - display_name: "Learning Data Science"
      description: "Datascience Environment with Sample Notebooks"
      kubespawner_override:
        image: jupyter/datascience-notebook:2343e33dec46
        lifecycle_hooks:
          postStart:
            exec:
              command:
                - "sh"
                - "-c"
                - >
                  gitpuller https://github.com/data-8/materials-fa17 master materials-fa;

Any help is appreciated.

2

Hi! I’ve written some parts about this that is relevant at https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/994, hope that can be helpful!

3

Thank you so much. But now, My the biggest challenge was Promen 1. When I restart the user’s pod, then I need to select image again, even though I had been selected an image at the initial time.