# Distribute GPU with jupyterhub

**URL:** <https://discourse.jupyter.org/t/distribute-gpu-with-jupyterhub/2181>\
**Category:** JupyterHub\
**Created:** [September 19, 2019, 12:57pm UTC](https://discourse.jupyter.org/t/distribute-gpu-with-jupyterhub/2181 "2019-09-19T12:57:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![RSchauf](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/rschauf/32/1188_2.png) [@RSchauf](https://discourse.jupyter.org/u/RSchauf)\
**Post date:** [September 19, 2019, 12:57pm UTC](https://discourse.jupyter.org/t/distribute-gpu-with-jupyterhub/2181/1 "2019-09-19T12:57:24Z")

</div>

## 1.) The environment

- Ubuntu 18.0.4. headless server
- Nvidia driver 430
- Miniconda installed on a per user base (separately under the /home//miniconda for all
- Cuda 10.1
- jupyterhub 1.0.0
- 4 Tesla P100 GPUs installed on the server
- docker version 19.03.2

## 2.) The Question

How can we let jupyterhub spawn (systemuser-spawner) docker containers with arbitrary GPUs ?  
At this time I can spawn a container with all or specific defined GPUs by the keyword in the Dockerfile:

```auto
ENV NVIDIA_VISIBLE_DEVICES all

```

or

```auto
ENV NVIDIA_VISIBLE_DEVICES 1, 2

```

What we want is something else:

- At spawn menu, one should be able to choose a container with or without GPU.
- If a container with GPU is chosen, the number of GPUs should be defined.
- We have 4 GPUs, and want to provide them as first come - first serve.
- So an unser can pick 4, 3, 2, or 1 GPU if he is the first.
- The next can pick whatever is left, all or some.

How can I create a docker file that is spawnable that receives any nunber of GPUs that are left in my set?

thanks and regards,  
rschauf

---

<div class="post-metadata">

**Author:** ![RSchauf](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/rschauf/32/1188_2.png) [@RSchauf](https://discourse.jupyter.org/u/RSchauf)\
**Post date:** [September 25, 2019, 5:35am UTC](https://discourse.jupyter.org/t/distribute-gpu-with-jupyterhub/2181/2 "2019-09-25T05:35:03Z")

</div>

Hello everybody,

**does no one have a clue, if there is a mechanism for spawnable docker containers to pass in any of the available GPUs into the container?**

AT this time I know only the possibliity, to pass in a particular GPU, i.e. with its GUID or number. So you always get a fixed GPU. This means, to distribute the GPUs with jupyterhub, I need to build an image for each container with another GPU, to make it spawnable with jupyterhub.

We want to offer a pool of GPUs to the user (students) that may spawn a container in jupyterhub with or without one or even more GPUs and according to the priciple “first come -first serve” a student gets whatever is left in the pool.

thanks and regards,  
rschauf

---

<div class="post-metadata">

**Author:** ![manniche](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manniche/32/799_2.png) [@manniche](https://discourse.jupyter.org/u/manniche)\
**Post date:** [November 14, 2019, 12:19pm UTC](https://discourse.jupyter.org/t/distribute-gpu-with-jupyterhub/2181/3 "2019-11-14T12:19:44Z")

</div>

Hi @RSchauf,

If the passing is solely relying on the setting of environment variables, you are able to do so in the Spawner by using the `c.Spawner.environment` configuration setting as described here: [https://jupyterhub.readthedocs.io/en/stable/api/spawner.html#jupyterhub.spawner.Spawner.environment](https://jupyterhub.readthedocs.io/en/stable/api/spawner.html#jupyterhub.spawner.Spawner.environment)

I hope this helps you out.

Best,  
Steen