# Implementing more functionality for storage?

**URL:** <https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612>\
**Category:** Zero to JupyterHub on Kubernetes\
**Created:** [April 16, 2025, 9:23am UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612 "2025-04-16T09:23:22Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Roy\_Dragseth](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/roy_dragseth/32/4212_2.png) [@Roy\_Dragseth](https://discourse.jupyter.org/u/Roy_Dragseth)\
**Post date:** [April 16, 2025, 9:23am UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/1 "2025-04-16T09:23:22Z")

</div>

Hi, I’m looking into how to get the extra storage part working more dynamically.

What I would like to achieve is to have something like

```auto
 storage:
    extraVolumes:
      - name: user-datadir
        persistentVolumeClaim:
          claimName: private-datadir-{username}
          spec:
              resources:
                requests:
                  storage: 10Ti
              storageClassName: managed-csi
    extraVolumeMounts:
      - name: user-datadir
        state: optional
        mountPath: /home/datadir

```

The state parameter would be something like precreated (the current default behaviour), dynamic (create on demand and persist) or ephemeral (autocreate and delete on pod exit).

I have a partially working blurb using extraConfig in the helm chart, but this should probably not be too hard to implement in a more general fashion in KubeSpawner.

Before I start diving into the code I just wanted to check if the proposed functionality is already achievable in the current Z2JH? Am I missing something obvious?

Regards,  
Roy

---

<div class="post-metadata">

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\
**Post date:** [April 16, 2025, 12:58pm UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/2 "2025-04-16T12:58:10Z")

</div>

KubeSpawner only supports the automatic creation of a single PVC

> <https://github.com/jupyterhub/kubespawner/blob/9e994a7ce7507196ec2258460836eef40c98d0e1/kubespawner/spawner.py#L3134-L3143>

so you’ll need to subclass KubeSpawner if you want to create additional PVCs.

---

<div class="post-metadata">

**Author:** ![Roy\_Dragseth](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/roy_dragseth/32/4212_2.png) [@Roy\_Dragseth](https://discourse.jupyter.org/u/Roy_Dragseth)\
**Post date:** [April 16, 2025, 2:14pm UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/3 "2025-04-16T14:14:54Z")

</div>

Yes, from what I can see in the code it shouldn’t be too hard. Would this be of general interest? E.g. would a PR for this be accepted into the mainline KubeSpawner/Z2JH? I’m just gauging the interest here to see if it is worth the effort to make the implementation generalized.

---

<div class="post-metadata">

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\
**Post date:** [April 16, 2025, 2:37pm UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/4 "2025-04-16T14:37:02Z")

</div>

Maybe, but I’m not sure. Are you able to share your current workaround with `extraConfig`, then we can decide whether to support it directly in KubeSpawner, or whether to add extension points/hooks to make it easier?

---

<div class="post-metadata">

**Author:** ![Roy\_Dragseth](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/roy_dragseth/32/4212_2.png) [@Roy\_Dragseth](https://discourse.jupyter.org/u/Roy_Dragseth)\
**Post date:** [April 16, 2025, 3:08pm UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/5 "2025-04-16T15:08:13Z")

</div>

Sure, as I said, this is just a partial implementation. If a pv with a matching name to the extra volume exists it will be mounted to the pod. If not, the pv and the mount is removed from the list of extra volumes and extra volume mounts. For now I create the extra volumes manually.

```auto
 extraConfig:
    checkvolumes: |
      async def check_pvcs(spawner):
        """Remove nonexisting pvcs (except default home dir) from pod specification.
            This makes it possible to provide an extra datadir for users."""
        # Assumptions:
        # Default user dir pvc name starts with "volume-"
        # Default user home path is "/home/jovyan"
        import copy
        spawner.log.info("check_pvcs: Doing a PVC check...")

        existing_pvcs = await spawner.api.list_namespaced_persistent_volume_claim(spawner.namespace)
        existing_pvc_names = [n.metadata.name for n in existing_pvcs.items]
        spawner.log.info("check_pvcs: existing_pvc-names %s" % list(existing_pvc_names))
        for i, volume in enumerate(copy.deepcopy(spawner.volumes)):
          expanded_volume = spawner._expand_all(volume)
          spawner.log.info(f"check_pvcs: expanded_volume {expanded_volume}")
          if 'persistentVolumeClaim' not in expanded_volume:
            continue
          evname = expanded_volume['name']
          claimName = expanded_volume['persistentVolumeClaim']['claimName']
          spawner.log.info("check_pvcs: claimname %s" % str(claimName))
          spawner.log.info("check_pvcs: volume %s" % str(volume))
          if not (evname.startswith("volume-") or
                  claimName in existing_pvc_names):
            spawner.log.info("check_pvcs: Removing nonexistant PVC %s" % str(spawner.volumes[i]))
            del spawner.volumes[i]
        filtered_pvc_names = [v['name'] for v in spawner.volumes]
        for i, volume_mount in enumerate(copy.deepcopy(spawner.volume_mounts)):
          expanded_volume_mount = spawner._expand_all(volume_mount)
          mountPath = expanded_volume_mount['mountPath']
          evmname = expanded_volume_mount['name']
          if not (mountPath == "/home/jovyan" or
                  evmname in filtered_pvc_names):
            spawner.log.info("check_pvcs: Nonexisting PVC, removing mountpath %s" % str(spawner.volume_mounts[i]))
            del spawner.volume_mounts[i]

      c.Spawner.pre_spawn_hook = check_pvcs

```

---

<div class="post-metadata">

**Author:** ![Roy\_Dragseth](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/roy_dragseth/32/4212_2.png) [@Roy\_Dragseth](https://discourse.jupyter.org/u/Roy_Dragseth)\
**Post date:** [April 17, 2025, 6:42am UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/6 "2025-04-17T06:42:48Z")

</div>

Oh, I see now that this doesn’t make any sense without the helm chart config

```auto
  storage:
    extraVolumes:
      - name: user-datadir
        persistentVolumeClaim:
          claimName: private-datadir-{username}

    extraVolumeMounts:
      - name: user-datadir
        mountPath: /home/datadir

```

If the a pvc named private-datadir-{username} exists it will get mounted on pod start, if it doesn’t the volume and mount will be deleted from the lists spawner.volumes and spawner.volume\_mounts.

---

<div class="post-metadata">

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\
**Post date:** [April 20, 2025, 10:02pm UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/7 "2025-04-20T22:02:43Z")

</div>

I think we should try and stick with the Kubernetes principle of declarative configuration- I don’t think we should try to mount volumes based on whether they exist or not. I think it’d be fine to make the dynamic volume creation more generic though, so instead of only supporting a single dynamic PVC for the home directory KubeSpawner could support multiple dynamic PVCs.

---

<div class="post-metadata">

**Author:** ![Roy\_Dragseth](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/roy_dragseth/32/4212_2.png) [@Roy\_Dragseth](https://discourse.jupyter.org/u/Roy_Dragseth)\
**Post date:** [April 22, 2025, 9:10am UTC](https://discourse.jupyter.org/t/implementing-more-functionality-for-storage/34612/8 "2025-04-22T09:10:57Z")

</div>

Yes, that would be great. The main thing for me is to be able to provide small and fast storage for home and large (and slow) storage for data repositories.