We’ve had our JupyterHub cluster running on EKS for a while now and it’s been great, but we recently attempted to install the EBS CSI add-on in order to upgrade the cluster to EKS v1.23, and now our pods can’t connect to their volumes, despite the PVCs still being there. Has anyone else had this issue upgrading?
Have you tried kubectl describe on your pods/volumes, or looking at your k8s events? Are you having problems with only pre-existing volumes, or does it occur with new ones too? Is the problem reproducible with a manually created pod/PVC? Were your nodes recycled at any point- IIRC EBS volumes are in a single availability zone so if a node appears in a different zone of the same region, or if your nodes are spread across multiple zones for redundancy, the volume won’t be accessible from some nodes.