Users are getting 504 Gateway Error

Hi @jupyterhub_users @JupyterLabUser

I’m hosting jupyterhub 2.2.2 version in openshift cluster. Hub and notebook pods are up and running. users are using actively but intermittently we are seeing 504 GateWay error when user trying to open notebook.

Thank you

We experienced a similar issue and it ended up being a failure in our networking layer (calico) on one dynamically provisioned node at start up so worth looking at all running pods not just the jupyter namespace. This gave us a 1/N failure rate on resolving k8s internal requests. We had fortunately enabled Traefik access logs and so were able to see the 504s were on 30s timeout on various calls within the k8s cluster.