I deployed Zero to Jupyterhub in my on-premise k8s cluster, which has 3 nodes and deployed through kubespray. this is a existing k8s cluster , I was using MetalLB BPG mode , not the MetalLB layer 2 mode which was recommended by Zero to Jupyterhub
after I deployed jupyterhub, all pods works fine except hub pods restarts again and again
k get pods -n jupyterhub
NAME READY STATUS RESTARTS AGE
continuous-image-puller-clf6d 1/1 Running 0 41h
continuous-image-puller-ldw6z 1/1 Running 0 41h
continuous-image-puller-mrlth 1/1 Running 0 41h
continuous-image-puller-pzcgk 1/1 Running 0 41h
continuous-image-puller-rb4m4 1/1 Running 0 41h
continuous-image-puller-vptfn 1/1 Running 0 41h
hub-6df78db8d8-49rlx 1/1 Running 1 (2m20s ago) 12m
proxy-75c547b6fb-kn7sl 1/1 Running 0 41h
user-scheduler-fd7895df5-9xkj4 1/1 Running 0 46h
user-scheduler-fd7895df5-fvn7m 1/1 Running 0 46h
so I checked the logs in hub pod, I got following msg , how can I solve this issue,
[I 2022-08-28 08:49:22.870 JupyterHub log:189] 302 GET /user/user/lab? -> /hub/user/user/lab? (@10.233.64.178) 1.72ms
[D 2022-08-28 08:49:23.087 JupyterHub base:328] Refreshing auth for admin
[W 2022-08-28 08:49:23.087 JupyterHub base:1449] User admin requested server for user, which they don't own
[I 2022-08-28 08:49:23.088 JupyterHub log:189] 302 GET /hub/user/user/lab? -> /user/admin/lab (admin@10.233.64.178) 44.28ms
[I 2022-08-28 08:49:23.292 JupyterHub log:189] 302 GET /user/admin/lab -> /hub/user/admin/lab (@10.233.64.178) 1.54ms
[D 2022-08-28 08:49:23.477 JupyterHub user:317] Creating <class 'kubespawner.spawner.KubeSpawner'> for admin:
[D 2022-08-28 08:49:23.483 JupyterHub spawner:194] Starting executor thread pool with 320 workers
WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x72ca43fccbb0>, 'Connection to 10.233.0.1 timed out. (connect timeout=60)')': /api/v1/namespaces/jupyterhub/pods?fieldSelector=&labelSelector=component%3Dsingleuser-server
WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x72ca43fccd90>, 'Connection to 10.233.0.1 timed out. (connect timeout=60)')': /api/v1/namespaces/jupyterhub/pods?fieldSelector=&labelSelector=component%3Dsingleuser-server
WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x72ca43fccee0>, 'Connection to 10.233.0.1 timed out. (connect timeout=60)')': /api/v1/namespaces/jupyterhub/pods?fieldSelector=&labelSelector=component%3Dsingleuser-server
thanks in advance
Sam