Hello, we have internal instance of Jupyterlab, running on k8s cluster. After the moment I open UI, I usually experience several consecutive errors like:
[E 2023-11-17 12:07:44.475 ServerApp] 500 GET <some_path>/jupyter/api/kernels/153ad49f-6d1f-4f68-aa44-1e848b2b2dd7/channels?session_id=c3780625-5bd9-42cd-9a02-2a6dc46e3749 (266b9f0237614ad6885dbb74d6558310@10.218.43.180) 33668.23ms referer=None
[E 2023-11-17 12:07:44.476 ServerApp] Uncaught exception GET <some_path>/jupyter/api/kernels/153ad49f-6d1f-4f68-aa44-1e848b2b2dd7/channels?session_id=c3780625-5bd9-42cd-9a02-2a6dc46e3749 (10.197.217.112)
HTTPServerRequest(protocol='http', host='<some_host>:8080', method='GET', uri='<some_path>/jupyter/api/kernels/153ad49f-6d1f-4f68-aa44-1e848b2b2dd7/channels?session_id=c3780625-5bd9-42cd-9a02-2a6dc46e3749', version='HTTP/1.1', remote_ip='10.197.217.112')
Traceback (most recent call last):
File "/opt/spark/work-dir/miniconda3/lib/python3.10/site-packages/tornado/web.py", line 1786, in _execute
result = await result
File "/opt/spark/work-dir/.local/lib/python3.10/site-packages/jupyter_server/services/kernels/websocket.py", line 65, in get
await super().get(kernel_id=kernel_id)
File "/opt/spark/work-dir/miniconda3/lib/python3.10/site-packages/tornado/websocket.py", line 272, in get
await self.ws_connection.accept_connection(self)
File "/opt/spark/work-dir/miniconda3/lib/python3.10/site-packages/tornado/websocket.py", line 862, in accept_connection
await self._accept_connection(handler)
File "/opt/spark/work-dir/miniconda3/lib/python3.10/site-packages/tornado/websocket.py", line 902, in _accept_connection
self.selected_subprotocol = handler.select_subprotocol(subprotocols)
File "/opt/spark/work-dir/.local/lib/python3.10/site-packages/jupyter_server/services/kernels/websocket.py", line 87, in select_subprotocol
preferred_protocol = self.connection.kernel_ws_protocol
AttributeError: 'NoneType' object has no attribute 'kernel_ws_protocol'
After those websocket is being opened and no further errors occure, nor any issues with functionality of jupyterlab I can detect:
[E 2023-11-17 12:07:44.477 ServerApp] 500 GET <some_path>/jupyter/api/kernels/153ad49f-6d1f-4f68-aa44-1e848b2b2dd7/channels?session_id=c3780625-5bd9-42cd-9a02-2a6dc46e3749 (2367bf49cb2f4e97a05df0b79a8909ea@10.197.217.112) 28037.39ms referer=None
[D 2023-11-17 12:07:44.477 ServerApp] 101 GET <some_path>/jupyter/api/kernels/153ad49f-6d1f-4f68-aa44-1e848b2b2dd7/channels?session_id=c3780625-5bd9-42cd-9a02-2a6dc46e3749 (2367bf49cb2f4e97a05df0b79a8909ea@10.197.217.112) 2884.24ms
[D 2023-11-17 12:07:44.477 ServerApp] Opening websocket <some_path>/jupyter/api/kernels/153ad49f-6d1f-4f68-aa44-1e848b2b2dd7/channels
I failed so far to locate the root-cause of this error, if anybody has any ideas on what can cause it, the help is greatly appreciated.