I have configured custom kernel to connect with pyspark which is running in docker container and install it successfully , below is the configuration of kernel
{
“argv”: [
“docker”,
“exec”,
“-i”,
“149e10d09b12”,
“/usr/local/bin/pyspark”,
“{connection_file}”
],
“display_name”: “PySpark (Docker)”,
“language”: “python”,
“metadata”: {
“debugger”: false
}
}
But when I am trying to connect this kernel from jupyter UI, I am getting below error in logs
[W 2023-11-07 18:07:07.046 SingleUserLabApp handlers:752] wrote error: ‘Kernel does not exist: be5bd613-3d4b-4e98-a805-95e2985882e8’
Traceback (most recent call last):
File “/root/datadrive/miniconda3/lib/python3.11/site-packages/tornado/web.py”, line 1786, in _execute
result = await result
^^^^^^^^^^^^
File “/root/datadrive/miniconda3/lib/python3.11/site-packages/jupyter_server/services/kernels/handlers.py”, line 75, in get
model = await ensure_async(km.kernel_model(kernel_id))
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/root/datadrive/miniconda3/lib/python3.11/site-packages/jupyter_server/services/kernels/kernelmanager.py”, line 500, in kernel_model
self._check_kernel_id(kernel_id)
File “/root/datadrive/miniconda3/lib/python3.11/site-packages/jupyter_server/services/kernels/kernelmanager.py”, line 531, in _check_kernel_id
raise web.HTTPError(404, “Kernel does not exist: %s” % kernel_id)
tornado.web.HTTPError: HTTP 404: Not Found (Kernel does not exist: be5bd613-3d4b-4e98-a805-95e2985882e8)