404 not found with custom kernel

JupyterHub
1

Hi,

I’m getting strange error while using custom kernel.

I have very simple code

! kinit -kt <userName>.keytab <userName>

import pyspark
import os
from pyspark import SparkContext
from pyspark.sql import SparkSession

import sys

spark = SparkSession.builder\
                    .master("yarn")\
                    .config("spark.sql.deployment","cluster")\
                    .config("spark.dynamicAllocation.enabled", "true")\
                    .appName("Tutorial")\
                    .getOrCreate()

But this code gives me this error

[E 2024-04-15 13:33:59.691 ServerApp] Uncaught exception GET /user/<username>/api/contents/.envs/<condaEnvName>/lib/python3.9/site-packages/pyspark/context.py?content=0&hash=0&1713188039531 (::ffff:127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:9999', method='GET', uri='/user/<username>/api/contents/.envs/<condaEnvName>/lib/python3.9/site-packages/pyspark/context.py?content=0&hash=0&1713188039531', version='HTTP/1.1', remote_ip='::ffff:127.0.0.1')
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 153, in get
        model = await ensure_async(
                ^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_core/utils/__init__.py", line 198, in ensure_async
        result = await obj
                 ^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/contents/filemanager.py", line 922, in get
        model = await self._file_model(
                ^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/contents/filemanager.py", line 823, in _file_model
        model = self._base_model(path)
                ^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/contents/filemanager.py", line 235, in _base_model
        raise web.HTTPError(404, four_o_four)
    tornado.web.HTTPError: HTTP 404: Not Found (file or directory does not exist: '.envs/<condaEnvName>/lib/python3.9/site-packages/pyspark/context.py')

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/auth/decorator.py", line 73, in inner
        return await out
               ^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 179, in get
        await self._finish_error(
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 113, in _finish_error
        self.write(message)
      File "/opt/conda/lib/python3.11/site-packages/tornado/web.py", line 914, in write
        raise RuntimeError("Cannot write() after finish()")
    RuntimeError: Cannot write() after finish()
[I 2024-04-15 13:33:59.776 ServerApp] 200 GET /user/<username>/api/kernels?1713188039635 (<username>@::ffff:127.0.0.1) 0.60ms
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

But the file exists
/home/jovyan/.envs/<condaEnvName>/lib/python3.9/site-packages/pyspark/context.py

I’ve even added mentioned path to PATH variable
I also tried to add this to path
export PATH=$PATH:/home/jovyan/.local/bin:/home/jovyan/.envs/<condaEnvName>:/home/jovyan/.envs/:/home/jovyan/.envs/<condaEnvName>/lib/python3.9/site-packages/

Any idea what I’m doing wrong ?

2

I’m experiencing a similar issue with JupyterLab… The notebook exists and I can open it and run it but I’m seeing a bunch of these errors on the command line.