How should we get source files from a remote kernel environment?

When an error occurs, a traceback is shown. It often includes a path to the files where the error occurred. Users would benefit from a way to open that file. The file can be either:

  • within the Jupyter root directory (this can be handled by jupyter-server’s ContentsManager if running with a server), or
  • in a remote location (when remote kernels are used)

While ContentsManager can also handle some remote files (if subclassed), it may not be aware of the library/package scripts specific to runtime environment if this is separate from storage which is available to the contents manager.

I wonder whether kernels should be able to expose (potentially read-only) some of their environment source files if needed (if those are remote and separate from the contents manager scope). We already do that for the purpose of the Debugger Adapter Protocol (via debug-request), but it is not currently exposed by kernels which are not debugger-enabled and requires the debugger to be on (which has an overhead). While we could “just” add a new message, it is not clear if we want to go this way (and will require a JEP consensus), especially because only a subset of kernels will ever require this (i.e. most kernels today are local, not remote).

Where there any discussions about for example proxying a subset of ContentsManager API by remote Kernel Provisioners? My question comes because I understand that provisioners will enable remote kernels in a clean way and replace the current implementation in Enterprise Gateway in the future, is this correct?

1 Like

It may be worth considering in this in the broader context of “any files a client might need from a kernel,” which would include JS/CSS assets, and even data files, and in the brave new era, files under collaboration (which is kinda scary vs an arbitrary file lookup mechanism).

Indeed, I believe the julia kernel already does something like this, as it doesn’t really rely on files-in-Jupyter-paths for much, and has (several) home-rolled solutions.

Unfortunately, in the JupyterLab+friends application architecture, these wouldn’t work that well, as having to restart a browser after a kernel is loaded to get some new assets would be… awkward. Scoped import maps (and the related adopted stylesheets) only just became generally available in mainstream browsers, which would be the missing piece to tying some of these things together.

And yes, running this through the Jupyter Contents API (and its “dumb” /files route) would likely be the weird way to expose this to a client, e.g. GET /api/contents/.kernel/{session-id}/etc/passwd and GET /files/.kernel/{session-id}, even if some custom messages (but probably better, comm) actually serviced it.

2 Likes