For those who hasn’t seen, JupyterLite is a version of JupyterLab that runs fully on the browser using Pyodide.
I am currently developing a new Jupyter kernel called ICortex that is based on ipykernel, the Jupyter kernel which runs IPython. In my current implementation, I simply subclass IPythonKernel and override some methods to add new functionalities.
My goal is to port ICortex to JupyterLite and run it fully in the browser. I’ve been toying around with different JupyterLite kernels & configurations:
I thought xeus-python would be a good place to start until I realized it’s a full reimplementation in C++ that’s separate from ipykernel. So that’s not viable for me unless I decide to port ICortex to xeus at a future date.
I see that Pyolite derives from ipykernel to create a kernel that uses Pyodide.
Should I do it the same way as Pyolite, to port ICortex to JupyterLite? What would be the best practice for this?
A main concern is to not end up having to maintain duplicate logic in different packages. If possible, I want to just be able to wrap my existing non-WASM package. Do you think this is possible?
Sadly, neither of these have been updated in a bit, and are probably behind on some of the API changes, and therefore have some nasty copy-paste reuse patterns.
We’re hoping to split pyolite out as a separate package and repo, partially so that you’d have the option of using only other python kernels, such as jupyterlite/xeus-python-kernel, rather than having both in a deployment by default. This would also improve some packaging discipline, as things are a bit… tangled inside core at the moment.
Thanks! These create a new kernel the same way I am.
I copied the folder lite from Pidgy and adapted it to my project, and can finally see my kernel in the list, however I’m still running into issues executing cells:
It prints the right banner when I start up a console, but when I execute a cell, it just hangs and says that the kernel is busy.
Pyolite also hangs, so the issue is not only with my kernel.
I tried to rebuild and host Pidgy without touching anything—same problem.
I’m using jupyterlite==0.1.0a20 as in the repo to prevent any errors due to API changes.
On the browser console, I get messages along the lines of:
ValueError: Couldn't find a pure Python 3 wheel for 'psutil'. You can use `micropip.install(..., keep_going=True)` to get a list of all packages with missing wheels.
Do you know what might be the issue?
Either case, I’m not too keen on depending on the old API. My previous issue was not being able to verify or see whether my kernel was really included in the build.
yep, no psutil, because no processes. It’s important to maintain all of the hacks and patches applied in pyolite.
Is the right way to define a new kernel, and include the field along the lines
No, it will need to be a full typescript package (even if it’s never published).
The argv is a lie, as what’s a terminal? The kernelspec is there to appease all the APIs and validation, but it’s the stuff in kernel.ts and worker.ts that are really running the show, and that’s what’s changed the most between what you’re seeing there and what’s on main… but you want those changes real bad, as they include the unified contents/kernel files. Still rough edges there, as well, but being able to upload scripts and import them (with a bit of care) as well as download data is worth the hassle.
I’ll circle back on pidgy at some point and update that… the robot one is a bit deeper.
This just made me realize it’s probably not straightforward to make HTTP requests either, which I need to in my kernel. I just tried with the old and the current version, and urllib.request raises host unreachable error in both. I’m guessing making that work, even for a single domain, would require a hack? Should I look into how piplite does it?
otherwise, the rest of that library is has a hard async requirement, so you could refactor your implementation to be a bit more general, and use something like tornado.AsyncHTTPClient or httpx when subclassing ipykernel.