How to async from sync?

Kernels
1

Registered to share the story:
I manage some data tools for the team, and some of the tools have UI widgets for ease of use (ipywidgets).
The problem - ipywidgets that i use synchronous, but i need to call async code form it, and here where the trouble begin.

For context our setup:
Python 3.12 ,
nest_asyncio==1.6
jupyter==1.1.1
jupyter-bokeh==4.0.5
jupyter-client==8.6.3
jupyter-console==6.6.3
jupyter-core==5.8.1
jupyter-events==0.12.0
jupyter-lsp==2.2.6
jupyter-server==2.16.0
jupyter-server-terminals==0.5.3
jupyterhub==5.4.3
jupyterlab==4.4.5
jupyterlab-pygments==0.3.0
jupyterlab-server==2.27.3
jupyterlab-widgets==3.0.15

Initial approach to sync in async was

running_loop = asyncio.get_running_loop()
nest_asyncio.apply(running_loop)
running_loop.run_until_complete(async_fuinction())

but this would kill the kernel after each execution…

Traceback (most recent call last):
File “”, line 198, in _run_module_as_main
File “”, line 88, in _run_code
File “/home/maksym/backoffice/.venv/lib/python3.12/site-packages/ipykernel_launcher.py”, line 18, in
app.launch_new_instance()
File “/home/maksym/backoffice/.venv/lib/python3.12/site-packages/traitlets/config/application.py”, line 1075, in launch_instance
app.start()
File “/home/maksym/backoffice/.venv/lib/python3.12/site-packages/ipykernel/kernelapp.py”, line 739, in start
self.io_loop.start()
File “/home/maksym/backoffice/.venv/lib/python3.12/site-packages/tornado/platform/asyncio.py”, line 211, in start
self.asyncio_loop.run_forever()
File “/usr/lib/python3.12/asyncio/base_events.py”, line 645, in run_forever
self._run_once()
File “/usr/lib/python3.12/asyncio/base_events.py”, line 1984, in _run_once
handle = self._ready.popleft()

Then Maybe thread executor?
Something like this:
with ThreadPoolExecutor(max_workers=1) as executor:
res = executor.submit(asyncio.run, async_func(…))
res.result()

while working, this create three problems:

  1. all logs printed into the cell , instead of log console
  2. memory seems leaking
  3. this slow down whole kernel \ server after a dozen of uses

and then while carefully checking the logs, the code of nest_asyncio (no longer maintained)
i found out that it is possible to enforce specific scripts to be executed during the start jupyter server

go to ~/.ipython/profile_default/startup
create file accordingly to provided README
i used 00-nest-asyncio.py
with the content:
import sys
sys.path.insert(0, “path/to_my_venv”)
import nest_asyncio
nest_asyncio.apply()

And It worked.

relevant PR into nest_asyncio2