IPython & Jupyterlab: kernel gets "stuck" after a long-running cell that loads a huge object. Bug?

After I execute a long-running cell that loads a huge object in Jupyterlab using the IPython kernel, the kernel seems to get “stuck” running some kind of background task that never finishes.

It’s unclear if it’s because the cell ran for a long time (~3 minutes) or because it loaded ~1.6 GB of JSON. Is this an issue with the variable inspector, or with the long-running process notifier?

I ensured that the indicator in the top right says “Kernel status: Idle” before running the command. I also made sure that the debugger mode was disabled; the little bug icon is gray, not orange.

After running, it says “Kernel Status: Busy”, and “Executed 1/2 requests”. It keeps a running clock of how long it’s been busy for. It never seems to finish, and as far as I can tell, all the actual computation is long since finished. See this screenshot if my description is not clear.

The result is that all subsequent cell executions end up in some kind of job queue, and the only way to get any output is to press “Interrupt” several times before running each cell. This is extremely annoying and tedious!

Is this a known problem of some kind? What kind of info can I provide in order to help reproduce this?

I am using MacOS 12.4, and Google Chrome 103.0.5060.114.

Originally I had thought that the problem was my use of tqdm.notebook and/or of %matplotlib widget, but I have since removed both of them from my notebook and I still get this problem.

$ mamba list | grep -E 'jupyter|python'

gitpython                 3.1.27             pyhd8ed1ab_0    conda-forge
ipython                   8.4.0           py310hbe9552e_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jupyter-lsp               1.5.1              pyhd8ed1ab_0    conda-forge
jupyter-server-mathjax    0.2.5              pyhc268e32_0    conda-forge
jupyter_client            7.3.1              pyhd8ed1ab_0    conda-forge
jupyter_core              4.10.0          py310hbe9552e_0    conda-forge
jupyter_server            1.17.0             pyhd8ed1ab_0    conda-forge
jupyterlab                3.4.2              pyhd8ed1ab_0    conda-forge
jupyterlab-fileopen       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab-git            0.37.1             pyhd8ed1ab_0    conda-forge
jupyterlab-lsp            3.10.1             pyhd8ed1ab_0    conda-forge
jupyterlab-notifications  0.4.1              pyhd8ed1ab_0    conda-forge
jupyterlab-topbar         0.6.1              pyhd8ed1ab_2    conda-forge
jupyterlab-unfold         0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab-variableinspector 3.0.9              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.14.0             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        1.1.0              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.3           py310hea002bf_1    conda-forge
python                    3.10.4          hfc7342c_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.15.3             pyhd8ed1ab_0    conda-forge
python-flatbuffers        2.0                pyhd8ed1ab_0    conda-forge
python-tzdata             2022.1             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge
snowflake-connector-python 2.7.8           py310hc2056c1_0    conda-forge
xeus-python               0.13.9          py310h7c1f55a_0    conda-forge
xeus-python-shell         0.2.0              pyhd8ed1ab_0    conda-forge
$ jupyter labextension list
JupyterLab v3.4.2
        jupyterlab-fileopen v0.2.2 enabled OK (python, jupyterlab-fileopen)
        jupyterlab-notifications v0.4.1 enabled OK (python, jupyterlab-notifications)
        jupyterlab_pygments v0.2.2 enabled OK (python, jupyterlab_pygments)
        jupyterlab-topbar-extension v0.6.1 enabled OK (python, jupyterlab-topbar)
        nbdime-jupyterlab v2.1.1 enabled OK
        jupyterlab-unfold v0.2.2 enabled OK (python, jupyterlab-unfold)
        jupyter-matplotlib v0.11.1 enabled OK
        @lckr/jupyterlab_variableinspector v3.0.9 enabled OK (python, lckr_jupyterlab_variableinspector)
        @krassowski/jupyterlab-lsp v3.10.1 enabled OK (python, jupyterlab-lsp)
        @jupyterlab/git v0.37.1 enabled OK (python, jupyterlab-git)
        @jupyter-widgets/jupyterlab-manager v3.1.0 enabled OK (python, jupyterlab_widgets)

Other labextensions (built into JupyterLab)
   app dir: /Users/.../.local/opt/mamba/envs/sandbox/share/jupyter/lab
        @jupyterlab/celltags v3.4.2 enabled OK
        @jupyterlab/debugger v3.4.2 enabled OK

Upgrading Jupyterlab to 3.4.3 and uninstalling lckr-jupyterlab-variableinspector appears to have fixed it!

Does 3.4.3 include Remove ipywidgets message count in the execution indicator model by trungleduc · Pull Request #12665 · jupyterlab/jupyterlab · GitHub? If so, it looks like Notebook execution indicator gets confused by Comms · Issue #12616 · jupyterlab/jupyterlab · GitHub in some cases was more than just a visual bug.

That, or, it was unrelated, and the real problem was some adverse behavior in lckr-jupyterlab-variableinspector. Or possibly some other bad interaction solved by reinstalling.