Plot interactivity broken, even in notebooks where it worked before

[arosko]

Earlier this week, I created a new notebook, and matplotlib plots were not interactive using %matplotlib widget. In particular for my use case, 3D plots aren’t rotatable, but zooming and panning doesn’t work either. The buttons for interaction show up next to the plot as usual, but do nothing (except for the save button, which does download a static image).

Thinking there was a problem in my code, I re-ran some old notebooks from months ago that also had plots, and they too are no longer interactive, even though they worked when I created them and I didn’t change a single line of code in a single cell in those notebooks. If I drag on the plots, the hourglass shows up for a few seconds, showing that the mouse movement is being detected and triggering some response, but nothing visible changes. Calling plt.isinteractive() returns True.

If I switch to %matplotlib notebook, I get the “iPython not found” error, and I even tried installing ipympl and using %matplotlib ipympl (which wasn’t necessary before) and they’re STILL not interactive. I’m suspecting that a Chrome update somehow broke interactivity with the Notebook renderer because nothing else changed, and everything else in Notebook still works fine.

Has anyone else encountered this issue recently? It’s Chrome in Windows 11, both fully up to date. The last time I know the interactivity worked was sometime in October.

[fomightez]

Can you please read Getting good answers to your questions and share code and version numbers so we can talk in specifics.

You may be right that things aren’t working in your case; however, you are not demonstrating that whereas I have code and offerings that work below when using the ipympl documentation with my fully updated Chrome on Mac (Version 143.0.7499.170 (Official Build) (arm64)).

One thing to try is to make sure you do a hard refresh of your browser page. This is necessary as ipympl affects a lot of things and sometimes if things break you need a hard browser refresh to propagate all the changes. On a Mac in Chrome, it is 'SHIFT + COMMAND + R`. I suggest looking up what it is in your version of Windows.

If you want to share code here, see about ‘block code formatting’ here. (Or see about ‘fenced code blocks’ here. They are both the same thing if you look into the details.) You can also use code snippet sharing resources like GitHub’s gists.

---

@arosko

I even tried installing ipympl and using %matplotlib ipympl (which wasn’t necessary before)

Just because it worked without ipympl before doesn’t mean it should still. ipympl is the current way and has been for years now.

As the ipympl github repo says, " ipympl enables the interactive features of matplotlib in the Jupyter notebook and in JupyterLab."

I have posted about it here and on StackOverflow fairly often. I can point you to examples that work right now in environments where things work. See here for code that you can run using the environment you’ll get following instructions at the bottom of here to get a mybinder-served session that is on a remote machine without needing to login or touch your own machine. The code referenced in option #2 there. These both work in Chrome on a Mac updated fully, specifically Version 143.0.7499.170 (Official Build) (arm64) .

As here makes it clear, now %matplotlib widget is just using ipympl behind the scenes in modern Jupyter. I would encourage you though to be explicit and use %matplotlib ipympl as it makes clear to you and others what is necssary to use the feature. (Others have even reported sometimes %matplotlib ipympl works when %matplotlib widget does not. That shouldn’t be the case; however, fortunately, the more explicit approach is the one that works in those cases.)

[arosko]

That linked code with the slider runs on the remote machine (after installing ipympl in that remote environment, which isn’t on there by default). The existing demo plots in that remote notebook also become interactive if I change %matplotlib inline to %matplotlib ipympl. However, on my own machine (in a new notebook) the code with the slider doesn’t plot anything at all–no axes, no points, nothing. The slider shows up and can be dragged but it also does nothing.

I looked into the console window from which the notebook was launched, and noticed there was an error that came up whenever I opened a notebook. It was a long cascading error whose ultimate cause was “ModuleNotFoundError: No module named 'notebook.services’” in a file called “collapsible_headings.py” in nbconvert.

After a bit of searching here on this error, I have discovered that this error results when the notebook is version 7+ but some of the dependencies are still version 6.x.

Therefore, I have gathered that the most logical course of action is to do a clean reinstall of Jupyter. Do you agree? Assuming that’s the case, what is the best way to do this to A) avoid creating MORE incompatibilities, and B) lose as little work as possible–not just keeping all my existing .ipynb files but also hopefully all the other non-Jupyter-related packages installed using pip in my system Python, many of which are required for my existing saved notebooks as well as other Python scripts I have. I also have the existing Jupyter connected to my Julia install, so that I get the Julia kernel option in addition to iPython when creating a notebook (I installed Julia after Jupyter and of course Python had been installed for years) and would like to not break any compatibility there either.

[fomightez]

@arosko

I looked into the console window from which the notebook was launched, and noticed there was an error that came up whenever I opened a notebook. It was a long cascading error whose ultimate cause was “ModuleNotFoundError: No module named 'notebook.services’” in a file called “collapsible_headings.py” in nbconvert.

Sounds like you got to the bottom of the issue there.

@arosko

installed using pip in my system Python

This is likely part of the issue.
From what you describe, you outgrew that being viable for you a long time ago.

You would be better served by keeping what still works, if it indeed does (You talk as if something does?), and using Anaconda/conda, or at least virtual environments, (or maybe uv if you look into it) to separately make environments where you can work with modern versions of what you need separate from your old install.

Or … is it possible you can use Anaconda Cloud for your modern stuff and keep what you have on your local system?

[arosko]

This is likely part of the issue.
From what you describe, you outgrew that being viable for you a long time ago.

What did I outgrow? Is running snippets of Python code in a notebook that much more of an advanced technology than running code in a regular Python script (using the same packages, like Numpy and matplotlib) that it requires a completely different setup?

@fomightez

You would be better served by keeping what still works, if it indeed does (You talk as if something does?)

Everything but plot interactivity works. Now, that IS rather important since a lot of what I’m working on depends on 3D visualization… but I mean that I can import Numpy and other packages I use (like biopython) and use them no problem, as long as I’m getting static output (printed text, 2D graphs).

I’m not sure exactly what you mean by “modern versions of what I need”–the packages I directly install (the ones I explicitly import in my code, whether in a notebook or a script) generally have a rather stable interface and I don’t think about updating them continuously unless there’s an important new feature or some major performance upgrade. The dependencies of Jupyter itself are something I trust Jupyter’s own install and/or update mechanism to keep in sync, because I never use those directly and in fact wouldn’t know what most of them do. The only time I have really had to think in detail about creating a self-contained snapshot of Python at one point in time was when building an application using PyInstaller for distribution through GitHub to people who don’t already use Python.

[arosko]

I fixed the ModuleNotFoundError, but it did NOT fix the plot interactivity issue.

What I did was temporarily downgrade notebook to 6.5.7. I then got a message that nglview (one of the packages I’ve used in some of my notebooks, which is a molecular structure viewer that’s also interactive) is not compatible with 6.5, so I reinstalled nglview and it upped the notebook version back to 7.5.1 So now I have 7.5.1 again but I can open any notebook I want without getting ANY error messages at all in the console. But as I said, the plot interactivity is still broken.

I checked one of my past notebooks that used nglview and it runs fine–including the nglview embeds themselves which are still interactive. I know for a fact that both nglview and interactive matplotlib worked simultanously in the past (because some of my notebooks used BOTH of them), so that proves that they don’t have compatibility requirements that conflict with one another. I have absolutely nothing to go on though now about what’s causing the matplotlib error since there are no error messages anywhere.

[arosko]

Looking around a bit more at the output of the notebook, I saw that while there are no errors, there were warnings, one of which was

“Got events for closed stream <zmq.eventloop.zmqstream.ZMQStream object at… >”

This caught my attention because the non-interactivity suggests that the event loop for the notebook widgets is broken somewhere.

This was preceded by a notice, that didn’t even reach “warning” level,

“Skipped non-installed server(s): basedpyright, bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyrefly, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server”

This all led me to this thread:

Jupyter Notebook 'zmq message arrived on closed channel' Error

Dear all, I’ve recently begun to experience near constant zmq/tornado errors when running Jupyter notebook from my conda environment (Jupyter, conda env, and traceback details below). When the error occurs, code cells in the Jupyter Notebook fail to either run entirely and/or the cells run but the output does not get printed to the Notebook. The same error also occurs while using JupyterLab but seemingly less frequently. On occasion, Jupyter will run for several hours without exhibiting the is…

Some people in that thread linked this to matplotlib interaction.

It seems from reading this thread that a fix was never found that worked for everyone, and even the fix that worked for the most people again involved restoring pre-v7 versions of components. Seeing as this was in 2023 when version 7 had barely been released, I’m starting to get the impression that version 7 has been buggy, error-prone software from the get-go. This is really a shame since the idea of being able to run snippets of code interactively is a great one–but if it’s subject to regular updates that break even relatively core functionality (matplotlib plotting has been ubiquitous in Python for over a decade now) then that negates its usefulness.

Anyway, is one of those server components that aren’t installed a critical one that could be causing this problem?

[fomightez]

@arosko

and I don’t think about updating them continuously unless there’s an important new feature or some major performance upgrade. The dependencies of Jupyter itself are something I trust Jupyter’s own install and/or update mechanism to keep in sync,

pip doesn’t have the ability to keep that stuff ‘in sync’ as you found.
Ideally, you would be updating things that go together and using them as you go along. For example, numpy and Pandas keep pretty well on each complementing each other. This is why installing the current versions of associated packages at the same time makes things go easier. This is what using environments allows to be more convenient. Plus, you’d have those past-established environments available in the future so what you had working can continue to work if you activate the correct environment.

What about Python? Are you updating that regularly to a well-vetted version? Do you use a version that is close to the version These are also things using Anaconda/conda would also make easier.

Then you have working environments that don’t get messed up when you try to update or add in one new package.

pip alone as a package manager is nice for starting from square one; however, you seem to be doing some work in the Python ecosystem that you want to go forward with for several years. So you want new stuff to work and your old stuff to work. This is what I meant by having outgrown your current method of dealing with versions.

@arosko

It seems from reading this thread that a fix was never found that worked for everyone, and even the fix that worked for the most people again involved restoring pre-v7 versions of components. Seeing as this was in 2023 when version 7 had barely been released, I’m starting to get the impression that version 7 has been buggy, error-prone software from the get-go. This is really a shame since the idea of being able to run snippets of code interactively is a great one–but if it’s subject to regular updates that break even relatively core functionality (matplotlib plotting has been ubiquitous in Python for over a decade now) then that negates its usefulness.

This again would likely be addressed by using package/environment managers that have more ability and features than pip. I mentioned several options in my previous reply. One I didn’t mention but pointed at is that you run tests using temporary remote sessions using MyBinder-served sessions to see what works together without touching your own system. Use of MyBinder-served sessions is only possible though with things that aren’t overly computationally demanding.

As for the last decade. There has been a lot of change and just because something is ubiquitous it doesn’t mean it is going to be stagnant. And there is a lot of moving parts and so things are going to break now and then.

Version 7 of the Notebook is very different than version 6. You are going to have a hard go of it if you have elements of both around. I suspect that is actually much of what you are encountering in that thread. And maybe your own experience? Having things isolated in environments makes it easier to go forward and have both old and new versions on your system, without them interfering with each other.

[arosko]

These are the versions of some of the key things (Python, matplotlib, and components I know to be associated with Jupyter

Python 3.11.1
numpy 1.26.24
matplotlib 3.10.7
matplotlib-inline 0.2.1
jupyter 1.1.1
jupyter-console 6.6.3
jupyter-contrib-core 0.4.2
jupyter-contrib-nbextensions 0.7.0
jupyter-events 0.12.0
jupyter-lsp 2.3.0
jupyter_client 7.4.9
jupyter_core 5.9.1
jupyter_nbextensions_configurator 0.6.4
jupyter_server 2.17.0
jupyter_server_terminals 0.5.3
jupyterlab 4.5.1
jupyterlab_pygments 0.3.0
jupyterlab_server 2.28.0
jupyterlab_widgets 3.0.16
nbclassic 1.3.3
nbclient 0.10.4
nbconvert 7.16.6
nbformat 5.10.4
notebook 7.5.1
notebook_shim 0.2.4

Would you recommend a separate environment for each notebook? a new environment every time I install a new package? The thing is, I never intentionally changed anything that would seem on the surface to relate to the integration between matplotlib and Notebook. I can certainly count on one hand the number of Python packages I installed since then, and most of them don’t conceivably have anything to do with the issue at hand (like pyaudio for audio output–which I might even have installed prior, just not used in a notebook until recently.

The biggest change by far was installing the Julia kernel, in that this was a whole new language support being added–possibly that messed with something?

[arosko]

After posting that last post I went ahead and updated Numpy because it seems to have been the farthest behind (and Scipy as well because they’re closely linked), in case that was going to eventually cause me problems with something. The other ones I checked seem to be close to current.

[fomightez]

@arosko

Would you recommend a separate environment for each notebook?

Separate environments for each project is the recommended way for most more-featured package managers.

They often don’t have quite the same they handle the specifics; however, they tend to be centered around that. For example, see the ’ Welcome Page’ and ’ Sessions and Projects’ sections of the JupyterLab Desktop 2023 update announcement. Or see ’ Working with Python Projects’ under the ’ Installing Python 3’ Guide in TalkPython’s ‘Training’ content. (While that last one doesn’t cover uv with Jupyter, you can see this thread or ’ One Line Command to Launch a Notebook with Pytorch’.)

However, one environment per notebook was suggested here.

@arosko

After posting that last post I went ahead and updated Numpy because it seems to have been the farthest behind

If that means you have only the latest numpy available, that will be good for current development. However, it isn’t ideal as you may then cause problems for your old projects. (Hypothetically, a possibility. Hopefully, not the actual case. If you later find it is the actual case, then it is a further example of why I said earlier you outgrew just relying on a single installation on your machine. Any why environments are popular with Python coders.)

We may never know what triggered the issue. You talked about swapping back and forth between Jupyter 6 and 7 and so likely it was in there. Maybe ipympl would have worked earlier than that with a browser page refresh; however, you didn’t know to try that at the time and now have changed things around and dug yourself a deeper hole. Probably not worth exploring that aspect too much now. You just want to get back to something you can use your notebooks and code with again.

---

---

As to your issue with Jupyter:
Are you able to try a fresh install that would be separate from your own? For example, do you have access to a similar machine as yours? Or can you make a new user account and start fresh to see if ipympl will work in Jupyter there to give you interactive plots? (Maybe use JupyterLab Desktop there as test installation?) This would help you know if it is something about your system beyond what is going on in your own user account? I suspect it is just your user account and that you have old files around because Jupyter update can have issues on computers with older installations having been previously present and not cleaned out prior. (Those won’t necessarily show up in the listing of versions.)

[arosko]

I have tried reloading the notebooks across multiple browser sessions, both before and after trying downgrading to Notebook <7 (as a direct result of seeing people who had issues with matplotlib interactivity in Jupyter reporting only being able to fix it by resetting components to <7 versions). My original install of Notebook (on this machine) was either in late 2023 or early 2024, i.e. not terribly ancient in the timeframe of Jupyter, so it’s not as if there would be really old versions of the components around.

Interestingly I looked in my site-packages folder and the folder for ipympl was created in September of 2024–so something must have installed ipympl back then, even though I hadn’t even heard of the %matplotlib ipympl magic until a week ago and none of my old notebooks used it. This is in contrast to all the other Jupyter-related package folders which have creation dates within the last week.

Would installing something like uv and creating a new environment definitely solve the problem of old files hanging around? If that’s really the way to do it, and it in turn then requires reinstalling all of Numpy/Scipy, matplotlib, biopython, etc. then I guess that’s just a bit of work I will need to invest in order to have a working system for the future. Though I guess that means I have to reinstall them all again each time I want to start a new project?

[fomightez]

Since your original installation wasn’t that long ago, maybe it isn’t/wasn’t that. However, I thought you mentioned trying 6 & downgrading. So maybe while older components being around wasn’t a problem at first, I was concerned it became an issue.
ipympl impacts a lot of things and so it is a trickier installation than, say Pandas. And so it is easy to get messed up or think it isn’t working because of browser caching. Troubleshooting can often then make things worse. Yet, this is pure speculation and you need to take it as such. Like I said, you may never know the cause at this point. Just getting things to a manageable state at this point should be the focus.

@arosko

Would installing something like uv and creating a new environment definitely solve the problem of old files hanging around?

Since we don’t really know the underlying issue, I can not say definitely.
It would though be a way try things that is less likely to mess with your current installation. And it would make it less likely your current installation would mess with the new attempts.

---

Plus there is an open issue that produces blank figures that was bing discussed as recent as last month, see here? Is that what you see?

[arosko]

Thanks for the suggestion about using uv, it worked. For others who might come across this issue, the most important part was to reinstall Notebook inside the uv venv. Even just installing a new ipykernel in the venv, and new copies of matplotlib and ipympl (and other packages needed for the notebook) and then using “uv run jupyter notebook” didn’t change anything.

To answer the question from directly above, with my own notebooks, the plots weren’t blank, just non-interactive. But the test example with the slider did in fact produce a completely blank figure, with just the “Figure 1” title and white space below, with the slider being the only other graphical element present.

[fomightez]

@arosko

Thanks for the suggestion about using uv, it worked. For others who might come across this issue, the most important part was to reinstall Notebook inside the uv venv.

Thanks for updating this with specifics of what helped you get it working.
I suspect it will indeed help others with the same issue sort it much faster.