Cripplingly slow UI: am I the only one?

JupyterLab
1

I’ve used JupyterLab off and on for the past couple years and heavily in the last couple weeks. I’ve noticed performance issues during this time which have brought me to a crossroads, where I am trying to decide whether to continue investing in this platform or whether I should look into other options.

First, the good: I really like the feature set that JupyterLab is offering-- flexible arrangement of notebooks, management of multiple kernels, embedded consoles/editors, etc etc. There is enormous potential in bringing these things all together in a unified and extensible graphical environment usable across languages. This is what attracted me to JupyterLab in the first place, and I still believe in the vision. The JupyterLab of my dreams is like (Neo)Vim but built with data visualization in mind: extremely efficient for manipulating text, capable of managing 10s to hundreds of open files/notebooks/terminals at once, extensible for developers, customizable by users, usable across many languages, large user community… Free is good but I would gladly pay a sizable sum for such an app.

However, the vast majority of the time I end up using JupyterLab to, at most, edit two notebooks side by side. This is a far cry from the way I would like to be using it. The reason for this discrepancy is JupyterLab’s UI performance. We’re talking:

  • very laggy scrolling: several seconds to quick scroll from the bottom to the top of a 50-cell notebook
  • 5 second lag to expand the sidebar (command palette area)
  • 10 second lag to move a tab
  • 1-second lag for keyboard input
  • etc

The lag is not very bad if I open a fresh instance with no open notebooks (although even then, the UI seems much more sluggish than it should be). But as I begin to work, the lag gets worse and worse. It is at least roughly a function of how many notebooks I have open and how many visualizations they contain (I use mostly Bokeh/Matplotlib for visualizations via Holoviews).

I’m not sure why having multiple idle notebooks open or kernels running should kneecap performance as badly as it does. It seems like any change in a visible element (typing in a notebook cell or scrolling a notebook) somehow has to cascade through JupyterLab’s entire saved state, including off-screen elements in totally separate documents. I have not looked at JupyterLab’s architecture, but naively it seems as if there is some fundamental foundational flaw.

I didn’t post this to complain, but rather to describe what I’m experiencing and to find out:

  • Are other people experiencing the same issues? Is there any kind of community consensus that this is a problem?
  • Does the JupyterLab team have plans to focus on performance in the near future? Is anyone paid to work on this project full-time (seems like someone should be! Tons of people are using Jupyter notebooks now)?

Like I said, I believe in the JupyterLab vision, and I really appreciate the hard work that’s already been put into this project. But the current performance is a dealbreaker. Searching this forum I could not find any existing thread focused on these issues, so I am opening this one.

5 Likes
2

I am using JupyterLab daily and not seeing the issues you are mentioning, but there may be some OS/browser/extension/etc combinations that are leading to degraded performance. Saul S. has done some initial work in benchmarking JupyterLab - we will try to dive into this and may need some additional information as we do.

1 Like
3

fwiw, we have had some students also non-deterministically report similar issues. Due to the pandemic, we haven’t really been able to isolate anything, so we just bounce them over to classic notebook…

4

Happy to provide any information I can to assist. Not sure how much of this is relevant, but here is my OS/browser/extension/other info:

  • Chrome Canary
  • MacOS 10.15.6, late 2016 Macbook Pro
  • browser extensions: Vimium, Apollo DevTools, MiniY, Vue DevTools, 1Password, AdBlock, Z-Context, Firenvim
  • JupyterLab extensions: @pyviz/jupyterlab_pyviz
  • display is 34" LG 5K2K display connected to my laptop by a Radeon 590 eGPU

The only browser extension I actually use within JupyterLab is Firenvim (which I know is not causing the problem, I added it recently), but the others are still “active”, as in I haven’t configured them to be explicitly disabled in JupyterLab.

5

I have also had similar issues, particularly with notebooks that are mostly made up of code cells. I haven’t done a ton of thorough testing of the issue (since it does not seem to appear consistently) but some of the things I found trying to debug this on my machine from time to time:

  • It can be impacted by the amount of displayed media in the notebook (e.g., high res matplotlib images) but is not limited to them. I seem to run into this problem primarily with notebooks made up of a large number of code cells
  • Switching from Chrome to Safari made a huge difference. Firefox had the same performance issues as Chrome. The same notebook that would freeze up and lag would open and scroll fine in Safari. Though Safari is extremely laggy when using the CSV viewer on any size file.
  • Restarting Jupyterlab from scratch helps reduce the issue. There seems to be some relationship to time as the longer I work the more likely the big notebooks will experience a big and lasting spike in lag
  • The issue seems to be limited to bigger notebooks (roughly 50-100 cells)
  • The slow responsiveness is not just limited to the large notebook itself but impacts everything in the UI (e.g., moving tiles around, resizing things, etc)

My Info:
Machine: MacBook Pro (2017)
Processor: 2.9 GHz Quad-Core Intel Core i7
RAM: 16 GB 2133 MHz LPDDR3
Graphics: Radeon Pro 560 4 GB, Intel HD Graphics 630 1536 MB

Jupyerlab: v2.2.0
Extensions:
bokeh/jupyter_bokeh v2.0.3
jupyter-voila/jupyterlab-preview v1.1.0
jupyter-widgets/jupyterlab-manager v2.0.0
jupyter-widgets/jupyterlab-sidecar v0.5.0
jupyterlab/git v0.20.0
jupyterlab/toc v4.0.0
karosc/jupyterlab_dracula v2.0.3
krassowski/jupyterlab_go_to_definition v1.0.0
lckr/jupyterlab_variableinspector v0.5.1
mohirio/jupyterlab-horizon-theme v2.0.0
oriolmirosa/jupyterlab_materialdarker v0.5.0
ryantam626/jupyterlab_code_formatter v1.3.5
telamonian/theme-darcula v2.0.0
yeebc/jupyterlab_neon_theme v1.0.6
bqplot v0.5.14
jupyter-leaflet v0.13.1
jupyter-matplotlib v0.7.3
nbdime-jupyterlab v2.0.0

2 Likes
6

I have started working on some benchmarks for loading and switching between larger notebooks: Benchmark Notebook Loads by saulshanabrook · Pull Request #8020 · jupyterlab/jupyterlab · GitHub

The next step after those benchmarks is for us to address the problem, either by virtualization (infinite scroll) or some other method.

However, I haven’t seen a few of the times you are talking about, like “5 second lag to expand the sidebar (command palette area)”

Would you be able to upload help us reproduce this kind of lag? Like by uploading the notebooks you have running to get that? Or, does that only come up after a long session?

Overall, I think addressing the issues you bring up will require adding some more rigorous benchmarking tools to JupyterLab and then having them run regularly as part of the contribution process to keep timings under control. It might also help to try to some automated tests to detect memory leaks.

There are plenty of people paid to work on this full time. However, we all come from different organizations with different priorities and we have limited bandwidth to devote to things that we know should be done but that aren’t essential to the folks paying us.

I can speak for myself, that at Quansight we have a client who cares a lot about the performance issues with larger notebooks so thats why I, and others at Quansight, are working towards addressing that at least.

If you have either funding or time and would like to help contribute to the effort more broadly, I am happy to help as I can in getting you up to speed.

1 Like
7

A meta issue here is that we currently don’t have a great/rigorous process for determining what we should prioritize in terms of user needs as a product.

You might be interested in contributing to the discussion recently opened on this topic in the team compass repo: https://github.com/jupyterlab/team-compass/issues/80

1 Like
8

@saulshanabrook Thanks for explaining a bit about the current issues with project governance. I don’t have funding but I can contribute some time to help analyze the issues and possibly some code.

Giving you the notebooks that have been causing the issue would be difficult at the moment, as they are coupled to some large datasets and unpublished libraries. However, I have some suspicions about the worst culprits in terms of causing lag and may be able to generate a contrived example that reproduces the problem soon.

The “five second lag to expand the sidebar” I referred to is an instance of the broader issue corroborated by @cezarymierzejek:

The slow responsiveness is not just limited to the large notebook itself but impacts everything in the UI (e.g., moving tiles around, resizing things, etc)

What are the mechanisms by which a large and loaded but idle notebook might cause lag in purportedly independent parts of the UI? Here’s my guess as to what’s happening: certain data visualizations are expensive to render. When you expand the sidebar or move a pane, it can alter the shape of an open notebook and trigger some kind of preemptive re-render, even in cells that aren’t visible. If this is indeed the case, a solution would be to perform this work asynchronously or delay the re-render until the visualization actually comes on screen again. Perhaps this is similar to what you mean by “virtualization (infinite scroll)”?

1 Like
9

In the absence of notebooks that reproduce the problem, a Chrome profile of the issue occurring may be helpful as well. Instructions for capturing a profile can be found at Chrome-Entwicklertools  |  DevTools  |  Chrome for Developers. Try to keep the profile fairly short (I normally try to set everything up then only capture 10s or so). Once the profile is completed then you can download it with the ‘save profile’ button.

Performance can be impacted by browser extensions as well- it’s always worth trying incognito mode which will normally run without extensions.

There can be sooo many sources of performance issues.

You mention 1s keypress times- for profiling these I typically start the profiler, quickly type a few words, then stop the profiler.

certain data visualizations are expensive to render.

Absolutely- an example that I’ve dealt with recently is a complex visualization creating a large number of render layers in the browser which lead to composition times of >300ms per frame, every frame. But there can also be unnecessary layout invalidation, code executing unnecessarily, etc.

1 Like
10

Yeah the tricky part here is it might be hard to separate JupyterLab performance from any extensions you have. As you say, if you have an extension that is doing some intense rendering and is maybe sub-optimal in some way for it’s layout, then as it changes everything slows down.

Virtualization could help here. The idea there is to basically not render outputs that aren’t in view.

11

Virtualization of outputs can be tricky to rely on for performance like this- removing outputs and re-rendering them when they are back in view can cause significant scrolling performance hiccups. Also many visualizations will have state which may be difficult to restore- such as a Vega plot zoomed into a position- it may reset the position when scrolled off the page then back into view.

1 Like
12

That’s true. It’s also complicated from the UI perspective (browser based find won’t work, and scrolling might jump if we don’t know the size of everything AOT).

It should be a last resort, but might be necessary if we are rendering output we don’t control that is slow.

13

Is there something about JupyterLab’s architecture or limitations of running in the browser that prevents the work from being done asychronously? The rerender problem is very similar to the problem of linting large codebases. It is computationally intensive work that is triggered by simple user interactions. Good editors solve this problem by performing the work out-of-process so that it doesn’t block the UI.

14

Just a note here. I’ve found that jupyter performance is very highly dependent on

  1. the network for remote notebooks
  2. memory for local notebooks

Because jupyter is passing a lot of data back for forth to the server, if there is anything that causes the server to drop packets or slow the connection, then jupyter will start to lag.

The other thing that causes lag is the browser. In particular, the browser will start to cache things in memory and if the machine runs out of memory things will get quite slow.

It turns out that performance issues are very hard to track because a lot depends on the local set up, but narrowing things down to memory or network has let me track down quite a few things.

There’s one particular bottleneck that I’m aware of. Jupyter will compile all of the extensions into one monster javascript file that gets loaded on startup and if that is not cached, then jupyter take a long time to startup.

15

Also one thing that will help a lot in debugger performance issues is browser “developer tools”

It dumps a huge about of information about what the browser is doing. In particular you get the see the traffic back and forth.

One suggestion is that you might try to run jupyter on a browser without adblock. Jupyter is sending across a lot of javascript which the adblock might be trying to process.

1 Like
16

My experience has been that it turns out that rendering isn’t usually the bottleneck. If you have a complicated 3-d volume the data for that gets send directly to the browser via webgl and that goes directly to hardware.

The big bottlenecks that I’ve seen are due to network. Redraws are fast, but I have a suspicion that what might be happening is that every time you scroll, something is being sent over the network and the client is waiting for something to come back.

17

Anecdotally, switching off Adblock helped a lot in my case. :slightly_smiling_face:

2 Likes
18

We were running into this issue after recent changes to our CSS used by _repr_html_ to display our objects. The problem got worse with the number of cells in a notebook, even if those cells were empty.

It turned out that putting CSS custom properties into :root was the cause (see Do not put things into :root to avoid bad performance of _repr_html_ by SimonHeybrock · Pull Request #1847 · scipp/scipp · GitHub). Placing the properties in a specific class brings back the original performance.

Note that this does not solve all problems: I still experience slow UI once about 100 or more cells are in a notebook, but it is not as bad as when CSS :root was modified.

19

I have exactly the same issue. I tried to switched off my AdBlocker but that didn’t really help. If anyone knows any other solutions, please let me know. Thanks!

20

I have the same experience. It is more severe with Chrome than with Firefox. Started this week. I tried rebooting server and client. No internet speed issues. No out-of-memory issue either. I updated the jupyer-lab which also did not help. I do not use custom CSS.

The devtools show lots of entries like:

Added non-passive event listener to a scroll-blocking ‘touchstart’ event. Consider marking event handler as ‘passive’ to make the page more responsive