Output stuck/buffered for 60 seconds / Uncaught exception in ZMQStream callback

General
1

We have recently upgraded from z2jh pre-1.0 to the latest version (jupyterlab 3.6.3). We use a mix of ipywidgets and plotly to visualise data for our clients, and we’re experiencing issues - mainly that the widgets don’t load properly, or our notebooks get ‘stuck’ during execution. I’m struggling to pin down the issue, but have managed to create a simple notebook that reproduces the problem after running it 5+ times:

import datetime

import ipywidgets as widgets # ipywidgets 8.0.6 but all 8.0.x versions exhibit the issue
import plotly.graph_objects as go # plotly 5.14.1


for i in range(20):
    f = go.FigureWidget()
    f.add_trace(go.Scatter(x=list(range(10_000)), y=list(range(10_000))))
       
    w = widgets.VBox([
        widgets.HTML(f"<b>Test {i} - {datetime.datetime.utcnow()}</b>"),
        widgets.HBox([
            widgets.Button(description=f"Button {i} a"),
            widgets.Button(description=f"Button {i} b"),
            widgets.Button(description=f"Button {i} c"),
            widgets.Button(description=f"Button {i} d"),
            widgets.Button(description=f"Button {i} e"),
            widgets.Button(description=f"Button {i} f"),            
        ]),
        f,
    ])
    t = widgets.Tab(titles=[f"{i}"], children=[w])
    display(t)

…by the 5th or 6th run (restart kernel and run all cells) only a handful of the widgets are output, and then ~a minute or so later, the remainder are rendered.

In the logs for the singleuser notebook I can see errors, but I dont know if this is a smoking gun or a symptom of an underlying issue:

[I 2023-05-26 08:16:21.173 ServerApp] 200 GET /user/mksttrcmfe53e59e/api/sessions?1685088980909 (mksttrcmfe53e59e@10.1.122.143) 1.14ms
[I 2023-05-26 08:16:21.183 ServerApp] 200 GET /user/mksttrcmfe53e59e/api/kernels?1685088980910 (mksttrcmfe53e59e@10.1.122.143) 2.07ms
[E 2023-05-26 08:16:21.277 ServerApp] Uncaught exception in ZMQStream callback
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 584, in _run_callback
        f = callback(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 308, in stream_callback
        return callback(self, msg)
               ^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 518, in handle_outgoing_message
        self._on_zmq_reply(stream, msg)
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 575, in _on_zmq_reply
        self.write_message(msg, binary=isinstance(msg, bytes))
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 334, in write_message
        return self.ws_connection.write_message(message, binary=binary)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1081, in write_message
        fut = self._write_frame(True, opcode, message, flags=flags)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1056, in _write_frame
        return self.stream.write(frame)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 539, in write
        self._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 1462, in _handle_write
        super()._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 965, in _handle_write
        self._write_buffer.advance(num_bytes)
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
               ^^^^^^^^^^^^^^^^^^^^^^
    AssertionError
[E 2023-05-26 08:16:21.278 ServerApp] Uncaught exception in zmqstream callback
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 634, in _handle_events
        self._handle_recv()
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 663, in _handle_recv
        self._run_callback(callback, msg)
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 584, in _run_callback
        f = callback(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 308, in stream_callback
        return callback(self, msg)
               ^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 518, in handle_outgoing_message
        self._on_zmq_reply(stream, msg)
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 575, in _on_zmq_reply
        self.write_message(msg, binary=isinstance(msg, bytes))
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 334, in write_message
        return self.ws_connection.write_message(message, binary=binary)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1081, in write_message
        fut = self._write_frame(True, opcode, message, flags=flags)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1056, in _write_frame
        return self.stream.write(frame)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 539, in write
        self._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 1462, in _handle_write
        super()._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 965, in _handle_write
        self._write_buffer.advance(num_bytes)
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
               ^^^^^^^^^^^^^^^^^^^^^^
    AssertionError
[E 2023-05-26 08:16:21.278 ServerApp] Exception in callback functools.partial(<function ZMQStream._update_handler.<locals>.<lambda> at 0x7f1794433880>)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/tornado/ioloop.py", line 738, in _run_callback
        ret = callback()
              ^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 718, in <lambda>
        self.io_loop.add_callback(lambda: self._handle_events(self.socket, 0))
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 634, in _handle_events
        self._handle_recv()
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 663, in _handle_recv
        self._run_callback(callback, msg)
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 584, in _run_callback
        f = callback(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 308, in stream_callback
        return callback(self, msg)
               ^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 518, in handle_outgoing_message
        self._on_zmq_reply(stream, msg)
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 575, in _on_zmq_reply
        self.write_message(msg, binary=isinstance(msg, bytes))
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 334, in write_message
        return self.ws_connection.write_message(message, binary=binary)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1081, in write_message
        fut = self._write_frame(True, opcode, message, flags=flags)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1056, in _write_frame
        return self.stream.write(frame)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 539, in write
        self._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 1462, in _handle_write
        super()._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 965, in _handle_write
        self._write_buffer.advance(num_bytes)
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
               ^^^^^^^^^^^^^^^^^^^^^^
    AssertionError
[I 2023-05-26 08:17:21.214 ServerApp] Starting buffering for 5cbd7296-7b4d-47e4-b293-236ee1fdf426:12f99ae3-390c-41ac-9d82-105bf7540efc
[I 2023-05-26 08:17:21.427 ServerApp] 101 GET /user/mksttrcmfe53e59e/api/kernels/5cbd7296-7b4d-47e4-b293-236ee1fdf426/channels?session_id=12f99ae3-390c-41ac-9d82-105bf7540efc (mksttrcmfe53e59e@10.1.122.143) 1.47ms
[I 2023-05-26 08:17:21.427 ServerApp] Connecting to kernel 5cbd7296-7b4d-47e4-b293-236ee1fdf426.
[I 2023-05-26 08:17:21.427 ServerApp] Restoring connection for 5cbd7296-7b4d-47e4-b293-236ee1fdf426:12f99ae3-390c-41ac-9d82-105bf7540efc
[I 2023-05-26 08:17:21.462 ServerApp] Replaying 48 buffered messages
[I 2023-05-26 08:17:27.152 ServerApp] 101 GET /user/mksttrcmfe53e59e/api/events/subscribe?token=[secret] (mksttrcmfe53e59e@10.1.122.143) 0.74ms
[I 2023-05-26 08:17:35.935 ServerApp] 200 GET /user/mksttrcmfe53e59e/api/kernels?1685089055884 (mksttrcmfe53e59e@10.1.122.143) 1.12ms

It’s almost as if the code gets executed before the user has connected to the restarted kernel, or are some messages sent on the ‘previous’ kernel connection before it’s restarted, and it takes 60 seconds for something to reconnect?

If anyone can shed some light on either the issue, or how to troubleshoot the issue, that would be great. We don’t see the issue in production where we are running old versions (like 2+ years old) of everything.

Edit: Looking at the logs a bit more, it seems that if left idle, the kernel will continuously disconnect/reconnect every 60 (63?) seconds. I don’t know why this is happening, but am wondering if this is the source of the problem:

[I 2023-05-26 10:13:49.460 ServerApp] Connecting to kernel 85e599ad-4a18-405f-88cc-636913470181.
[I 2023-05-26 10:13:49.461 ServerApp] Restoring connection for 85e599ad-4a18-405f-88cc-636913470181:4221ff9f-edc2-40de-ac1b-797f697a02e1
[I 2023-05-26 10:14:52.532 ServerApp] Starting buffering for 85e599ad-4a18-405f-88cc-636913470181:4221ff9f-edc2-40de-ac1b-797f697a02e1
[I 2023-05-26 10:14:52.744 ServerApp] Connecting to kernel 85e599ad-4a18-405f-88cc-636913470181.
[I 2023-05-26 10:14:52.744 ServerApp] Restoring connection for 85e599ad-4a18-405f-88cc-636913470181:4221ff9f-edc2-40de-ac1b-797f697a02e1
[I 2023-05-26 10:15:55.214 ServerApp] Connecting to kernel 85e599ad-4a18-405f-88cc-636913470181.
[I 2023-05-26 10:15:55.214 ServerApp] Restoring connection for 85e599ad-4a18-405f-88cc-636913470181:4221ff9f-edc2-40de-ac1b-797f697a02e1
[I 2023-05-26 10:16:57.416 ServerApp] Connecting to kernel 85e599ad-4a18-405f-88cc-636913470181.
[I 2023-05-26 10:16:57.417 ServerApp] Restoring connection for 85e599ad-4a18-405f-88cc-636913470181:4221ff9f-edc2-40de-ac1b-797f697a02e1
2

Can you reproduce your error without JupyterHub, e.g. by running the container image locally?

3

I do not see the kernel disconnecting/reconnecting when running the singleuser image locally, no.

I think that the example notebook I gave has little to do with the issue, and its more that (part of) the execution happens during the kernel being disconnected, hence why the output gets stuck until the kernel reconnects.

It would be great to understand:

  1. why it’s disconnecting in the first place
  2. why it takes ~63 seconds to reconnect (is there a timer, can this be set to 1 second, perhaps that would mitigate the issue in terms of user experience, even if the underlying problem still exists)
4

Can’t figure out how to edit posts (I edited my original, but maybe there is a time limit?). In any case, I’ve tried upgrading to jupyter_server==2.6.0 and that seems to stop the disconnect/reconnect every 60 seconds… but doesn’t fix the issue itself. I’m still getting the Uncaught exception error, and my notebooks are miserable to use.

5

Do you only see the error when using ipywidgets, or plotly, or is it a more general issue?

6

Edit:
I have downgraded to jupyter_client==7.4.9 and looks like that has resolved the issue. I’ll create an Issue in the jupyter-client repo, although there seem to be similar-ish issues around websockets and 8.x…

Old content:
I’ve so far only been able to reproduce the issue with the combination of z2jh + ipywidgets + plotly.

Running the same notebooks in a local jupyter single-user instance work flawlessly. I don’t know if there is some race condition that isn’t occurring locally because the latency is effectively zero as its running localhost.

Logs when restarting kernel in localhost instance:

[I 2023-05-31 08:46:40.741 ServerApp] Kernel restarted: 33b887a3-39c1-4ec4-85a9-da455e3b3d5c
[I 2023-05-31 08:46:40.748 ServerApp] Starting buffering for 33b887a3-39c1-4ec4-85a9-da455e3b3d5c:d162a583-87c3-47e2-9d74-815725538161
[I 2023-05-31 08:46:40.754 ServerApp] Connecting to kernel 33b887a3-39c1-4ec4-85a9-da455e3b3d5c.
[I 2023-05-31 08:46:40.754 ServerApp] Restoring connection for 33b887a3-39c1-4ec4-85a9-da455e3b3d5c:d162a583-87c3-47e2-9d74-815725538161

Logs when restarting kernel in deployed instance:

[I 2023-05-31 08:48:08.069 ServerApp] Kernel restarted: 3c1db283-a6cb-4ad2-8ee6-23ffc9fbdf3b
[I 2023-05-31 08:48:08.071 ServerApp] 200 POST /user/mksttrcmfe53e59e/api/kernels/3c1db283-a6cb-4ad2-8ee6-23ffc9fbdf3b/restart?1685522887656 (mksttrcmfe53e59e@10.1.122.149) 226.20ms
[I 2023-05-31 08:48:08.275 ServerApp] 101 GET /user/mksttrcmfe53e59e/api/kernels/3c1db283-a6cb-4ad2-8ee6-23ffc9fbdf3b/channels?session_id=ddf5b861-55ed-4d88-b6e8-ca48ba5d0622 (mksttrcmfe53e59e@10.1.122.149) 2.05ms
[I 2023-05-31 08:48:08.275 ServerApp] Connecting to kernel 3c1db283-a6cb-4ad2-8ee6-23ffc9fbdf3b.
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-05-31 08:48:10.376 ServerApp] Loading directory [ /community/tradefeedr-resources/<my-email-address>/shared/prod/algo_widget_docs ]
[I 2023-05-31 08:48:10.589 ServerApp] 200 GET /user/mksttrcmfe53e59e/api/contents/community/tradefeedr-resources/<my-email-address>/shared/prod/algo_widget_docs?content=1&1685522890266 (mksttrcmfe53e59e@10.1.122.149) 271.88ms
[I 2023-05-31 08:48:11.135 ServerApp] 200 GET /user/mksttrcmfe53e59e/api/sessions?1685522891072 (mksttrcmfe53e59e@10.1.122.149) 3.66ms
[I 2023-05-31 08:48:11.150 ServerApp] 200 GET /user/mksttrcmfe53e59e/api/kernels?1685522891085 (mksttrcmfe53e59e@10.1.122.149) 1.30ms
[E 2023-05-31 08:48:11.878 ServerApp] Uncaught exception in ZMQStream callback
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 584, in _run_callback
        f = callback(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 308, in stream_callback
        return callback(self, msg)
               ^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 505, in handle_outgoing_message
        self._on_zmq_reply(stream, msg)
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 562, in _on_zmq_reply
        self.write_message(msg, binary=isinstance(msg, bytes))
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 334, in write_message
        return self.ws_connection.write_message(message, binary=binary)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1081, in write_message
        fut = self._write_frame(True, opcode, message, flags=flags)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1056, in _write_frame
        return self.stream.write(frame)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 539, in write
        self._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 1462, in _handle_write
        super()._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 965, in _handle_write
        self._write_buffer.advance(num_bytes)
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
               ^^^^^^^^^^^^^^^^^^^^^^
    AssertionError
[E 2023-05-31 08:48:11.882 ServerApp] Uncaught exception in zmqstream callback
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 634, in _handle_events
        self._handle_recv()
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 663, in _handle_recv
        self._run_callback(callback, msg)
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 584, in _run_callback
        f = callback(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 308, in stream_callback
        return callback(self, msg)
               ^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 505, in handle_outgoing_message
        self._on_zmq_reply(stream, msg)
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 562, in _on_zmq_reply
        self.write_message(msg, binary=isinstance(msg, bytes))
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 334, in write_message
        return self.ws_connection.write_message(message, binary=binary)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1081, in write_message
        fut = self._write_frame(True, opcode, message, flags=flags)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1056, in _write_frame
        return self.stream.write(frame)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 539, in write
        self._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 1462, in _handle_write
        super()._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 965, in _handle_write
        self._write_buffer.advance(num_bytes)
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
               ^^^^^^^^^^^^^^^^^^^^^^
    AssertionError
[E 2023-05-31 08:48:11.883 ServerApp] Exception in callback functools.partial(<function ZMQStream._update_handler.<locals>.<lambda> at 0x7fb9343ad440>)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/tornado/ioloop.py", line 738, in _run_callback
        ret = callback()
              ^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 718, in <lambda>
        self.io_loop.add_callback(lambda: self._handle_events(self.socket, 0))
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 634, in _handle_events
        self._handle_recv()
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 663, in _handle_recv
        self._run_callback(callback, msg)
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 584, in _run_callback
        f = callback(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/zmq/eventloop/zmqstream.py", line 308, in stream_callback
        return callback(self, msg)
               ^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 505, in handle_outgoing_message
        self._on_zmq_reply(stream, msg)
      File "/usr/local/lib/python3.11/site-packages/jupyter_server/services/kernels/connection/channels.py", line 562, in _on_zmq_reply
        self.write_message(msg, binary=isinstance(msg, bytes))
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 334, in write_message
        return self.ws_connection.write_message(message, binary=binary)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1081, in write_message
        fut = self._write_frame(True, opcode, message, flags=flags)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/websocket.py", line 1056, in _write_frame
        return self.stream.write(frame)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 539, in write
        self._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 1462, in _handle_write
        super()._handle_write()
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 965, in _handle_write
        self._write_buffer.advance(num_bytes)
      File "/usr/local/lib/python3.11/site-packages/tornado/iostream.py", line 182, in advance
        assert 0 < size <= self._size
               ^^^^^^^^^^^^^^^^^^^^^^
    AssertionError
1 Like
7

Great! Could you link the GitHub issue here, so that if anyone finds this thread in future they can follow it up?

8

Created issue 8.x - Uncaught exception in ZMQStream callback · Issue #950 · jupyter/jupyter_client · GitHub

1 Like
9

Hello,
I received the same “Uncaught exception in ZMQStream callback”.
The exception is no longer encountered if I use: “%matplotlib inline”
instead of: “%matplotlib notebook; from mpl_toolkits.mplot3d import Axes3D”
The issue is not fixed but the notebook does not stuck anymore.

Cheers