A similar problem here, and I noticed this issue when using %matplotlib notebook.
Specifically, when I tried to adjust the plot using the pan/zoom tools in the matplotlib figure, this “Assertion Error” popped up in the terminal and the kernel seems to disconnect for several seconds, and the figure widget becomes broken at that point.
Edit)
This problem happens with jupyter 8.0.2 and tornado 6.2. If I roll back the tornado to 6.1, and consequently the jupyter 7.3.2, this problem does not occur.