At the JupyterCon coding sprint today, @kolibril13 initiated today a brainstorm with
(? sorry, I am bad with names and should have written it down) and myself about
screencasting jupyterlab notebook sessions in a better way than with a plain video,
so that:
- the screen cast file would be much more lightweight
- the user could interrupt at any time the screen cast and start live coding from there; and get back to the screen cast.
- the screen cast would (ideally) self adapt to the user’s device and configuration
- accessibility! the user is facing a real running application, and can use screen readers & such.
@kolibril13 had gotten the inspiration from application XXX (@kolibril13, can you remind me?)
Ideas that were floated around:
- Use RRWeb ; rrweb is about pixel-perfect replay, not self-adapting replay
- Use Jupyter-graffiti: jupyter 6 only at this stage
- Use Jupyter telemetry to record the user actions, and replay them
At the end, we realized that a very natural candidate would be to use JupyterLab’s Real Time Collaboration: after all, RTC is all about replaying on another device the user’s actions on a Jupyter notebook. Here the replay is just delayed in time.
Here is some feedback from @SylvainCorlay :
Indeed we already considered using RTC to replay a session in a Jupyter notebook, though we only had in mind the use case of debugging RTC. This should be feasible since there is a full history of the ystore in the backend. All that is missing is a user interface. @kolibril13: if you’d be interested in exploring this, please get in touch!
@SylvainCorlay : are there timestamps in the history of the ystore so that one would know at which rate to replay the events?
Caveats:
- RTC is not tracking the user’s mouse; without additional work this form of screencasting would not be adequate for showing user interactions involving the mouse; and interactions outside of the notebook for that matter.
- I am not sure how RTC handles interactions with the kernel: here we really need to replay these interactions to have a kernel in the proper state.
Cheers,
Nicolas
Thoughts anyone?