Hello there,
I want to use SageMath within my already set-up Jupyterlab. I do not know exactly where the problem is, a solution could involve any or all of:
- bash
- cygwin
- kernelspec
- jupyter kernel protocol
- jupyter advanced settings
SageMath out of the box
SageMath is a superlanguage of Python geared towards mathematical computations. It is available on Windows systems yet it is developped for Unix-like systems. This is why it is invoked in mintty, a cygwin terminal. A shell, a console and a jupyter notebook server are provided as shortcuts that resolve to:
"C:\Program Files\SageMath 9.2\runtime\bin\mintty.exe" -t 'SageMath 9.2 Shell' -i sagemath.ico /bin/bash --login -c '/opt/sagemath-9.2/sage -sh'
"C:\Program Files\SageMath 9.2\runtime\bin\mintty.exe" -t 'SageMath 9.2 Console' -i sagemath.ico /bin/bash --login -c '/opt/sagemath-9.2/sage'
"C:\Program Files\SageMath 9.2\runtime\bin\mintty.exe" -t 'SageMath 9.2 Notebook Server' -i sagemath.ico /bin/bash --login -c '/opt/sagemath-9.2/sage --notebook jupyter'
that are set to execute at C:\Program Files\SageMath 9.2.
While I can code in such a way, I would find it way more comfortable to centralise my coding projects in a single Windows installation of Jupyter.
Kernelspec
Skimming through SageMath’s files one finds kernel specifications for Python and SageMath.
{"argv": ["/opt/sagemath-9.2/sage", "--python", "-m", "sage.repl.ipython_kernel", "-f", "{connection_file}"], "display_name": "SageMath 9.2", "language": "sage","env":{"SAGE_ROOT":"C:/Program Files/SageMath 9.2/runtime/opt/sagemath-9.2"}}
The env field is something I needed to set after installation to get it working.
My hope was to adapt this to another kernelspec in my Windows Jupyter. This is what I’m currently working with.
{"argv": ["C:\\Program Files\\SageMath 9.2\\runtime\\bin\\mintty.exe","--hold","always","--dir","C:\\Program Files\\SageMath 9.2","/bin/bash","--login","/opt/sagemath-9.2/external_kernel.txt","{connection_file}","--python","-m","sage.repl.ipython_kernel","-f"], "display_name": "SageMath 9.2", "language": "sage","env":{"SAGE_ROOT":"C:/Program Files/SageMath 9.2/runtime/opt/sagemath-9.2"}}
The hold flag is used to keep the terminal open to see error messages. Note that I do not directly call the /opt/sagemath-9.2/sage script but a script I use to format the connection file path. This script is:
#!/bin/bash
path="$1"
path=${path/"C:\\Users\\REDACTED\\AppData\\Roaming\\jupyter\runtime\\"/"/jump/"}
echo "$path"
shift
echo ${PWD}
exec -l "${PWD}/runtime/opt/sagemath-9.2/sage" $@ "$path"
The echos are there to make sure this step is reached. jump is a “joint” symbolic link to where my Windows Jupyter generates connection files.
Errors and stuff
Now what happens when I try to use this kernel in Jupyterlab is this. On the server’s side I get:
[I 2022-06-14 19:32:27.144 ServerApp] Kernel started: 0599e48b-7207-4cfd-9a5d-e9e83e2ba1d7
[I 2022-06-14 19:32:30.126 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (1/5), new random ports
[I 2022-06-14 19:32:33.257 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (2/5), new random ports
[I 2022-06-14 19:32:36.347 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (3/5), new random ports
[I 2022-06-14 19:32:39.643 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (4/5), new random ports
[I 2022-06-14 19:32:42.767 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (5/5), new random ports
[W 2022-06-14 19:32:45.860 ServerApp] AsyncIOLoopKernelRestarter: restart failed
[W 2022-06-14 19:32:45.861 ServerApp] Kernel 0599e48b-7207-4cfd-9a5d-e9e83e2ba1d7 died, removing from map.
[E 2022-06-14 19:33:24.078 ServerApp] Error opening stream: HTTP 404: Not Found (Kernel does not exist: 0599e48b-7207-4cfd-9a5d-e9e83e2ba1d7)
While this happens six terminals open, each bearing the results of my echos. After a while most of them (randomly with regards to order of opening as far as I can tell) will spout error tracebacks:
"/jump/kernel-0599e48b-7207-4cfd-9a5d-e9e83e2ba1d7.json"
/cygdrive/c/Program Files/SageMath 9.2
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/repl/ipython_kernel/__main__.py", line 3, in <module>
IPKernelApp.launch_instance(kernel_class=SageKernel)
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/traitlets/config/application.py", line 663, in launch_instance
app.initialize(argv)
File "</opt/sagemath-9.2/local/lib/python3.7/site-packages/decorator.py:decorator-gen-124>", line 2, in initialize
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 547, in initialize
self.init_sockets()
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 266, in init_sockets
self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 213, in _bind_socket
return self._try_bind_socket(s, port)
File "/opt/sagemath-9.2/local/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 189, in _try_bind_socket
s.bind("tcp://%s:%i" % (self.ip, port))
File "zmq/backend/cython/socket.pyx", line 550, in zmq.backend.cython.socket.Socket.bind
File "zmq/backend/cython/checkrc.pxd", line 26, in zmq.backend.cython.checkrc._check_rc
raise ZMQError(errno)
zmq.error.ZMQError: Address already in use
/bin/bash: Exit 1.
The remaining two let themselves be closed without complaining. Closing a terminal early blares warnings about processes that would be terminated. This indicates that the scripts reached their end, for one meaning or another. Trying that on a regular Jupyter notebook gives the same effects, but with some more verbose errors Jupyter server-side.
[W 20:48:09.203 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20220614204647 (127.0.0.1) 51.860000ms referer=http://127.0.0.1:8890/notebooks/Documents/Carnets%20jupyter/Stage%20m1/Untitled2.ipynb
Exception in callback <TaskWakeupMethWrapper object at 0x0000020744E8AF70>(<Future finis...4026\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x0000020744E8AF70>(<Future finis...4026\r\n\r\n'>)>
Traceback (most recent call last):
File "c:\users\redacted\appdata\local\programs\python\python39\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-35' coro=<HTTP1ServerConnection._server_request_loop() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\http1connection.py:823> wait_for=<Future finished result=b'GET /api/co...54026\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\ioloop.py:688]> while another task <Task pending name='Task-2' coro=<KernelManager._async_start_kernel() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\jupyter_client\manager.py:75>> is being executed.
Exception in callback <TaskWakeupMethWrapper object at 0x0000020744E47C70>(<Future finis...4026\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x0000020744E47C70>(<Future finis...4026\r\n\r\n'>)>
Traceback (most recent call last):
File "c:\users\redacted\appdata\local\programs\python\python39\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-40' coro=<HTTP1ServerConnection._server_request_loop() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\http1connection.py:823> wait_for=<Future finished result=b'GET /static...54026\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\ioloop.py:688]> while another task <Task pending name='Task-2' coro=<KernelManager._async_start_kernel() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\jupyter_client\manager.py:75>> is being executed.
Exception in callback <TaskWakeupMethWrapper object at 0x0000020744D063D0>(<Future finis...4026\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x0000020744D063D0>(<Future finis...4026\r\n\r\n'>)>
Traceback (most recent call last):
File "c:\users\redacted\appdata\local\programs\python\python39\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-39' coro=<HTTP1ServerConnection._server_request_loop() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\http1connection.py:823> wait_for=<Future finished result=b'GET /static...54026\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\ioloop.py:688]> while another task <Task pending name='Task-1' coro=<MultiKernelManager._async_start_kernel() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\jupyter_client\multikernelmanager.py:212>> is being executed.
Exception in callback <TaskWakeupMethWrapper object at 0x0000020744E8AC40>(<Future finis...4026\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x0000020744E8AC40>(<Future finis...4026\r\n\r\n'>)>
Traceback (most recent call last):
File "c:\users\redacted\appdata\local\programs\python\python39\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-41' coro=<HTTP1ServerConnection._server_request_loop() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\http1connection.py:823> wait_for=<Future finished result=b'GET /static...54026\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\ioloop.py:688]> while another task <Task pending name='Task-1' coro=<MultiKernelManager._async_start_kernel() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\jupyter_client\multikernelmanager.py:212>> is being executed.
[I 20:48:09.608 NotebookApp] Kernel started: f8a04e76-4cac-4638-8606-59db0aa88803, name: sagemath
[I 20:48:12.560 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
Exception in callback <TaskWakeupMethWrapper object at 0x0000020744E8A670>(<Future finished result=None>)
handle: <Handle <TaskWakeupMethWrapper object at 0x0000020744E8A670>(<Future finished result=None>)>
Traceback (most recent call last):
File "c:\users\redacted\appdata\local\programs\python\python39\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-29' coro=<HTTP1ServerConnection._server_request_loop() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\http1connection.py:823> wait_for=<Future finished result=None> cb=[IOLoop.add_future.<locals>.<lambda>() at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\tornado\ioloop.py:688]> while another task <Task pending name='Task-13' coro=<KernelManager._async_shutdown_kernel() running at c:\users\redacted\appdata\local\programs\python\python39\lib\site-packages\jupyter_client\manager.py:75>> is being executed.
[I 20:48:15.745 NotebookApp] KernelRestarter: restarting kernel (2/5), new random ports
[I 20:48:18.860 NotebookApp] KernelRestarter: restarting kernel (3/5), new random ports
[I 20:48:22.048 NotebookApp] KernelRestarter: restarting kernel (4/5), new random ports
[I 20:48:25.139 NotebookApp] KernelRestarter: restarting kernel (5/5), new random ports
[W 20:48:28.240 NotebookApp] KernelRestarter: restart failed
[W 20:48:28.285 NotebookApp] Kernel f8a04e76-4cac-4638-8606-59db0aa88803 died, removing from map.
[W 20:49:12.620 NotebookApp] Timeout waiting for kernel_info reply from f8a04e76-4cac-4638-8606-59db0aa88803
[E 20:49:12.635 NotebookApp] Error opening stream: HTTP 404: Not Found (Kernel does not exist: f8a04e76-4cac-4638-8606-59db0aa88803)
I have not enough knowledge of jupyter’s protocols, of its architecture or of general web development to make sense of these errors.
Where is the problem?
An interesting variation is instead of having in my homebrew script jump/ instead of /jump/. The path echo behaves as expected but now all but one mintty terminals show sage errors, and the Jupyter notebook server has those long-winded errors only for the first kernel startup try.
- Maybe my problem stems from cygwin pathing?
- Maybe I did not call mintty with the right flags?
Another interesting observation is that the Jupyter servers try to restart the kernel and eventually aborts long before errors appear on Sage’s side.
- Maybe Jupyter doesn’t wait long enough for the kernel to startup?
I found traces in old documentations of a heartbeat parameter to delay the first startup but it seems to be deprecated. - Maybe the problem stems from how Jupyter accesses the kernel?
Having basically two environments, one Windows, one Unix-like, disrupts the communication?
These are all possible to me. I think at this stage only someone with in-depth knowledge of each of those systems could help me rig up this kernel. Any helpful insight greatly appreciated!