Jupyter notebook with julia kernel in Amazon SageMaker

Kernels
1

I want to use a jupyter notebook in Amazon’s Sagemaker jupyterlab with a julia kernel. There are instructions on how to do this here:

https://d1.awsstatic.com/whitepapers/julia-on-sagemaker.pdf?did=wp_card&trk=wp_card

However, it doesn’t work. I created a notebook instance in SageMaker and ran the following on the terminal:

source activate
conda create --yes -n julia
conda activate julia
conda install --yes -c conda-forge Julia=1.7.1

I was then able to start the julia REPL. As per instructions, I installed IJulia.

using Pkg
Pkg.add("IJulia")
using IJulia
jupyterlab(detached=true)

Everything appears to be working. The Julia 1.7.1 icon even shows up in the Launcher for the Jupyterlab notebook. I select it and open a workbook. I try to run a julia command, like

versioninfo()

But jupyterlab cannot connect to the kernel.

conda info --envs

gives

base                     /home/ec2-user/anaconda3
JupyterSystemEnv      *  /home/ec2-user/anaconda3/envs/JupyterSystemEnv
R                        /home/ec2-user/anaconda3/envs/R
amazonei_mxnet_p27       /home/ec2-user/anaconda3/envs/amazonei_mxnet_p27
amazonei_mxnet_p36       /home/ec2-user/anaconda3/envs/amazonei_mxnet_p36
amazonei_pytorch_latest_p36     /home/ec2-user/anaconda3/envs/amazonei_pytorch_latest_p36
amazonei_tensorflow2_p27     /home/ec2-user/anaconda3/envs/amazonei_tensorflow2_p27
amazonei_tensorflow2_p36     /home/ec2-user/anaconda3/envs/amazonei_tensorflow2_p36
amazonei_tensorflow_p27     /home/ec2-user/anaconda3/envs/amazonei_tensorflow_p27
amazonei_tensorflow_p36     /home/ec2-user/anaconda3/envs/amazonei_tensorflow_p36
chainer_p27              /home/ec2-user/anaconda3/envs/chainer_p27
chainer_p36              /home/ec2-user/anaconda3/envs/chainer_p36
julia                    /home/ec2-user/anaconda3/envs/julia
mxnet_latest_p37         /home/ec2-user/anaconda3/envs/mxnet_latest_p37
mxnet_p27                /home/ec2-user/anaconda3/envs/mxnet_p27
mxnet_p36                /home/ec2-user/anaconda3/envs/mxnet_p36
python2                  /home/ec2-user/anaconda3/envs/python2
python3                  /home/ec2-user/anaconda3/envs/python3
pytorch_latest_p36       /home/ec2-user/anaconda3/envs/pytorch_latest_p36
pytorch_p27              /home/ec2-user/anaconda3/envs/pytorch_p27
pytorch_p36              /home/ec2-user/anaconda3/envs/pytorch_p36
tensorflow2_p36          /home/ec2-user/anaconda3/envs/tensorflow2_p36
tensorflow_p27           /home/ec2-user/anaconda3/envs/tensorflow_p27
tensorflow_p36           /home/ec2-user/anaconda3/envs/tensorflow_p36

so I am wondering if Jupyter runs in the JupyterSystemEnv environment and can’t connect to julia (although if that’s true, why does Julia 1.7.1 show up in the Launcher?). So I tried the above steps after running

conda activate JupyterSystemEnv

and indeed julia runs fine on the command line in the JupyterSystemEnv environment afterwards. Jupyterlab, however, still cannot connect to the julia kernel.

Here is a log message:

`

[W 18:17:23.586 NotebookApp] 404 GET /api/kernels/e9365004-3aa3-4109-bec2-10d84d9ba4ab/channels (10.0.0.64): Kernel does not exist: e9365004-3aa3-4109-bec2-10d84d9ba4ab

`

Do I need to install nb_conda_kernels manually? I tried, but didn’t succeed as I didn’t find clear instructions how to do this for julia. R also shows up in the Launcher and works just fine in Jupyterlab.

2

I also posted this question here, and a valiant helper found a (somewhat hacky) solution which ultimately, however, made the Jupyter notebook with a Julia kernel work in Amazon SageMaker.

1 Like