WebSocket closed unexpectedly

I have a binderhub/jupyterhub setup on GKE. When I launch a notebook and create a kernel, I keep getting disconnected about every 30s, with this behaviour:

18:25:17.829 main.min.js[..] Session: kernel_created (56dcd51a-5384-42b9-980d-6c0126fa10fa)
18:25:17.830 main.min.js[..] Starting WebSockets: wss://<jupyterhub_url>/api/kernels/2c4b8f7f-6eed-43f4-990f-75f62ea37534
[..]
18:25:18.296 extension.js?v=20220127172111:1 Setting requirejs-esri-loader's config:
18:25:18.296 extension.js?v=20220127172111:1 Object
18:25:18.296 extension.js?v=20220127172111:1 Initializing esriLoader for quicker load times...
18:25:18.326 main.min.js[..] Kernel: kernel_connected (2c4b8f7f-6eed-43f4-990f-75f62ea37534)
18:25:18.435 main.min.js[..] Kernel: kernel_ready (2c4b8f7f-6eed-43f4-990f-75f62ea37534)
18:25:22.949 extension.js?v=20220127172111:1 esriLoader initialization completed successfully!
[..]
18:25:48.452 main.min.js[..] WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
18:25:48.453 main.min.js[..] Kernel: kernel_disconnected (2c4b8f7f-6eed-43f4-990f-75f62ea37534)
18:25:48.454 main.min.js[..] Connection lost, reconnecting in 1 seconds.
18:25:49.466 main.min.js[..] Kernel: kernel_reconnecting (2c4b8f7f-6eed-43f4-990f-75f62ea37534)
18:25:49.467 main.min.js[..] Starting WebSockets: wss://<jupyterhub_url>/api/kernels/2c4b8f7f-6eed-43f4-990f-75f62ea37534
18:25:49.724 main.min.js[..] Kernel: kernel_connected (2c4b8f7f-6eed-43f4-990f-75f62ea37534)
18:25:49.839 main.min.js[..] Kernel: kernel_ready (2c4b8f7f-6eed-43f4-990f-75f62ea37534)
18:26:19.720 main.min.js[..] WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
...

and it keeps repeating. Funny enough, I can use the kernel and compute ok.

My image was actually built with a previous jupyterhub deployment that was working fine, so I don’t think that is the culprit. I have read around that it could be a proxy problem, but I don’t know how to investigate.

My GKE setup involves an ingress (https, uses google managed-certificate) directing to the proxy-public LoadBalancer service as created by the deployment. It’s a recent helm chart from earlier this month.

Any ideas what this could be or how I could debug into it?

Are those you browser logs? Do you have any custom setup in your GKE cluster, or in Z2JH?

Can you try turning on debug logging in Z2JH (for JupyterHub and the singleuser server), and share the logs from the notebook and maybe proxy? It might also be worth looking at your ingress controller logs if you can.

1 Like

Hi, Yes, those are my browser logs.

I have a jupyterhub domain pointing to a static IP and the GKE ingress redirects all traffic to the proxy-public service. The domain has a google-managed certificate, https traffic is routed internally in the cluster to http.

My relevant jupyterhub yaml (debug enabled) is the following:

jupyterhub:
  hub:
    config:
      BinderSpawner:
        cors_allow_origin: '*'
  debug:
    enabled: true
  scheduling:
    userScheduler:
      enabled: true
    podPriority:
      enabled: true
    userPlaceholder:
      # Specify three dummy user pods will be used as placeholders
      enabled: true
      replicas: 3
  singleuser:
    cpu:
      limit: 1
      guarantee: 0.01
    memory:
      limit: 1024M
      guarantee: 192M
  cull:
    every: 300
    timeout: 900
    enabled: true
    users: true
  rbac:
    enabled: true
  hub:
    extraConfig:
      binder: |
        import os
        import sys
        import yaml
        from tornado import web
        # get cors config from values.custom.cors
        import z2jh
        cors = z2jh.get_config('custom.cors', {})
        # image & token are set via spawn options
        from kubespawner import KubeSpawner
        class BinderSpawner(KubeSpawner):
          def get_args(self):
              args = [
                  '--ip=0.0.0.0',
                  '--port=%i' % self.port,
                  '--NotebookApp.base_url=%s' % self.server.base_url,
                  '--NotebookApp.token=%s' % self.user_options['token'],
                  '--NotebookApp.trust_xheaders=True',
                  '--NotebookApp.iopub_data_rate_limit=1000000000',
                  '--NotebookApp.rate_limit_window=10.0',
              ]
              allow_origin = cors.get('allowOrigin')
              if allow_origin:
                  args.append('--NotebookApp.allow_origin=' + allow_origin)
              return args + self.args
          def start(self):
              if 'token' not in self.user_options:
                raise web.HTTPError(400, "token required")
              if 'image' not in self.user_options:
                raise web.HTTPError(400, "image required")
              self.image_spec = self.user_options['image']
              return super().start()
        c.JupyterHub.spawner_class = BinderSpawner

imageCleaner:
  enabled: true
  image:
    name: jupyterhub/k8s-image-cleaner
    tag: local
  delay: 5
  imageGCThresholdHigh: 95
  imageGCThresholdLow: 60

I launched an image and started a kernel, here are the logs for the 10’ session or so. I don’t see anything interesting in the ingress logs. Here are the browser logs from the notebook session:

07:45:13.393 main.min.js[..]:38824 Session: kernel_created (e35176b8-7450-4cdf-bcaf-ead0edb79fe6)
07:45:13.394 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:45:13.465 main.min.js[..]:24167 Loading extension: jupyter-js-widgets/extension
07:45:13.535 main.min.js[..]:24167 Loading extension: jupyter-offlinenotebook/main
07:45:13.620 extension.js?v=20220129064502:1 Setting requirejs-esri-loader's config:
07:45:13.620 extension.js?v=20220129064502:1 Object
07:45:13.620 extension.js?v=20220129064502:1 Initializing esriLoader for quicker load times...
07:45:13.624 offlinenotebook.js:25 offline-notebook repoid not found, disabled
07:45:13.624 offlinenotebook.js:28 offline-notebook repoLabel: Repo
07:45:13.624 offlinenotebook.js:30 offline-notebook binderRefUrl: 
07:45:13.624 offlinenotebook.js:32 offline-notebook binderPersistentUrl: 
07:45:13.624 main.min.js[..]:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
get @ main.min.js[..]:49253
07:45:13.624 main.min.js[..]:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
get @ main.min.js[..]:49253
07:45:13.624 main.min.js[..]:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
get @ main.min.js[..]:49253
07:45:13.624 main.min.js[..]:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
get @ main.min.js[..]:49253
07:45:13.624 main.min.js[..]:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
get @ main.min.js[..]:49253
07:45:13.926 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:45:14.048 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:45:17.631 extension.js?v=20220129064502:1 esriLoader initialization completed successfully!
07:45:18.914 DevTools failed to load source map: Could not load content for https://<my-image-url>-6wyfrmyt/static/notebook/js/main.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
07:45:18.919 DevTools failed to load source map: Could not load content for https://<my-image-url>-6wyfrmyt/nbextensions/arcgis/extension.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
07:45:44.068 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:45:44.070 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:45:44.070 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:45:45.427 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:45:45.428 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:45:45.682 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:45:45.781 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:15.704 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:46:15.706 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:15.706 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:46:17.422 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:17.422 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:46:17.691 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:17.795 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:47.687 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:46:47.687 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:47.687 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:46:49.411 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:49.411 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:46:49.687 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:46:49.785 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:19.915 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:47:19.916 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:19.916 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:47:21.415 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:21.415 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:47:21.691 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:21.792 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:51.809 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:47:51.810 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:51.810 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:47:53.415 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:53.415 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:47:53.685 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:47:53.794 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:23.731 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:48:23.733 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:23.733 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:48:25.416 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:25.416 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:48:25.683 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:25.785 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:55.759 main.min.js[..]:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
07:48:55.760 main.min.js[..]:37612 Kernel: kernel_disconnected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:55.761 main.min.js[..]:38070 Connection lost, reconnecting in 1 seconds.
07:48:56.764 main.min.js[..]:37612 Kernel: kernel_reconnecting (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:56.765 main.min.js[..]:37969 Starting WebSockets: wss://<my-image-url>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582
07:48:57.033 main.min.js[..]:37612 Kernel: kernel_connected (cdd1b699-6441-4415-bfd0-9854c5149582)
07:48:57.131 main.min.js[..]:37612 Kernel: kernel_ready (cdd1b699-6441-4415-bfd0-9854c5149582)

And the proxy logs (timestamps 1h behind because of timezone diff):

06:42:28.940 [ConfigProxy] info: 200 GET /api/routes
06:43:28.941 [ConfigProxy] info: 200 GET /api/routes
06:44:28.941 [ConfigProxy] info: 200 GET /api/routes
06:44:59.666 [ConfigProxy] debug: PROXY WEB /hub/api/users/<my-image>-6wyfrmyt to http://hub:8081
06:44:59.796 [ConfigProxy] debug: PROXY WEB /hub/api/users/<my-image>-6wyfrmyt/servers/ to http://hub:8081
06:44:59.905 [ConfigProxy] debug: PROXY WEB /hub/api/users/<my-image>-6wyfrmyt/server/progress to http://hub:8081
06:45:03.766 [ConfigProxy] info: Adding route /user/<my-image>-6wyfrmyt -> http://10.44.0.35:8888
06:45:03.766 [ConfigProxy] info: Route added /user/<my-image>-6wyfrmyt -> http://10.44.0.35:8888
06:45:03.767 [ConfigProxy] info: 201 POST /api/routes/user/<my-image>-6wyfrmyt
06:45:04.153 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/ to http://10.44.0.35:8888
06:45:04.157 [ConfigProxy] debug: Not recording activity for status 302 on /user/<my-image>-6wyfrmyt
06:45:04.284 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/tree to http://10.44.0.35:8888
06:45:04.516 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css to http://10.44.0.35:8888
06:45:04.519 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/style/style.min.css to http://10.44.0.35:8888
06:45:04.521 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/jquery-typeahead/dist/jquery.typeahead.min.css to http://10.44.0.35:8888
06:45:04.522 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/es6-promise/promise.min.js to http://10.44.0.35:8888
06:45:04.524 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/react/react-dom.production.min.js to http://10.44.0.35:8888
06:45:04.525 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/custom/custom.css to http://10.44.0.35:8888
06:45:04.526 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/react/react.production.min.js to http://10.44.0.35:8888
06:45:04.541 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/create-react-class/index.js to http://10.44.0.35:8888
06:45:04.547 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/requirejs/require.js to http://10.44.0.35:8888
06:45:05.023 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/tree/js/main.min.js to http://10.44.0.35:8888
06:45:05.026 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/base/images/logo.png to http://10.44.0.35:8888
06:45:05.827 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/services/contents.js to http://10.44.0.35:8888
06:45:05.956 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/config/tree to http://10.44.0.35:8888
06:45:05.959 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/config/common to http://10.44.0.35:8888
06:45:05.980 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/terminals to http://10.44.0.35:8888
06:45:05.982 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/kernelspecs to http://10.44.0.35:8888
06:45:05.984 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/sessions to http://10.44.0.35:8888
06:45:05.987 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/terminals to http://10.44.0.35:8888
06:45:06.046 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/font-awesome/fonts/fontawesome-webfont.woff2 to http://10.44.0.35:8888
06:45:06.048 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/custom/custom.js to http://10.44.0.35:8888
06:45:06.103 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/contents to http://10.44.0.35:8888
06:45:06.208 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/base/images/favicon.ico to http://10.44.0.35:8888
06:45:11.619 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/contents to http://10.44.0.35:8888
06:45:11.760 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/notebooks/Untitled.ipynb to http://10.44.0.35:8888
06:45:12.582 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/notebook/css/override.css to http://10.44.0.35:8888
06:45:12.583 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/custom/custom.css to http://10.44.0.35:8888
06:45:12.584 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/MathJax.js to http://10.44.0.35:8888
06:45:12.585 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/codemirror/lib/codemirror.css to http://10.44.0.35:8888
06:45:12.588 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/bootstrap-tour/build/css/bootstrap-tour.min.css to http://10.44.0.35:8888
06:45:12.596 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:12.828 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/notebook/js/main.min.js to http://10.44.0.35:8888
06:45:12.831 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/text-encoding/lib/encoding.js to http://10.44.0.35:8888
06:45:12.851 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js to http://10.44.0.35:8888
06:45:14.008 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/config/notebook to http://10.44.0.35:8888
06:45:14.013 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/config/common to http://10.44.0.35:8888
06:45:14.017 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/custom/custom.js to http://10.44.0.35:8888
06:45:14.019 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/contents/Untitled.ipynb to http://10.44.0.35:8888
06:45:14.021 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/kernelspecs to http://10.44.0.35:8888
06:45:14.026 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:14.028 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/config/Safe.js to http://10.44.0.35:8888
06:45:14.038 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:14.113 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-datawidgets/extension.js to http://10.44.0.35:8888
06:45:14.116 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-offlinenotebook/main.js to http://10.44.0.35:8888
06:45:14.117 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-threejs/extension.js to http://10.44.0.35:8888
06:45:14.121 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter_resource_usage/main.js to http://10.44.0.35:8888
06:45:14.122 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-js-widgets/extension.js to http://10.44.0.35:8888
06:45:14.124 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/arcgis/extension.js to http://10.44.0.35:8888
06:45:14.173 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/contents/Untitled.ipynb/checkpoints to http://10.44.0.35:8888
06:45:14.178 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/sessions to http://10.44.0.35:8888
06:45:14.183 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/kernelspecs/python3/logo-64x64.png to http://10.44.0.35:8888
06:45:14.198 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/extensions/TeX/newcommand.js to http://10.44.0.35:8888
06:45:14.204 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/extensions/TeX/begingroup.js to http://10.44.0.35:8888
06:45:14.217 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/extensions/MathML/content-mathml.js to http://10.44.0.35:8888
06:45:14.227 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js to http://10.44.0.35:8888
06:45:14.255 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/components/MathJax/extensions/Safe.js to http://10.44.0.35:8888
06:45:14.256 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:14.257 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js to http://10.44.0.35:8888
06:45:14.525 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/offlinenotebook/config to http://10.44.0.35:8888
06:45:14.538 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:45:14.551 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/base/images/favicon-notebook.ico to http://10.44.0.35:8888
06:45:14.935 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/base/images/favicon-notebook.ico to http://10.44.0.35:8888
06:45:19.268 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:19.321 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/custom/custom.css to http://10.44.0.35:8888
06:45:19.325 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:19.757 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/notebook/js/main.min.js.map to http://10.44.0.35:8888
06:45:19.768 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/arcgis/extension.js.map to http://10.44.0.35:8888
06:45:19.771 [ConfigProxy] debug: Not recording activity for status 404 on /user/<my-image>-6wyfrmyt
06:45:19.779 [ConfigProxy] debug: Not recording activity for status 404 on /user/<my-image>-6wyfrmyt
06:45:19.782 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-js-widgets/extension.js.map to http://10.44.0.35:8888
06:45:19.785 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js.map to http://10.44.0.35:8888
06:45:19.912 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/style/style.min.css.map to http://10.44.0.35:8888
06:45:24.278 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:28.940 [ConfigProxy] info: 200 GET /api/routes
06:45:29.267 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:34.269 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:34.275 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:39.258 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:39.262 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:46.584 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:45:48.138 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:48.142 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:49.248 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:49.252 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:45:54.309 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:45:54.313 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:46:18.593 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:46:28.941 [ConfigProxy] info: 200 GET /api/routes
06:46:50.597 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:47:14.489 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/contents/Untitled.ipynb to http://10.44.0.35:8888
06:47:14.591 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/contents/Untitled.ipynb to http://10.44.0.35:8888
06:47:22.599 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:47:28.940 [ConfigProxy] info: 200 GET /api/routes
06:47:47.892 [ConfigProxy] debug: PROXY WEB /incl/image_test.shtml to http://hub:8081
06:47:47.898 [ConfigProxy] debug: Not recording activity for status 302 on /
06:47:54.595 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:48:22.045 [ConfigProxy] debug: PROXY WEB / to http://hub:8081
06:48:22.048 [ConfigProxy] debug: Not recording activity for status 302 on /
06:48:22.101 [ConfigProxy] debug: PROXY WEB /hub/ to http://hub:8081
06:48:22.105 [ConfigProxy] debug: Not recording activity for status 302 on /
06:48:22.158 [ConfigProxy] debug: PROXY WEB /hub/login to http://hub:8081
06:48:22.199 [ConfigProxy] debug: Not recording activity for status 403 on /
06:48:26.592 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:48:28.941 [ConfigProxy] info: 200 GET /api/routes
06:48:49.598 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:48:54.279 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:48:54.282 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:48:57.937 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:48:59.268 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:48:59.272 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:49:04.309 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:49:04.313 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:49:09.288 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:49:09.294 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:49:28.941 [ConfigProxy] info: 200 GET /api/routes
06:49:29.603 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:49:31.818 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:49:31.824 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:50:01.594 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:50:28.944 [ConfigProxy] info: 200 GET /api/routes
06:50:33.577 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:51:05.567 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:51:28.940 [ConfigProxy] info: 200 GET /api/routes
06:51:37.604 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:51:49.313 [ConfigProxy] debug: PROXY WEB / to http://hub:8081
06:51:49.319 [ConfigProxy] debug: Not recording activity for status 405 on /
06:51:49.485 [ConfigProxy] debug: PROXY WEB /.env to http://hub:8081
06:51:49.490 [ConfigProxy] debug: Not recording activity for status 302 on /
06:52:09.568 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:52:28.940 [ConfigProxy] info: 200 GET /api/routes
06:52:41.604 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:52:53.609 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:52:54.268 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:52:54.273 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:52:59.269 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:52:59.277 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:53:04.289 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:53:04.292 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:53:08.172 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/interrupt to http://10.44.0.35:8888
06:53:09.260 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:53:09.264 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:53:12.937 [ConfigProxy] debug: PROXY WS /user/<my-image>-6wyfrmyt/api/kernels/cdd1b699-6441-4415-bfd0-9854c5149582/channels to http://10.44.0.35:8888
06:53:14.258 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:53:14.264 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:53:19.278 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/api/metrics/v1 to http://10.44.0.35:8888
06:53:19.282 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:53:20.549 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/logout to http://10.44.0.35:8888
06:53:20.707 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/auth/css/override.css to http://10.44.0.35:8888
06:53:20.716 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/custom/custom.css to http://10.44.0.35:8888
06:53:20.718 [ConfigProxy] debug: Not recording activity for status 304 on /user/<my-image>-6wyfrmyt
06:53:20.937 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/style/style.min.css.map to http://10.44.0.35:8888
06:53:20.939 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/auth/js/main.min.js to http://10.44.0.35:8888
06:53:21.327 [ConfigProxy] debug: PROXY WEB /user/<my-image>-6wyfrmyt/static/auth/js/main.min.js.map to http://10.44.0.35:8888
06:53:21.331 [ConfigProxy] debug: Not recording activity for status 404 on /user/<my-image>-6wyfrmyt
06:53:28.940 [ConfigProxy] info: 200 GET /api/routes
06:54:28.941 [ConfigProxy] info: 200 GET /api/routes
06:55:28.941 [ConfigProxy] info: 200 GET /api/routes

I forgot to include the singleuser server logs. I post them below for another session, so timestamps are not comparable. I don’t know how to turn on debugging for that though. Any hint?

[I 11:49:51.409 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
[I 11:49:52.061 NotebookApp] JupyterLab extension loaded from /srv/conda/envs/notebook/lib/python3.7/site-packages/jupyterlab
[I 11:49:52.062 NotebookApp] JupyterLab application directory is /srv/conda/envs/notebook/share/jupyter/lab
[I 11:49:52.066 NotebookApp] nteract extension loaded from /srv/conda/envs/notebook/lib/python3.7/site-packages/nteract_on_jupyter
[I 11:49:52.069 NotebookApp] Serving notebooks from local directory: /home/jovyan
[I 11:49:52.069 NotebookApp] Jupyter Notebook 6.3.0 is running at:
[I 11:49:52.069 NotebookApp] http://<my-image>:8888/user/<my-image>/?token=...
[I 11:49:52.069 NotebookApp]  or http://127.0.0.1:8888/user/<my-image>/?token=...
[I 11:49:52.069 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 11:49:52.073 NotebookApp] No web browser found: could not locate runnable browser.
[I 11:49:56.205 NotebookApp] 302 GET /user/<my-image>/ (10.44.0.34) 0.750000ms
[I 11:49:56.524 NotebookApp] 302 GET /user/<my-image>/?token=yvgaVOd1RFmna5izR2NjOA (::ffff:35.191.1.73) 0.890000ms
[I 11:50:01.857 NotebookApp] Creating new notebook in
[I 11:50:01.873 NotebookApp] Writing notebook-signing key to /home/jovyan/.local/share/jupyter/notebook_secret
[I 11:50:04.305 NotebookApp] Kernel started: 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1, name: python3
[W 11:50:09.144 NotebookApp] 404 GET /user/<my-image>/static/notebook/js/main.min.js.map (::ffff:35.191.1.79) 8.630000ms referer=None
[W 11:50:09.165 NotebookApp] 404 GET /user/<my-image>/nbextensions/arcgis/extension.js.map (::ffff:35.191.9.35) 2.800000ms referer=None
[I 11:50:34.868 NotebookApp] Starting buffering for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:50:36.904 NotebookApp] Restoring connection for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:51:06.907 NotebookApp] Starting buffering for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:51:08.134 NotebookApp] Restoring connection for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:51:38.136 NotebookApp] Starting buffering for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:51:39.945 NotebookApp] Restoring connection for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:52:04.940 NotebookApp] Saving file at /Untitled.ipynb
[I 11:52:09.947 NotebookApp] Starting buffering for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:52:11.937 NotebookApp] Restoring connection for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:52:41.941 NotebookApp] Starting buffering for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:52:43.932 NotebookApp] Restoring connection for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:53:13.938 NotebookApp] Starting buffering for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506
[I 11:53:15.996 NotebookApp] Restoring connection for 210a7cf7-ccd5-4312-b9ed-d74e1536dfb1:2f667e1c458945d1b0a7945b61aa4506

To enable singleuser debug logs you could try adding --debug to your spawner args? Could you also show us your hub logs?

If that doesn’t give you any clues it might be worth simplifying things:

  • Install just Z2JH instead of BinderHub with a simple Docker image
  • Try a different browser
  • Check whether you have any firewalling, proxies, or network management tools that might be inteferring with your connection.

Hi, I have enabled more debug logs. Here is a full session. First, the browser logs

12:00:43.707 main.min.js?v=:45677 actions jupyter-notebook:find-and-replace does not exist, still binding it in case it will be defined later...
12:00:43.741 main.min.js?v=:33602 Loaded moment locale en
12:00:43.774 main.min.js?v=:24190 load_extensions Arguments(6) ['jupyter-datawidgets/extension', 'jupyter-threejs/extension', 'jupyter_resource_usage/main', 'jupyter-offlinenotebook/main', 'jupyter-js-widgets/extension', 'arcgis/extension', callee: (...), Symbol(Symbol.iterator): ƒ]
12:00:43.829 DevTools failed to load source map: Could not load content for https://<my-image-url>/static/notebook/js/main.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
12:00:43.941 main.min.js?v=:24167 Loading extension: jupyter-datawidgets/extension
12:00:43.946 main.min.js?v=:24167 Loading extension: jupyter_resource_usage/main
12:00:43.968 extension.js?v=20220207110028:1 nbextension path = /nbextensions/arcgis/
12:00:43.976 main.min.js?v=:24167 Loading extension: arcgis/extension
12:00:43.995 main.min.js?v=:24167 Loading extension: jupyter-threejs/extension
12:00:44.042 extension.js?v=20220207110028:1 Setting requirejs-esri-loader's config:
12:00:44.043 extension.js?v=20220207110028:1 {map: {…}, config: {…}, packages: Array(13)}
12:00:44.043 extension.js?v=20220207110028:1 Initializing esriLoader for quicker load times...
12:00:44.047 main.min.js?v=:38824 Session: kernel_created (62a29ad5-351b-4e7c-9921-d37affa100b3)
12:00:44.047 main.min.js?v=:37969 Starting WebSockets: wss://<my-image-url>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15
12:00:44.197 main.min.js?v=:24167 Loading extension: jupyter-js-widgets/extension
12:00:44.211 DevTools failed to load source map: Could not load content for https://<my-image-url>/nbextensions/arcgis/extension.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
12:00:44.246 main.min.js?v=:24167 Loading extension: jupyter-offlinenotebook/main
12:00:44.720 main.min.js?v=:37612 Kernel: kernel_connected (e10773be-d68e-4b52-8823-e9146684cc15)
12:00:44.824 offlinenotebook.js:25 offline-notebook repoid not found, disabled
12:00:44.824 offlinenotebook.js:28 offline-notebook repoLabel: Repo
12:00:44.824 offlinenotebook.js:30 offline-notebook binderRefUrl: 
12:00:44.824 offlinenotebook.js:32 offline-notebook binderPersistentUrl: 
12:00:44.824 main.min.js?v=:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
12:00:44.825 main.min.js?v=:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
12:00:44.825 main.min.js?v=:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
12:00:44.826 main.min.js?v=:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
12:00:44.826 main.min.js?v=:49253 accessing "actions" on the global IPython/Jupyter is not recommended. Pass it to your objects constructors at creation time
12:00:44.903 main.min.js?v=:37612 Kernel: kernel_ready (e10773be-d68e-4b52-8823-e9146684cc15)
12:00:49.946 extension.js?v=20220207110028:1 esriLoader initialization completed successfully!
12:01:14.731 main.min.js?v=:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
12:01:14.734 main.min.js?v=:37612 Kernel: kernel_disconnected (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:14.734 main.min.js?v=:38070 Connection lost, reconnecting in 1 seconds.
12:01:15.752 main.min.js?v=:37612 Kernel: kernel_reconnecting (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:15.753 main.min.js?v=:37969 Starting WebSockets: wss://<my-image-url>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15
12:01:15.999 main.min.js?v=:37612 Kernel: kernel_connected (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:16.093 main.min.js?v=:37612 Kernel: kernel_ready (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:46.008 main.min.js?v=:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}isTrusted: truebubbles: falsecancelBubble: falsecancelable: falsecode: 1006composed: falsecurrentTarget: WebSocket {url: 'wss://mars.edbr.io/user/<my-image-url>-u…nnels?session_id=2100c8dccdb346c58eec62beb1623a24', readyState: 3, bufferedAmount: 0, onopen: ƒ, onerror: ƒ, …}defaultPrevented: falseeventPhase: 0path: []reason: ""returnValue: truesrcElement: WebSocket {url: 'wss://mars.edbr.io/user/<my-image-url>-u…nnels?session_id=2100c8dccdb346c58eec62beb1623a24', readyState: 3, bufferedAmount: 0, onopen: ƒ, onerror: ƒ, …}target: WebSocket {url: 'wss://mars.edbr.io/user/<my-image-url>-u…nnels?session_id=2100c8dccdb346c58eec62beb1623a24', readyState: 3, bufferedAmount: 0, onopen: ƒ, onerror: ƒ, …}timeStamp: 64434.60000002384type: "close"wasClean: false[[Prototype]]: CloseEvent
12:01:46.010 main.min.js?v=:37612 Kernel: kernel_disconnected (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:46.011 main.min.js?v=:38070 Connection lost, reconnecting in 1 seconds.
12:01:47.022 main.min.js?v=:37612 Kernel: kernel_reconnecting (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:47.022 main.min.js?v=:37969 Starting WebSockets: wss://<my-image-url>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15
12:01:47.368 main.min.js?v=:37612 Kernel: kernel_connected (e10773be-d68e-4b52-8823-e9146684cc15)
12:01:47.454 main.min.js?v=:37612 Kernel: kernel_ready (e10773be-d68e-4b52-8823-e9146684cc15)
12:02:17.440 main.min.js?v=:37999 WebSocket closed unexpectedly CloseEvent {isTrusted: true, wasClean: false, code: 1006, reason: '', type: 'close', …}
12:02:17.441 main.min.js?v=:37612 Kernel: kernel_disconnected (e10773be-d68e-4b52-8823-e9146684cc15)
12:02:17.442 main.min.js?v=:38070 Connection lost, reconnecting in 1 seconds.
12:02:18.464 main.min.js?v=:37612 Kernel: kernel_reconnecting (e10773be-d68e-4b52-8823-e9146684cc15)
12:02:18.464 main.min.js?v=:37969 Starting WebSockets: wss://<my-image-url>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15
12:02:18.992 main.min.js?v=:37612 Kernel: kernel_connected (e10773be-d68e-4b52-8823-e9146684cc15)
12:02:19.084 main.min.js?v=:37612 Kernel: kernel_ready (e10773be-d68e-4b52-8823-e9146684cc15)

Then the proxy logs

11:00:09.766 [ConfigProxy] e[32minfoe[39m: 200 GET /api/routes 
11:00:25.325 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /hub/api/users/<my-image> to http://10.48.11.200:8081
11:00:25.370 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /hub/api/users/<my-image>/servers/ to http://10.48.11.200:8081
11:00:25.455 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /hub/api/users/<my-image> to http://10.48.11.200:8081
11:00:26.486 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /hub/api/users/<my-image> to http://10.48.11.200:8081
11:00:27.915 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /hub/api/users/<my-image> to http://10.48.11.200:8081
11:00:29.265 [ConfigProxy] e[32minfoe[39m: Adding route /user/<my-image> -> http://10.44.1.35:8888
11:00:29.265 [ConfigProxy] e[32minfoe[39m: Route added /user/<my-image> -> http://10.44.1.35:8888
11:00:29.266 [ConfigProxy] e[32minfoe[39m: 201 POST /api/routes/user/<my-image> 
11:00:29.904 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /hub/api/users/<my-image> to http://10.48.11.200:8081
11:00:30.299 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/?token=80zV0JMNSkCY4rwDkSMiDA to http://10.44.1.35:8888
11:00:30.390 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/tree?token=80zV0JMNSkCY4rwDkSMiDA to http://10.44.1.35:8888
11:00:30.587 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=fb45616eef2c454960f91fcd2a04efeda84cfacccf0c5d741ba2793dc1dbd6d3ab01aaae6485222945774c7d7a9a2e9fb87e0d8ef1ea96893aa6906147a371bb to http://10.44.1.35:8888
11:00:30.590 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=5edf53bf6bb9c3b1ddafd8594825a7e2ed621f19423e569c985162742f63911c09eba2c529f8fb47aebf27fafdfe287d563347f58c1126b278189a18871b6a9a to http://10.44.1.35:8888
11:00:30.591 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/style/style.min.css?v=56dfd556850eb17b7998c6828467598a322b41593edc758739c66cb2c3fea98f23d0dd8bf8b9b0f5d67bb976a50e4c34f789fe640cbb440fa089e1bf5ec170bd to http://10.44.1.35:8888
11:00:30.593 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/custom/custom.css to http://10.44.1.35:8888
11:00:30.594 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/es6-promise/promise.min.js?v=bea335d74136a63ae1b5130f5ac9a50c6256a5f435e6e09fef599491a84d834a8b0f011ca3eaaca3b4ab6a2da2d3e1191567a2f171e60da1d10e5b9d52f84184 to http://10.44.1.35:8888
11:00:30.596 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 to http://10.44.1.35:8888
11:00:30.598 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 to http://10.44.1.35:8888
11:00:30.598 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/requirejs/require.js?v=d37b48bb2137faa0ab98157e240c084dd5b1b5e74911723aa1d1f04c928c2a03dedf922d049e4815f7e5a369faa2e6b6a1000aae958b7953b5cc60411154f593 to http://10.44.1.35:8888
11:00:30.599 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec to http://10.44.1.35:8888
11:00:30.839 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/tree/js/main.min.js?v=e37855cd299f6edbfb33deaf95a41c5c5e2a2057426822be7e214be679b8c8e8f031ecf6ba0b1a4f0aa8864d58e470c6e433f8f245c81d9518f9fe472ec2d53a to http://10.44.1.35:8888
11:00:30.853 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/base/images/logo.png?v=a2a176ee3cee251ffddf5fa21fe8e43727a9e5f87a06f9c91ad7b776d9e9d3d5e0159c16cc188a3965e00375fb4bc336c16067c688f5040c0c2d4bfdb852a9e4 to http://10.44.1.35:8888
11:00:31.459 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/base/images/favicon.ico?v=50afa725b5de8b00030139d09b38620224d4e7dba47c07ef0e86d4643f30c9bfe6bb7e1a4a1c561aa32834480909a4b6fe7cd1e17f7159330b6b5914bf45a880 to http://10.44.1.35:8888
11:00:31.460 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/services/contents.js?v=20220207110028 to http://10.44.1.35:8888
11:00:31.576 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/config/tree?_=1644231631141 to http://10.44.1.35:8888
11:00:31.579 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/config/common?_=1644231631142 to http://10.44.1.35:8888
11:00:31.583 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/terminals?_=1644231631143 to http://10.44.1.35:8888
11:00:31.587 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/terminals?_=1644231631145 to http://10.44.1.35:8888
11:00:31.588 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/kernelspecs to http://10.44.1.35:8888
11:00:31.589 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/sessions?_=1644231631144 to http://10.44.1.35:8888
11:00:31.607 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/custom/custom.js?v=20220207110028 to http://10.44.1.35:8888
11:00:31.610 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 to http://10.44.1.35:8888
11:00:31.695 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/contents?type=directory&_=1644231631146 to http://10.44.1.35:8888
11:00:41.520 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/contents to http://10.44.1.35:8888
11:00:41.652 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/notebooks/Untitled.ipynb?kernel_name=python3 to http://10.44.1.35:8888
11:00:42.484 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/codemirror/lib/codemirror.css?v=81fecb54f83101e2bbe6d2e3131e252ac83f2910366100ca83ba4834f5d41754c837f306eecfdceed05f9c9111614942e2ced5acdd8040746b66c6bef0141d0e to http://10.44.1.35:8888
11:00:42.485 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/custom/custom.css to http://10.44.1.35:8888
11:00:42.486 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured to http://10.44.1.35:8888
11:00:42.487 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/bootstrap-tour/build/css/bootstrap-tour.min.css?v=95c93e52db61ab29625defe55361384ce6776a7d303b97da5a73fef5ddf8e391a6223599a0b58669476bd71645a4f0022df0517c88b0c05df80ba465e36f5417 to http://10.44.1.35:8888
11:00:42.488 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/notebook/css/override.css?v=16733f6ba5f2224692fe4e654f3cbb2e3cae82f1df06ca53aa1cb88b147465f16c968c0898e2b0203a7ad3a469f82b959e26bb4b27b790f7f364c4336449b0aa to http://10.44.1.35:8888
11:00:42.626 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/text-encoding/lib/encoding.js?v=737ac6f9f978afb6348b5914877e7d7136de7465cd4cdf389bad9a6b3ad5ceffbfb23febc75c23378967d7d36f98f5388208e8eb78c80f2bf47e8f8c000481ad to http://10.44.1.35:8888
11:00:42.628 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/notebook/js/main.min.js?v=7af39fe9ec3585e958e8637ace5e96fe0bf3ced51a8b8b5875c571f941299779fe746bf74f02bab0f3c6e08281d6ecadafcb5b08120e98f081c9c7dd824bdfdc to http://10.44.1.35:8888
11:00:42.650 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js?V=2.7.9 to http://10.44.1.35:8888
11:00:42.684 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/style/style.min.css.map to http://10.44.1.35:8888
11:00:43.188 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/config/Safe.js?V=2.7.9 to http://10.44.1.35:8888
11:00:43.740 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/config/notebook?_=1644231643272 to http://10.44.1.35:8888
11:00:43.742 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/notebook/js/main.min.js.map to http://10.44.1.35:8888
11:00:43.742 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/config/common?_=1644231643273 to http://10.44.1.35:8888
11:00:43.753 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/extensions/TeX/newcommand.js?V=2.7.9 to http://10.44.1.35:8888
11:00:43.768 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/extensions/MathML/content-mathml.js?V=2.7.9 to http://10.44.1.35:8888
11:00:43.771 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/extensions/TeX/begingroup.js?V=2.7.9 to http://10.44.1.35:8888
11:00:43.780 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/kernelspecs to http://10.44.1.35:8888
11:00:43.803 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/contents/Untitled.ipynb?type=notebook&_=1644231643274 to http://10.44.1.35:8888
11:00:43.806 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/custom/custom.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.811 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js?V=2.7.9 to http://10.44.1.35:8888
11:00:43.813 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/components/MathJax/extensions/Safe.js?V=2.7.9 to http://10.44.1.35:8888
11:00:43.853 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-datawidgets/extension.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.868 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-offlinenotebook/main.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.869 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-threejs/extension.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.870 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-js-widgets/extension.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.871 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter_resource_usage/main.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.873 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/arcgis/extension.js?v=20220207110028 to http://10.44.1.35:8888
11:00:43.938 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/sessions to http://10.44.1.35:8888
11:00:43.943 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/contents/Untitled.ipynb/checkpoints?_=1644231643275 to http://10.44.1.35:8888
11:00:43.952 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/kernelspecs/python3/logo-64x64.png to http://10.44.1.35:8888
11:00:44.019 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:00:44.038 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/arcgis/extension.js.map to http://10.44.1.35:8888
11:00:44.050 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js?v=20220207110028 to http://10.44.1.35:8888
11:00:44.269 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-js-widgets/extension.js.map to http://10.44.1.35:8888
11:00:44.285 [ConfigProxy] e[34mdebuge[39m: PROXY WS /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 to http://10.44.1.35:8888
11:00:44.314 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js.map to http://10.44.1.35:8888
11:00:44.315 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/offlinenotebook/config to http://10.44.1.35:8888
11:00:49.058 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:00:50.016 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/static/base/images/favicon-notebook.ico to http://10.44.1.35:8888
11:00:54.070 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:00:59.051 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:04.038 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:09.051 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:09.764 [ConfigProxy] e[32minfoe[39m: 200 GET /api/routes 
11:01:14.070 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:15.949 [ConfigProxy] e[34mdebuge[39m: PROXY WS /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 to http://10.44.1.35:8888
11:01:19.060 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:24.070 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:29.051 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:34.088 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:39.062 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:44.072 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:47.342 [ConfigProxy] e[34mdebuge[39m: PROXY WS /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 to http://10.44.1.35:8888
11:01:49.052 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:54.070 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888
11:01:59.068 [ConfigProxy] e[34mdebuge[39m: PROXY WEB /user/<my-image>/api/metrics/v1 to http://10.44.1.35:8888

Then the hub logs

[I 2022-02-07 11:00:01.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.89ms
[D 2022-02-07 11:00:05.742 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:00:05.742 JupyterHub reflector:202] Connecting pods watcher
[D 2022-02-07 11:00:09.763 JupyterHub proxy:766] Proxy: Fetching GET http://10.48.2.65:8001/api/routes
[I 2022-02-07 11:00:09.780 JupyterHub proxy:320] Checking routes
[I 2022-02-07 11:00:11.487 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.86ms
[D 2022-02-07 11:00:15.765 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:00:15.765 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:00:21.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.88ms
[I 2022-02-07 11:00:25.362 JupyterHub log:174] 201 POST /hub/api/users/<my-image> (binder@35.191.15.145) 34.93ms
[D 2022-02-07 11:00:25.389 JupyterHub user:242] Creating <class 'BinderSpawner'> for <my-image>:
[D 2022-02-07 11:00:25.395 JupyterHub base:825] Initiating spawn for <my-image>
[D 2022-02-07 11:00:25.395 JupyterHub base:832] 0/64 concurrent spawns
[D 2022-02-07 11:00:25.396 JupyterHub base:837] 3 active servers
[D 2022-02-07 11:00:25.439 JupyterHub user:556] Calling Spawner.start for <my-image>
[W 2022-02-07 11:00:25.439 JupyterHub spawner:1287] KubeSpawner.image_spec is deprecated in 0.10. Use KubeSpawner.image
[W 2022-02-07 11:00:25.444 JupyterHub base:950] User <my-image> is slow to start (timeout=0)
[I 2022-02-07 11:00:25.445 JupyterHub log:174] 202 POST /hub/api/users/<my-image>/servers/ (binder@35.191.15.153) 71.75ms
[I 2022-02-07 11:00:25.476 JupyterHub log:174] 200 GET /hub/api/users/<my-image> (binder@35.191.15.155) 17.95ms
[D 2022-02-07 11:00:25.787 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:00:25.787 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:00:26.505 JupyterHub log:174] 200 GET /hub/api/users/<my-image> (binder@35.191.15.147) 17.30ms
[I 2022-02-07 11:00:27.934 JupyterHub log:174] 200 GET /hub/api/users/<my-image> (binder@35.191.15.151) 16.75ms
[D 2022-02-07 11:00:29.246 JupyterHub spawner:1113] Polling subprocess every 30s
[D 2022-02-07 11:00:29.261 JupyterHub utils:218] Server at http://10.44.1.35:8888/user/<my-image>/ responded with 302
[W 2022-02-07 11:00:29.262 JupyterHub _version:34] Single-user server has no version header, which means it is likely < 0.8. Expected 1.1.0
[I 2022-02-07 11:00:29.262 JupyterHub base:855] User <my-image> took 3.867 seconds to start
[I 2022-02-07 11:00:29.262 JupyterHub proxy:262] Adding user <my-image> to proxy /user/<my-image>/ => http://10.44.1.35:8888
[D 2022-02-07 11:00:29.262 JupyterHub proxy:766] Proxy: Fetching POST http://10.48.2.65:8001/api/routes/user/<my-image>
[I 2022-02-07 11:00:29.917 JupyterHub log:174] 200 GET /hub/api/users/<my-image> (binder@35.191.15.155) 11.46ms
[I 2022-02-07 11:00:31.487 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 1.04ms
[D 2022-02-07 11:00:35.814 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:00:35.814 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:00:41.487 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 1.06ms
[D 2022-02-07 11:00:45.838 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:00:45.838 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:00:51.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 1.02ms
[D 2022-02-07 11:00:55.864 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:00:55.864 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:01:01.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.99ms
[D 2022-02-07 11:01:05.886 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:01:05.886 JupyterHub reflector:202] Connecting pods watcher
[D 2022-02-07 11:01:09.763 JupyterHub proxy:766] Proxy: Fetching GET http://10.48.2.65:8001/api/routes
[I 2022-02-07 11:01:09.780 JupyterHub proxy:320] Checking routes
[I 2022-02-07 11:01:11.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.89ms
[D 2022-02-07 11:01:15.910 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:01:15.910 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:01:21.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.99ms
[D 2022-02-07 11:01:25.933 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:01:25.933 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:01:31.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.89ms
[D 2022-02-07 11:01:35.955 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:01:35.956 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:01:41.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.90ms
[D 2022-02-07 11:01:45.981 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:01:45.982 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:01:51.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.86ms
[D 2022-02-07 11:01:56.004 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:01:56.004 JupyterHub reflector:202] Connecting pods watcher
[I 2022-02-07 11:02:01.486 JupyterHub log:174] 200 GET /hub/health (@10.44.1.1) 0.90ms
[D 2022-02-07 11:02:06.028 JupyterHub reflector:270] pods watcher timeout
[D 2022-02-07 11:02:06.029 JupyterHub reflector:202] Connecting pods watcher

Finally the single-user server logs

[D 11:00:28.060 NotebookApp] Searching ['/home/jovyan', '/home/jovyan/.jupyter', '/srv/conda/envs/notebook/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 11:00:28.061 NotebookApp] Looking for jupyter_config in /etc/jupyter
[D 11:00:28.061 NotebookApp] Looking for jupyter_config in /usr/local/etc/jupyter
[D 11:00:28.061 NotebookApp] Looking for jupyter_config in /srv/conda/envs/notebook/etc/jupyter
[D 11:00:28.062 NotebookApp] Looking for jupyter_config in /home/jovyan/.jupyter
[D 11:00:28.062 NotebookApp] Looking for jupyter_config in /home/jovyan
[D 11:00:28.063 NotebookApp] Looking for jupyter_notebook_config in /etc/jupyter
[D 11:00:28.063 NotebookApp] Looking for jupyter_notebook_config in /usr/local/etc/jupyter
[D 11:00:28.064 NotebookApp] Looking for jupyter_notebook_config in /srv/conda/envs/notebook/etc/jupyter
[D 11:00:28.064 NotebookApp] Loaded config file: /srv/conda/envs/notebook/etc/jupyter/jupyter_notebook_config.json
[D 11:00:28.064 NotebookApp] Looking for jupyter_notebook_config in /home/jovyan/.jupyter
[D 11:00:28.064 NotebookApp] Looking for jupyter_notebook_config in /home/jovyan
[D 11:00:28.078 NotebookApp] Paths used for configuration of jupyter_notebook_config: 
    	/etc/jupyter/jupyter_notebook_config.json
[D 11:00:28.079 NotebookApp] Paths used for configuration of jupyter_notebook_config: 
    	/usr/local/etc/jupyter/jupyter_notebook_config.json
[D 11:00:28.079 NotebookApp] Paths used for configuration of jupyter_notebook_config: 
    	/srv/conda/envs/notebook/etc/jupyter/jupyter_notebook_config.d/jupyter_resource_usage.json
    	/srv/conda/envs/notebook/etc/jupyter/jupyter_notebook_config.d/jupyterlab.json
    	/srv/conda/envs/notebook/etc/jupyter/jupyter_notebook_config.d/nteract_on_jupyter.json
    	/srv/conda/envs/notebook/etc/jupyter/jupyter_notebook_config.d/offlinenotebook_nbserverextension.json
    	/srv/conda/envs/notebook/etc/jupyter/jupyter_notebook_config.json
[D 11:00:28.081 NotebookApp] Paths used for configuration of jupyter_notebook_config: 
    	/home/jovyan/.jupyter/jupyter_notebook_config.json
[I 11:00:28.085 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
[I 11:00:28.751 NotebookApp] JupyterLab extension loaded from /srv/conda/envs/notebook/lib/python3.7/site-packages/jupyterlab
[I 11:00:28.751 NotebookApp] JupyterLab application directory is /srv/conda/envs/notebook/share/jupyter/lab
[I 11:00:28.756 NotebookApp] nteract extension loaded from /srv/conda/envs/notebook/lib/python3.7/site-packages/nteract_on_jupyter
[I 11:00:28.758 NotebookApp] Serving notebooks from local directory: /home/jovyan
[I 11:00:28.758 NotebookApp] Jupyter Notebook 6.3.0 is running at:
[I 11:00:28.758 NotebookApp] http://<my-image-url>:8888/user/<my-image>/?token=...
[I 11:00:28.758 NotebookApp]  or http://127.0.0.1:8888/user/<my-image>/?token=...
[I 11:00:28.758 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 11:00:28.762 NotebookApp] No web browser found: could not locate runnable browser.
[I 11:00:29.256 NotebookApp] 302 GET /user/<my-image>/ (10.44.1.29) 0.880000ms
[I 11:00:30.302 NotebookApp] 302 GET /user/<my-image>/?token=80zV0JMNSkCY4rwDkSMiDA (35.191.12.167) 0.680000ms
[D 11:00:30.393 NotebookApp] Accepting token-authenticated connection from 35.191.12.161
[D 11:00:30.395 NotebookApp] Using contents: services/contents
[D 11:00:30.456 NotebookApp] Path base/images/favicon.ico served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/base/images/favicon.ico
[D 11:00:30.456 NotebookApp] Path components/jquery-ui/themes/smoothness/jquery-ui.min.css served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css
[..]
[D 11:00:30.466 NotebookApp] 200 GET /user/<my-image>/tree?token=80zV0JMNSkCY4rwDkSMiDA (35.191.12.161) 73.590000ms
[D 11:00:30.595 NotebookApp] 200 GET /user/<my-image>/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=..bb (35.191.12.165) 5.270000ms
[D 11:00:30.603 NotebookApp] 200 GET /user/<my-image>/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=..9a (35.191.12.169) 6.310000ms
[D 11:00:30.604 NotebookApp] Path custom.css served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/custom/custom.css
[D 11:00:30.606 NotebookApp] 200 GET /user/<my-image>/custom/custom.css (35.191.12.167) 5.450000ms
[D 11:00:30.607 NotebookApp] 200 GET /user/<my-image>/static/components/es6-promise/promise.min.js?v=..84 (35.191.12.167) 5.670000ms
[D 11:00:30.612 NotebookApp] 200 GET /user/<my-image>/static/components/react/react-dom.production.min.js?v=..84 (35.191.12.169) 4.230000ms
[D 11:00:30.613 NotebookApp] 200 GET /user/<my-image>/static/components/react/react.production.min.js?v=..35 (35.191.12.161) 3.890000ms
[D 11:00:30.614 NotebookApp] 200 GET /user/<my-image>/static/components/requirejs/require.js?v=..93 (35.191.12.165) 4.100000ms
[D 11:00:30.615 NotebookApp] 200 GET /user/<my-image>/static/components/create-react-class/index.js?v=..ec (35.191.12.171) 4.720000ms
[D 11:00:30.616 NotebookApp] 200 GET /user/<my-image>/static/style/style.min.css?v=..bd (35.191.12.163) 17.700000ms
[D 11:00:30.856 NotebookApp] 200 GET /user/<my-image>/static/base/images/logo.png?v=..e4 (35.191.12.165) 1.320000ms
[D 11:00:30.868 NotebookApp] 200 GET /user/<my-image>/static/tree/js/main.min.js?v=..3a (35.191.12.163) 27.110000ms
[D 11:00:31.463 NotebookApp] 200 GET /user/<my-image>/static/base/images/favicon.ico?v=..80 (35.191.12.171) 1.510000ms
[D 11:00:31.464 NotebookApp] Path services/contents.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/services/contents.js
[D 11:00:31.465 NotebookApp] 200 GET /user/<my-image>/static/services/contents.js?v=20220207110028 (35.191.12.165) 1.500000ms
[D 11:00:31.580 NotebookApp] Paths used for configuration of tree: 
    	/etc/jupyter/nbconfig/tree.json
[D 11:00:31.581 NotebookApp] Paths used for configuration of tree: 
    	/usr/local/etc/jupyter/nbconfig/tree.json
[D 11:00:31.582 NotebookApp] Paths used for configuration of tree: 
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/tree.json
[D 11:00:31.583 NotebookApp] Paths used for configuration of tree: 
    	/home/jovyan/.jupyter/nbconfig/tree.json
[D 11:00:31.584 NotebookApp] 200 GET /user/<my-image>/api/config/tree?_=1644231631141 (35.191.12.169) 5.770000ms
[D 11:00:31.588 NotebookApp] Paths used for configuration of common: 
    	/etc/jupyter/nbconfig/common.json
[D 11:00:31.589 NotebookApp] Paths used for configuration of common: 
    	/usr/local/etc/jupyter/nbconfig/common.json
[D 11:00:31.589 NotebookApp] Paths used for configuration of common: 
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/common.json
[D 11:00:31.591 NotebookApp] Paths used for configuration of common: 
    	/home/jovyan/.jupyter/nbconfig/common.json
[D 11:00:31.592 NotebookApp] 200 GET /user/<my-image>/api/config/common?_=1644231631142 (35.191.12.163) 6.340000ms
[D 11:00:31.592 NotebookApp] 200 GET /user/<my-image>/api/terminals?_=1644231631143 (35.191.12.167) 6.010000ms
[D 11:00:31.597 NotebookApp] 200 GET /user/<my-image>/api/terminals?_=1644231631145 (35.191.12.165) 2.720000ms
[D 11:00:31.598 NotebookApp] Found kernel python3 in /srv/conda/envs/notebook/share/jupyter/kernels
[D 11:00:31.602 NotebookApp] 200 GET /user/<my-image>/api/kernelspecs (35.191.12.161) 6.510000ms
[D 11:00:31.603 NotebookApp] 200 GET /user/<my-image>/api/sessions?_=1644231631144 (35.191.12.171) 7.500000ms
[D 11:00:31.613 NotebookApp] Path components/font-awesome/fonts/fontawesome-webfont.woff2 served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/font-awesome/fonts/fontawesome-webfont.woff2
[D 11:00:31.615 NotebookApp] 200 GET /user/<my-image>/static/components/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 (35.191.12.169) 3.090000ms
[D 11:00:31.616 NotebookApp] Path custom.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/custom/custom.js
[D 11:00:31.617 NotebookApp] 200 GET /user/<my-image>/custom/custom.js?v=20220207110028 (35.191.12.163) 4.820000ms
[D 11:00:31.702 NotebookApp] 200 GET /user/<my-image>/api/contents?type=directory&_=1644231631146 (35.191.12.167) 4.290000ms
[I 11:00:41.523 NotebookApp] Creating new notebook in 
[D 11:00:41.524 NotebookApp] Saving /home/jovyan/Untitled.ipynb
[I 11:00:41.540 NotebookApp] Writing notebook-signing key to /home/jovyan/.local/share/jupyter/notebook_secret
[D 11:00:41.552 NotebookApp] 201 POST /user/<my-image>/api/contents (35.191.12.163) 29.890000ms
[D 11:00:41.655 NotebookApp] Using contents: services/contents
[D 11:00:41.724 NotebookApp] Path base/images/favicon-notebook.ico served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/base/images/favicon-notebook.ico
[D 11:00:41.725 NotebookApp] Path components/bootstrap-tour/build/css/bootstrap-tour.min.css served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/bootstrap-tour/build/css/bootstrap-tour.min.css
[D 11:00:41.725 NotebookApp] Path components/codemirror/lib/codemirror.css served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/codemirror/lib/codemirror.css
[D 11:00:41.725 NotebookApp] Path notebook/css/override.css served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/notebook/css/override.css
[D 11:00:42.347 NotebookApp] Path components/text-encoding/lib/encoding.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/text-encoding/lib/encoding.js
[D 11:00:42.348 NotebookApp] Path notebook/js/main.min.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/notebook/js/main.min.js
[D 11:00:42.354 NotebookApp] 200 GET /user/<my-image>/notebooks/Untitled.ipynb?kernel_name=python3 (35.191.12.167) 700.260000ms
[D 11:00:42.490 NotebookApp] 200 GET /user/<my-image>/static/components/codemirror/lib/codemirror.css?v=81fecb54f83101e2bbe6d2e3131e252ac83f2910366100ca83ba4834f5d41754c837f306eecfdceed05f9c9111614942e2ced5acdd8040746b66c6bef0141d0e (35.191.12.167) 3.110000ms
[D 11:00:42.491 NotebookApp] 304 GET /user/<my-image>/custom/custom.css (35.191.12.171) 2.510000ms
[D 11:00:42.495 NotebookApp] Path components/MathJax/MathJax.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/MathJax.js
[D 11:00:42.496 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured (35.191.12.163) 3.450000ms
[D 11:00:42.497 NotebookApp] 200 GET /user/<my-image>/static/components/bootstrap-tour/build/css/bootstrap-tour.min.css?v=..17 (35.191.12.161) 3.650000ms
[D 11:00:42.498 NotebookApp] 200 GET /user/<my-image>/static/notebook/css/override.css?v=..aa (35.191.12.165) 3.590000ms
[D 11:00:42.630 NotebookApp] 200 GET /user/<my-image>/static/components/text-encoding/lib/encoding.js?v=..ad (35.191.12.167) 1.750000ms
[D 11:00:42.653 NotebookApp] Path components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js
[D 11:00:42.657 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js?V=2.7.9 (35.191.12.161) 4.760000ms
[D 11:00:42.659 NotebookApp] 200 GET /user/<my-image>/static/notebook/js/main.min.js?v=7af39fe9ec3585e958e8637ace5e96fe0bf3ced51a8b8b5875c571f941299779fe746bf74f02bab0f3c6e08281d6ecadafcb5b08120e98f081c9c7dd824bdfdc (35.191.12.163) 28.320000ms
[D 11:00:42.686 NotebookApp] Path style/style.min.css.map served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/style/style.min.css.map
[D 11:00:42.688 NotebookApp] 200 GET /user/<my-image>/static/style/style.min.css.map (35.191.12.171) 2.180000ms
[D 11:00:43.190 NotebookApp] Path components/MathJax/config/Safe.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/config/Safe.js
[D 11:00:43.192 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/config/Safe.js?V=2.7.9 (35.191.12.161) 2.280000ms
[D 11:00:43.744 NotebookApp] Paths used for configuration of notebook: 
    	/etc/jupyter/nbconfig/notebook.json
[D 11:00:43.745 NotebookApp] Paths used for configuration of notebook: 
    	/usr/local/etc/jupyter/nbconfig/notebook.json
[D 11:00:43.746 NotebookApp] Paths used for configuration of notebook: 
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/notebook.d/jupyter-datawidgets.json
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/notebook.d/jupyter-threejs.json
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/notebook.d/jupyter_resource_usage.json
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/notebook.d/offlinenotebook_nbextension.json
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/notebook.d/widgetsnbextension.json
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/notebook.json
[D 11:00:43.747 NotebookApp] Paths used for configuration of notebook: 
    	/home/jovyan/.jupyter/nbconfig/notebook.json
[D 11:00:43.749 NotebookApp] 200 GET /user/<my-image>/api/config/notebook?_=1644231643272 (35.191.12.167) 6.580000ms
[D 11:00:43.751 NotebookApp] Using contents: services/contents
[W 11:00:43.760 NotebookApp] 404 GET /user/<my-image>/static/notebook/js/main.min.js.map (35.191.12.171) 10.150000ms referer=None
[D 11:00:43.761 NotebookApp] Paths used for configuration of common: 
    	/etc/jupyter/nbconfig/common.json
[D 11:00:43.762 NotebookApp] Paths used for configuration of common: 
    	/usr/local/etc/jupyter/nbconfig/common.json
[D 11:00:43.763 NotebookApp] Paths used for configuration of common: 
    	/srv/conda/envs/notebook/etc/jupyter/nbconfig/common.json
[D 11:00:43.764 NotebookApp] Paths used for configuration of common: 
    	/home/jovyan/.jupyter/nbconfig/common.json
[D 11:00:43.764 NotebookApp] 200 GET /user/<my-image>/api/config/common?_=1644231643273 (35.191.12.161) 13.820000ms
[D 11:00:43.766 NotebookApp] Path components/MathJax/extensions/TeX/newcommand.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/extensions/TeX/newcommand.js
[D 11:00:43.767 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/extensions/TeX/newcommand.js?V=2.7.9 (35.191.12.165) 1.680000ms
[D 11:00:43.775 NotebookApp] Path components/MathJax/extensions/MathML/content-mathml.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/extensions/MathML/content-mathml.js
[D 11:00:43.776 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/extensions/MathML/content-mathml.js?V=2.7.9 (35.191.12.163) 2.870000ms
[D 11:00:43.776 NotebookApp] Path components/MathJax/extensions/TeX/begingroup.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/extensions/TeX/begingroup.js
[D 11:00:43.777 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/extensions/TeX/begingroup.js?V=2.7.9 (35.191.12.169) 2.680000ms
[D 11:00:43.783 NotebookApp] Found kernel python3 in /srv/conda/envs/notebook/share/jupyter/kernels
[D 11:00:43.785 NotebookApp] 304 GET /user/<my-image>/api/kernelspecs (35.191.12.161) 2.400000ms
[D 11:00:43.815 NotebookApp] 200 GET /user/<my-image>/api/contents/Untitled.ipynb?type=notebook&_=1644231643274 (35.191.12.171) 9.940000ms
[D 11:00:43.819 NotebookApp] 304 GET /user/<my-image>/custom/custom.js?v=20220207110028 (35.191.12.165) 2.970000ms
[D 11:00:43.820 NotebookApp] Path components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js
[D 11:00:43.821 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js?V=2.7.9 (35.191.12.169) 3.180000ms
[D 11:00:43.821 NotebookApp] Path components/MathJax/extensions/Safe.js served from /srv/conda/envs/notebook/lib/python3.7/site-packages/notebook/static/components/MathJax/extensions/Safe.js
[D 11:00:43.822 NotebookApp] 200 GET /user/<my-image>/static/components/MathJax/extensions/Safe.js?V=2.7.9 (35.191.12.163) 3.530000ms
[D 11:00:43.856 NotebookApp] Path jupyter-datawidgets/extension.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-datawidgets/extension.js
[D 11:00:43.857 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-datawidgets/extension.js?v=20220207110028 (35.191.12.167) 2.000000ms
[D 11:00:43.873 NotebookApp] Path jupyter-js-widgets/extension.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-js-widgets/extension.js
[D 11:00:43.879 NotebookApp] Path jupyter_resource_usage/main.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter_resource_usage/main.js
[D 11:00:43.880 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter_resource_usage/main.js?v=20220207110028 (35.191.12.169) 3.760000ms
[D 11:00:43.880 NotebookApp] Path arcgis/extension.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/arcgis/extension.js
[D 11:00:43.881 NotebookApp] 200 GET /user/<my-image>/nbextensions/arcgis/extension.js?v=20220207110028 (35.191.12.163) 3.850000ms
[D 11:00:43.881 NotebookApp] Path jupyter-offlinenotebook/main.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-offlinenotebook/main.js
[D 11:00:43.882 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-offlinenotebook/main.js?v=20220207110028 (35.191.12.161) 4.140000ms
[D 11:00:43.882 NotebookApp] Path jupyter-threejs/extension.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-threejs/extension.js
[D 11:00:43.883 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-threejs/extension.js?v=20220207110028 (35.191.12.171) 4.600000ms
[D 11:00:43.885 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-js-widgets/extension.js?v=20220207110028 (35.191.12.165) 13.420000ms
[D 11:00:43.962 NotebookApp] Starting kernel: ['/srv/conda/envs/notebook/bin/python', '-m', 'ipykernel_launcher', '-f', '/home/jovyan/.local/share/jupyter/runtime/kernel-e10773be-d68e-4b52-8823-e9146684cc15.json']
[D 11:00:43.968 NotebookApp] Connecting to: tcp://127.0.0.1:60113
[D 11:00:43.969 NotebookApp] Connecting to: tcp://127.0.0.1:48163
[I 11:00:43.972 NotebookApp] Kernel started: e10773be-d68e-4b52-8823-e9146684cc15, name: python3
[D 11:00:43.972 NotebookApp] Kernel args: {'kernel_name': 'python3', 'cwd': '/home/jovyan'}
[D 11:00:43.979 NotebookApp] 201 POST /user/<my-image>/api/sessions (35.191.12.171) 38.500000ms
[D 11:00:43.981 NotebookApp] 200 GET /user/<my-image>/api/contents/Untitled.ipynb/checkpoints?_=1644231643275 (35.191.12.165) 6.010000ms
[D 11:00:43.983 NotebookApp] Serving kernel resource from: /srv/conda/envs/notebook/share/jupyter/kernels/python3
[D 11:00:43.984 NotebookApp] 200 GET /user/<my-image>/kernelspecs/python3/logo-64x64.png (35.191.12.169) 7.760000ms
[D 11:00:44.024 NotebookApp] 200 GET /user/<my-image>/api/metrics/v1 (35.191.12.163) 2.370000ms
[D 11:00:44.044 NotebookApp] Using contents: services/contents
[W 11:00:44.048 NotebookApp] 404 GET /user/<my-image>/nbextensions/arcgis/extension.js.map (35.191.12.167) 7.930000ms referer=None
[D 11:00:44.053 NotebookApp] Path jupyter-offlinenotebook/jslib/offlinenotebook.js served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js
[D 11:00:44.055 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js?v=20220207110028 (35.191.12.161) 2.380000ms
[D 11:00:44.272 NotebookApp] Path jupyter-js-widgets/extension.js.map served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-js-widgets/extension.js.map
[D 11:00:44.287 NotebookApp] Initializing websocket connection /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:00:44.290 NotebookApp] Requesting kernel info from e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.290 NotebookApp] Connecting to: tcp://127.0.0.1:56617
[D 11:00:44.317 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-js-widgets/extension.js.map (35.191.12.167) 46.420000ms
[D 11:00:44.321 NotebookApp] Path jupyter-offlinenotebook/jslib/offlinenotebook.js.map served from /srv/conda/envs/notebook/share/jupyter/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js.map
[D 11:00:44.324 NotebookApp] OfflineNotebook config:{"repoid": "", "binder_repo_label": "", "binder_ref_url": "", "binder_persistent_url": ""} 
[D 11:00:44.324 NotebookApp] 200 GET /user/<my-image>/offlinenotebook/config (35.191.12.171) 4.360000ms
[D 11:00:44.328 NotebookApp] 200 GET /user/<my-image>/nbextensions/jupyter-offlinenotebook/jslib/offlinenotebook.js.map (35.191.12.161) 8.860000ms
[D 11:00:44.588 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (starting)
[D 11:00:44.591 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:00:44.593 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:00:44.594 NotebookApp] Received kernel info: {'status': 'ok', 'protocol_version': '5.3', 'implementation': 'ipython', 'implementation_version': '7.26.0', 'language_info': {'name': 'python', 'version': '3.7.10', 'mimetype': 'text/x-python', 'codemirror_mode': {'name': 'ipython', 'version': 3}, 'pygments_lexer': 'ipython3', 'nbconvert_exporter': 'python', 'file_extension': '.py'}, 'banner': "Python 3.7.10 | packaged by conda-forge | (default, Oct 13 2021, 20:51:14) \nType 'copyright', 'credits' or 'license' for more information\nIPython 7.26.0 -- An enhanced Interactive Python. Type '?' for help.\n", 'help_links': [{'text': 'Python Reference', 'url': 'https://docs.python.org/3.7'}, {'text': 'IPython Reference', 'url': 'https://ipython.org/documentation.html'}, {'text': 'NumPy Reference', 'url': 'https://docs.scipy.org/doc/numpy/reference/'}, {'text': 'SciPy Reference', 'url': 'https://docs.scipy.org/doc/scipy/reference/'}, {'text': 'Matplotlib Reference', 'url': 'https://matplotlib.org/contents.html'}, {'text': 'SymPy Reference', 'url': 'http://docs.sympy.org/latest/index.html'}, {'text': 'pandas Reference', 'url': 'https://pandas.pydata.org/pandas-docs/stable/'}]}
[D 11:00:44.595 NotebookApp] 101 GET /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 (35.191.9.34) 308.230000ms
[D 11:00:44.596 NotebookApp] Opening websocket /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:00:44.596 NotebookApp] Getting buffer for e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.596 NotebookApp] Connecting to: tcp://127.0.0.1:48163
[D 11:00:44.597 NotebookApp] Connecting to: tcp://127.0.0.1:56617
[D 11:00:44.598 NotebookApp] Connecting to: tcp://127.0.0.1:60113
[D 11:00:44.599 NotebookApp] Connecting to: tcp://127.0.0.1:52113
[D 11:00:44.600 NotebookApp] Connecting to: tcp://127.0.0.1:56617
[D 11:00:44.601 NotebookApp] Nudge: attempt 1 on kernel e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.603 NotebookApp] Nudge: IOPub received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.603 NotebookApp] Nudge: resolving iopub future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.605 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:00:44.607 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:00:44.607 NotebookApp] Nudge: shell info reply received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.607 NotebookApp] Nudge: resolving shell future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:00:44.850 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:00:44.855 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:00:44.857 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:00:44.860 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:00:49.061 NotebookApp] 200 GET /user/<my-image>/api/metrics/v1 (35.191.12.165) 1.550000ms
[D 11:00:50.020 NotebookApp] 200 GET /user/<my-image>/static/base/images/favicon-notebook.ico (35.191.12.163) 1.500000ms
[D 11:00:54.074 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.163) 1.560000ms
[D 11:00:59.054 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.161) 1.410000ms
[D 11:01:04.042 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.161) 1.570000ms
[D 11:01:09.054 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.165) 1.390000ms
[D 11:01:14.073 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.171) 1.510000ms
[D 11:01:14.599 NotebookApp] Websocket closed e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[I 11:01:14.599 NotebookApp] Starting buffering for e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[D 11:01:14.599 NotebookApp] Clearing buffer for e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:15.951 NotebookApp] Initializing websocket connection /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:01:15.953 NotebookApp] 101 GET /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 (35.191.19.102) 2.720000ms
[D 11:01:15.953 NotebookApp] Opening websocket /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:01:15.954 NotebookApp] Getting buffer for e10773be-d68e-4b52-8823-e9146684cc15
[I 11:01:15.954 NotebookApp] Restoring connection for e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[D 11:01:15.954 NotebookApp] Connecting to: tcp://127.0.0.1:56617
[D 11:01:15.955 NotebookApp] Nudge: attempt 1 on kernel e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:15.958 NotebookApp] Nudge: IOPub received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:15.958 NotebookApp] Nudge: resolving iopub future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:15.960 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:01:15.960 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:01:15.961 NotebookApp] Nudge: shell info reply received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:15.961 NotebookApp] Nudge: resolving shell future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:16.064 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:01:16.066 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:01:19.063 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.163) 1.440000ms
[D 11:01:24.073 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.167) 1.450000ms
[D 11:01:29.054 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.165) 1.460000ms
[D 11:01:34.092 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.171) 1.830000ms
[D 11:01:39.065 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.169) 1.400000ms
[D 11:01:44.076 NotebookApp] 200 GET /user/<my-image>/api/metrics/v1 (35.191.12.171) 1.610000ms
[D 11:01:45.956 NotebookApp] Websocket closed e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[I 11:01:45.957 NotebookApp] Starting buffering for e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[D 11:01:45.958 NotebookApp] Clearing buffer for e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:47.344 NotebookApp] Initializing websocket connection /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:01:47.347 NotebookApp] 101 GET /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 (35.191.11.82) 4.140000ms
[D 11:01:47.348 NotebookApp] Opening websocket /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:01:47.349 NotebookApp] Getting buffer for e10773be-d68e-4b52-8823-e9146684cc15
[I 11:01:47.350 NotebookApp] Restoring connection for e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[D 11:01:47.350 NotebookApp] Connecting to: tcp://127.0.0.1:56617
[D 11:01:47.352 NotebookApp] Nudge: attempt 1 on kernel e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:47.357 NotebookApp] Nudge: IOPub received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:47.358 NotebookApp] Nudge: resolving iopub future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:47.359 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:01:47.360 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:01:47.361 NotebookApp] Nudge: shell info reply received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:47.361 NotebookApp] Nudge: resolving shell future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:01:47.433 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:01:47.435 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:01:49.055 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.161) 1.780000ms
[D 11:01:54.073 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.163) 1.420000ms
[D 11:01:59.072 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.161) 1.790000ms
[D 11:02:04.083 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.167) 1.860000ms
[D 11:02:09.051 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.169) 1.450000ms
[D 11:02:14.064 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.169) 1.490000ms
[D 11:02:17.352 NotebookApp] Websocket closed e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[I 11:02:17.352 NotebookApp] Starting buffering for e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[D 11:02:17.353 NotebookApp] Clearing buffer for e10773be-d68e-4b52-8823-e9146684cc15
[D 11:02:18.960 NotebookApp] Initializing websocket connection /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:02:18.962 NotebookApp] 101 GET /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels?session_id=2100c8dccdb346c58eec62beb1623a24 (35.191.12.176) 2.570000ms
[D 11:02:18.962 NotebookApp] Opening websocket /user/<my-image>/api/kernels/e10773be-d68e-4b52-8823-e9146684cc15/channels
[D 11:02:18.962 NotebookApp] Getting buffer for e10773be-d68e-4b52-8823-e9146684cc15
[I 11:02:18.962 NotebookApp] Restoring connection for e10773be-d68e-4b52-8823-e9146684cc15:2100c8dccdb346c58eec62beb1623a24
[D 11:02:18.962 NotebookApp] Connecting to: tcp://127.0.0.1:56617
[D 11:02:18.963 NotebookApp] Nudge: attempt 1 on kernel e10773be-d68e-4b52-8823-e9146684cc15
[D 11:02:18.966 NotebookApp] Nudge: IOPub received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:02:18.966 NotebookApp] Nudge: resolving iopub future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:02:18.967 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:02:18.968 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:02:18.968 NotebookApp] Nudge: shell info reply received: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:02:18.969 NotebookApp] Nudge: resolving shell future: e10773be-d68e-4b52-8823-e9146684cc15
[D 11:02:19.032 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.163) 2.170000ms
[D 11:02:19.062 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (busy)
[D 11:02:19.063 NotebookApp] activity on e10773be-d68e-4b52-8823-e9146684cc15: status (idle)
[D 11:02:24.073 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.167) 1.530000ms
[D 11:02:29.081 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.163) 1.410000ms
[D 11:02:34.044 NotebookApp] 304 GET /user/<my-image>/api/metrics/v1 (35.191.12.165) 1.500000ms

I have tried different browsers, same result. I am not aware of any other firewalls or network redirections that might interfere.

Frankly I don’t see any clues from the logs, just this repeating message:

[D 11:01:45.956 NotebookApp] Websocket closed 
[I 11:01:45.957 NotebookApp] Starting buffering 
[D 11:01:45.958 NotebookApp] Clearing buffer

Or have I missed something?

Ok, I have found the solution, it’s all explained in this post

GCP LoadBalancers are not by default configured to handle websockets and optimized for http calls, because by default the load balancers have 30 second timeouts in place that causes connections to close.

As explained there the solution is to create a BackendConfig, and let the proxy-public service know about it. For example in our values.yaml

  proxy:
    service:
      annotations:
        beta.cloud.google.com/backend-config: '{"ports": {"80":"proxy-public-backendconfig"}}'
2 Likes