503 : service unavailable (services)

JupyterHub
,
1

Hi,

I upgraded jupyterhub to 1.3.0 (jinja2 2.11.3), but all my services stopped showing up. For example the jupyterhub_announcement service, I get 503 : service unavailable and in the log file:

15:34:30.180 [ConfigProxy] debug: PROXY WEB /services/announcement/ to http://127.0.0.1:8888
jupyterhub[31684]: ERROR:tornado.application:Uncaught exception
jupyterhub[31684]: Traceback (most recent call last):
jupyterhub[31684]: File "/usr/local/lib64/python3.6/site-packages/tornado/http1connection.py", line 273, in _read_message
jupyterhub[31684]: delegate.finish()
jupyterhub[31684]: File "/usr/local/lib64/python3.6/site-packages/tornado/routing.py", line 268, in finish
jupyterhub[31684]: self.delegate.finish()
jupyterhub[31684]: File "/usr/local/lib64/python3.6/site-packages/tornado/web.py", line 2297, in finish
jupyterhub[31684]: self.execute()
jupyterhub[31684]: File "/usr/local/lib64/python3.6/site-packages/tornado/web.py", line 2317, in execute
jupyterhub[31684]: self.application, self.request, **self.handler_kwargs
jupyterhub[31684]: File "/usr/local/lib64/python3.6/site-packages/tornado/web.py", line 233, in __init__
jupyterhub[31684]: self.initialize(**kwargs)  # type: ignore
jupyterhub[31684]: File "/usr/local/lib/python3.6/site-packages/jupyterhub_announcement/announcement.py", line 132, in initialize
jupyterhub[31684]: self.template = self.env.get_template("index.html")
jupyterhub[31684]: File "/home/jupyterhub/.local/lib/python3.6/site-packages/jinja2/environment.py", line 883, in get_template
jupyterhub[31684]: return self._load_template(name, self.make_globals(globals))
jupyterhub[31684]: File "/home/jupyterhub/.local/lib/python3.6/site-packages/jinja2/environment.py", line 857, in _load_template
jupyterhub[31684]: template = self.loader.load(self, name, globals)
jupyterhub[31684]: File "/home/jupyterhub/.local/lib/python3.6/site-packages/jinja2/loaders.py", line 429, in load
jupyterhub[31684]: raise TemplateNotFound(name)
jupyterhub[31684]: jinja2.exceptions.TemplateNotFound: index.html
jupyterhub[31684]: 15:34:30.183 [ConfigProxy] error: 503 GET /services/announcement/ socket hang up
jupyterhub[31684]: [D 2021-04-08 15:34:30.186 JupyterHub pages:648] No template for 503

I got the same error with similar services. Downgrading jinja2 to 2.10.1 for example fixes temporary the problem.

Any idea?

1 Like
2

If you have time would you mind figuring out which version of jinja2 first broke things?

It would also be helpful if you could try to reproduce this with a fresh installation of JupyterHub (in a new virtualenv).

3

@manics the version of jinja2 that broke is 2.11.3. Actually, it was upgraded automatically when upgrading jupyterhub.

4

Iā€™m getting similar errors having accidentally upgraded to beyond jupyterhub 1.3.

Aug 30 14:29:23  jupyterhub[5755]:     Traceback (most recent call last):
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/tornado/web.py", line 1704, in _execute
Aug 30 14:29:23  jupyterhub[5755]:         result = await result
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jupyterhub/handlers/login.py", line 135, in get
Aug 30 14:29:23  jupyterhub[5755]:         self.finish(self._render(username=username))
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jupyterhub/handlers/login.py", line 102, in _render
Aug 30 14:29:23  jupyterhub[5755]:         {'next': self.get_argument('next', '')},
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 1176, in render_template
Aug 30 14:29:23  jupyterhub[5755]:         template = self.get_template(name)
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 1170, in get_template
Aug 30 14:29:23  jupyterhub[5755]:         return self.settings['jinja2_env'].get_template(name)
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jinja2/environment.py", line 883, in get_template
Aug 30 14:29:23  jupyterhub[5755]:         return self._load_template(name, self.make_globals(globals))
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jinja2/environment.py", line 857, in _load_template
Aug 30 14:29:23  jupyterhub[5755]:         template = self.loader.load(self, name, globals)
Aug 30 14:29:23  jupyterhub[5755]:       File "/etc/jupyterhub/.venv/lib/python3.6/site-packages/jinja2/loaders.py", line 429, in load
Aug 30 14:29:23  jupyterhub[5755]:         raise TemplateNotFound(name)
Aug 30 14:29:23  jupyterhub[5755]:     jinja2.exceptions.TemplateNotFound: login.html

I tried downgrading the environment to jupyterhub < 1.3 and jinja < 2.11.3 but this persists

5

Never mind, had a bunch of directory permission issues that needed resolving.

1 Like