Bad config encountered during initialization: The authenticator_class trait of... instance must be a type, but jhub_remote_user_authenticator... could not be imported

Hello together,
I have extended the Jupyterhub hub using a Dockerfile. I am attaching the code below.
For a while I had no problems with it. Then I had to reset and rebuild the Kubernetes cluster. The jhub_remote_user_authenticator used to be recognized, but now I get the following error message. (See below)
I changed jhub_remote_user_authenticator a bit, why it is copied locally and then installed.
I hope you guys can help me further. Thanks a lot in advance.

kubectl -n jhub logs hub-65f9bb8d6-4cll5

[D 2022-10-05 08:26:23.362 JupyterHub application:837] Looking for /usr/local/etc/jupyterhub/jupyterhub_config in /srv/jupyterhub
Loading /usr/local/etc/jupyterhub/secret/values.yaml
No config at /usr/local/etc/jupyterhub/existing-secret/values.yaml
Loading extra config:
[D 2022-10-05 08:26:23.641 JupyterHub application:858] Loaded config file: /usr/local/etc/jupyterhub/
[C 2022-10-05 08:26:23.693 JupyterHub application:112] Bad config encountered during initialization: The 'authenticator_class' trait of < object at 0x7f8551791910> instance must be a type, but 'jhub_remote_user_authenticator.remote_user_auth.RemoteUserAuthenticator' could not be imported
[D 2022-10-05 08:26:23.693 JupyterHub application:113] Config at the time: {'JupyterHub': {'config_file': '/usr/local/etc/jupyterhub/', 'log_level': 10, 'upgrade_db': True}}
[D 2022-10-05 08:26:23.693 JupyterHub application:963] Exiting application: jupyterhub


ARG JUPYTERHUB_K8S_VERSION=1.1.3-n721.h0acbcf43

USER root

RUN mkdir -p /jhub_remote_user_authenticator

COPY jhub_remote_user_authenticator/ /jhub_remote_user_authenticator/

RUN pip3 install --upgrade --force-reinstall /jhub_remote_user_authenticator

RUN mkdir -p /templates
COPY templates/* /templates/


Can you tell us exactly what’s changed between the two deployments? For example are you using the same container image or did you rebuild it? If you rebuilt it are you using the same versions of all components including jhub_remote_user_authenticator or have you updated something?

No, I haven’t actually changed anything, at least I wouldn’t know. I use the same yaml files with the same versions. The only thing that changed is that I don’t build the docker image locally anymore, I build it on gitlab now.

It sounds like jhub_remote_user_authenticator wasn’t installed properly. You could try running the Docker container manually, and check whether it’s importable.

I connected to the container and tested this out.

pip3 list lists this:
“jhub-remote-user-authenticator 0.1.0”

In Python itself, import jhub_remote_user_authenticator does not make an error.

I checked everything again and remembered that I had to change once so that I can build the Docker image locally and use jhub_remote_user_authenticator. I undid the change and the authenticator is recognised.
But now I have a different error message.
Should I create a new thread for this one because I’m stuck there too?

[D 2022-10-08 08:22:38.496 JupyterHub application:837] Looking for /usr/local/etc/jupyterhub/jupyterhub_config in /srv/jupyterhub
Loading /usr/local/etc/jupyterhub/secret/values.yaml
No config at /usr/local/etc/jupyterhub/existing-secret/values.yaml
Loading extra config:
[D 2022-10-08 08:22:38.788 JupyterHub application:858] Loaded config file: /usr/local/etc/jupyterhub/
[I 2022-10-08 08:22:38.847 JupyterHub app:2771] Running JupyterHub version 2.3.1
[I 2022-10-08 08:22:38.847 JupyterHub app:2801] Using Authenticator: jhub_remote_user_authenticator.remote_user_auth.RemoteUserAuthenticator
[I 2022-10-08 08:22:38.847 JupyterHub app:2801] Using Spawner: kubespawner.spawner.KubeSpawner-4.1.0
[I 2022-10-08 08:22:38.847 JupyterHub app:2801] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-2.3.1
[D 2022-10-08 08:22:38.848 JupyterHub app:1775] Connecting to db: sqlite:///jupyterhub.sqlite
[I 2022-10-08 08:22:38.861 JupyterHub dbutil:130] Upgrading sqlite:///jupyterhub.sqlite
[I 2022-10-08 08:22:38.861 JupyterHub dbutil:99] Backing up jupyterhub.sqlite => jupyterhub.sqlite.2022-10-08-082238
[I 2022-10-08 08:22:39.250 alembic.runtime.migration migration:204] Context impl SQLiteImpl.
[I 2022-10-08 08:22:39.251 alembic.runtime.migration migration:207] Will assume non-transactional DDL.
[E 2022-10-08 08:22:39.254 alembic.util.messaging messaging:65] Can't locate revision identified by '651f5419b74d'
FAILED: Can't locate revision identified by '651f5419b74d'
[E 2022-10-08 08:22:39.303 JupyterHub app:3297]
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/site-packages/jupyterhub/", line 3294, in launch_instance_async
        await self.initialize(argv)
      File "/usr/local/lib/python3.9/site-packages/jupyterhub/", line 2813, in initialize
      File "/usr/local/lib/python3.9/site-packages/jupyterhub/", line 1777, in init_db
        dbutil.upgrade_if_needed(self.db_url, log=self.log)
      File "/usr/local/lib/python3.9/site-packages/jupyterhub/", line 135, in upgrade_if_needed
      File "/usr/local/lib/python3.9/site-packages/jupyterhub/", line 84, in upgrade
        check_call(['alembic', '-c', alembic_ini, 'upgrade', revision])
      File "/usr/local/lib/python3.9/", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['alembic', '-c', '/tmp/tmpl6mcn2a1/alembic.ini', 'upgrade', 'head']' returned non-zero exit status 255.

[D 2022-10-08 08:22:39.304 JupyterHub application:961] Exiting application: jupyterhub

Sounds like something has gone wrong with your database. Can you try running your server without an existing database (so a new one is created), if that works then we can try and work out what’s gone wrong during the upgrade.

I made a copy of jupyterhub.sqlite and then deleted jupyterhub.sqlite. Jupyterhub then started without any problems.

What version of JupyterHub was used to create the old database file?

That’s the funny thing. I have not changed the version. It was always 1.1.3-n721.h0acbcf43.

RUN pip3 install --upgrade --force-reinstall /jhub_remote_user_authenticator

I suspect this caused jupyterhub to be upgraded because it was a dependency somewhere, and then you had issues related to the switch to a new JupyterHub version.

1 Like