Kubespawner cert Error

JupyterHub
,
1

Hi all,

im getting the following Error and trying to fix it for weeks now. Hope someone can Help me on this.

[D 2022-06-30 10:36:11.183 JupyterHub reflector:281] Connecting events watcher
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-19' coro=<ResourceReflector.start() done, defined at /usr/local/lib/python3.9/site-packages/kubespawner/reflector.py:370> exception=ClientOSError(1, '[X509: CERT_ALREADY_IN_HASH_TABLE] cert already in hash table (_ssl.c:2633)')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kubespawner/reflector.py", line 383, in start
    await self._list_and_update()
  File "/usr/local/lib/python3.9/site-packages/kubespawner/reflector.py", line 228, in _list_and_update
    initial_resources_raw = await list_method(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/kubernetes_asyncio/client/api_client.py", line 185, in __call_api
    response_data = await self.request(
  File "/usr/local/lib/python3.9/site-packages/kubernetes_asyncio/client/rest.py", line 193, in GET
    return (await self.request("GET", url,
  File "/usr/local/lib/python3.9/site-packages/kubernetes_asyncio/client/rest.py", line 177, in request
    r = await self.pool_manager.request(**args)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 898, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ClientOSError: [Errno 1] [X509: CERT_ALREADY_IN_HASH_TABLE] cert already in hash table (_ssl.c:2633)

I already disabled interlan_ssl and kubespawner ssl but it doesn’t affect the error.

Thanks,
Endrit

2

Hi! Please could you give us enough information to reproduce your problem? For example:

  • How is your K8S cluster configured?
  • What versions of JupyterHub and KubeSpawner are you using- are you using Z2JH or something else?
  • Can you show us your configuration with secrets redacted
  • What triggers the above errors, and does it break your deployment or is it a harmless error?
4

The normal recommendation for running JupyterHub on K8s is to use the official Helm chart. Zero to JupyterHub with Kubernetes — Zero to JupyterHub with Kubernetes documentation . It doesn’t work straight away with OpenShift, but there are several posts on this forum about getting it to work.

Regardless, you should be able to get this working without Z2JH using your own manifests, but you’ll have to do some investigation yourself. For example, try simplifying your deployment by removing as much configuration as possible.

JupyterHub uses a local sqlite database by default. Maybe it’s not being written to a persistent volume?

One obvious issue is that your Kubernetes version 1.11.0 is extremely old, KubeSpawner is tested on 1.20+.

1 Like