Jubyterhub ads integration failing

JupyterHub
1

Hi,

I’m new to jupyterhub and i installed jupyterhub through helm on my open source kubernetes cluster. I’m trying to integrate ads authentication for my jupyterhub login.
It say’s invalid CA public key file. I have a adsldap-combined.pem file in /etc/openldap/certs directory. The same file is working for rancher ads authentication and vault authetication with proper ads details.
But the same thing is failing for jupyterhub authentication. Below is the error.

[E 2025-05-12 15:51:00.264 JupyterHub web:1875] Uncaught exception POST /hub/login?next=%2Fhub%2F (::ffff:10.245.26.192)
    HTTPServerRequest(protocol='http', host='aeamxpv600:8273', method='POST', uri='/hub/login?next=%2Fhub%2F', version='HTTP/1.1', remote_ip='::ffff:10.245.26.192')
    Traceback (most recent call last):
      File "/usr/local/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/jupyterhub/handlers/login.py", line 164, in post
        user = await self.login_user(data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/jupyterhub/handlers/base.py", line 964, in login_user
        authenticated = await self.authenticate(data)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/jupyterhub/auth.py", line 688, in get_authenticated_user
        authenticated = await maybe_future(self.authenticate(handler, data))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/ldapauthenticator/ldapauthenticator.py", line 613, in authenticate
        resolved_username, resolved_dn = self.resolve_username(login_username)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/ldapauthenticator/ldapauthenticator.py", line 425, in resolve_username
        conn = self.get_connection(
               ^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/ldapauthenticator/ldapauthenticator.py", line 524, in get_connection
        tls = Tls(**self.tls_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.12/site-packages/ldap3/core/tls.py", line 97, in __init__
        raise LDAPSSLConfigurationError('invalid CA public key file')
    ldap3.core.exceptions.LDAPSSLConfigurationError: invalid CA public key file
type or paste code here

ADS configuration details:

hub:
  revisionHistoryLimit:
  config:
    JupyterHub:
#      admin_access: true
      authenticator_class: ldapauthenticator.LDAPAuthenticator
    LDAPAuthenticator:
      bind_dn_template:
        - CN=xx-system,OU=ServiceAccounts,OU=Process,DC=ads,DC=abc,DC=com
      escape_userdn: false
      lookup_dn: true
      tls_strategy: on_connect
      lookup_dn_search_filter: ({login_attr}={login})
      lookup_dn_search_password: <password>
      server_address: ldaps://adserver.com
      server_port: 636
      user_attribute: sAMAccountName
      user_search_base: dc=ads,dc=abc,dc=com
      user_search_filter: '({login_attr}={login})'
      group_search_filter: dc=ads,dc=abc,dc=com
      use_ssl: true

      tls_kwargs: {
      "ca_certs_file": "/etc/openldap/certs/adsldap-combined.pem"
      }
2

LDAPAuthenticator uses ldap3 under the hood and the error you are seeing comes from here. The error in your case is due to the file does not exist at the path you specified. Did you create a Secret and mounted it on the hub pod?

1 Like
3

Hi Mahendra,

Thanks for the reply.
Why do we need to mount the secret, if we are passing the ad configuration details in the hub config in values.yaml file and running helm install?

4

The hub runs in a pod/container, which by default has no access to the host filesystem. You’ll therefore need to make the CA available inside the container. Mounting a secret or volume is one way, another is to rebuild the image to incorporate your CA.

1 Like
5

Is there any specific path i need to mount the CA file inside the hub pod?

6

After mounting the CA inside the pod, i’m getting below error.

[D 2025-05-13 22:19:27.809 JupyterHub ldapauthenticator:532] Attempting to bind svc-id
[D 2025-05-13 22:19:28.476 JupyterHub ldapauthenticator:559] Successfully bound svc-id
[D 2025-05-13 22:19:28.476 JupyterHub ldapauthenticator:446] Looking up user with:
        search_base = 'dc=ads,dc=abc,dc=com'
        search_filter = '(sAMAccountName=<username>)'
        attributes = '[sAMAccountName]'
[D 2025-05-13 22:19:28.514 JupyterHub ldapauthenticator:532] Attempting to bind CN=svc-id,OU=ServiceAccounts,OU=Process,DC=ads,DC=abc,DC=com
[D 2025-05-13 22:19:28.762 JupyterHub ldapauthenticator:550] Failed to bind CN=svc.acxdvldap-system,OU=ServiceAccounts,OU=Process,DC=ads,DC=abc,DC=com
    LDAPBindError: automatic bind not successful - invalidCredentials
[W 2025-05-13 22:19:28.762 JupyterHub ldapauthenticator:638] Failed to bind user 'username' to an LDAP user.
[W 2025-05-13 22:19:28.762 JupyterHub base:979] Failed login for username
7

Are you able to connect to the server with those parameters using the ldapsearch command line utility?

8

@manics Yes..I’m able to do ldapsearch with service id & password which i passed to the Jupyter hub for ad integration. I suspect .pem file which i created as a secret is not taken properly by the Jupter hub pod.

hub:
  extraVolumes:
    - name: adsldap-cert
      secret:
        secretName: ad-cert-secret-jupyterhub
  extraVolumeMounts:
    - name: adsldap-cert
      mountPath: /etc/ssl/certs
      readOnly: true
9

Is the pem file just for the CA (it sounds like you’ve fixed that problem), or is it needed for something else?

10

This combined pem file is not picked by the hub. I tried created a secret, but there is no proper variable for the pem file to be passed inside the jupyter hub config. I tried to copy the pem file to a directory inside the container, by building customized docker image. That is also not working.

11

Could you post the manifests on how you are trying to create secret and mouting it on hub pods? That will help to debug the issue!

12 
kubectl create secret generic ad-cert-secret-jupyterhub --from-file=adsldap-combined.pem=/etc/openldap/certs/adsldap-combined.pem

NAME                                 TYPE                 DATA   AGE
ad-cert-secret-jupyterhub            Opaque               1      11d

hub:
  extraVolumes:
    - name: adsldap-cert
      secret:
        secretName: ad-cert-secret-jupyterhub
  extraVolumeMounts:
    - name: adsldap-cert
      mountPath: /app/certs/adsldap-combined.pem
      subPath: adsldap-combined.pem
      readOnly: true
  revisionHistoryLimit:
  config:
    debug:
      enabled: true
    JupyterHub:
#      admin_access: true
#      authenticator_class: ldapauthenticator.LDAPAuthenticator
13

Can you tell us which errors or behaviour indicates the CA is the problem? In your first post you reported the error

but it sounds like that error has gone.

14

@manics When i use below config, it throws that error.

#      tls_kwargs: {
#      "ca_certs_file": "/app/certs/adsldap-combined.pem"
#      }
15

I copied the adsldap-combined.pem file to the container and used the below ldap parameter.
ldap_tls_cacert: /app/certs/adsldap-combined.pem

16

Can you start a shell inside your hub pod kubectl exec -it deploy/hub -- bash, then:

  • check that CA file is readable, and that it is correctly formatted
    ls -l /app/certs/adsldap-combined.pem
cat /app/certs/adsldap-combined.pem
  • If it is then inside the pod start a python shell/REPL and try to connect to your LDAP/AD server using SSL and TLS — ldap3 2.10.2 documentation as a guide
    import ldap3
tls = ldap3.Tls(ca_certs_file="/app/certs/adsldap-combined.pem")
server = ldap3.Server(..., tls=tls)
connection = ldap3.Connection(server, ...)
1 Like
17

I get output as None when i use the below code.

import ldap3
tls = ldap3.Tls(ca_certs_file="/app/certs/adsldap-combined.pem")
server = ldap3.Server('abc.ads.com', use_ssl=True, port=636, tls=tls)
connection = ldap3.Connection(server, ...)
connection.open()
print(connection.extend.standard.who_am_i())
18

That means the certificate is good and you can connect to LDAP server. Are you making an anonymous connection or did you use user credentials? So, when you use with JupyterHub what sort of errors do you get? Could you post the logs of hub in DEBUG mode?

19

@mahendrapaipuri I use service id & password of the ads server for authentication. For login to Jupyter hub i use my ads credentials.

As per manics , If i use svc-id & password in the above script inside the hub pod, i get an error saying invalid server address.

[D 2025-05-21 16:08:30.119 JupyterHub ldapauthenticator:532] Attempting to bind svc-id
[D 2025-05-21 16:08:30.893 JupyterHub ldapauthenticator:559] Successfully bound svc-id
[D 2025-05-21 16:08:30.893 JupyterHub ldapauthenticator:446] Looking up user with:
        search_base = 'DC=ads,DC=abc,DC=com'
        search_filter = '(sAMAccountName= userid)'
        attributes = '[sAMAccountName]'
[D 2025-05-21 16:08:30.931 JupyterHub ldapauthenticator:532] Attempting to bind CN=svc-id,OU=ServiceAccounts,OU=Process,DC=ads,DC=abc,DC=com
[D 2025-05-21 16:08:31.192 JupyterHub ldapauthenticator:550] Failed to bind CN=svc-id,OU=ServiceAccounts,OU=Process,DC=ads,DC=abc,DC=com
    LDAPBindError: automatic bind not successful - invalidCredentials
[W 2025-05-21 16:39:10.469 JupyterHub ldapauthenticator:638] Failed to bind user yyy to an LDAP user
20

Seems like your service account svc-id bound to the LDAP successfully from this log Successfully bound svc-id. It is your user account that is failing. I think you might need to look into your configuration provided to LDAPAuthenticator to ensure if it is looking for the user in the right directory.

1 Like