ldap3.core.exceptions.LDAPSocketReceiveError: with 500 : Internal Server Error

JupyterHub Zero to JupyterHub on Kubernetes
1

Hello everyone,

I followed the instruction(Zero to JupyterHub with Kubernetes — Zero to JupyterHub with Kubernetes documentation) to deploy JupyterHub in our on-premise K8s infra. The JupyterHub works pretty smoothly. But I couldn’t authenticate by my AD server.

Here is my config.yaml, I have tested my AD server information successfully by ldapsearch :

hub:
  config:
    JupyterHub:
      authenticator_class: ldapauthenticator.LDAPAuthenticator
    LDAPAuthenticator:
      bind_dn_template:
         - cn={username},dc=mydc1,dc=mydc2,dc=mydc3,dc=com
      use_ssl: false
      server_port: 389
      escape_userdn: false
      lookup_dn: true
      lookup_dn_search_filter: ({login_attr}={login})
      lookup_dn_search_password: mypassword
      lookup_dn_search_user: cn=ad-query_account,cn=users,dc=mydc1,dc=mydc2,dc=mydc3,dc=com
      lookup_dn_user_dn_attribute: cn
      server_address: mydc1.mydc2.mydc3.com
      user_attribute: sAMAccountName
      user_search_base: dc=mydc1,dc=mydc2,dc=mydc3,dc=com
  db:
    type: sqlite-memory
singleuser:
  defaultUrl: "/lab"
  storage:
    type: none

This is my Helm command:

RELEASE=jhub
NAMESPACE=jhub

helm upgrade --cleanup-on-fail
–install $RELEASE jupyterhub/jupyterhub
–namespace $NAMESPACE
–create-namespace
–version=1.1.3
–values config.yaml

I can see the login portal but got 500 : Internal Server Error after login by my Microsoft AD credential.

Here is the log from kubectl logs:

[I 2021-09-23 09:54:26.710 JupyterHub log:189] 200 GET /hub/api/users (cull-idle@::1) 14.62ms
[I 2021-09-23 09:54:46.577 JupyterHub log:189] 302 GET / -> /hub/ (@::ffff:192.168.145.0) 1.44ms
[I 2021-09-23 09:54:46.586 JupyterHub log:189] 302 GET /hub/ -> /hub/login?next=%2Fhub%2F (@::ffff:192.168.145.0) 1.20ms
[I 2021-09-23 09:54:46.633 JupyterHub log:189] 200 GET /hub/login?next=%2Fhub%2F (@::ffff:192.168.145.0) 40.64ms
[E 2021-09-23 09:54:58.841 JupyterHub web:1789] Uncaught exception POST /hub/login?next=%2Fhub%2F (::ffff:192.168.145.0)
    HTTPServerRequest(protocol='http', host='10.138.1.181:30615', method='POST', uri='/hub/login?next=%2Fhub%2F', version='HTTP/1.1', remote_ip='::ffff:192.168.145.0')
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/tornado/web.py", line 1704, in _execute
        result = await result
      File "/usr/local/lib/python3.8/dist-packages/jupyterhub/handlers/login.py", line 151, in post
        user = await self.login_user(data)
      File "/usr/local/lib/python3.8/dist-packages/jupyterhub/handlers/base.py", line 754, in login_user
        authenticated = await self.authenticate(data)
      File "/usr/local/lib/python3.8/dist-packages/jupyterhub/auth.py", line 469, in get_authenticated_user
        authenticated = await maybe_future(self.authenticate(handler, data))
      File "/usr/local/lib/python3.8/dist-packages/ldapauthenticator/ldapauthenticator.py", line 361, in authenticate
        username, resolved_dn = self.resolve_username(username)
      File "/usr/local/lib/python3.8/dist-packages/ldapauthenticator/ldapauthenticator.py", line 236, in resolve_username
        conn = self.get_connection(
      File "/usr/local/lib/python3.8/dist-packages/ldapauthenticator/ldapauthenticator.py", line 314, in get_connection
        conn = ldap3.Connection(
      File "/usr/local/lib/python3.8/dist-packages/ldap3/core/connection.py", line 363, in __init__
        self._do_auto_bind()
      File "/usr/local/lib/python3.8/dist-packages/ldap3/core/connection.py", line 391, in _do_auto_bind
        if self.start_tls(read_server_info=False):
      File "/usr/local/lib/python3.8/dist-packages/ldap3/core/connection.py", line 1314, in start_tls
        if self.server.tls.start_tls(self) and self.strategy.sync:  # for asynchronous connections _start_tls is run by the strategy
      File "/usr/local/lib/python3.8/dist-packages/ldap3/core/tls.py", line 280, in start_tls
        return self._start_tls(connection)
      File "/usr/local/lib/python3.8/dist-packages/ldap3/core/tls.py", line 289, in _start_tls
        raise start_tls_exception_factory(e)(connection.last_error)
    ldap3.core.exceptions.LDAPStartTLSError: wrap socket error: [Errno 104] Connection reset by peer
    
[E 2021-09-23 09:54:58.874 JupyterHub log:181] {
      "X-Forwarded-Host": "10.138.1.181:30615",
      "X-Forwarded-Proto": "http",
      "X-Forwarded-Port": "30615",
      "X-Forwarded-For": "::ffff:192.168.145.0",
      "Accept-Language": "en-US,en;q=0.9",
      "Accept-Encoding": "gzip, deflate",
      "Referer": "http://10.138.1.181:30615/hub/login?next=%2Fhub%2F",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36",
      "Content-Type": "application/x-www-form-urlencoded",
      "Origin": "http://10.138.1.181:30615",
      "Upgrade-Insecure-Requests": "1",
      "Cache-Control": "max-age=0",
      "Content-Length": "40",
      "Connection": "close",
      "Host": "10.138.1.181:30615"
    }

I am very new to K8s & Jupyter, any help is appreciated!

Thanks

2

Unfortunately JupyterHub LDAP can be very hard to debug as it’s dependent on your LDAP/AD server config. You can try turning on debug logging to see if that gives you more information.

Otherwise try searching this forum for other posts: e.g. Search results for 'ldap3' - Jupyter Community Forum