Jupyterhub + ldapauthenticator + HashiCorp Vault to manage shared and dynamic secrets

JupyterHub
,
1

Hello everyone,

I hope you’re doing well. I wanted to share some progress I’ve made with our Jupyterhub setup, particularly regarding LDAP authentication. Users with access can now log in and initiate container instances successfully.

However, I’m currently facing a challenge as I collaborate with my team: we need a secure way to share secrets and API keys within these containers. While researching solutions, I came across HashiCorp’s Vault, a tool that centralizes and dynamically manages secrets. It seems like a promising solution for our needs.

My goal is to set up the Vault client inside the spawned container using LDAP user credentials, like this:

# Initialize the HVAC client
client = hvac.Client(url='https://vault-server-url', token='your-vault-token')

# Authenticate HVAC client using LDAP user credentials
def authenticate_ldap(username, password):
    client.auth.ldap.login(username=username, password=password)
    if client.is_authenticated():
        return True
    else:
        return False

ldap_username = 'ldap-username'
ldap_password = 'ldap-password'

Hello everyone,

I hope you’re doing well. I wanted to share some progress I’ve made with our Jupyterhub setup, particularly regarding LDAP authentication. Users with access can now log in and initiate container instances successfully.

However, I’m currently facing a challenge as I collaborate with my team: we need a secure way to share secrets and API keys within these containers. While researching solutions, I came across HashiCorp’s Vault, a tool that centralizes and dynamically manages secrets. It seems like a promising solution for our needs.

My goal is to set up the Vault client inside the spawned container using LDAP user credentials, like this:

python

# Initialize the HVAC client
client = hvac.Client(url='https://vault-server-url', token='your-vault-token')

# Authenticate HVAC client using LDAP user credentials
def authenticate_ldap(username, password):
    client.auth.ldap.login(username=username, password=password)
    if client.is_authenticated():
        return True
    else:
        return False

ldap_username = 'ldap-username'
ldap_password = 'ldap-password'

Once a user is authenticated, they’ll be able to securely access secrets from our central server. My question is, can we achieve this using Jupyterhub? I’ve attempted to install the Vault client via pip (pip install hvac), but I’m encountering difficulties accessing the LDAP credentials to initialize the client.

I’m reaching out to seek your support in accomplishing this goal. Additionally, I’d love to know if there are any similar tools or approaches for securely sharing secrets and API keys among our team within the Jupyterhub environment.

Your guidance and assistance would be greatly appreciated.
Thank you in advance!

2

You will not have access to the LDAP credentials in the spawner to initialise your HVAC client. You can pass the LDAP credentials via auth_state to spawner and then authenticate the client in spawner. But I dont think it is a very sensible solution.

Instead, you can authenticate your HVAC client in the authenticator (as you have access to user’s LDAP credentials) and then ask your client to generate an API token with a TTL (Time To Live), say, 1 day and then pass this API token in auth_state to the spawner. Then you can inject this API token in spawner’s environment and use it to initialise HVAC client and work collaboratively with your team. To generate a new token, your users need to re-authenticate with hub and that can be forced by setting your cookie_max_age_days to 1 day as well. This lets you not to sprawl your user’s LDAP credentials and work safely with limited time API tokens of vault.

1 Like
3

Could you please provider more details?

This means that I could use the the HVAC client in the jupyterhub_config.py with the LDAP bind users credentials, like in the below configuration file?

# Configuration file for JupyterHub
import os

# Import LDAP authenticator
from ldapauthenticator import LDAPAuthenticator

c = get_config()
## LDAP Authenticator Configuration
c.JupyterHub.authenticator_class = LDAPAuthenticator

# The LDAP server URI
c.LDAPAuthenticator.server_address = 'ldaps://lab-XXX-XXX.XXX:636'

# The base DN to search for users
c.LDAPAuthenticator.bind_dn_template = [
        'uid={username},cn=users,cn=accounts,dc=lab']

c.LDAPAuthenticator.allowed_groups = [
        'cn=developers,cn=groups,cn=accounts,dc=lab',
        'cn=admins,cn=groups,cn=accounts,dc=lab']

# The attribute to use as the user name
c.LDAPAuthenticator.user_attribute = 'uid'

# The DN of the LDAP user to bind as to perform the search
c.LDAPAuthenticator.lookup_dn_search_user = 'uid=<service-account>,cn=sysaccounts,cn=etc,dc=lab'

# The password for the LDAP user to bind as to perform the search
c.LDAPAuthenticator.lookup_dn_search_password = <service-account-password>

# Set up the LDAP SSL certificate
c.LDAPAuthenticator.use_ssl = True

c.LDAPAuthenticator.user_search_base = 'dc=lab'
c.LDAPAuthenticator.admin_users = {'lab-user'}

# HVAC configuration

import hvac
client = hvac.Client()

client.auth.ldap.configure(
    user_dn='dc=users,dc=hvac,dc=network',
    group_dn='ou=groups,dc=hvac,dc=network',
    url='ldaps://ldap.hvac.network:12345',
    bind_dn='cn=admin,dc=hvac,dc=network',
    bind_pass='ourverygoodadminpassword'
    user_attr='uid',
    group_attr='cn',
)

client.auth.ldap.login(
    username=<service-account>,
    password=<service-account-password>,
    mount_point='prod-ldap'
)
print(client.is_authenticated())  # => True

if client.is_authenticated():
    token = client.auth.token.create(policies=['root'], ttl='1d')

I think the next steps on passing the API token to the spawner and using it in the spawner is not quiet clear for me.

could you give me a snippet on how to do this?
I’m using docker-compose. Thanks

4

I am not quite sure what do you mean by service-account and service-account-password in the HVAC client. The way I understood your use case is that the same user who authenticates itself with JupyterHub should authenticate with HVAC client as well that is using same LDAP server. In the config you posted above, you are using two different LDAP servers (potentially having different auth credentials for the same user). Are you using two different LDAP servers, one for hub and one for Vault?

A config like below should work for the use case where a single LDAP server is used to authenticate against JupyterHub and Vault:

# Import LDAP authenticator
from ldapauthenticator import LDAPAuthenticator

# Subclass the LDAPAuthenticator to add vault authentication
# and get an API token
class LDAPAuthenticatorWithVault(LDAPAuthenticator):

    async def authenticate(self, handler, data):
        super().__init__()

        import hvac
        client = hvac.Client()

        self.log.debug("Configuring HVAC client")

        # Confgigure LDAP
        client.auth.ldap.configure()

        self.log.debug("Authenticating HVAC client with user's LDAP credentials")

        client.auth.ldap.login(
            username=data['username'],
            password=data['password'],
            mount_point='prod-ldap'
        )

        if client.is_authenticated():
            self.log.info("HVAC client authenticated successfully. Generating a API token")
            token = client.auth.token.create(policies=['root'], ttl='1d')
            return {"name": data['username'], "auth_state": {'vault_token': token}}
        else:
            self.log.warning("HVAC client authentication failed")
            return data['username']
    
    async def pre_spawn_start(self, user, spawner):
        """Pass vault_token to spawner via environment variable"""
        auth_state = await user.get_auth_state()
        if not auth_state:
            # auth_state not enabled
            return
        spawner.environment['VAULT_TOKEN'] = auth_state['vault_token']

# LDAP Authenticator Configuration
c.JupyterHub.authenticator_class = LDAPAuthenticatorWithVault

# Persist auth_state hook to be able to use token
# Dont for get to setup a JUPYTERHUB_CRYPT_KEY env variable for hub 
# that will be used to encrypt auth_state in the DB
c.Authenticator.enable_auth_state = True

In the above config, we create a new authenticator based on LDAPAuthenticator class and add the HVAC auth flow using same credentials that user provided at hub login. So, Hub and Vault should bind to same LDAP server. If auth is successful, we generate an API token and inject that token in spawner environment using pre_spawn_start hook. So, once you spawn a single user server, the API token will be available at VAULT_TOKEN with which the user’s can authenticate with Vault and work with shared secrets.

1 Like
5

I’m using a bind authenticated user to perform LDAP search queries. In the LDAP server, I’ve created a service account (bind user) dedicated to run only queries.

I’m not using two different ldap servers, the ldap config of the HVAC client is from the example provided in the HVAC documentation.

You are correct, this is my use-case.
To configure LDAPAuthenticator params, now i need to call LDAPAuthenticatorWithVault like the below code? The jupyterhub_config.py contains the following code:

# Import LDAP authenticator
from ldapauthenticator import LDAPAuthenticator

# Subclass the LDAPAuthenticator to add vault authentication
# and get an API token
class LDAPAuthenticatorWithVault(LDAPAuthenticator):

    async def authenticate(self, handler, data):
        super().__init__()

        import hvac
        client = hvac.Client()

        self.log.debug("Configuring HVAC client")

        # Confgigure LDAP
        client.auth.ldap.configure(
          user_dn=self.user_search_base ,
          group_dn=self.allowed_groups ,
          url=self.server_address ,
          bind_dn=self.lookup_dn_search_user,
          bind_pass=self.lookup_dn_search_password ,
          user_attr='uid',
          group_attr='cn',
      )

        self.log.debug("Authenticating HVAC client with user's LDAP credentials")

        client.auth.ldap.login(
            username=data['username'],
            password=data['password'],
            mount_point='prod-ldap'
        )

        if client.is_authenticated():
            self.log.info("HVAC client authenticated successfully. Generating a API token")
            token = client.auth.token.create(policies=['root'], ttl='1d')
            return {"name": data['username'], "auth_state": {'vault_token': token}}
        else:
            self.log.warning("HVAC client authentication failed")
            return data['username']
    
    async def pre_spawn_start(self, user, spawner):
        """Pass vault_token to spawner via environment variable"""
        auth_state = await user.get_auth_state()
        if not auth_state:
            # auth_state not enabled
            return
        spawner.environment['VAULT_TOKEN'] = auth_state['vault_token']

# LDAP Authenticator Configuration
c.JupyterHub.authenticator_class = LDAPAuthenticatorWithVault

# Persist auth_state hook to be able to use token
# Dont for get to setup a JUPYTERHUB_CRYPT_KEY env variable for hub 
# that will be used to encrypt auth_state in the DB
if 'JUPYTERHUB_CRYPT_KEY' not in os.environ:
    warnings.warn(
        "Need JUPYTERHUB_CRYPT_KEY env for persistent auth_state.\n"
        "    export JUPYTERHUB_CRYPT_KEY=$(openssl rand -hex 32)"
    )

c.Authenticator.enable_auth_state = True

# The LDAP server URI
c.LDAPAuthenticatorWithVault.server_address = 'ldaps://lab-XXX-XXX.XXX:636'

# The base DN to search for users
c.LDAPAuthenticatorWithVault.bind_dn_template = [
        'uid={username},cn=users,cn=accounts,dc=lab']

c.LDAPAuthenticatorWithVault.allowed_groups = [
        'cn=developers,cn=groups,cn=accounts,dc=lab',
        'cn=admins,cn=groups,cn=accounts,dc=lab']

# The attribute to use as the user name
c.LDAPAuthenticatorWithVault.user_attribute = 'uid'

# The DN of the LDAP user to bind as to perform the search
c.LDAPAuthenticatorWithVault.lookup_dn_search_user = 'uid=<service-account>,cn=sysaccounts,cn=etc,dc=lab'

# The password for the LDAP user to bind as to perform the search
c.LDAPAuthenticatorWithVault.lookup_dn_search_password = <service-account-password>

# Set up the LDAP SSL certificate
c.LDAPAuthenticatorWithVault.use_ssl = True

c.LDAPAuthenticatorWithVault.user_search_base = 'dc=lab'
c.LDAPAuthenticatorWithVault.admin_users = {'lab-user'}

In the Dockerfile.jupyterhub, I’ve created the env JUPYTERHUB_CRYPT_KEY as requested:

FROM jupyterhub/jupyterhub:latest

# Install dockerspawner, ldapauth
RUN python3 -m pip install --no-cache-dir \
        dockerspawner==12.* \
        hvac \
        jupyterhub-ldapauthenticator 

RUN export JUPYTERHUB_CRYPT_KEY=$(openssl rand -hex 32)

CMD ["jupyterhub", "-f", "/srv/jupyterhub/jupyterhub_config.py"]
6

Yes, you can directly define your custom authenticator and spawner in the hub config file. That would be a good starting point to test your setup. You will have to refine the custom authenticator for production by catching possible exceptions from HVAC client though.