Access S3 Bucket from Jupyter Notebook

Hello, I am very new to Jupyterhub and I want to be able to access S3 bucket from my Jupyter Notebook. I have authentication done through Keycloak, and I have found some resources where I can retrieve the Keycloak access token, but I can’t figure out where to make the STS get session token call to retrieve temporary credentials for AWS S3 for the authenticated keycloak user. Could someone provide some insight on this? Thanks.

You should be able to configure a pre_spawn_start hook in your Authenticator that can make arbitrary calls and store the results to auth_state which can be accessed by the spawner.

Starting single-user notebook with our custom ldap docker image - #4 by manics is a bit out of date, but shows the basic principals of a spawner passing user-specific variables to a spawner (in this case a UID) which makes them available to the user as environment variables.

1 Like

So for testing purposes I’ve been able to pass in AWS credentials to the Jupyter Notebook environment with that pre_spawn_start hook and have access to AWS S3, but I would like those credentials to be tied with the authenticated Keycloak user. I was thinking about doing a AssumeRoleWithWebIdentity call with the Keycloak access token to get temporary AWS credentials, but I’m getting a “couldn’t retrieve verification key from identity provider” when trying to make that call. Is there a better way of getting AWS credentials associated with a Keycloak user?

I’ve never integrated KeyCloak with AWS before. Can you share your configuration/code with secrets redacted? If I still can’t help then someone else might be able to.