OAuthenticator 500 internal server error

JupyterHub The Littlest JupyterHub
1

The error on UI shows up as:

Error in Authenticator.pre_spawn_start: CalledProcessError Command ‘[‘useradd’, ‘–create-home’, ‘jupyter-mahesh ramchandra -722d7’]’ returned non-zero exit status 3.

You can try restarting your server from the [home page]

tljh config is:

users:
  admin:
  - maheshmhatre
user_environment:
  TLJH_DISABLE_AUTO_SCROLL: false
https:
  enabled: true
  letsencrypt:
    email: abc@xyz.com
    domains:
    - osjupyter.xxx.azure.com
auth:
  OAuthenticator:
    oauth_callback_url: https://osjupyter.westus2.cloudapp.azure.com/hub/oauth_callback
    client_id: xxxxxx-xxx-xxxx-xxxx-xxxxxxxxxxx
    client_secret: xxxxxxxxxxxxx
    allow_all: true
  AzureAdOAuthenticator:
    tenant_id: xxxxxxxxxxxxxx
    scope:
    - openid
    - profile
    admin_users:
    - maheshmhatre@xyz.com
  type: azuread

logs:
Jan 03 19:53:25 JupyterHub python3[22270]: [I 2024-01-03 19:53:25.355 JupyterHub provider:574] Creating oauth client jupyterhub-user-mahesh%20ramchandra%20mhatre%20%28maheshmhatre%29
Jan 03 19:53:25 JupyterHub python3[22270]: useradd: invalid user name ‘jupyter-mahesh ramchandra -722d7’
Jan 03 19:53:25 JupyterHub python3[22270]: [E 2024-01-03 19:53:25.385 JupyterHub user:720] Unhandled error starting mahesh ramchandra mhatre (maheshmhatre)'s server: Command ‘[‘useradd’, ‘–create-home’, ‘jupyter-mahesh ramchandra -722d7’]’ returned non-zero exit status 3.
Jan 03 19:53:25 JupyterHub python3[22270]: [W 2024-01-03 19:53:25.428 JupyterHub web:1787] 500 GET /hub/spawn (131.107.1.157): Error in Authenticator.pre_spawn_start: CalledProcessError Command ‘[‘useradd’, ‘–create-home’, ‘jupyter-mahesh ramchandra -722d7’]’ returned non-zero exit status 3.
Jan 03 19:53:25 JupyterHub python3[22270]: [E 2024-01-03 19:53:25.430 JupyterHub log:181] {
Jan 03 19:53:25 JupyterHub python3[22270]: “Host”: “osjupyter.westus2.cloudapp.azure.com”,
Jan 03 19:53:25 JupyterHub python3[22270]: “User-Agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Accept-Encoding”: “gzip, deflate, br”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Accept-Language”: “en-US,en;q=0.9”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Cache-Control”: “max-age=0”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Cookie”: “jupyterhub-hub-login=[secret]; jupyterhub-session-id=[secret]”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Ch-Ua”: “"Not_A Brand";v="8", "Chromium";v="120", "Microsoft Edge";v="120"”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Ch-Ua-Mobile”: “?0”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Ch-Ua-Platform”: “"Windows"”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Fetch-Dest”: “document”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Fetch-Mode”: “navigate”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Fetch-Site”: “none”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Sec-Fetch-User”: “?1”,
Jan 03 19:53:25 JupyterHub python3[22270]: “Upgrade-Insecure-Requests”: “1”,
Jan 03 19:53:25 JupyterHub python3[22270]: “X-Forwarded-For”: “131.107.1.157”,
Jan 03 19:53:25 JupyterHub python3[22270]: “X-Forwarded-Host”: “osjupyter.westus2.cloudapp.azure.com”,
Jan 03 19:53:25 JupyterHub python3[22270]: “X-Forwarded-Port”: “443”,
Jan 03 19:53:25 JupyterHub python3[22270]: “X-Forwarded-Proto”: “https”,
Jan 03 19:53:25 JupyterHub python3[22270]: “X-Forwarded-Server”: “JupyterHub”,
Jan 03 19:53:25 JupyterHub python3[22270]: “X-Real-Ip”: “131.107.1.157”
Jan 03 19:53:25 JupyterHub python3[22270]: }
Jan 03 19:53:25 JupyterHub python3[22270]: [E 2024-01-03 19:53:25.430 JupyterHub log:189] 500 GET /hub/spawn (mahesh ramchandra mhatre (maheshmhatre)@131.107.1.157) 105.62ms

2

I found a similar post for ldap authenticator: LDAP Authenticator error 500 creating user - JupyterHub / The Littlest JupyterHub - Jupyter Community Forum I wonder if similar setting to use_lookup_dn_username = False and use sAMA account name is available in OAuthenticator

3

Seems like your auth provider is sending your username claim as mahesh ramchandra -722d7. Spaces are not allowed in Linux usernames and that is what the error is saying.

You will have to either fix it on your auth provider side to pass proper username claim or override the AzureAdOAuthenticator to remove spaces from username.

class MyAzureAdOAuthenticator(AzureAdOAuthenticator):

    def normalize_username(self, username):
       return username.replace(' ', '_')

Something like this should work!!

4

@mahendrapaipuri Thanks for your answer. I tried overriding the AzureAdOAuthenticator class as you suggested. I modified the file /opt/tljh/hub/lib/python3.6/site-packages/azuread.py and added a class MyAzureAdOAuthenticator as you comment mentioned. In tljh-config I changed the setting from auth.AzureAdOAuthenticator to auth.MyAzureAdOAuthenticator and re-loaded tljh-config. I don’t see any errors in jourrnalctl logs or azure logs. It tries to redirect and fails. I notice that the url which it tries to redirect is different than before. After clicking ‘Signin with Azure AD’ button now it redirects to https://login.microsoftonline.com//oauth2/authorize?response_type=code&redirect_uri=https%3A%2F%2Fosjupyter.westus2.cloudapp.azure.com%2Fhub%2Foauth_callback&client_id=05e90bac-90b6-49ce-9fc7-8516ca9cd50c&state=eyJzdGF0ZV9pZCI6ICI5ZDY3NmZiZTVmY2E0MjhmOTRjMDdmZWY4ODQzMzBlMCIsICJuZXh0X3VybCI6ICIifQ%3D%3D and ends up in 404 not found.

Before this when i used AzureAdOauthenticator class the url used to be https://login.microsoftonline.com/975f013f-7f24-47e8-a7d3-abc4752bf346/oauth2/authorize?response_type=code&redirect_uri=https%3A%2F%2Fosjupyter.westus2.cloudapp.azure.com%2Fhub%2Foauth_callback&client_id=05e90bac-90b6-49ce-9fc7-8516ca9cd50c&state=eyJzdGF0ZV9pZCI6ICJmOGFlOGM5YTAwNDA0YjhlYmE4MmU3MGI1NTZkMzA4NiIsICJuZXh0X3VybCI6ICIifQ%3D%3D&scope=profile+openid

I wonder if I did something wrong when overriding AzureAdOAuthenticator?

image
image1048×576 49.3 KB

5

Could you share the hub logs with new authenticator class if possible with debugging on?

Looking into the authorize URL you posted after you changed the config, it seems that tenant_id is missing from the URL. So, I guess you have not modified auth.type which points to azuread. It should be updated as well to use new class MyAzureAdOAuthenticator .

Could you try with type: oauthenticator.azuread.MyAzureAdOAuthenticator?

1 Like
6

I tried enabling debug logs by adding

debug:
    enabled: true

in tljh-config. Not sure if thats the correct way to do it. I don’t see any extra logs than I posted before. Please let me know if enabling debug logs is possible in tljh

7

@mahendrapaipuri changing type: oauthenticator.azuread.MyAzureAdOAuthenticator did the trick. It works with overridden class.

Thanks a lot for looking into it :slight_smile:

2 Likes