I am in the list under Authenticator.admin_users. But I do not see full admin options (only admin tab and the jhub version. Under manage groups I can only edit existing groups) nor am I able to start/stop user servers (not even my own). When click on the Admin tab I get logs on hub:
JupyterHub base:89] Blocking Cross Origin API request. Referer: https://dev.com/myenv/hub/admin, Host: dev.com, Host URL: http://dev.com/myenv/hub/
[W 2023-02-02 05:00:43.682 JupyterHub scopes:804] Not authorizing access to /myenv/hub/api/users. Requires any of [list:users], not derived from scopes []
[W 2023-02-02 05:00:43.682 JupyterHub web:1796] 403 GET /myenv/hub/api/users?include_stopped_servers&offset=0&limit=50&name_filter= (10.97.11.164): Action is not authorized with current scopes; requires any of [list:users]
[W 2023-02-02 05:00:43.683 JupyterHub log:186] 403 GET /myenv/hub/api/users?include_stopped_servers&offset=0&limit=50&name_filter= (@10.97.11.164) 5.06ms
Why do I get warnings like these? I think we will need it to cull idle terminals
base:1409] Failing suspected API request to not-running server: /myenv/hub/user/username/api/terminals
[W 2023-02-02 05:00:40.339 JupyterHub log:186] 424 GET /myenv/hub/user/username/api/terminals?1675314039247 (username@10.97.11.170) 4.47ms
I am unable to read the user-server spawn progress. I think this is also causing the redirection to /lab (that I see in the
single user server logs) never happen and the hub shows that the operation timed out. But actually, the single-user server is up and running as expected. This is the error that I see for spawn progress:
JupyterHub scopes:804] Not authorizing access to /myenv/hub/api/users/username/server/progress. Requires any of [read:servers], not derived from scopes []
[W 2023-02-02 05:04:26.817 JupyterHub web:1796] 403 GET /myenv/hub/api/users/username/server/progress (172.16.0.1): Action is not authorized with current scopes; requires any of [read:servers]
How do I make sure : a) All users are able to list start and stop their own servers. b) The hub is able to view spawn progress. The hub doesnt see the warnings that I mentioned above c) Admin users are able to do what they are supposed to do
This is not a scope issue, but rather a proxy header issue causing some false-positive CORS detection (we’ve changed how all this works to avoid these problems in the next Hub release). The error is message is misleading because you have no scopes because auth was rejected due to the CORS issue, and granted no scopes to the request, not because your user has no scopes.
The Hub thinks the requests are coming to http://dev.com/myenv/hub/admin from http_s_://dev.com/myenv/hub/admin, which is not allowed. Ultimately, we need to make sure your proxy sets the appropriate proxy headers (Likely Forwarded) so that it gets the true URL that the browser is requesting. JupyterHub’s mechanism for getting the protocol is here, but the first priority is the Forwarded header, so if that is set with the right value by your frontmost layer, it should get the right answer.
How is JupyterHub deployed, and what is between the web and JupyterHub?
EDIT: (lots of) other folks have had similar issues, e.g. here
We have an Application gateway and Nginx Ingress & Controller in between web and juputerhub
And as per your reply, am I correct in inferring that if I set proxy_set_header X-Scheme https; on nginx, the user, admin roles will be set properly as a result of the CORS errors being resolved?
X-Scheme will work as long as no proxy layer is also setting the Forwarded Header, which will take priority. If any layer is setting Forwarded, that’s where the info needs to go.
Hi @minrk , I get the following logs on jupyterhub after applying the above fix.
[W 2023-02-06 06:37:41.468 JupyterHub auth:298] Service Server at /myenv/user/username/ requested scopes access:servers!server=username/,read:users:name!user,read:users:groups!user,access:servers!user=username for user username, granting only access:servers!server=username/,read:users:name!user,read:users:groups!user.
.
.
.
JupyterHub api:38] Blocking Cross Origin API request. Referer: https://dev.com/myenv/user/username/lab?, Host: dev.com/myenv/hub/, User host: dev.com/user/
How do I set appropriate scope to ‘username’ user which is already an admin
Is it correct that the CORS warning is because of “access:servers!user=username” no being assigned to ‘username’?
