I am aware of the recent changes of the role-based access right, which was introduced in 2.0.0 version. But I noticed that in the recent update of 2.0.1, /home/admin# page returns me an almost blank page! Is this a bug in my configuration of my role or the admin panel is removed totally?
You wouldn’t happen to be proxying your JupyterHub behind Nginx would you? I ran into a similar problem and was able to resolve it by following the suggestions by the helpful users in this GitHub issue:
opened 09:15AM - 25 Dec 21 UTC
bug
### Bug description
Merry Christmas!
The Admin settings of jupyterhub v.… 2.0.1 are not shown if the hub is behind a reverse nginx proxy.
The symptoms appear to be similar to #3711 except that they are now not caused by `base_url`.
#### Expected behaviour
Admin page when the hub is accessed directly:
<img width="684" alt="Screenshot 2021-12-25 at 09 32 58" src="https://user-images.githubusercontent.com/2140361/147381008-67c6e49d-a583-47d0-aab2-bc1246976b1e.png">
#### Actual behaviour
Admin page when the hub is accessed through a proxy:
<img width="683" alt="Screenshot 2021-12-25 at 09 34 48" src="https://user-images.githubusercontent.com/2140361/147381033-be241b38-db3f-410c-a439-2d4eb3f71f33.png">
This message looks suspicious (see the full log below):
```
jupyterhub_1 | [W 2021-12-25 08:58:25.156 JupyterHub base:89] Blocking Cross Origin API request. Referer: http://127.0.0.1:8080/hub/admin, Host: 127.0.0.1, Host URL: http://127.0.0.1/hub/
```
There is no problem with `jupyterhub/jupyterhub:1.5.0`
### How to reproduce
1. Ensure the following directory structure:
```
.
├── docker-compose.yml
├── jupyterhub
│ ├── Dockerfile
│ └── jupyterhub_config.py
└── nginx
├── Dockerfile
└── nginx.conf
```
2. Ensure the following file contents:
- `docker-compose.yml`:
```
version: '3'
services:
nginx:
image: nginx
build: nginx
ports:
- 8080:80
jupyterhub:
image: jupyterhub
build: jupyterhub
expose:
- "8000"
```
- `jupyterhub/Dockerfile`:
```
FROM jupyterhub/jupyterhub:2.0.1
COPY jupyterhub_config.py /etc/jupyterhub/
CMD jupyterhub -f /etc/jupyterhub/jupyterhub_config.py --debug
```
- `jupyterhub/jupyterhub_config.py`:
```
c.Authenticator.admin_users = {'user'}
c.JupyterHub.authenticator_class = 'dummy'
```
- `nginx/Dockerfile`:
```
FROM nginx:1.21.4
COPY nginx.conf /etc/nginx/
```
- `nginx/nginx.conf` ([as documented](https://jupyterhub.readthedocs.io/en/stable/reference/config-proxy.html#nginx) but without ssl):
```
events {}
http {
server {
listen 0.0.0.0:80;
server_name localhost;
location / {
proxy_pass http://jupyterhub:8000/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# websocket headers
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
proxy_set_header X-Scheme $scheme;
proxy_buffering off;
}
real_ip_header proxy_protocol;
set_real_ip_from 127.0.0.1;
}
}
```
3. Build containers:
`docker-compose build`
4. Start the containers:
`docker-compose up`
5. Visit the page: http://127.0.0.1:8080/hub/admin
6. Enter the user name `user`
### Your personal set up
<!--
Tell us a little about the system you're using.
Please include information about how you installed,
e.g. are you using a distribution such as zero-to-jupyterhub or the-littlest-jupyterhub.
-->
- OS: MacOS 12.1
- Version(s):
```
% docker --version
Docker version 20.10.11, build dea9396
% docker-compose --version
Docker Compose version v2.2.1
```
<details><summary>Logs</summary>
- docker-compose:
```
% docker-compose up
[+] Running 2/0
⠿ Container jupyterhub-test1_jupyterhub_1 Created 0.0s
⠿ Container jupyterhub-test1_nginx_1 Created 0.0s
Attaching to jupyterhub_1, nginx_1
nginx_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx_1 | 10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx_1 | /docker-entrypoint.sh: Configuration complete; ready for start up
jupyterhub_1 | [D 2021-12-25 08:58:14.902 JupyterHub application:731] Looking for /etc/jupyterhub/jupyterhub_config in /srv/jupyterhub
jupyterhub_1 | [D 2021-12-25 08:58:14.902 JupyterHub application:753] Loaded config file: /etc/jupyterhub/jupyterhub_config.py
jupyterhub_1 | [I 2021-12-25 08:58:14.912 JupyterHub app:2766] Running JupyterHub version 2.0.1
jupyterhub_1 | [I 2021-12-25 08:58:14.912 JupyterHub app:2796] Using Authenticator: jupyterhub.auth.DummyAuthenticator-2.0.1
jupyterhub_1 | [I 2021-12-25 08:58:14.912 JupyterHub app:2796] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-2.0.1
jupyterhub_1 | [I 2021-12-25 08:58:14.912 JupyterHub app:2796] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-2.0.1
jupyterhub_1 | [I 2021-12-25 08:58:14.924 JupyterHub app:1606] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_1 | [D 2021-12-25 08:58:14.930 JupyterHub app:1773] Connecting to db: sqlite:///jupyterhub.sqlite
jupyterhub_1 | [D 2021-12-25 08:58:14.959 JupyterHub orm:955] database schema version found: 833da8570507
jupyterhub_1 | [I 2021-12-25 08:58:15.017 JupyterHub proxy:496] Generating new CONFIGPROXY_AUTH_TOKEN
jupyterhub_1 | [D 2021-12-25 08:58:15.017 JupyterHub app:2019] Loading roles into database
jupyterhub_1 | [I 2021-12-25 08:58:15.035 JupyterHub app:1924] Not using allowed_users. Any authenticated user will be allowed.
jupyterhub_1 | [D 2021-12-25 08:58:15.042 JupyterHub app:2278] Purging expired APITokens
jupyterhub_1 | [D 2021-12-25 08:58:15.047 JupyterHub app:2278] Purging expired OAuthCodes
jupyterhub_1 | [D 2021-12-25 08:58:15.053 JupyterHub app:2111] Loading role assignments from config
jupyterhub_1 | [D 2021-12-25 08:58:15.076 JupyterHub app:2424] Initializing spawners
jupyterhub_1 | [D 2021-12-25 08:58:15.080 JupyterHub app:2555] Loaded users:
jupyterhub_1 |
jupyterhub_1 | [I 2021-12-25 08:58:15.081 JupyterHub app:2835] Initialized 0 spawners in 0.006 seconds
jupyterhub_1 | [W 2021-12-25 08:58:15.087 JupyterHub proxy:565] Found proxy pid file: /srv/jupyterhub/jupyterhub-proxy.pid
jupyterhub_1 | [W 2021-12-25 08:58:15.088 JupyterHub proxy:577] Proxy no longer running at pid=12
jupyterhub_1 | [D 2021-12-25 08:58:15.088 JupyterHub proxy:618] Removing proxy pid file jupyterhub-proxy.pid
jupyterhub_1 | [W 2021-12-25 08:58:15.090 JupyterHub proxy:687] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
jupyterhub_1 | [I 2021-12-25 08:58:15.090 JupyterHub proxy:691] Starting proxy @ http://:8000
jupyterhub_1 | [D 2021-12-25 08:58:15.090 JupyterHub proxy:692] Proxy cmd: ['configurable-http-proxy', '--ip', '', '--port', '8000', '--api-ip', '127.0.0.1', '--api-port', '8001', '--error-target', 'http://127.0.0.1:8081/hub/error']
jupyterhub_1 | [D 2021-12-25 08:58:15.098 JupyterHub proxy:610] Writing proxy pid file: jupyterhub-proxy.pid
jupyterhub_1 | 08:58:16.203 [ConfigProxy] info: Proxying http://*:8000 to (no default)
jupyterhub_1 | 08:58:16.210 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
jupyterhub_1 | [D 2021-12-25 08:58:16.219 JupyterHub proxy:728] Proxy started and appears to be up
jupyterhub_1 | [D 2021-12-25 08:58:16.221 JupyterHub proxy:821] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
jupyterhub_1 | 08:58:16.244 [ConfigProxy] info: 200 GET /api/routes
jupyterhub_1 | [I 2021-12-25 08:58:16.251 JupyterHub app:3084] Hub API listening on http://127.0.0.1:8081/hub/
jupyterhub_1 | [D 2021-12-25 08:58:16.252 JupyterHub proxy:343] Fetching routes to check
jupyterhub_1 | [D 2021-12-25 08:58:16.252 JupyterHub proxy:821] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
jupyterhub_1 | 08:58:16.265 [ConfigProxy] info: 200 GET /api/routes
jupyterhub_1 | [D 2021-12-25 08:58:16.271 JupyterHub proxy:346] Checking routes
jupyterhub_1 | [I 2021-12-25 08:58:16.273 JupyterHub proxy:431] Adding route for Hub: / => http://127.0.0.1:8081
jupyterhub_1 | [D 2021-12-25 08:58:16.276 JupyterHub proxy:821] Proxy: Fetching POST http://127.0.0.1:8001/api/routes/
jupyterhub_1 | 08:58:16.282 [ConfigProxy] info: Adding route / -> http://127.0.0.1:8081
jupyterhub_1 | 08:58:16.287 [ConfigProxy] info: Route added / -> http://127.0.0.1:8081
jupyterhub_1 | 08:58:16.290 [ConfigProxy] info: 201 POST /api/routes/
jupyterhub_1 | [I 2021-12-25 08:58:16.293 JupyterHub app:3150] JupyterHub is now running at http://:8000
jupyterhub_1 | [D 2021-12-25 08:58:16.296 JupyterHub app:2759] It took 1.401 seconds for the Hub to start
jupyterhub_1 | [I 2021-12-25 08:58:17.444 JupyterHub log:189] 302 GET /hub/admin -> /hub/login?next=%2Fhub%2Fadmin (@172.20.0.1) 1.23ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:17 +0000] "GET /hub/admin HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [I 2021-12-25 08:58:17.534 JupyterHub log:189] 200 GET /hub/login?next=%2Fhub%2Fadmin (@172.20.0.1) 43.03ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:17 +0000] "GET /hub/login?next=%2Fhub%2Fadmin HTTP/1.1" 200 5473 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:17.563 JupyterHub log:189] 304 GET /hub/static/js/admin-react.js (@172.20.0.1) 1.67ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:17 +0000] "GET /hub/static/js/admin-react.js HTTP/1.1" 304 0 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:24.862 JupyterHub roles:449] Assigning default role to User user
jupyterhub_1 | [D 2021-12-25 08:58:24.875 JupyterHub base:557] Setting cookie jupyterhub-session-id: {'httponly': True, 'path': '/'}
jupyterhub_1 | [D 2021-12-25 08:58:24.876 JupyterHub base:561] Setting cookie for user: jupyterhub-hub-login
jupyterhub_1 | [D 2021-12-25 08:58:24.877 JupyterHub base:557] Setting cookie jupyterhub-hub-login: {'httponly': True, 'path': '/hub/'}
jupyterhub_1 | [I 2021-12-25 08:58:24.878 JupyterHub base:797] User logged in: user
jupyterhub_1 | [I 2021-12-25 08:58:24.884 JupyterHub log:189] 302 POST /hub/login?next=%2Fhub%2Fadmin -> /hub/admin (user@172.20.0.1) 30.40ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:24 +0000] "POST /hub/login?next=%2Fhub%2Fadmin HTTP/1.1" 302 0 "http://127.0.0.1:8080/hub/login?next=%2Fhub%2Fadmin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:24.937 JupyterHub scopes:488] Checking access via scope admin:users
jupyterhub_1 | [D 2021-12-25 08:58:24.937 JupyterHub scopes:386] Unrestricted access to /hub/admin via admin:users
jupyterhub_1 | [D 2021-12-25 08:58:24.937 JupyterHub scopes:488] Checking access via scope admin:servers
jupyterhub_1 | [D 2021-12-25 08:58:24.938 JupyterHub scopes:386] Unrestricted access to /hub/admin via admin:servers
jupyterhub_1 | [D 2021-12-25 08:58:24.954 JupyterHub user:347] Creating <class 'jupyterhub.spawner.LocalProcessSpawner'> for user:
jupyterhub_1 | [I 2021-12-25 08:58:24.963 JupyterHub log:189] 200 GET /hub/admin (user@172.20.0.1) 48.74ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:24 +0000] "GET /hub/admin HTTP/1.1" 200 5162 "http://127.0.0.1:8080/hub/login?next=%2Fhub%2Fadmin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /logo HTTP/1.1" 302 0 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [I 2021-12-25 08:58:25.021 JupyterHub log:189] 302 GET /logo -> /hub/logo (@172.20.0.1) 3.75ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/static/js/admin-react.js HTTP/1.1" 304 0 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:25.062 JupyterHub log:189] 304 GET /hub/static/js/admin-react.js (@172.20.0.1) 2.92ms
jupyterhub_1 | [D 2021-12-25 08:58:25.079 JupyterHub log:189] 200 GET /hub/logo (@172.20.0.1) 3.64ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/logo HTTP/1.1" 200 11453 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:25.142 JupyterHub log:189] 200 GET /hub/static/js/admin.js?v=20211225085815 (@172.20.0.1) 4.37ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/static/js/admin.js?v=20211225085815 HTTP/1.1" 200 8013 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [W 2021-12-25 08:58:25.156 JupyterHub base:89] Blocking Cross Origin API request. Referer: http://127.0.0.1:8080/hub/admin, Host: 127.0.0.1, Host URL: http://127.0.0.1/hub/
jupyterhub_1 | [D 2021-12-25 08:58:25.165 JupyterHub scopes:488] Checking access via scope list:groups
jupyterhub_1 | [D 2021-12-25 08:58:25.168 JupyterHub scopes:383] No access to /hub/api/groups via list:groups
jupyterhub_1 | [W 2021-12-25 08:58:25.169 JupyterHub scopes:496] Not authorizing access to /hub/api/groups. Requires any of [list:groups], not derived from scopes []
jupyterhub_1 | [W 2021-12-25 08:58:25.177 JupyterHub web:1787] 403 GET /hub/api/groups?offset=0&limit=50 (172.20.0.1): Action is not authorized with current scopes; requires any of [list:groups]
jupyterhub_1 | [W 2021-12-25 08:58:25.184 JupyterHub log:189] 403 GET /hub/api/groups?offset=0&limit=50 (@172.20.0.1) 40.88ms
jupyterhub_1 | [W 2021-12-25 08:58:25.190 JupyterHub base:89] Blocking Cross Origin API request. Referer: http://127.0.0.1:8080/hub/admin, Host: 127.0.0.1, Host URL: http://127.0.0.1/hub/
jupyterhub_1 | [D 2021-12-25 08:58:25.190 JupyterHub scopes:488] Checking access via scope list:users
jupyterhub_1 | [D 2021-12-25 08:58:25.190 JupyterHub scopes:383] No access to /hub/api/users via list:users
jupyterhub_1 | [W 2021-12-25 08:58:25.191 JupyterHub scopes:496] Not authorizing access to /hub/api/users. Requires any of [list:users], not derived from scopes []
jupyterhub_1 | [W 2021-12-25 08:58:25.191 JupyterHub web:1787] 403 GET /hub/api/users?offset=0&limit=50 (172.20.0.1): Action is not authorized with current scopes; requires any of [list:users]
jupyterhub_1 | [W 2021-12-25 08:58:25.192 JupyterHub log:189] 403 GET /hub/api/users?offset=0&limit=50 (@172.20.0.1) 43.34ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/api/groups?offset=0&limit=50 HTTP/1.1" 403 105 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/api/users?offset=0&limit=50 HTTP/1.1" 403 104 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:25.227 JupyterHub log:189] 200 GET /hub/static/js/jhapi.js?v=20211225085815 (@172.20.0.1) 2.09ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/static/js/jhapi.js?v=20211225085815 HTTP/1.1" 200 4446 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
jupyterhub_1 | [D 2021-12-25 08:58:25.236 JupyterHub log:189] 200 GET /hub/static/components/moment/moment.js?v=20211225085815 (@172.20.0.1) 10.13ms
jupyterhub_1 | [D 2021-12-25 08:58:25.243 JupyterHub log:189] 200 GET /hub/static/js/utils.js?v=20211225085815 (@172.20.0.1) 17.01ms
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/static/components/moment/moment.js?v=20211225085815 HTTP/1.1" 200 173902 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
nginx_1 | 172.20.0.1 - - [25/Dec/2021:08:58:25 +0000] "GET /hub/static/js/utils.js?v=20211225085815 HTTP/1.1" 200 4383 "http://127.0.0.1:8080/hub/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15"
```
- Errors in the browser console (Safari 15.2):
```
[Error] Failed to load resource: the server responded with a status of 403 (Forbidden) (users, line 0)
[Error] Failed to load resource: the server responded with a status of 403 (Forbidden) (groups, line 0)
[Error] TypeError: undefined is not a function (near '...u.map...') — admin-react.js:2:28870
ri (admin-react.js:2:146088)
(anonymous function) (admin-react.js:2:148187)
co (admin-react.js:2:112238)
si (admin-react.js:2:149923)
Cu (admin-react.js:2:166325)
Cu
(anonymous function) (admin-react.js:2:194357)
xu (admin-react.js:2:163064)
du (admin-react.js:2:159263)
du
(anonymous function) (admin-react.js:2:108745)
(anonymous function) (admin-react.js:2:194357)
Va (admin-react.js:2:108691)
Ha (admin-react.js:2:108626)
pu (admin-react.js:2:159359)
notify (admin-react.js:2:919)
(anonymous function) (admin-react.js:2:494)
(anonymous function) (admin-react.js:2:569)
(anonymous function)
p (admin-react.js:2:52697)
promiseReactionJob
```
</details>
Thanks for your reply, but I do not use nginx as a reverse proxy!
I was able to fix this by stopping the jupyterhub server, deleting jupyterhub.sqlite, and then restarting the server.
I am facing the same problem with jupyterhub on kubernetes, do you know where is the jupyterhub.sqlite file or how to delete it.
Derrik
April 12, 2023, 3:38pm
7
Looks like the latest dev release fixed the admin page and permissions that I was experiencing and am using ingress-nginx on k8s. 3.0.0-0.dev.git.6126.h73773b85