I need to make httponly flag for _xsrf false because some extensions like nbgrader need to work with it in the javascript (i think) it causes some issues in nbgrader extension.
Which version of JupyterHub are you using? In our deployment, they both are set to false for _xsrf cookies and we are not using any special cookie options in our deployment. We are on v5.2.1
Hmm, with a quick test using JupyterHub 4.0.2 locally, I see both are set to false as well. In your screenshot, I see that you are looking at the _xsrf token on a service (as path is /service..). Are you sure that it is not your service handler that is managing the cookie flags?
Thank you for the response. I am using nbgrader as service. How can i test that my service is setting httponly to true?
How about my nginx ingress controller?
Ahh, I missed your nginx ingress. Probably, the configuration is not being applied on the ingress controller. Can you check your final nginx.conf in the pod? Something like kubectl exec <pod> -n <ns> -- cat /etc/nginx/nginx.conf should give you the current config.
Thanks. Here is nginx config from the nginx controller pod. kubectl exec ingress-nginx-controller-7f5bfdb96d-4wf67 -n ingress-nginx -- cat /etc/nginx/nginx.conf
I added the secure flag to cookies but not httponly! but the xsrf cookie is httponly in the browser.
I think this part is related to the jupyterhub
## start server jupyterhub<redacted>
server {
server_name jupyterhub<redacted> ;
listen 80 ;
listen [::]:80 ;
listen 443 ssl http2 ;
listen [::]:443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "default";
set $ingress_name "jupyterhub";
set $service_name "proxy-public";
set $service_port "http";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = true,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
plugins.run()
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "default-proxy-public-http";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Forwarded-Scheme $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
proxy_cookie_flags ~ Secure;
# proxy_cookie_flags ~ Secure HttpOnly;
# proxy_cookie_flags ~ Secure HttpOnly SameSite=Strict;
more_set_headers "x-jupyterhub-version: X";
# add_header Strict-Transport-Security "max-age=31557600; includeSubDomains; preload" always;
add_header X-Content-Type-Options nosniff always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Robots-Tag "noindex, nofollow" always;
add_header X-Download-Options noopen always;
add_header X-Permitted-Cross-Domain-Policies none always;
add_header Referrer-Policy same-origin always;
add_header Permissions-Policy "microphone=(), geolocation=(), camera=()" always;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
}
## end server jupyterhub<redacted>
Ok, in that case my guess is that it is nbgrader who is setting that cookie flag. I dont have any experience with nbgrader but I assume you are running it as JupyterHub service. So, I would suggest you to set the environment variable JUPYTERHUB_COOKIE_OPTIONS in the service definition of nbgrader. Using example provided in nbgrader docs it would look as
Thanks. How can make sure that the nbgrader is setting the httponly on xsrf?
I’m asking this because when I do a get request to /hub/home (which is probably unrelated to service), I get httponly xsrf cookie in the response:
Well, I am looking at the source code. For example, for 4.0.2, here is the place Hub sets the _xsrf cookie options and apart from path, it does not set anything else. Eventually, downstream it is tornado that sets the _xsrf cookie with the options passed by JupyterHub. So, I am not sure who else might be fiddling with cookie options if not ingress controller!!
I agree. it’s very strange. It was working just fine but then it stopped working (i didn’t change any script). could an update or change in the server have caused that?
Here is the complete ingress part:
You mentioned that all the cookies of JupyterHub (_xsrf, hub-login, single-server, oauth,…) have httponly set, right?!
for example send a request directly to jupyterhub without involving ingress and checking the cookies?
Depends on the network policy of the JupyterHub deployment. If you can access JupyterHub service via CluserIP, yes, that would be a good test by port forwarding the cluster IP of JupyterHub service and see how cookies are set.
i did it already but nothing changed. It seems that regardless of the changes I make in the ingress.annotations the cookies always have secure and httponly flags.
For example, I made changes like proxy_cookie_flags _xsrf "", proxy_cookie_flags ~ Secure SameSite=Strict in the ingress.annotations for testing but cookies don’t change (see screenshot please).
