Actually, I can confirm this helm chart configuration does work!
So anyone else doing something similar, make sure you not only update your Helm chart config.yaml, but also add CSP headers to the parent site, so you can embed your jupyterhub site in an iframe on that site:
<meta http-equiv="Content-Security-Policy" content="frame-src 'self' (your jupyterhub domain)">
Also, if you’re doing the LTIv1.3 Open ID connect sequence, you have to make sure that the LMS pages shown inside the iframe as part of the Open ID connect redirect sequence have the reverse CSP to the ancestor:
<meta http-equiv="Content-Security-Policy" content="frame-ancestors 'self';"/>
Finally, if you’re using Django and the older XFrameOptionsMiddleware, make sure you set the X_FRAME_OPTIONS configuration setting to “SAMEORIGIN” (not “DENY”, which is the default if you used Django Cookiecutter to set up your project).