This assumption turned out to be incorrect. When I had been testing with an example token, I made a mistake that prevented the token auth from working properly. Though it is not explicitly documented, you can use a token to authenticate and hit any of the internal pages/resources (not just the API routes). So, it can be configured as follows.
First, create a token for some dedicated user. I created a prometheus user in JupyterHub for this.
jupyterhub token prometheus >/etc/prometheus/secrets/jupyterhub
Then, add a section to your scrape config for the JupyterHub job to use this token:
scrape_configs:
- job_name: 'jupyterhub-exporter'
bearer_token_file: /etc/prometheus/secrets/jupyterhub
static_configs:
- targets: ['10.10.0.2:8075'] # Or whatever <host>