Jupyter-server-proxy and port

I’m using Dask from JupyterLab (running in Docker). I have installed jupyter-server-proxy to allow access to a remote web server (= the dashboard of the dask-scheduler running on a remote host ), and by using an url like /proxy/remote_host:port via the dask-labextension

I’ve granted the access to remote_host, via the host_allowlist method set in jupyter_notebook_config.py,
I talk about something like :

def my_host_allowlist(handler, host):

c.ServerProxy.host_allowlist = my_host_allowlist

It works fine

Nethertheless I’d like to get control on the port, in order to allow access to a specific port only
Is there a way to do that ? or is it an irrelevant question ?
The objective beeing to denied a non authorized user to see that dashboard.