[TLJH] How to set env vars for users _after_ but not for the spawner?

JupyterHub
1

We have a setup where the machine running TLJH (but I guess this is true for arbitrary JupyterHubs) behind a firewall that blocks all outgoing http(s) and provides a proxy for these protocols.

Manually setting $http_proxy and $https_proxy after the user server started, correctly enables all outgoing HTTP(S) connections.

Setting the env vars in the JupyterHub config via the c.Spawner.environment dict (as suggested in this comment) works for setting arbitrary env vars.

But setting http_proxy in the spawner config seems to break the communication of the jupyter-user-server with the proxy of the hub (guessing here) and leads to a 502 bad gateway error after the user server is spawned.

So my question is: Is there a TLJH way to define env variables for the users that does not affect the spawner?

2 Likes
2

Hey @willirath, did you find a solution for this problem in the meantime? Facing the same issue…

3

We didn’t manage to set the http_proxy variable. But setting https_proxy as described above didn’t seem to interfere with JupyterHub. For most of our use cases, this was sufficient.

1 Like
4

Does setting no_proxy to include the hub ip work? On TLJH, that should be 127.0.0.1, I believe, so this config:

c.Spawner.environment = {
  "http_proxy": "http://your proxy",
  "https_proxy": "https://your proxy",
  "no_proxy": "127.0.0.1,localhost",
}

ought to work, assuming the http client(s) in question supports $no_proxy. Which are they, btw?

If you still need to, kernelspecs can define an env dict to specify any environment variables that should be set when launching the kernel, for example:

{
 "argv": ["python3", "-m", "IPython.kernel",
          "-f", "{connection_file}"],
 "display_name": "Python 3",
 "language": "python",
 "env": {"http_proxy": "..."}
}

so patching kernelspec .json files is one way to set envs. Another is to subclass MappingKernelManager, extending .start_kernel() to pass additional env variables at kernel start time.

But modifying kernelspecs won’t affect terminals, so the no_proxy exception for the hub ip is probably preferable, if it works.

3 Likes
5

Thank you @willirath and @minrk. Setting http_proxy and https_proxy together with no_proxy seems to work in the terminal :slight_smile:

6

Great! I’ll mark this as a solution.

7

assuming the http client(s) in question supports $no_proxy. Which are they, btw?

I aimed at using git, curl, urllib (via cartopy). All of these work with the no_proxy solution. (And I guess they all link to the same system libraries in the end?)

8

These env variables aren’t system-level configuration, support is only by convention and implemented separately in each client library (urllib’s is here), which means it can be spotty, especially for some of the less-used edge cases (see longstanding issues in requests for instance).

It’s good to know that your case is covered!

1 Like