@betatim I really appreciate the help. Good idea to setup a test scenario.
I’m seeing the same thing. I setup your test containers, and was able to 1) curl https://jovyan.example.com and then 2) configure the AsyncHTTPClient and use that to successfully hit the same url.
I also went one step further and created a cert using a CA external to the certificate. This changes the flow to
- Create CA .pem and .key
- Create jovyan.example.com .key and .csr
- Sign the .csr using the CA stuff, producing a .crt for nginx to use
- Add CA .pem to client CA cert store
I’m not seeing any difference in behavior between the two patterns. Both curl and the AsyncHTTPClient can get content from https://jovyan.example.com
This tells me that the tornado client, by default, does pick up CA certs from the default OS store.
When I run similar tests in my live environment (within the binderhub pod deployed via helm), however, I still see SSL issues. I’ve installed my company’s CA cert, so I know I have the required certs in the store.
So, what could be different between the test case and the deployed copy? Just brainstorming here, no confidence in any of these paths:
- The pod container environment has a variable set that changes the behavior of libcurl.
- My company proxies all http(s) trafic by default. Something about the proxy setup is screwing up the flow
- Something else is happening in the binderhub code that is causing pycurl to look away from the default ca store.
I’ll try to falsify each of these one at a time. I think we’re getting closer here. Thanks again for the help