Need guidance on setting up hub sidecar container

Hello all! I just found this forum today and wanted to thank you all for maintaining the JupyterHub on Kubernetes.

As I am a bit of a novice when it comes to Kubernetes I was hoping to get some advice on my specific setup. I currently have a dockerized auth application that I am trying to stand up in front my JupyterHub installation. What the auth application does is:

  1. Check if cookie is set and is valid. If cookie is set and is valid, continue to JupyterHub. If not, go to step 2.
  2. Redirect to URL to set cookie and go back to step 1.

I did a proof of concept and stood this auth application up in front of a “React Hello World” application. The config files looked like this:

---
apiVersion: v1
kind: Service
metadata:
  labels:
    tier: primary
  name: docker-react
spec:
  ports:
  - name: auth-port
    port: 8080
    targetPort: 8080
  - name: port-80
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    tier: primary
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    tier: primary
  name: docker-react
spec:
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      tier: primary
  template:
    metadata:
      labels:
        sdr.appname: docker-react-primary
        tier: primary
    spec:
      containers:
      - image: REACT HELLO WORLD DOCKER IMAGE
        imagePullPolicy: IfNotPresent
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 30
          timeoutSeconds: 30
        name: docker-react
        ports:
        - containerPort: 80
          protocol: TCP
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 30
          timeoutSeconds: 30
        resources:
          limits:
            cpu: "2"
            memory: 1Gi
          requests:
            cpu: 500m
            memory: 1Gi
      - args:
        - -e=http://localhost:80
        - -p=8080
        env:
        - name: APC_APP_ADMIN_PASSWORD
          valueFrom:
            secretKeyRef:
              key: APC_APP_ADMIN_PASSWORD
              name: docker-react-auth-primary-secrets
        envFrom:
        - configMapRef:
            name: docker-react-auth-primary-env-config
        image: DOCKER AUTH IMAGE
        imagePullPolicy: IfNotPresent
        livenessProbe:
          httpGet:
            path: /__health
            port: 8080
          timeoutSeconds: 30
        name: docker-react-auth
        ports:
        - containerPort: 8080
          protocol: TCP
        resources: {}
      priorityClassName: p4
      restartPolicy: Always

So essentially what this proof of concept did was:

user → auth → react hello world

I am currently trying to stand this auth application in front of a JupyterHub installation. I was able to deploy the auth application along side the hub using the extraContainers property here: zero-to-jupyterhub-k8s/schema.yaml at a9ae11913d7052d2ca59383200c434e695172fc5 · jupyterhub/zero-to-jupyterhub-k8s · GitHub

I am now trying to figure out how to tell the proxy-public service to route to the auth application rather than directly to the hub.

This is my current config file:

hub: 
  extraContainers:
- args:
  - -e=http://localhost:80
  - -p=8080
  env:
  - name: APC_APP_ADMIN_PASSWORD
    valueFrom:
      secretKeyRef:
        key: APC_APP_ADMIN_PASSWORD
        name: docker-react-auth-primary-secrets
  envFrom:
  - configMapRef:
      name: docker-react-auth-primary-env-config
  image: <DOCKER AUTH IMAGE>
  imagePullPolicy: IfNotPresent
  livenessProbe:
    httpGet:
      path: /__health
      port: 8080
    timeoutSeconds: 30
  name: docker-react-auth
  ports:
  - containerPort: 8080
    protocol: TCP
  resources: {}
proxy:
  secretToken: "token"
  service:
type: ClusterIP
extraPorts:
- name: auth-port
  port: 8080
  targetPort: 8080
debug:
  enabled: true

Any and all help is very, very appreciated! Thank you all so much.

I hope that all made sense. What I wanted to do at first was:

user -> auth application that sets cookie -> proxy-public -> hub

But I wasn’t able to find any documentation on how to set up a sidecar container in the proxy-public service. So right now I am trying to do:

user -> proxy-public -> auth application that sets cookie -> hub

Hi! Would you mind expanding on your overall goal? At the moment it sounds like you want two layers of authentication, one to reach the JupyerHub login page, followed by JupyerHub’s own authentication. Is that correct?

If what you actually want is for JupyterHub to use your auth application then you can write your own authenticator: Authenticators — JupyterHub 1.3.0 documentation
You could then run your auth application as a standalone k8s service, and have your custom JupyterHub authenticator connect to it directly, or redirect users to it, depending on exactly what you want.

Hi Manics, thanks for the reply!

Yes that’s what I want. Basically my company has a single sign on service which has been dockerized.

So my overall goal is when people at my company visit my JupyterHub installation it first redirects to our dockerized auth application. Our auth application, once authenticated, would redirect back. So it would look like

user -> https://my-jupyterhub.com -> auth service -> https://my-jupyterhub.com -> proxy-public

I hope that made sense. So would in this case would I implement a custom authenticator?

Manics,

Ideally it would be great if it was possible to install a sidecar container inside of the proxy-public pod. And I would have the auth container would listen on port 8080 and the proxy-public would listen on 80.

Is that possible?

Yes, you would set some cookie or whatever and then the JupyterHub Authenticator only reads this cookie instead of allowing a login page. You can see this shibboleth authenticator as an example pattern, which relies on headers set in an apache or nginx shibboleth plugin. This pattern sounds like what you are after.

There are two main choices of pattern:

  1. the regular hub proxy is the public entrypoint, and your custom LoginHandler redirects to your auth service if the cookie is not set
  2. your auth service is actually the only public endpoint and it proxies requests to the hub proxy only if they are authenticated

In both cases, instead of a sidecar container on an existing pod, I would use a separate service and deployment for the auth. In the proxy case, the hub proxy-public does not need to be publicly exposed, it can be a private service that is only accessible from your proxy (or at least from inside your cluster).

It would probably be best to use a JWT or other reasonably trusted signed value, if that’s an option available to you.

1 Like