Best practices for ssh credentials on cloud-based jupyterhubs

JupyterHub Zero to JupyterHub on Kubernetes
1

I am trying to do something very simple: push to a git repo from the jupyterlab terminal in K8S-based jupyterhub environment.

Because I have two-factor authentication enabled on github, I need to either use an SSH key or a personal access token.

The way I solve this on all other remote systems is by using ssh key forwarding. But that doesn’t work here, since I’m not logging in via SSH. My options seem to be

  • Generate the personal access token, and write it on a piece of paper
  • Generate the personal access token and store it plain text on the hub
  • Copy my existing SSH key to the hub
  • Create a new SSH key on the hub and register it with github

None of these is particularly appealing. But it would help to know how “secure” a K8S jhub is. Can I safely store secrets like this in my home directory?

2 Likes
2

If you put ssh secrets in your home directory, the attack vectors I can think of are:

  1. All JupyterHub admins can now see your key, and assume your identity
  2. Kubernetes admins can also do the same
  3. If your home directories are on NFS, anyone who has access to that can see your key
  4. Most importantly, you might forget you have an ssh key here, and someone might discover it a few years down the line in some archived backup.

The most secure way I can think of is:

  1. Generate an ssh key locally, and use a strong passphrase
  2. Copy this key to your hub, and use one key per hub, so you can revoke these easily
  3. Write in your passphrase every time you need to push to GitHub.

This still leaves you open to attacks (1) and (2), but there’s probably no way around that at all. It does protect your key at rest, which is the place it’s most vulnerable in.

In an ideal world, we’d have some time-limited token (like an OAuth token) that can be requested from GitHub post authentication, and be valid only as long as your current Jupyter Notebook server is up. This would force you to authenticate to GitHub from your notebook each time your server starts, but is probably the most secure thing and easy thing we can do.

1 Like
3

Have you tried using a GitHub Deploy key? It’s an SSH key that’s tied to a single GitHub repository, which would limit the damage an attacker could do.

2 Likes