Spawn failures owing to pod security policies

JupyterHub Zero to JupyterHub on Kubernetes
1

I’m seeing symptoms similar to Single User Container Not Creating due to Pod Policies i.e. spawn failures, with the error given as:

HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"jupyter-user1\" is forbidden: PodSecurityPolicy: unable to admit pod: []","reason":"Forbidden","details":{"name":"jupyter-user1","kind":"pods"},"code":403}

I’ve tried the suggestion there i.e.

singleuser:
  #...
  cloudMetadata:
      blockWithIptables: false

and that hasn’t helped.

These are the network policies:

 kubectl get networkPolicy -n jupyterhub
NAME         POD-SELECTOR                                                   AGE
autohttps    app=jupyterhub,component=autohttps,release=z2jh-0.14           31m
hub          app=jupyterhub,component=hub,release=z2jh-0.14                 3h28m
proxy        app=jupyterhub,component=proxy,release=z2jh-0.14               3h28m
singleuser   app=jupyterhub,component=singleuser-server,release=z2jh-0.14   3h28m

This is using Helm chart 1.2.0, the k8s cluster is running 1.23.4, on OpenStack via Magnum.

This is the pod security policy:

kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME                PRIV   CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
magnum.privileged   true   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *
2

I was able to work around this by applying a rolebinding which authorized service accounts.

3

Hi,

I am actually facing the same issue, could you please provide more details about your solution?

Thanks!

4

Sure, we apply this:

5

Thanks! Is it me or there is a missing part of your answer?

6

Hmm, seems it stripped out the code from my e-mail.

The YAML code is:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jupyterhub-rolebinding
  namespace: jhub
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: magnum:podsecuritypolicy:privileged
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts
2 Likes
7

Thank you! that worked for me as well!