How to send jupyter-server's hide_globs param to singleuser to hide lost+found dir

[mr_z_ro]

Currently in our singleuser JupyterLab servers, the lost+found directory appears which is confusing for our users.

Outside z2jh, the common way to manage this is to send a ContentsManager.hide_globs flag with jupyter-server to hide folders: Block certain filenames from showing up even if hidden files shown · Issue #16517 · jupyterlab/jupyterlab · GitHub

However, it’s not immediately clear how to send that as a config option through z2jh helm config.yaml file. Is there a suggested manner in which to do this?

[mr_z_ro]

Feels like there might be something to do here: Configuration Reference — Zero to JupyterHub with Kubernetes documentation

Possibly set c.singleuser.cmd=NULL then run a custom jupyter-server start command with hide_clobs arg on the server itself?

[mahendrapaipuri]

You can set it on singleuser.cmd like c.singleuser.cmd = ['--ContentsManager.hide_globs=<opt>'] which should be propagated to jupyter_server

[mr_z_ro]

Great, thank you for this @mahendrapaipuri !

For posterity, here is the default hide_globs in the jupyter-server source code, which is truncated in the documentation I’ve linked above: jupyter_server/jupyter_server/services/contents/manager.py at 74655ce66f36ed85a83591e6658e70ba91232580 · jupyter-server/jupyter_server · GitHub

[mr_z_ro]

@mahendrapaipuri it looks as though it’s now being run as it’s own command rather than being passed through to jupyter-server. I’m getting the following error on the pod post-spawn:

[FATAL tini (7)] exec --ContentsManager.hide_globs='["__pycache__", "*.pyc", "*.pyo", ".DS_Store", "*.so", "*.dylib", "*~", "lost+found",]' failed: No such file or directory

Should I instead be adding a first element in that array such that it’s c.singleuser.cmd = ['jupyter-singleuser', '--ContentsManager.hideglobs=<opt>'] ?

[mahendrapaipuri]

Oh yeah you need to add full command i.e., you need to add jupyter-singleuser as you rightly noticed. Sorry my bad!!

[mr_z_ro]

@mahendrapaipuri thank you for your help so far.

Two things:

1. I realized the actual command is jupyterhub-singleuser (sorry I’d missed the “hub” part of that in my last comment)

2. I’ve added cmd: ["jupyterhub-singleuser", "--ContentsManager.hide_globs='[\"__pycache__\", \"*.pyc\", \"*.pyo\", \".DS_Store\", \"*.so\", \"*.dylib\", \"*~\", \"lost+found\",]'"] to the singleuser attribute in the helm chart and the servers are spawning nicely now. However, despite this I am still seeing the lost+found directory appearing in my jupyterlab instances. Any thoughts on what I’m doing wrong?

[mahendrapaipuri]

As it is a list of strings, maybe something strange happening during escaping? I guess jupyter_server is attempting to read it as a string and not list of string and so ignoring it? If you enable debug logging and check single user pod logs, you ll see if the config parameter is being taken into account or not.

A better solution would be to add a config file to one of the config paths of jupyter.

$ jupyter --paths
config:
    /home/jovyan/.jupyter
    /home/jovyan/.local/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/jovyan/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/jovyan/.local/share/jupyter/runtime

A file as follows:

$ cat jupyter_server_config.py
c.ContentsManager.hide_globs = ["lost+found", ...]

I dont know if there is way to inject files into single user pods at runtime without needing to create a custom image.

[mr_z_ro]

Thank you as always @mahendrapaipuri - while it’s admittedly frustrating this cmd approach doesn’t work (based on the logs the escaping appears to be received just fine), I do seem to recall reading somewhere that jupyterhub-singleuser does not actually process cmd line args as part of some design decision.

So this alternative approach is not only helpful in working around that, but provides me more customizability than just this one param too. Moreover, I’m already pulling a custom image for singleuser instances, so copying a file to take care of this is time consuming, but will work for me.

For anyone coming across this in the future, I am taking the following approach:

1. ssh into a running singleuser jupyterlab server
2. run jupyter server --generate-config to generate a config file with defaults and comments
3. customize the config file using
4. putting into /home/jovyan/.jupyter/jupyter_server_config.py on my custom Docker image
5. building and deploying that image

Given the amount of time step 5 typically takes, I’m hopeful this works

[mahendrapaipuri]

@mr_z_ro

jupyterhub-singleuser does not actually process cmd line args as part of some design decision.

No, it does. We just it in our production JupyterHub. I guess the argument is being passed as string and not list and so it is getting ignored (price of not having type safety)

You should be able to use singleUserFiles to inject this config file at runtime without having to build custom image.

[mr_z_ro]

Unfortunately, it appears this config param is being ignored. Despite following the process I outlined above, I’m still seeing my “lost+found” dir in jupyterlab.

[mr_z_ro]

I think the new docker I created has not been pulled by the spawner and it’s still using the previous manifestation, despite pullPolicy: Always and tag: latest – either of which should force an update, but neither of which seem to work for me (and apparently many others who’ve posted about this).

Will try with another tag to force the download.