I recall that this behaviour varies between file systems (and thus is different by default on Windows as compared to Linux). I think setting FileContentsManager.always_delete_dir to true in jupyter-server configuration might help you, but there are caveats. See Config file and command line options — Jupyter Server documentation