If you want to start from jupyter/base-notebook try following these instructions to build your own image without repo2docker:
https://jupyter-docker-stacks.readthedocs.io/en/latest/using/recipes.html#using-mamba-install-recommended-or-pip-install-in-a-child-docker-image