Hi,
Following your suggestion, I built the Docker image locally, pushed it to Docker Hub and successfully launched RStudio instance using the pre-built image. When launching the RStudio instance via binder, it was actually A LOT faster (minutes) using the pre-built Docker image than building it in the binder server (hours). I think this will the default way of making my repo binder-ready for my new projects.
Many thanks for the great tips!