Hello,
I managed to create a Binder and was able to launch it. However, now after some time I was trying to rebuild it and struggle with the following error. Does anyone have an idea? The IRkernel package is not coming from my specification.
.binder files: aquainfra-usecase-Daugava/.binder at containerize · AstraLabuce/aquainfra-usecase-Daugava · GitHub
Binder: Binder
It looks like you based your approach on here. If you click here to try to launch from there, you’ll see the example encounters the same thing.
So for now, this way isn’t working. I don’t have time to look around on this forum right now, but I thought this was pointed out a few weeks or months ago. (It may have been related to use of ‘Shiny’?) If so, looking there may offer you more insight.
If you go to the section ‘Specifying an R environment with a runtime.txt file’ of the ’ Sample Binder Repositories’ documentation page in the Binder User guide, you’ll see there is a note warning against using that as a basis." We recommend to follow r-conda instead." That leads to here.
And I can attest that currently works well to get an R kernel as it both works in launches from the guide version (see note below though) and works here in launches from my repo based on it.
Note: With the r-conda demo site there is an issue with the example ‘Jupyter + R’ that I have asked to be fixed.
1 Like
Sorry, it was Dash that was the issue here. So what I was thinking of didn’t involve that repo using runtime to run R.
Looking at your repo though it looks like you can possibly convert it to using environment.yml with guidance from my last post?
I know it doesn’t pertain to OP, yet because I raised ‘Shiny’ in this thread though, for the sake of others getting here:
Sorry, it was Dash that was the issue here. Not Shiny.
I had though raised issues about Shiny a long time ago here. I don’t know if it ever was workin in the interim there.
I will point out in regards to Shiny presently, there is py.cafe. It says ‘Shiny for Python’ though and I don’t know how easy the conversions are though.
There are a few pending repo2docker PRs related to R
https://github.com/jupyterhub/repo2docker/pulls
Mostly waiting on someone to build and run the PR to confirm it’s working as expected since we don’t have any UI tests for Rshiny or Rstudio.
1 Like
Thanks so far! I am currently trying to convert it to an environment.yml but struggle with some version conflicts because of r-terra, which comes in version 1.7-46 though 1.8.5 is required by the library mapview. Anyway, I will keep trying.
I just looked at a version you had updated 5 minutes ago (this one here I believe).
A couple of things:
- Ideally, do not pin things at the outset. Or at least keep it to minimum. Let conda sort it out. Only pin once you determine a combination that works & only if you are sure you’ll need it. It may be best to at least just record the combination that works so you can recover later if things change out of sync, as sometimes happens.
- I also thought you couldn’t have
defaultsthere under channel anymore and have it build completely? That was the case when I had investigated here. Or at least it is definitely not recommended.
You can still install with R if that helps you in your case, too. My repo has an example where I use an R script to install a library as well.
Alright, I think I am one step further. I removed the versions and defaults from the env.yml. It’s just strange that the libs mapview and tmap don’t work as they expect r-terra>=1.8.5.
If I install mapview locally, it installs r-terra=1.8.15 but when I install r-tmap afterwards, it wants to downgrade r-terra to 1.7_83, although it also actually requires 1.8.5. I will continue playing a bit with the env.yml :).
1 Like
So, I couldn’t find a solution and start believing that there is a more fundamental compatibility problem. If I don’t set r-terra to 1.8._15, it installs 1.7_sth. and if I set the version, it cannot resolve the compatibility issue. Strange though.
Sorry. I don’t know what to suggest for that issue. Since it is offered in conda-forge, I wonder if you could learn what combination may work by using Anaconda Cloud?
I understand that environment.yml is the recommended way, but do I get it right that runtime.txt + install.R is not going to work anymore at all? Or is this a temporary issue that will be fixed? I also tried some other Binders that used runtime.txt and they also didn’t work. So, all Binders using runtime.txt will need to convert?
Thanks so far for your suggestions!
You’d have to look into the details that @manics posted above. All I know is that the runtime approach hasn’t worked for a while now and the recommended way is with conda. You also may want to check out Jupyter’s Zulip to maybe see what was discussed there and get more input.
Sorry you’re having issues, @MarkusKonkol. Our unit and integration tests for runtime.txt do pass, so I opened Investigate potential issues with our `runtime.txt` implementation for R · Issue #1415 · jupyterhub/repo2docker · GitHub to try to investigate what the issues are.
If you (and perhaps @fomightez?) could link to some repos that are currently failing, that would help identify issues and see what needs to be fixed.
1 Like
I was trying Binder Examples repo: ‘Specifying an R environment with a runtime.txt file’.
I just tried to launch a session from that repo again right now and it gave the same issue. Build fails with:
...
> install.packages(c('devtools', 'IRkernel', 'shiny'), repos=Sys.getenv("EXPANDED_CRAN_MIRROR_URL"))
Installing packages into ‘/srv/rlibs’
(as ‘lib’ is unspecified)
Warning: unable to access index for repository https://packagemanager.posit.co/all/__linux__/jammy/2024-01-10+kZwhsmgb/src/contrib:
cannot open URL 'https://packagemanager.posit.co/all/__linux__/jammy/2024-01-10+kZwhsmgb/src/contrib/PACKAGES'
>
>
Warning message:
packages ‘devtools’, ‘IRkernel’, ‘shiny’ are not available for this version of R
Versions of these packages for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages
> IRkernel::installspec(prefix=Sys.getenv("NB_PYTHON_PREFIX"))
Error in loadNamespace(x) : there is no package called ‘IRkernel’
Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted
---> Removed intermediate container ed72fee468b9
The command '/bin/sh -c export EXPANDED_CRAN_MIRROR_URL="$(. /etc/os-release && echo https://packagemanager.posit.co/all/__linux__/${VERSION_CODENAME}/2024-01-10+kZwhsmgb | envsubst)" && R --quiet -e "install.packages(c('devtools', 'IRkernel', 'shiny'), repos=Sys.getenv(\"EXPANDED_CRAN_MIRROR_URL\"))" && R --quiet -e "IRkernel::installspec(prefix=Sys.getenv(\"NB_PYTHON_PREFIX\"))"' returned a non-zero code: 1Error in event stream: Error
ty @fomightez! I’ve opened Add a test for runtime.txt with full R version specified by yuvipanda · Pull Request #1416 · jupyterhub/repo2docker · GitHub to add a test with that runtime.txt, let’s see if that fails! If it does that should allow us to figure out how to fix it.
1 Like
Thanks for looking into this! In case you need another example, feel free to check this one: GitHub - AstraLabuce/aquainfra-usecase-Daugava at 48f0485ef513f7271f472e09c2e2d2b629ecf58c. In the meantime I changed the dependencies to conda, but in that commit you will find runtime.txt/install.R.
1 Like
Ty @MarkusKonkol too!
I am investigating, as the PR I opened now succeeds, and repo2docker running locally on my machine works!
It looks like something is blocking connections from binder to packagemanager.posit.co! Let me poke around to see what that may be.
Fascinating. So connecting to packagemanager.posit.co (but not to posit.co or other things) fails only when run within a docker build, and nowhere else. not during docker run, not from kubernetes pod, nor from the host.
With the following dockerfile:
FROM buildpack-deps:24.04
RUN curl -vvv "https://packagemanager.posit.co/" -o water
RUN head water
This command fails on the mybinder, when run from inside the dind container:
$ docker buildx build --progress=plain -t wat .
[...]
#5 [2/3] RUN curl -vvv "https://packagemanager.posit.co/" -o water
#5 0.279 % Total % Received % Xferd Average Speed Time Time Time Current
#5 0.279 Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Host packagemanager.posit.co:443 was resolved.
#5 0.293 * IPv6: (none)
#5 0.293 * IPv4: 3.147.162.64, 3.132.14.200, 18.221.183.63
#5 0.293 * Trying 3.147.162.64:443...
#5 0.402 * Connected to packagemanager.posit.co (3.147.162.64) port 443
#5 0.405 * ALPN: curl offers h2,http/1.1
#5 0.405 } [5 bytes data]
#5 0.405 * TLSv1.3 (OUT), TLS handshake, Client hello (1):
#5 0.405 } [512 bytes data]
#5 0.444 * CAfile: /etc/ssl/certs/ca-certificates.crt
#5 0.444 * CApath: /etc/ssl/certs
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0^C
eventually timing out after 5min
However, docker buildx build --progress=plain -t wat --network=host . immediately succeeds.
But this is only an issue for packagemanager.posit.co. Every other website works just fine.
So this is some cursed network situation we need to figure out!
Update: I’ve made more progress! By creating a docker buildkit builder of type docker-container, I’m able to exec into it and reproduce this behavior.
So it’s some networking thing in the buildkit layer that’s causing this 
@MarkusKonkol @fomightez success! This was not an issue in repo2docker (so the tests didn’t catch it), but an issue at the tcp/ip layer on docker’s default configuration in a specific set of circumstances interacting with posit packagemanager’s edge node configuration… after a lot of digging, I’ve fixed it now with Explicitly set docker's MTU to 1450 by yuvipanda · Pull Request #3235 · jupyterhub/mybinder.org-deploy · GitHub! I definitely feel like I fought the computer today and won 
Both the binder-examples/r repo and the AstraLabuce repo work now on mybinder \o/
Thank you for reporting it, and thank you for being patient as I debugged this complex infrastructure (than software) issue!
1 Like
I can confirm that it’s working. Great job, thanks!