Kernel for Spark + R

Kernels
1

I am trying to create kernel for R with Spark. So far I was able to install IRkernel package and create config without Spark like below, and it works

{
    "display_name": "R",
    "language": "R",
    "argv": ["R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"]
}

but while I try to add Spark (by replacing R with sparkR for instance in argv), kernel fails to connect with error Fatal error: you must specify '--save', '--no-save' or '--vanilla', which I failed to fix (tried to pass env var “SPARKR_SUBMIT_ARGS”: “–no-save”). Any suggestions, how to set up such kernel correctly?

1 Like