Changing yarn queue for Jhub

JupyterHub
, ,
1

Hi All,
We use jupyterhub in a hadoop cluster, At the moment all the jupyterhub jobs are submitted as pyspark-shell to default yarn queue. We have created a new yarn queue and we want to use this new yarn queue for Jhub jobs. Where and how I can achieve this? Please advise.

Thanks

2

I fixed this by adding spark.yarn.queue =new queue name in the kernel.json’s pyspark-submit command. It is working fine.