Hello,
I’m trying to use Enterprise Gateway(EG) for my K8S environment project.
For structure simplicity, I wish that just one EG objects can manage many other clusters. It means that EG objects at “Management Cluster” have to swarm kernels to “A Work Cluster” or “B Work Cluster” or “C work Cluster” or any cluster.
But I found something at the EG docs that
"Within the base framework, the IP address must be a local IP address meaning that the kernel cannot be remote from the library launching the kernel. " (link)
I wonder how to swarm kernels from EG cluster to another cluster.
Thank you very much.
Hyunwoo
“kubernetes”, “enterprise gateway”, “jupyter”, “swarm”
Hi @Hyunwoo_Kim - thank you for the question and welcome to the community!
The Kubernetes support in EG allows for the management of “kernel pods” within a single cluster. Our 2.6 release does introduce a feature that could enable the ability for the K8s-hosted EG to manage kernels within other external clusters (like Hadoop YARN), although I would classify that as experimental.
Wrt to K8s-cluster-A managing kernels targetted for K8s-cluster-B, we might be able to leverage the referenced feature (single-response address) to support this, but that we would be something that would likely need to be contributed. If you’d like to pursue this further, please consider opening an issue or discussion item in the repo.
The text you cite regarding the enforcement of local IPs is merely to introduce the concepts behind Enterprise Gateway and its ability to manage kernel lifecycles within resource-managed clusters. With the introduction of Kernel Provisioners (in jupyter_client 7.0) the “base framework” can support kernels associated with non-local IPs.
I hope you find this helpful.
1 Like