Hi everyone, We are evaluating having multiple ku...
# general
a
Hi everyone, We are evaluating having multiple kubernetes clusters within the same network for redundancy. We're trying to find our way for service discovery. What if service A runs only a single cluster and be unavailable on cluster B? Maybe try to make such services always available on both and keep things simple? How do you guys do this? Any suggestions?
s
it really depends where you are running the clusters, are you using routable pods or not, and which CNI you are using
the upstream APIs for this that are purpose built for this use case is called MCS or in the long form Multi Cluster Service API https://multicluster.sigs.k8s.io/concepts/multicluster-services-api/ like with ingress or gateway API you need a tool which implements this. a great OSS solution many use is https://submariner.io/ if you are using Antrea as a CNI which i personally love, it has a builtin implementation for this: https://github.com/antrea-io/antrea/tree/main/multicluster if you are in GKE this is also supported natively: https://cloud.google.com/kubernetes-engine/docs/concepts/multi-cluster-services and in EKS this can be done via an AWS OSS project: https://github.com/aws/aws-cloud-map-mcs-controller-for-k8s it can also be done with cilium if you are using that as the CNI, using what they call a cluster mesh. it has beta support for the upstream APIs as initially and still the GA option is a custom implementation: https://docs.cilium.io/en/stable/network/clustermesh/services/ Azure have there own implementation as well as part of the Azure Fleet product which does not align with the upstream APIs, at least not yet
a
We use EKS and VPC CNI. We already have multiple clusters which communicate via internal ALB when necessary but now we'll be transforming this into multiple active active clusters. I'll checkout the links you shared. Thank you 🙏🏻
e
Cilium thinking