Kubernetes Multi-Cluster Ingress With Config Connector – part 1

Disclaimer: This blog contains opinions about Google technology. While I work at Google, this is my personal blog. Opinions stated here are my own, not those of my company.

If you want to expose your cluster to receive external traffic, there are multiple ways to do it. Before we get to how you can configure Kubernetes multi-cluster ingress with Config Connector, let’s review other load balancing concepts. Specifically, you can use the following to expose a single cluster to receive external traffic.

  • ClusterIP – default Kubernetes Service type that exposes a service for other workloads in the cluster. It doesn’t expose the traffic externally by default, but you can, using kubectl proxy, which can be helpful for debugging and other diagnostics scenarios.
  • NodePort – in addition to what ClusterIP does, exposes the service on all nodes.
  • LoadBalancer – L4 load balancer. On GKE, it will provision an IP address that you can use to directly expose a service externally.
  • Ingress – L7 load balancer that, on GKE will create forwarding rules, target proxy, url map and health checks.

To learn more on different Kubernetes services I recommend this and this article.

If you have a single Kubernetes cluster on GKE and would like a reliable way to route HTTP(s) traffic to it, using Ingress K8s resource is a standard approach. In fact, this is what I am doing in my WordPress example.

However, what if you have multiple Kubernetes clusters in different regions? How would you route the traffic to the region closest to the user?

As of the time of this writing, there is no standard approach to do it by initializing a single Kubernetes resource, like you can do with Ingress or LoadBalancer. A temporary solution is to use kubemci tool, that provisions all the required GCP resources.

Config Connector aims to provide universal declarative support for all GCP resources. Thus, it is now almost (with some gaps that we will describe) possible to configure multi-cluster ingress declaratively. Let’s review how you can do it. While it is not giving you the ease and full support of Ingress resource, it will provide, once the gaps are addressed, declarative Kubernetes model that will seamlessly integrate with the rest of your Kubernetes configuration.

Multi-cluster Ingress – easy with Config Connector

Let’s see how you can provision resources required for Kubernetes multi-cluster ingress with Config Connector.

First, let’s look at the diagram showing the objects that we are about to create:

First we setup Config Connector on the "service cluster", which then provisions all the required multi-cluster ingress resources.

We will start with the initial cluster, and we will create it without using Config Connector. We call it genesis cluster, or, on my diagram, service cluster. It will have Config Connector enabled and we will use it create all subsequent GCP objects: clusters and all the other resources that we need to set up multi-cluster load-balancing solution.

We’ll continue on this in the next post.

Leave a Comment