Skip to content

Cluster Inter-Connection with Submariner

Submariner is an open source networking component that connects multiple Kubernetes cluster Pod and Service networks which can help Kube-OVN interconnect multiple clusters.

Compared to OVN-IC, Submariner can connect Kube-OVN and non-Kube-OVN cluster networks, and Submariner can provide cross-cluster capability for services. However, Submariner currently only enables the default subnets to be connected, and cannot selectively connect multiple subnets.

Prerequisites

  • The Service CIDRs of the two clusters and the CIDR of the default Subnet cannot overlap.

Install Submariner

Note

Submariner uses nftables as the default forwarding backend since v0.22.0, which will conflict with Kube-OVN rules. Please execute the following commands in all clusters that need cluster interconnection before installing Submariner:

kubectl create namespace submariner-operator
kubectl create configmap submariner-global --namespace=submariner-operator --from-literal=use-nftables=false

Download the subctl binary and deploy it to the appropriate path:

curl -Ls https://get.submariner.io | bash
export PATH=$PATH:~/.local/bin
echo export PATH=\$PATH:~/.local/bin >> ~/.profile

Change kubeconfig context to the cluster that needs to deploy submariner-broker:

subctl deploy-broker

In this document the default subnet CIDR for cluster0 is 10.16.0.0/16 and the join subnet CIDR for cluster0 is 100.64.0.0/16, the default subnet CIDR for cluster1 is 11.16.0.0/16 and the join subnet CIDR for cluster1 is 100.68.0.0/16.

Switch kubeconfig to cluster0 to register the cluster to the broker, and register the gateway node:

subctl  join broker-info.subm --clusterid  cluster0 --clustercidr 100.64.0.0/16,10.16.0.0/16  --natt=false --cable-driver vxlan --health-check=false
kubectl label nodes cluster0 submariner.io/gateway=true

Switch kubeconfig to cluster1 to register the cluster to the broker, and register the gateway node:

subctl  join broker-info.subm --clusterid  cluster1 --clustercidr 100.68.0.0/16,11.16.0.0/16  --natt=false --cable-driver vxlan --health-check=false
kubectl label nodes cluster1 submariner.io/gateway=true

If no new gateway or routeagent pods appear after executing the join command, please add the following permissions to the submariner-operator clusterrole:

- apiGroups:
  - "apps"
  resources:
  - daemonsets
  verbs:
  - create
  - get
  - list
  - watch
  - update

For multi-node clusters, you need to change the gateway configuration of the default subnet ovn-default to centralized. The gateway nodes configured for submariner need to be exactly the same as the subnet nodes.

Next, you can start Pods in each of the two clusters and try to access each other using IPs.

Network communication problems can be diagnosed by using the subctl command:

subctl show all
subctl diagnose all

For more Submariner operations please read Submariner Usage.

PDF Slack Support

Comments