OVN Remote Port Mirroring¶
This feature provides ability to mirror the traffic of the specified Pod and direction, and to send the mirrored traffic to a remote destination.
This feature requires Kube-OVN version not lower than v1.12.
Install Multus-CNI¶
Install Multus-CNI by referring the Multus-CNI Document.
Create NetworkAttachmentDefinition¶
Create the following NetworkAttachmentDefinition:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: attachnet
namespace: default
spec:
config: |
{
"cniVersion": "0.3.1",
"type": "kube-ovn",
"server_socket": "/run/openvswitch/kube-ovn-daemon.sock",
"provider": "attachnet.default.ovn"
}
Format of the provider
field is <NAME>.<NAMESPACE>.ovn
.
Create Underlay Network¶
The mirrored traffic is encapsulated before transmition, so MTU of the network used to transmit the traffic should be greater than the mirrored LSP/Pod. Here we are using an underlay network.
Create the following underlay network:
apiVersion: kubeovn.io/v1
kind: ProviderNetwork
metadata:
name: net1
spec:
defaultInterface: eth1
---
apiVersion: kubeovn.io/v1
kind: Vlan
metadata:
name: vlan1
spec:
id: 0
provider: net1
---
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
name: subnet1
spec:
protocol: IPv4
cidrBlock: 172.19.0.0/16
excludeIps:
- 172.19.0.2..172.19.0.20
gateway: 172.19.0.1
vlan: vlan1
provider: attachnet.default.ovn
The subnet's provider
MUST be the same as the provider
of the NetworkAttachmentDefinition created above.
Create Receiving Pod¶
Create the following Pod:
apiVersion: v1
kind: Pod
metadata:
name: pod1
annotations:
k8s.v1.cni.cncf.io/networks: default/attachnet
spec:
containers:
- name: bash
image: docker.io/kubeovn/kube-ovn:v1.12.26
args:
- bash
- -c
- sleep infinity
securityContext:
privileged: true
After the Pod has been created, checkout the IP addresses:
$ kubectl get ips | grep pod1
pod1.default 10.16.0.12 00:00:00:FF:34:24 kube-ovn-worker ovn-default
pod1.default.attachnet.default.ovn 172.19.0.21 00:00:00:A0:30:68 kube-ovn-worker subnet1
The IP address 172.19.0.21
will be used later.
Create OVN Remote Port Mirroring¶
Create the following OVN remote port mirroring:
kubectl ko nbctl mirror-add mirror1 gre 99 from-lport 172.19.0.21
kubectl ko nbctl lsp-attach-mirror coredns-787d4945fb-gpnkb.kube-system mirror1
coredns-787d4945fb-gpnkb.kube-system
is the OVN LSP name with a format <POD_NAME>.<POD_NAMESPACE>
.
Here is the OVN command usage:
ovn-nbctl mirror-add <NAME> <TYPE> <INDEX> <FILTER> <IP>
NAME - add a mirror with given name
TYPE - specify TYPE 'gre' or 'erspan'
INDEX - specify the tunnel INDEX value
(indicates key if GRE, erpsan_idx if ERSPAN)
FILTER - specify FILTER for mirroring selection
('to-lport' / 'from-lport')
IP - specify Sink / Destination i.e. Remote IP
ovn-nbctl mirror-del [NAME] remove mirrors
ovn-nbctl mirror-list print mirrors
ovn-nbctl lsp-attach-mirror PORT MIRROR attach source PORT to MIRROR
ovn-nbctl lsp-detach-mirror PORT MIRROR detach source PORT from MIRROR
Configure Receiving Pod¶
Execute the following commands in the Pod:
root@pod1:/kube-ovn# ip link add mirror1 type gretap local 172.19.0.21 key 99 dev net1
root@pod1:/kube-ovn# ip link set mirror1 up
Now you can capture the mirrored packets:
root@pod1:/kube-ovn# tcpdump -i mirror1 -nnve
tcpdump: listening on mirror1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
05:13:30.328808 00:00:00:a3:f5:e2 > 00:00:00:97:0f:6e, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.16.0.7 tell 10.16.0.4, length 28
05:13:30.559167 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 212: (tos 0x0, ttl 64, id 57364, offset 0, flags [DF], proto UDP (17), length 198)
10.16.0.4.53 > 10.16.0.6.50472: 34511 NXDomain*- 0/1/1 (170)
05:13:30.559343 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 212: (tos 0x0, ttl 64, id 57365, offset 0, flags [DF], proto UDP (17), length 198)
10.16.0.4.53 > 10.16.0.6.45177: 1659 NXDomain*- 0/1/1 (170)
05:13:30.560625 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 200: (tos 0x0, ttl 64, id 57367, offset 0, flags [DF], proto UDP (17), length 186)
10.16.0.4.53 > 10.16.0.6.43848: 2636*- 0/1/1 (158)
05:13:30.562774 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 191: (tos 0x0, ttl 64, id 57368, offset 0, flags [DF], proto UDP (17), length 177)
10.16.0.4.53 > 10.16.0.6.37755: 48737 NXDomain*- 0/1/1 (149)
05:13:30.563523 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 187: (tos 0x0, ttl 64, id 57369, offset 0, flags [DF], proto UDP (17), length 173)
10.16.0.4.53 > 10.16.0.6.53887: 45519 NXDomain*- 0/1/1 (145)
05:13:30.564940 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 201: (tos 0x0, ttl 64, id 57370, offset 0, flags [DF], proto UDP (17), length 187)
10.16.0.4.53 > 10.16.0.6.40846: 25745 NXDomain*- 0/1/1 (159)
05:13:30.565140 00:00:00:a3:f5:e2 > 00:00:00:89:d5:cc, ethertype IPv4 (0x0800), length 201: (tos 0x0, ttl 64, id 57371, offset 0, flags [DF], proto UDP (17), length 187)
10.16.0.4.53 > 10.16.0.6.45214: 61875 NXDomain*- 0/1/1 (159)
05:13:30.566023 00:00:00:a3:f5:e2 > 00:00:00:55:e4:4e, ethertype IPv4 (0x0800), length 80: (tos 0x0, ttl 64, id 45937, offset 0, flags [DF], proto UDP (17), length 66)
10.16.0.4.44116 > 172.18.0.1.53: 16025+ [1au] AAAA? kube-ovn.io. (38)
Notice¶
- If you are using ERSPAN as the encapsulation protocol, the Linux kernel version of the OVN nodes and remote devices must not be lower than 4.14. If you are using ERSPAN as the encapsulation protocol and using IPv6 as the transport network, the Linux kernel version must not be lower than 4.16.
- The transmission of mirrored traffic is unidirectional, so you only need to ensure that the OVN node can access the remote device.
微信群 Slack Twitter Support Meeting