跳转至

SecurityGroup 使用

Kube-OVN 支持了安全组的配置,配置安全组使用的 CRD 为 SecurityGroup。

安全组示例

apiVersion: kubeovn.io/v1
kind: SecurityGroup
metadata:
  name: sg-example
spec:
  allowSameGroupTraffic: true
  egressRules:
  - ipVersion: ipv4
    policy: allow
    priority: 1
    protocol: all
    remoteAddress: 10.16.0.13 # 10.16.0.0/16 配置网段
    remoteType: address
  ingressRules:
  - ipVersion: ipv4
    policy: deny
    priority: 1
    protocol: icmp
    remoteAddress: 10.16.0.14
    remoteType: address

安全组各字段的具体含义,可以参考 Kube-OVN 接口规范

Pod 通过添加 annotation 来绑定安全组,使用的 annotation 有两个:

  • port_security: 源地址校验,如果开启,只有 kube-ovn ipam 分配到的 ip 源地址的包可以从 pod 网卡出去,关闭后, 任意 ip 都可以。
  • security_groups: 安全组列表,包含一系列 ACL 规则。

这两个 annotation 负责的功能是互相独立的。

    ovn.kubernetes.io/port_security: "true"
    ovn.kubernetes.io/security_groups: sg-example

注意事项

  • 安全组最后是通过设置 ACL 规则来限制访问的,OVN 文档中提到,如果匹配到的两个 ACL 规则拥有相同的优先级,实际起作用的是哪个 ACL 是不确定的。因此设置安全组规则的时候,需要注意区分优先级。

  • 当添加安全组的时候,要清楚的知道是在添加什么限制。Kube-OVN 作为 CNI,创建 Pod 后会进行 Pod 到网关的连通性测试,如果访问不通网关,就会导致 Pod 一直处于 ContainerCreating 状态,无法顺利切换到 Running 状态。

实际测试

利用以下 yaml 创建 Pod,在 annotation 中指定绑定示例中的安全组:

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: static
  annotations:
    ovn.kubernetes.io/port_security: 'true'
    ovn.kubernetes.io/security_groups: 'sg-example'
  name: sg-test-pod
  namespace: default
spec:
  nodeName: kube-ovn-worker
  containers:
  - image: docker.io/library/nginx:alpine
    imagePullPolicy: IfNotPresent
    name: qatest

实际测试结果显示如下:

# kubectl get pod -o wide
NAME                   READY   STATUS              RESTARTS   AGE     IP           NODE                     NOMINATED NODE   READINESS GATES
sg-test-pod            0/1     ContainerCreating   0          5h32m   <none>       kube-ovn-worker          <none>           <none>
test-99fff7f86-52h9r   1/1     Running             0          5h41m   10.16.0.14   kube-ovn-control-plane   <none>           <none>
test-99fff7f86-qcgjw   1/1     Running             0          5h43m   10.16.0.13   kube-ovn-worker          <none>           <none>

执行 kubectl describe pod 查看 Pod 的信息,可以看到报错提示:

# kubectl describe pod sg-test-pod
Name:         sg-test-pod
Namespace:    default
Priority:     0
Node:         kube-ovn-worker/172.18.0.2
Start Time:   Tue, 28 Feb 2023 10:29:36 +0800
Labels:       app=static
Annotations:  ovn.kubernetes.io/allocated: true
              ovn.kubernetes.io/cidr: 10.16.0.0/16
              ovn.kubernetes.io/gateway: 10.16.0.1
              ovn.kubernetes.io/ip_address: 10.16.0.15
              ovn.kubernetes.io/logical_router: ovn-cluster
              ovn.kubernetes.io/logical_switch: ovn-default
              ovn.kubernetes.io/mac_address: 00:00:00:FA:17:97
              ovn.kubernetes.io/pod_nic_type: veth-pair
              ovn.kubernetes.io/port_security: true
              ovn.kubernetes.io/routed: true
              ovn.kubernetes.io/security_groups: sg-allow-reject
Status:       Pending
IP:
IPs:          <none>
·
·
·
Events:
  Type     Reason                  Age                    From     Message
  ----     ------                  ----                   ----     -------
  Warning  FailedCreatePodSandBox  5m3s (x70 over 4h59m)  kubelet  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "40636e0c7f1ade5500fa958486163d74f2e2300051a71522a9afd7ba0538afb6": plugin type="kube-ovn" failed (add): RPC failed; request ip return 500 configure nic failed 10.16.0.15 network not ready after 200 ping 10.16.0.1

修改安全组的规则,添加到网关的访问规则,参考如下:

apiVersion: kubeovn.io/v1
kind: SecurityGroup
metadata:
  name: sg-gw-both
spec:
  allowSameGroupTraffic: true
  egressRules:
  - ipVersion: ipv4
    policy: allow
    priority: 2
    protocol: all
    remoteAddress: 10.16.0.13
    remoteType: address
  - ipVersion: ipv4
    policy: allow
    priority: 1
    protocol: all
    remoteAddress: 10.16.0.1
    remoteType: address
  ingressRules:
  - ipVersion: ipv4
    policy: deny
    priority: 2
    protocol: icmp
    remoteAddress: 10.16.0.14
    remoteType: address
  - ipVersion: ipv4
    policy: allow
    priority: 1
    protocol: icmp
    remoteAddress: 10.16.0.1
    remoteType: address

分别在入方向和出方向规则中,添加允许到网关的访问规则,并且设置该规则的优先级最高。

利用以下 yaml 绑定安全组,部署 Pod 后,确认 Pod 可以正常运行:

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: static
  annotations:
    ovn.kubernetes.io/port_security: 'true'
    ovn.kubernetes.io/security_groups: 'sg-gw-both'
  name: sg-gw-both
  namespace: default
spec:
  nodeName: kube-ovn-worker
  containers:
  - image: docker.io/library/nginx:alpine
    imagePullPolicy: IfNotPresent
    name: qatest

部署后查看 Pod 信息:

# kubectl get pod -o wide
NAME                   READY   STATUS              RESTARTS   AGE     IP           NODE                     NOMINATED NODE   READINESS GATES
sg-test-pod            0/1     ContainerCreating   0          5h41m   <none>       kube-ovn-worker          <none>           <none>
sg-gw-both             1/1     Running             0          5h37m   10.16.0.19   kube-ovn-worker          <none>           <none>

因此对于安全组的使用,要特别明确添加的限制规则的作用。如果单纯是限制流量访问,可以考虑使用网络策略实现。

微信群 Slack Twitter Support Meeting

评论