大型电商网站开发价格,旬阳做网站,wordpress+镜像下载,网络营销理论Linux ipvlan详解#xff0c;测试l2、l3、l3s和bridge、private和vepa模式。 最近在看Docker的网络#xff0c;看到关于ipvlan网络的介绍。查阅了相关资料#xff0c;记录如下。
参考
1.图解几个与Linux网络虚拟化相关的虚拟网卡-VETH/MACVLAN/MACVTAP/IPVLAN 2.IPVlan 详…Linux ipvlan详解测试l2、l3、l3s和bridge、private和vepa模式。 最近在看Docker的网络看到关于ipvlan网络的介绍。查阅了相关资料记录如下。
参考
1.图解几个与Linux网络虚拟化相关的虚拟网卡-VETH/MACVLAN/MACVTAP/IPVLAN 2.IPVlan 详解 3.IPVLAN Driver HOWTO 3.IPVlan 源码探秘 4.ipvlan内核代码流程 5.从 VLAN 到 IPVLAN: 聊聊虚拟网络设备及其在云原生中的应用 6.Linux网络协议栈6–ipvlan 7.ipvlan-l3s模式
环境
操作系统
Ubuntu22.04 kernel 5.15
1. IPVALN 介绍
本节内容参考 链接1、链接2 和 链接3
1.1 IPVLAN虚拟网卡技术
IPVLAN与MACVLAN类似。IPVLAN和MACVLAN的区别在于它在IP层进行流量分离而不是基于MAC地址因此你可以看到同属于一块宿主以太网卡的所有IPVLAN虚拟网卡的MAC地址都是一样的因为宿主以太网卡根本不是用MAC地址来分流IPVLAN虚拟网卡的流量的。具体的流程如下图所示 由于所有的虚拟接口共享同个mac地址因此有些地方需要注意当使用 DHCP 协议分配 ip 时一般会用 mac 地址作为机器的标识因此需要配置唯一的 ClientID 字段作为机器的标识 DHCP server 配置 ip 时需使用该字段作为机器标识而不是使用 mac 地址。
1.2 三种模式mode
ipvlan 有三种不同的工作模式L2 、L3 和 L3s。一个父接口只能选择其中一种模式不能采用混用模式依附于它的所有虚拟接口都会运行在这个模式下。 1、L2 模式 Ipvlan 的 L2 模式和 macvlan 的 bridge 模式工作原理很相似父接口作为交换机来转发子接口的数据。同一个网络的子接口可以通过父接口来转发数据而如果想发送到其他网络报文则会通过父接口的路由转发出去。 2、L3 模式 L3 模式下ipvlan 有点像路由器的功能它在各个虚拟网络和主机网络之间进行不同网络报文的路由转发工作。只要父接口相同即使虚拟机/容器不在同一个网络也可以互相 ping 通对方因为 ipvlan 会在中间做报文的转发工作。该模式把父接口当成一个路由器完全不支持广播这个模式下的接口也比l2模式下的ipvlan接口多了一个 NOARP属性也不会发送广播报文 2、L3s 模式 L3s 模式下与L3 模式类似区别在于启用了iptables (conn-tracking)
上述三种MODE不能混用。
4.1 L2 mode:In this mode TX processing happens on the stack instance attached to the
slave device and packets are switched and queued to the master device to send
out. In this mode the slaves will RX/TX multicast and broadcast (if applicable)
as well.4.2 L3 mode:In this mode TX processing up to L3 happens on the stack instance attached
to the slave device and packets are switched to the stack instance of the
master device for the L2 processing and routing from that instance will be
used before packets are queued on the outbound device. In this mode the slaves
will not receive nor can send multicast / broadcast traffic.4.3 L3S mode:This is very similar to the L3 mode except that iptables (conn-tracking)
works in this mode and hence it is L3-symmetric (L3s). This will have slightly less
performance but that shouldnt matter since you are choosing this mode over plain-L3
mode to make conn-tracking work.1.3 三种FLAGS
ipvlan 有三种不同的flagbridge 、private 和 vepa。bridge允许同一父接口下的子接口直接通讯private禁止子接口之间通讯vepa禁止子接口之间直接通讯需要外部交换机开启hairpin802.1q转发通讯。
上述三种FLAG不能混用。
5. Mode flags:At this time following mode flags are available5.1 bridge:This is the default option. To configure the IPvlan port in this mode,
user can choose to either add this option on the command-line or dont specify
anything. This is the traditional mode where slaves can cross-talk among
themselves apart from talking through the master device.5.2 private:If this option is added to the command-line, the port is set in private
mode. i.e. port wont allow cross communication between slaves.5.3 vepa:If this is added to the command-line, the port is set in VEPA mode.
i.e. port will offload switching functionality to the external entity as
described in 802.1Qbg
Note: VEPA mode in IPvlan has limitations. IPvlan uses the mac-address of the
master-device, so the packets which are emitted in this mode for the adjacent
neighbor will have source and destination mac same. This will make the switch /
router send the redirect message.说明IPVlan 和 MACVlan 会有各种模式mode和 flag比如 bridgeVEPAvirtual ethernet port aggregatorprivatepassthrough。形象举例假如父接口是一个聊天群所有的群成员都可以向外发消息那么可以这样理解
bridge 模式所有群成员可以在群内发言。private 模式所有群成员禁言的既不能在群内发言也不能在群外发言。vepa 模式所有群成员在群内禁言但是可以在群外互相私聊需要外部交换支持802.1q。passthrough 模式只有群主可以发言其他人禁言。
1.4 收发包流程
本节内容参考 链接2、链接5
1.4.1 收包流程
IPVlan 子设备的三种 mode 分别有不同的收包处理流程在内核的流程如下 首先会经过__netif_receive_skb_core 进入到创建时注册的 ipvlan_handle_frame 的处理流程此时数据包依然是主设备所拥有。
rx_handler_result_t ipvlan_handle_frame(struct sk_buff **pskb)
{struct sk_buff *skb *pskb;struct ipvl_port *port ipvlan_port_get_rcu(skb-dev);if (!port)return RX_HANDLER_PASS;switch (port-mode) {case IPVLAN_MODE_L2:return ipvlan_handle_mode_l2(pskb, port);case IPVLAN_MODE_L3:return ipvlan_handle_mode_l3(pskb, port);
#ifdef CONFIG_IPVLAN_L3Scase IPVLAN_MODE_L3S:return RX_HANDLER_PASS;
#endif}/* Should not reach here */WARN_ONCE(true, %s called for mode [%x]\n, __func__, port-mode);kfree_skb(skb);return RX_HANDLER_CONSUMED;
}ipvlan l2 对于 mode l2 模式的报文处理只处理多播的报文将报文放进前面创建子设备时初始化的多播处理的队列对于单播报文会直接交给 ipvlan_handle_mode_l3 进行处理
static rx_handler_result_t ipvlan_handle_mode_l2(struct sk_buff **pskb,struct ipvl_port *port)
{struct sk_buff *skb *pskb;struct ethhdr *eth eth_hdr(skb);rx_handler_result_t ret RX_HANDLER_PASS;// 多播if (is_multicast_ether_addr(eth-h_dest)) {if (ipvlan_external_frame(skb, port)) {struct sk_buff *nskb skb_clone(skb, GFP_ATOMIC);/* External frames are queued for device local* distribution, but a copy is given to master* straight away to avoid sending duplicates later* when work-queue processes this frame. This is* achieved by returning RX_HANDLER_PASS.*/if (nskb) {ipvlan_skb_crossing_ns(nskb, NULL);ipvlan_multicast_enqueue(port, nskb, false);}}// 单播} else {/* Perform like l3 mode for non-multicast packet */ret ipvlan_handle_mode_l3(pskb, port);}return ret;
}ipvlan l3 对于 mode l3 或者单播的 mode l2 报文进入 ipvlan_handle_mode_l3 处理流程首先通过 ipvlan_get_L3_hdr 获取到网络层的头信息然后根据 ip 地址去查找到对应的子设备最后调用 ipvlan_rcv_frame将报文的 dev 设置为 IPVlan 子设备并返回 RX_HANDLER_ANOTHER进行下一次收包。
static rx_handler_result_t ipvlan_handle_mode_l3(struct sk_buff **pskb,struct ipvl_port *port)
{void *lyr3h;int addr_type;struct ipvl_addr *addr;struct sk_buff *skb *pskb;rx_handler_result_t ret RX_HANDLER_PASS;lyr3h ipvlan_get_L3_hdr(port, skb, addr_type);if (!lyr3h)goto out;// 寻找目标地址子设备 addr ipvlan_addr_lookup(port, lyr3h, addr_type, true);if (addr)// 对应子设备收包ret ipvlan_rcv_frame(addr, pskb, false);out:// 走主设备路由return ret;
}ipvlan l3s 对于 mode l3s在 ipvlan_handle_frame 中会直接返回 RX_HANDLER_PASS也就是说mode l3s 的报文会在主设备就进入到网络层的处理阶段对于 mode l3s 来说预先注册的 nf_hook 会在 NF_INET_LOCAL_IN 时触发执行 ipvlan_l3_rcv 操作通过 addr 找到子设备更换报文的网络层目的地址然后直接进入 ip_local_deliver 进行网络层余下的操作。 case IPVLAN_MODE_L3S:return RX_HANDLER_PASS;具体请参考ipvlan-l3s模式
1.4.2 发包流程 ipvlan_queue_xmit 根据子设备的模式选择不同的发送方法mode l2 通过 ipvlan_xmit_mode_l2 发送mode l3 和 mode l3s 进行 ipvlan_xmit_mode_l3 发送。
int ipvlan_queue_xmit(struct sk_buff *skb, struct net_device *dev)
{struct ipvl_dev *ipvlan netdev_priv(dev);struct ipvl_port *port ipvlan_port_get_rcu_bh(ipvlan-phy_dev);if (!port)goto out;if (unlikely(!pskb_may_pull(skb, sizeof(struct ethhdr))))goto out;switch(port-mode) {// l2case IPVLAN_MODE_L2:return ipvlan_xmit_mode_l2(skb, dev);// l3 case IPVLAN_MODE_L3:
#ifdef CONFIG_IPVLAN_L3S// ls3case IPVLAN_MODE_L3S:
#endifreturn ipvlan_xmit_mode_l3(skb, dev);}/* Should not reach here */WARN_ONCE(true, %s called for mode [%x]\n, __func__, port-mode);
out:kfree_skb(skb);return NET_XMIT_DROP;
}ipvlan_xmit_mode_l2 1. 对于 ipvlan_xmit_mode_l2首先判断是否是本地地址或者 VEPA 模式如果不是 VEPA 模式的本地报文则首先通过 ipvlan_addr_lookup 查找是否是相同主设备下的 IPVlan 子设备如果是相同主设备下的 IPVlan 子设备则通过 ipvlan_rcv_frame 让对应子设备进行收包处理如果不是则通过 dev_forward_skb 让主设备进行处理。 2. 接下来 ipvlan_xmit_mode_l2 会对多播报文进行处理在处理之前通过 ipvlan_skb_crossing_ns 清理掉数据包的 netns 相关的信息包括 priority 等最后将数据包放到 ipvlan_multicast_enqueue触发上述的多播处理流程。 3. 对于非本地的数据包通过主设备的 dev_queue_xmit 进行发送。
static int ipvlan_xmit_mode_l2(struct sk_buff *skb, struct net_device *dev)
{const struct ipvl_dev *ipvlan netdev_priv(dev);struct ethhdr *eth skb_eth_hdr(skb);struct ipvl_addr *addr;void *lyr3h;int addr_type;// ipvlan模式不是vepa 且 源mac 与 目标mac 相同if (!ipvlan_is_vepa(ipvlan-port) ether_addr_equal(eth-h_dest, eth-h_source)) {lyr3h ipvlan_get_L3_hdr(ipvlan-port, skb, addr_type);if (lyr3h) {// 寻找目标地址addr ipvlan_addr_lookup(ipvlan-port, lyr3h, addr_type, true);if (addr) {// ipvlan模式是 private 那么 dropif (ipvlan_is_private(ipvlan-port)) {consume_skb(skb);return NET_XMIT_DROP;}// 调用该目标地址的子设备收包ipvlan_rcv_frame(addr, skb, true);return NET_XMIT_SUCCESS;}}skb skb_share_check(skb, GFP_ATOMIC);if (!skb)return NET_XMIT_DROP;/* Packet definitely does not belong to any of the* virtual devices, but the dest is local. So forward* the skb for the main-dev. At the RX side we just return* RX_PASS for it to be processed further on the stack.*/// 如果不属于子设备直接送主设备处理dev_forward_skb(ipvlan-phy_dev, skb);return NET_XMIT_SUCCESS;// 处理多播包} else if (is_multicast_ether_addr(eth-h_dest)) {skb_reset_mac_header(skb);ipvlan_skb_crossing_ns(skb, NULL);ipvlan_multicast_enqueue(ipvlan-port, skb, true);return NET_XMIT_SUCCESS;}// 是vepa 或 源mac与目标mac不同且不是多播包直接送主设备发出skb-dev ipvlan-phy_dev;return dev_queue_xmit(skb);
}ipvlan_xmit_mode_l3 1. ipvlan_xmit_mode_l3 的处理首先也是对 VEPA 进行判断对于非 VEPA 模式的数据包通过ipvlan_addr_lookup 查找是否是其他子设备如果是其他子设备则调用 ipvlan_rcv_frame 触发对应子设备进行收包处理。 2. 对于 VEPA 模式或目标地址不是本地的数据包首先进行 ipvlan_skb_crossing_ns 的处理然后进行 ipvlan_process_outbound的操作此时根据数据包的网络层协议选择 ipvlan_process_v4_outbound 或者 ipvlan_process_v6_outbound 进行处理。 3. 以 ipvlan_process_v4_outbound 为例首先会通过 ip_route_output_flow 进行路由的查找然后直接通过网络层的 ip_local_out在主设备的网络层继续进行发包操作。
static int ipvlan_xmit_mode_l3(struct sk_buff *skb, struct net_device *dev)
{const struct ipvl_dev *ipvlan netdev_priv(dev);void *lyr3h;struct ipvl_addr *addr;int addr_type;lyr3h ipvlan_get_L3_hdr(ipvlan-port, skb, addr_type);if (!lyr3h)goto out;// 如果ipvlan模式不是vepaif (!ipvlan_is_vepa(ipvlan-port)) {// 寻找目标地址 addr ipvlan_addr_lookup(ipvlan-port, lyr3h, addr_type, true);// 如果找到了目标地址if (addr) {// 如果ipvlan模式是 private 那么丢弃if (ipvlan_is_private(ipvlan-port)) {consume_skb(skb);return NET_XMIT_DROP;}// 只剩下bridge模式调用该目标地址的子设备收包ipvlan_rcv_frame(addr, skb, true);return NET_XMIT_SUCCESS;}}
out:// 是vepa 或 目标地址不是本子设备交给主设备ip_route_output_flow查找路由表走三层转发ipvlan_skb_crossing_ns(skb, ipvlan-phy_dev);return ipvlan_process_outbound(skb);
}2. IPVLAN 测试
注意 IPVLAN的model2、l3、l3s与flagsbridge、private、vepa均不能混用。
2.1 创建 IPVLAN 命令
Linux 创建 IPVLAN
rootubuntu22-25:~# ip link add help
Usage: ip link add [link DEV | parentdev NAME] [ name ] NAME[ txqueuelen PACKETS ][ address LLADDR ][ broadcast LLADDR ][ mtu MTU ] [index IDX ][ numtxqueues QUEUE_COUNT ][ numrxqueues QUEUE_COUNT ]type TYPE [ ARGS ]
......ip link help [ TYPE ]TYPE : { bareudp | bond | bond_slave | bridge | bridge_slave |dummy | erspan | geneve | gre | gretap | ifb |ip6erspan | ip6gre | ip6gretap | ip6tnl |ipip | ipoib | ipvlan | ipvtap |macsec | macvlan | macvtap |netdevsim | nlmon | rmnet | sit | team | team_slave |vcan | veth | vlan | vrf | vti | vxcan | vxlan | wwan |xfrm }IPVLAN 可选的 MODE 和 FLAGS
rootubuntu22-25:~# ip link help ipvlan
Usage: ... ipvlan [ mode MODE ] [ FLAGS ]MODE: l3 | l3s | l2
FLAGS: bridge | private | vepa
(first values are the defaults if nothing is specified).
rootubuntu22-25:~# 2.2 测试 IPVLAN L2
2.2.1 IPVLAN l2 bridge 同父接口下子接口通讯情况
1.测试步骤以enp0s5为父接口创建两个ipvlan子接口mode:l2、flags:bridgeipvlan_1和ipvlan_2分别加入到网络命名空间ns1和ns2中。在命名空间中配置两个ipvlan子接口IPipvlan_1 10.211.55.100ipvlan_2 10.211.55.200并启用。测试两个ipvlan子接口网络通讯情况如下图
2.测试结果 IPVLAN l2 bridge模式下同一父接口下的子接口直接通讯 2.2.1.1 配置 IPVLAN l2 bridge 环境
查看当前网卡enp0s5作为父接口
rootubuntu22-25:~# ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: enp0s5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ffinet 10.211.55.25/24 brd 10.211.55.255 scope global enp0s5valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:21c:42ff:fe1c:d10f/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591731sec preferred_lft 604531secinet6 fe80::21c:42ff:fe1c:d10f/64 scope link valid_lft forever preferred_lft forever
rootubuntu22-25:~# 创建网络命名空间ns1 和 ns2
rootubuntu22-25:~# ip netns add ns1
rootubuntu22-25:~# ip netns add ns2
rootubuntu22-25:~# ip netns list
ns2
ns1创建父接口enp0s5的两个ipvlan子接口ipvlan_1 和 ipvlan_2
rootubuntu22-25:~# ip link add link enp0s5 name ipvlan_1 type ipvlan mode l2 bridge
rootubuntu22-25:~# ip link add link enp0s5 name ipvlan_2 type ipvlan mode l2 bridgeipvlan_1 加入 ns1配置IP 10.211.55.100启用
// ipvlan_1 加入 ns1
rootubuntu22-25:~# ip link set ipvlan_1 netns ns1
// 配置IP
rootubuntu22-25:~# ip netns exec ns1 ip addr add 10.211.55.100/24 dev ipvlan_1
rootubuntu22-25:~# ip netns exec ns1 ip link set ipvlan_1 up
rootubuntu22-25:~# ip netns exec ns1 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns1 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
4: ipvlan_1if2: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.100/24 scope global ipvlan_1valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:1c:4200:11c:d10f/64 scope global dynamic mngtmpaddr valid_lft 2591992sec preferred_lft 604792secinet6 fe80::1c:4200:11c:d10f/64 scope link valid_lft forever preferred_lft forever
rootubuntu22-25:~# ipvlan_2 加入 ns2配置IP 10.211.55.200启用
// ipvlan_2 加入 ns2
rootubuntu22-25:~# ip link set ipvlan_2 netns ns2
// 配置IP
rootubuntu22-25:~# ip netns exec ns2 ip addr add 10.211.55.200/24 dev ipvlan_2
rootubuntu22-25:~# ip netns exec ns2 ip link set ipvlan_2 up
rootubuntu22-25:~# ip netns exec ns2 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns2 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
5: ipvlan_2if2: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.200/24 scope global ipvlan_2valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:1c:4200:21c:d10f/64 scope global dynamic mngtmpaddr valid_lft 2591992sec preferred_lft 604792secinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft forever
rootubuntu22-25:~# 2.2.1.2 测试 IPVLAN l2 bridge 子网卡通讯情况
子接口ipvlan_1 Ping 子接口ipvlan_2通子接口ipvlan_1 Ping 外部网关通子接口ipvlan_1 Ping 父接口不通
// 同一父接口下两个子接口互相Ping通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.200 -c3
PING 10.211.55.200 (10.211.55.200): 56 data bytes
64 bytes from 10.211.55.200: icmp_seq0 ttl64 time0.110 ms
64 bytes from 10.211.55.200: icmp_seq1 ttl64 time0.074 ms
64 bytes from 10.211.55.200: icmp_seq2 ttl64 time0.073 ms
--- 10.211.55.200 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.073/0.086/0.110/0.000 ms
// 子接口 Ping 外部网关通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.1 -c3
PING 10.211.55.1 (10.211.55.1): 56 data bytes
64 bytes from 10.211.55.1: icmp_seq0 ttl128 time0.387 ms
64 bytes from 10.211.55.1: icmp_seq1 ttl128 time0.446 ms
64 bytes from 10.211.55.1: icmp_seq2 ttl128 time0.351 ms
--- 10.211.55.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.351/0.395/0.446/0.039 ms
// 子接口与父接口隔离 Ping 父接口IP不通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.25 -c3
PING 10.211.55.25 (10.211.55.25): 56 data bytes
92 bytes from 10.211.55.100: Destination Host Unreachable
92 bytes from 10.211.55.100: Destination Host Unreachable
92 bytes from 10.211.55.100: Destination Host Unreachable
--- 10.211.55.25 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
rootubuntu22-25:~# 2.2.2 IPVLAN l2 private 同父接口下子接口通讯情况
1.测试步骤以enp0s5为父接口创建两个ipvlan子接口mode:l2、flags:privateeth_private1和eth_private2分别加入到网络命名空间ns_private1和ns_private2中。在命名空间中配置两个ipvlan子接口IPeth_private1 10.211.55.100eth_private2 10.211.55.200并启用。测试两个ipvlan子接口网络通讯情况如下图
2.测试结果 IPVLAN l2 private模式下同一父接口下的子接口网络隔离不能通讯
补充ipvlan l2 private 模式下同父接口下子接口之间数据包直接drop。详见上述 ”1.4.2 发包流程” 代码 2.2.2.1 配置 IPVLAN l2 private 环境
创建父接口enp0s5的两个ipvlan子接口eth_private1 和 eth_private2
rootubuntu22-25:~# ip link add link enp0s5 name eth_private1 type ipvlan mode l2 private
rootubuntu22-25:~# ip link add link enp0s5 name eth_private2 type ipvlan mode l2 private创建网络命名空间ns_private1 和 ns_private2
rootubuntu22-25:~# ip netns add ns_private1
rootubuntu22-25:~# ip netns add ns_private2eth_private1加入ns_private1配置IP 10.211.55.100并启用
// eth_private1加入ns_private1
rootubuntu22-25:~# ip link set eth_private1 netns ns_private1
// 配置IP
rootubuntu22-25:~# ip netns exec ns_private1 ip addr add 10.211.55.100/24 dev eth_private1
rootubuntu22-25:~# ip netns exec ns_private1 ip link set eth_private1 up
rootubuntu22-25:~# ip netns exec ns_private1 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns_private1 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3: eth_private1if2: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.100/24 scope global eth_private1valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:1c:4200:11c:d10f/64 scope global dynamic mngtmpaddr valid_lft 2591954sec preferred_lft 604754secinet6 fe80::1c:4200:11c:d10f/64 scope link valid_lft forever preferred_lft forevereth_private2加入ns_private2配置IP 10.211.55.200并启用
// eth_private2加入ns_private2
rootubuntu22-25:~# ip link set eth_private2 netns ns_private2
// 配置IP
rootubuntu22-25:~# ip netns exec ns_private2 ip addr add 10.211.55.200/24 dev eth_private2
rootubuntu22-25:~# ip netns exec ns_private2 ip link set eth_private2 up
rootubuntu22-25:~# ip netns exec ns_private2 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns_private2 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
4: eth_private2if2: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.200/24 scope global eth_private2valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:1c:4200:21c:d10f/64 scope global dynamic mngtmpaddr valid_lft 2592000sec preferred_lft 604800secinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft forever
rootubuntu22-25:~# 2.2.2.2 测试 IPVLAN l2 private 子网卡通讯情况
eth_private2 Ping 外部网关通eth_private2 Ping eth_private1private模式下不通ipvlan子接口eth_private2 Ping 父接口enp0s5不通
// eth_private2 Ping 外部网关通
rootubuntu22-25:~# ip netns exec ns_private2 ping 10.211.55.1 -c3
PING 10.211.55.1 (10.211.55.1): 56 data bytes
64 bytes from 10.211.55.1: icmp_seq0 ttl128 time0.355 ms
64 bytes from 10.211.55.1: icmp_seq1 ttl128 time0.368 ms
64 bytes from 10.211.55.1: icmp_seq2 ttl128 time0.369 ms
--- 10.211.55.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.355/0.364/0.369/0.000 ms
// eth_private2 Ping eth_private1不通
rootubuntu22-25:~# ip netns exec ns_private2 ping 10.211.55.100 -c3
PING 10.211.55.100 (10.211.55.100): 56 data bytes
92 bytes from 10.211.55.200: Destination Host Unreachable
92 bytes from 10.211.55.200: Destination Host Unreachable
92 bytes from 10.211.55.200: Destination Host Unreachable
--- 10.211.55.100 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
// 子接口eth_private2 Ping 父接口enp0s5不通
rootubuntu22-25:~# ip netns exec ns_private2 ping 10.211.55.25 -c3
PING 10.211.55.25 (10.211.55.25): 56 data bytes
92 bytes from 10.211.55.200: Destination Host Unreachable
92 bytes from 10.211.55.200: Destination Host Unreachable
92 bytes from 10.211.55.200: Destination Host Unreachable
--- 10.211.55.25 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss2.2.3 IPVLAN l2 vepa 同父接口下子接口通讯情况
1.测试步骤创建一个Linux bridgebr0和一对虚拟网卡veth1、veth1_br。veth1_br加入br0用于控制开启关闭hairpin功能。veth1作为父接口主设备创建两个ipvlan子接口mode:l2、flags:vepaeth_vepa1和eth_vepa2分别加入到网络命名空间ns_vepa1和ns_vepa2中。在命名空间中配置两个ipvlan子接口IPeth_vepa1 10.211.55.100eth_vepa2 10.211.55.200并启用。测试两个ipvlan子接口网络通讯的情况如下图。
2.测试结果IPVLAN l2 vepa模式下外部交换br0开启hairpin支持802.1gbg功能时同一父接口下的子接口可以经外部交换转发通讯
补充ipvlan l2 vepa 模式下子接口数据包由父接口直接外发。详见上述 ”1.4.2 发包流程” 代码
2.2.3.1 配置 Linux bridge 和 Veth pair
创建Linux bridge br0创建veth pairveth1和veth1_brveth1_br加入br0
// 创建Linux bridge br0并启用
rootubuntu22-25:~# brctl addbr br0
rootubuntu22-25:~# ip link set br0 up
// 创建veth pairveth1和veth1_br并启用
rootubuntu22-25:~# ip link add veth1 type veth peer veth1_br
rootubuntu22-25:~# ip link set veth1 up
rootubuntu22-25:~# ip link set veth1_br up
// veth1_br加入br0
rootubuntu22-25:~# brctl addif br0 veth1_br
rootubuntu22-25:~# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.ca19426b60e5 no veth1_br
rootubuntu22-25:~# 父接口veth1配置IP 10.211.55.254
rootubuntu22-25:~# ip addr add 10.211.55.254/24 dev veth1
rootubuntu22-25:~# ip addr show veth1
5: veth1veth1_br: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UP group default qlen 1000link/ether 9a:72:89:8d:28:29 brd ff:ff:ff:ff:ff:ffinet 10.211.55.254/24 scope global veth1valid_lft forever preferred_lft foreverinet6 fe80::9872:89ff:fe8d:2829/64 scope link valid_lft forever preferred_lft forever
rootubuntu22-25:~# 2.2.3.2 配置 IPVLAN l2 vepa 环境
创建父接口veth1的两个ipvlan子接口eth_vepa1 和 eth_vepa2
rootubuntu22-25:~# ip link add link veth1 name eth_vepa1 type ipvlan mode l2 vepa
rootubuntu22-25:~# ip link add link veth1 name eth_vepa2 type ipvlan mode l2 vepa创建网络命名空间ns_vepa1和ns_vepa2
rootubuntu22-25:~# ip netns add ns_vepa1
rootubuntu22-25:~# ip netns add ns_vepa2eth_vepa1加入ns_vepa1配置IP 10.211.55.100并启用
rootubuntu22-25:~# ip link set eth_vepa1 netns ns_vepa1
rootubuntu22-25:~# ip netns exec ns_vepa1 ip addr add 10.211.55.100/24 dev eth_vepa1
rootubuntu22-25:~# ip netns exec ns_vepa1 ip link set eth_vepa1 up
rootubuntu22-25:~# ip netns exec ns_vepa1 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns_vepa1 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
6: eth_vepa1if5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 9a:72:89:8d:28:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.100/24 scope global eth_vepa1valid_lft forever preferred_lft foreverinet6 fe80::9a72:8900:18d:2829/64 scope link valid_lft forever preferred_lft forevereth_vepa2加入ns_vepa2配置IP 10.211.55.200并启用
rootubuntu22-25:~# ip link set eth_vepa2 netns ns_vepa2
rootubuntu22-25:~# ip netns exec ns_vepa2 ip addr add 10.211.55.200/24 dev eth_vepa2
rootubuntu22-25:~# ip netns exec ns_vepa2 ip link set eth_vepa2 up
rootubuntu22-25:~# ip netns exec ns_vepa2 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns_vepa2 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
7: eth_vepa2if5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 9a:72:89:8d:28:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.200/24 scope global eth_vepa2valid_lft forever preferred_lft foreverinet6 fe80::9a72:8900:28d:2829/64 scope link valid_lft forever preferred_lft forever2.2.3.3 测试 IPVLAN l2 vepa 子网卡通讯情况
br0的接口veth1_br hairpin功能off
// 查看 bridgeveth1_br接口 hairpin off
rootubuntu22-25:~# bridge -d link
3: br0: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 master br0
4: veth1_brveth1: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 master br0 state forwarding priority 32 cost 2 hairpin off guard off root_block off fastleave off learning on flood on mcast_flood on mcast_to_unicast off neigh_suppress off vlan_tunnel off isolated off
rootubuntu22-25:~# eth_vepa2 Ping eth_vepa1vepa模式下不通
rootubuntu22-25:~# ip netns exec ns_vepa2 ping 10.211.55.100 -c3
PING 10.211.55.100 (10.211.55.100): 56 data bytes
--- 10.211.55.100 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
rootubuntu22-25:~# br0的接口veth1_br 开启hairpin功能
rootubuntu22-25:~# brctl hairpin br0 veth1_br on
rootubuntu22-25:~# bridge -d link
3: br0: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 master br0
4: veth1_brveth1: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 master br0 state forwarding priority 32 cost 2 hairpin on guard off root_block off fastleave off learning on flood on mcast_flood on mcast_to_unicast off neigh_suppress off vlan_tunnel off isolated off eth_vepa2 Ping eth_vepa1vepa模式下通 通过外部开启hairpin功能的 br0 实现转发
rootubuntu22-25:~# ip netns exec ns_vepa2 ping 10.211.55.100 -c3
PING 10.211.55.100 (10.211.55.100): 56 data bytes
64 bytes from 10.211.55.100: icmp_seq0 ttl64 time0.124 ms
64 bytes from 10.211.55.100: icmp_seq1 ttl64 time0.083 ms
64 bytes from 10.211.55.100: icmp_seq2 ttl64 time0.091 ms
--- 10.211.55.100 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.083/0.099/0.124/0.000 ms子接口eth_vepa2 Ping 父接口不通
rootubuntu22-25:~# ip netns exec ns_vepa2 ping 10.211.55.254 -c3
PING 10.211.55.254 (10.211.55.254): 56 data bytes
92 bytes from 10.211.55.200: Destination Host Unreachable
92 bytes from 10.211.55.200: Destination Host Unreachable
92 bytes from 10.211.55.200: Destination Host Unreachable
--- 10.211.55.254 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss2.2.4 IPVLAN l2 bridge子接口与父接口通讯情况
1.测试步骤在父接口enp0s510.211.55.25下创建两个ipvlan子接口mode:l2、flags:bridgeiplv2_bridge1和iplv2_bridge2其中子接口iplv2_bridge1加入到网络命名空间ns1配置IP10.211.55.100并启用子接口iplv2_bridge2与父接口在相同网络命名空间中配置与父接口同段IP10.211.55.254并启用。测试子接口iplv2_bridge1与父接口enp0s5网络通讯情况如下图
2.测试结果IPVLAN l2 bridge模式下父接口与子接口网络隔离不能直接通讯但是子接口可以通过与父接口同网络命名空间下的其它子接口转发通讯
补充vepa与private模式下ipvlan子接口之间无法直接通讯无法实现中转 2.2.4.1 配置IPVLAN l2 bridge 子接口与父接口通讯环境
创建网络命名空间ns1
rootubuntu22-25:~# ip netns add ns1创建两个ipvlan子接口mode l2flags bridgeipvl2_bridge1 和 ipvl2_bridge2
rootubuntu22-25:~# ip link add link enp0s5 name ipvl2_bridge1 type ipvlan mode l2 bridge
rootubuntu22-25:~#
rootubuntu22-25:~# ip link add link enp0s5 name ipvl2_bridge2 type ipvlan mode l2 bridge配置ipvlan子接口ipvl2_bridge1并启用 ipvl2_bridge1加入ns1配置IP 10.211.55.100/24
// ipvl2_bridge1加入ns1
rootubuntu22-25:~# ip link set ipvl2_bridge1 netns ns1
// 配置IP
rootubuntu22-25:~# ip netns exec ns1 ip addr add 10.211.55.100/24 dev ipvl2_bridge1
rootubuntu22-25:~# ip netns exec ns1 ip link set ipvl2_bridge1 up
rootubuntu22-25:~# ip netns exec ns1 ip link set lo up// 查看配置结果
rootubuntu22-25:~# ip netns exec ns1 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3: ipvl2_bridge1if2: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 10.211.55.100/24 scope global ipvl2_bridge1valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:1c:4200:11c:d10f/64 scope global dynamic mngtmpaddr valid_lft 2591959sec preferred_lft 604759secinet6 fe80::1c:4200:11c:d10f/64 scope link valid_lft forever preferred_lft forever配置ipvlan子接口ipvl2_bridge2并启用 配置IP 10.211.55.254/24与父接口在同一命名空间且IP地址段相同
// 配置IP
rootubuntu22-25:~# ip addr add 10.211.55.254/24 dev ipvl2_bridge2
rootubuntu22-25:~# ip link set ipvl2_bridge2 up
// 查看所有网卡IP
rootubuntu22-25:~# ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: enp0s5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ffinet 10.211.55.25/24 brd 10.211.55.255 scope global enp0s5valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:21c:42ff:fe1c:d10f/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591632sec preferred_lft 604432secinet6 fe80::21c:42ff:fe1c:d10f/64 scope link valid_lft forever preferred_lft forever
4: ipvl2_bridge2enp0s5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ffinet 10.211.55.254/24 scope global ipvl2_bridge2valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft forever2.2.4.2 测试IPVLAN l2 bridge 子接口与父接口通讯
ipvlan子接口ipvl2_bridge1经父接口enp0s5与外部网关通讯 ns1下ipvl2_bridge110.211.55.100 Ping 外部网关10.211.55.1通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.1 -c2
PING 10.211.55.1 (10.211.55.1): 56 data bytes
64 bytes from 10.211.55.1: icmp_seq0 ttl128 time0.363 ms
64 bytes from 10.211.55.1: icmp_seq1 ttl128 time0.209 ms
--- 10.211.55.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.209/0.286/0.363/0.077 ms
rootubuntu22-25:~# ipvlan子接口ipvl2_bridge1不能与父接口enp0s5直接通讯 ns1下ipvl2_bridge110.211.55.100Ping 父接口10.211.55.25不通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.25 -c3
PING 10.211.55.25 (10.211.55.25): 56 data bytes
92 bytes from 10.211.55.100: Destination Host Unreachable
92 bytes from 10.211.55.100: Destination Host Unreachable
92 bytes from 10.211.55.100: Destination Host Unreachable
--- 10.211.55.25 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss宿主机默认命名空间下配置访问10.211.55.100的静态路由
// 配置静态路由
rootubuntu22-25:~# ip route add 10.211.55.100 dev ipvl2_bridge2
rootubuntu22-25:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.25
10.211.55.0/24 dev ipvl2_bridge2 proto kernel scope link src 10.211.55.254
10.211.55.100 dev ipvl2_bridge2 scope link ipvlan子接口ipvl2_bridge1经ipvl2_bridge2转发与父接口enp0s5通讯 ns1下ipvl2_bridge110.211.55.100Ping 父接口10.211.55.25通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.25
PING 10.211.55.25 (10.211.55.25): 56 data bytes
64 bytes from 10.211.55.25: icmp_seq0 ttl64 time0.094 ms
64 bytes from 10.211.55.25: icmp_seq1 ttl64 time0.061 ms
^C--- 10.211.55.25 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.061/0.077/0.094/0.000 ms
rootubuntu22-25:~# ipvlan子接口ipvl2_bridge2上抓包查看中转结果 tcpdump -nn -e -i ipvl2_bridge2 not host 10.211.55.2 not host 抓包排除ssh客户端 10.211.55.2 -nn 两个n 表示不解析域名和端口 -e 在输出行打印出数据链路层的头部信息 -i 抓指定接口(网卡)的数据包
rootubuntu22-25:~# tcpdump -nn -e -i ipvl2_bridge2 not host 10.211.55.2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ipvl2_bridge2, link-type EN10MB (Ethernet), snapshot length 262144 bytes
12:26:13.924694 00:1c:42:1c:d1:0f ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.211.55.25 tell 10.211.55.100, length 28
12:26:13.924706 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype ARP (0x0806), length 42: Reply 10.211.55.25 is-at 00:1c:42:1c:d1:0f, length 28
12:26:13.924757 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 10.211.55.100: ICMP echo reply, id 15825, seq 0, length 64
12:26:14.926271 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 10.211.55.100: ICMP echo reply, id 15825, seq 1, length 64
12:26:15.928470 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 10.211.55.100: ICMP echo reply, id 15825, seq 2, length 64
12:26:16.930605 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 10.211.55.100: ICMP echo reply, id 15825, seq 3, length 64
12:26:17.932781 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 10.211.55.100: ICMP echo reply, id 15825, seq 4, length 64
12:26:18.979459 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype ARP (0x0806), length 42: Request who-has 10.211.55.100 tell 10.211.55.254, length 28
12:26:18.979579 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype ARP (0x0806), length 42: Reply 10.211.55.100 is-at 00:1c:42:1c:d1:0f, length 28
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel2.3 测试 IPVLAN L3
2.3.1 IPVLAN l3 bridge 同父接口下子接口通讯情况
1.测试步骤以enp0s5为父接口创建两个ipvlan子接口mode:l3、flags:bridgeipvl3_br1和ipvl3_br2分别加入到网络命名空间ipvl3_br1和ipvl3_br2中。在命名空间中配置两个ipvlan子接口IPipvl3_br1 192.168.10.100ipvl3_br2 172.16.1.200并启用。测试两个ipvlan子接口网络通讯的情况。
注意l3与l2不同的是子接口需要配置路由。详见下图
2.测试结果 IPVLAN l3 bridge模式下同一父接口下的子接口通过三层转发通讯 2.3.1.1 配置 IPVLAN l3 bridge 环境
创建父接口enp0s5的两个Ipvlan子接口ipvl3_br1 和 ipvl3_br2
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_br1 type ipvlan mode l3 bridge
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_br2 type ipvlan mode l3 bridge创建两个网络命名空间ns_bridge1 和 ns_bridge2
rootubuntu22-25:~# ip netns add ns_bridge1
rootubuntu22-25:~# ip netns add ns_bridge2ipvl3_br1加入ns_bridge1配置IP 192.168.10.100并启用
rootubuntu22-25:~# ip link set ipvl3_br1 netns ns_bridge1
rootubuntu22-25:~# ip netns exec ns_bridge1 ip addr add 192.168.10.100/24 dev ipvl3_br1
rootubuntu22-25:~# ip netns exec ns_bridge1 ip link set ipvl3_br1 up
rootubuntu22-25:~# ip netns exec ns_bridge1 ip link set lo upipvl3_br2加入ns_bridge2配置IP 172.16.1.200并启用
rootubuntu22-25:~# ip link set ipvl3_br2 netns ns_bridge2
rootubuntu22-25:~# ip netns exec ns_bridge2 ip addr add 172.16.1.200/24 dev ipvl3_br2
rootubuntu22-25:~# ip netns exec ns_bridge2 ip link set ipvl3_br2 up
rootubuntu22-25:~# ip netns exec ns_bridge2 ip link set lo upns_bridge1 和 ns_bridge2 添加默认路由
rootubuntu22-25:~# ip netns exec ns_bridge1 ip route add default dev ipvl3_br1
rootubuntu22-25:~# ip netns exec ns_bridge2 ip route add default dev ipvl3_br2
// 查看ns_bridge2默认路由
rootubuntu22-25:~# ip netns exec ns_bridge2 ip route
default dev ipvl3_br2 scope link
172.16.1.0/24 dev ipvl3_br2 proto kernel scope link src 172.16.1.200 2.3.1.2 测试 IPVLAN l3 bridge 子网卡通讯情况
测试ipvl3_br1 ping ipvl3_br2通
rootubuntu22-25:~# ip netns exec ns_bridge2 ping 192.168.10.100 -c3
PING 192.168.10.100 (192.168.10.100): 56 data bytes
64 bytes from 192.168.10.100: icmp_seq0 ttl64 time0.063 ms
64 bytes from 192.168.10.100: icmp_seq1 ttl64 time0.073 ms
64 bytes from 192.168.10.100: icmp_seq2 ttl64 time0.072 ms
--- 192.168.10.100 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.063/0.069/0.073/0.000 ms
rootubuntu22-25:~# 2.3.2 IPVLAN l3 private 同父接口下子接口通讯情况
1.测试步骤以enp0s5为父接口创建两个ipvlan子接口mode:l3、flags:privateipvl3_private1和ipvl3_private2分别加入到网络命名空间ipvl3_private1和ipvl3_private2中。在命名空间中配置两个ipvlan子接口IPipvl3_private1 192.168.10.100ipvl3_private2 172.16.1.200并启用。测试两个ipvlan子接口网络通讯情况。
注意l3与l2不同的是子接口需要配置路由。详见下图
2.测试结果 IPVLAN l3 private模式下同一父接口下的子接口禁止互相通讯
补充“1.4.2 发包流程”中已经很清楚只要是private那么包就被drop。
2.3.2.1 配置 IPVLAN l3 private 环境 创建父接口enp0s5的两个Ipvlan子接口ipvl3_private1 和 ipvl3_private2
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_private1 type ipvlan mode l3 private
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_private2 type ipvlan mode l3 private创建两个网络命名空间ns_private1 和 ns_private2
rootubuntu22-25:~# ip netns add ns_private1
rootubuntu22-25:~# ip netns add ns_private2ipvl3_private1加入ns_private1配置IP 192.168.10.100并启用ns_private1配置默认路由
rootubuntu22-25:~# ip link set ipvl3_private1 netns ns_private1
rootubuntu22-25:~# ip netns exec ns_private1 ip addr add 192.168.10.100/24 dev ipvl3_private1
rootubuntu22-25:~# ip netns exec ns_private1 ip link set ipvl3_private1 up
rootubuntu22-25:~# ip netns exec ns_private1 ip link set lo up
// 配置默认路由
rootubuntu22-25:~# ip netns exec ns_private1 ip route add default dev ipvl3_private1ipvl3_private2加入ns_private2配置IP 172.16.1.200并启用ns_private2配置默认路由
rootubuntu22-25:~# ip link set ipvl3_private2 netns ns_private2
rootubuntu22-25:~# ip netns exec ns_private2 ip addr add 172.16.1.200/24 dev ipvl3_private2
rootubuntu22-25:~# ip netns exec ns_private2 ip link set ipvl3_private2 up
rootubuntu22-25:~# ip netns exec ns_private2 ip link set lo up
// 配置默认路由
rootubuntu22-25:~# ip netns exec ns_private2 ip route add default dev ipvl3_private22.3.2.2 测试 IPVLAN l3 private 子网卡通讯情况
l3 private模式下子接口网络隔离测试Ping不通
// 查看ns_private2下ipvl3_private2的IP
rootubuntu22-25:~# ip netns exec ns_private2 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
4: ipvl3_private2if2: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 172.16.1.200/24 scope global ipvl3_private2valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft forever// 查看ns_private2下默认路由
rootubuntu22-25:~# ip netns exec ns_private2 ip route
default dev ipvl3_private2 scope link
172.16.1.0/24 dev ipvl3_private2 proto kernel scope link src 172.16.1.200 // 测试子接口互相 Ping不通
rootubuntu22-25:~# ip netns exec ns_private2 ping 192.168.10.100 -c3
PING 192.168.10.100 (192.168.10.100): 56 data bytes
--- 192.168.10.100 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss2.3.3 IPVLAN l3 vepa 同父接口下子接口通讯情况
1.测试步骤
使用宿主机ubuntu22-25 网卡enp0s5 创建两个ipvlan子接口mode:l3、flags:vepaipvl3_vepa1和ipvl3_vepa2分别加入到网络命名空间ns1和ns2中。在命名空间中配置两个ipvlan子接口IPipvl3_vepa1 192.168.10.100ipvl3_vepa2 192.168.20.200和默认路由。在宿主机ubuntu22-25 添加静态路由将两个ipvlan接口IP的外发路由指向宿主机ubuntu22-24需要ubuntu22-24做外部路由转发IP在宿主机ubuntu22-24 添加静态路由将两个ipvlan接口IP的返回路由指向宿主机ubuntu22-25ubuntu22-24完成转发后需要返回数据包在宿主机ubuntu22-24 上开启路由转发功能测试两个ipvlan子接口网络通讯的情况。
注意l3与l2不同的是子接口需要配置路由。详见下图
2.测试结果 IPVLAN l3 vepa模式下父接口不转发子接口之间数据包但外部路由支持时相同父接口下的子接口可以通过外部路由转发通讯
补充ipvlan l3 vepa 模式下子接口数据包由父接口直接外发。详见上述 ”1.4.2 发包流程” 代码
2.3.3.1 配置宿主机 ubuntu22-25
创建父接口enp0s5的两个Ipvlan子接口ipvl3_vepa1 和 ipvl3_vepa2
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_vepa1 type ipvlan mode l3 vepa
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_vepa2 type ipvlan mode l3 vepa创建两个网络命名空间ns1 和 ns2
rootubuntu22-25:~# ip netns add ns1
rootubuntu22-25:~# ip netns add ns2ipvl3_vepa1加入ns1配置IP 192.168.10.100并启用
rootubuntu22-25:~# ip link set ipvl3_vepa1 netns ns1
rootubuntu22-25:~# ip netns exec ns1 ip addr add 192.168.10.100/24 dev ipvl3_vepa1
rootubuntu22-25:~# ip netns exec ns1 ip link set lo up
rootubuntu22-25:~# ip netns exec ns1 ip link set ipvl3_vepa1 up
rootubuntu22-25:~# ip netns exec ns1 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3: ipvl3_vepa1if2: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 192.168.10.100/24 scope global ipvl3_vepa1valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:11c:d10f/64 scope link valid_lft forever preferred_lft foreverns1 使用 ipvl3_vepa1 配置默认路由
rootubuntu22-25:~# ip netns exec ns1 ip route add default dev ipvl3_vepa1
rootubuntu22-25:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.25
rootubuntu22-25:~# ip netns exec ns1 ip route
default dev ipvl3_vepa1 scope link
192.168.10.0/24 dev ipvl3_vepa1 proto kernel scope link src 192.168.10.100 ipvl3_vepa2加入ns2配置IP 192.168.2.200并启用
rootubuntu22-25:~# ip link set ipvl3_vepa2 netns ns2
rootubuntu22-25:~# ip netns exec ns2 ip addr add 192.168.20.200/24 dev ipvl3_vepa2
rootubuntu22-25:~# ip netns exec ns2 ip link set lo up
rootubuntu22-25:~# ip netns exec ns2 ip link set ipvl3_vepa2 up
rootubuntu22-25:~# ip netns exec ns2 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
4: ipvl3_vepa2if2: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 192.168.20.200/24 scope global ipvl3_vepa2valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft foreverns2 使用 ipvl3_vepa2 配置默认路由
rootubuntu22-25:~# ip netns exec ns2 ip route add default dev ipvl3_vepa2
rootubuntu22-25:~# ip netns exec ns2 ip route
default dev ipvl3_vepa2 scope link
192.168.20.0/24 dev ipvl3_vepa2 proto kernel scope link src 192.168.20.200 2.3.3.2 测试 IPVLAN l3 vepa 默认情况下子接口通讯情况
测试 IPVLAN 子接口间通信ns2 Ping 192.168.10.100不通 ipvlan l3 vepa 模式下子接口数据包由父接口直接外发父接口不做子接口之间转发。详见上述 ”1.4.2 发包流程” 代码
rootubuntu22-25:~# ip netns exec ns2 ping 192.168.10.100 -c3
PING 192.168.10.100 (192.168.10.100): 56 data bytes
--- 192.168.10.100 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss测试 IPVLAN 子接口 Ping 父接口不通
// 父网卡配合与ns1 和 ns2 同网段IP地址
rootubuntu22-25:~# ip addr add 192.168.10.254/24 dev enp0s5
rootubuntu22-25:~# ip addr add 192.168.20.254/24 dev enp0s5
// 父网卡与子网卡不能直接通讯
rootubuntu22-25:~# ping 192.168.10.100 -c2
PING 192.168.10.100 (192.168.10.100): 56 data bytes
92 bytes from ubuntu22-25 (192.168.10.254): Destination Host Unreachable
92 bytes from ubuntu22-25 (192.168.10.254): Destination Host Unreachable
--- 192.168.10.100 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
rootubuntu22-25:~# ping 192.168.20.200 -c2
PING 192.168.20.200 (192.168.20.200): 56 data bytes
92 bytes from ubuntu22-25 (192.168.20.254): Destination Host Unreachable
92 bytes from ubuntu22-25 (192.168.20.254): Destination Host Unreachable
--- 192.168.20.200 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss宿主机ubuntu22-25 配置静态路由 利用外部ubuntu22-24的路由测试 ipvlan l3 vepa 子接口间通讯ubuntu22-25 将192.168.10.100 和 192.168.20.200 路由指向ubuntu22-24的IP 10.211.55.24。
rootubuntu22-25:~# ip route add 192.168.10.100 via 10.211.55.24 dev enp0s5
rootubuntu22-25:~# ip route add 192.168.20.200 via 10.211.55.24 dev enp0s5
rootubuntu22-25:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.25
192.168.10.100 via 10.211.55.24 dev enp0s5
192.168.20.200 via 10.211.55.24 dev enp0s5 2.3.3.3 配置宿主机 ubuntu22-24
宿主机ubuntu22-24 配置前情况
rootubuntu22-24:~# ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: enp0s5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:1c:42:5e:4d:21 brd ff:ff:ff:ff:ff:ffinet 10.211.55.24/24 brd 10.211.55.255 scope global enp0s5valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:21c:42ff:fe5e:4d21/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591931sec preferred_lft 604731secinet6 fe80::21c:42ff:fe5e:4d21/64 scope link valid_lft forever preferred_lft forever
rootubuntu22-24:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.24
rootubuntu22-24:~# 宿主机ubuntu22-24 配置静态路由 ubuntu22-24 添加反向路由与ubuntu22-25配置的静态路由对应。将192.168.10.100 和 192.168.20.200 路由指回ubuntu22-25的IP 10.211.55.25。
rootubuntu22-24:~# ip route add 192.168.10.100 via 10.211.55.25 dev enp0s5
rootubuntu22-24:~# ip route add 192.168.20.200 via 10.211.55.25 dev enp0s5
rootubuntu22-24:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.24
192.168.10.100 via 10.211.55.25 dev enp0s5
192.168.20.200 via 10.211.55.25 dev enp0s5 2.3.3.4 测试 IPVLAN l3 vepa 经外部路由转发下子接口通讯情况
测试宿主机ubuntu22-24与ipvlan l3 vepa子网卡通讯 ubuntu22-24 可以Ping 192.168.10.100 和 192.168.20.200 通。说明ubuntu22-25 上的 ipvlan l3 vepa 子接口经过父接口三层转发、宿主机静态路由转发可以与外部通讯。
ubuntu22-24 -- ubuntu22-24 静态路由 -- ubuntu22-25 ipvlan父接口 -- ipvlan子接口
rootubuntu22-24:~# ping 192.168.10.100 -c3
PING 192.168.10.100 (192.168.10.100): 56 data bytes
64 bytes from 192.168.10.100: icmp_seq0 ttl64 time0.688 ms
64 bytes from 192.168.10.100: icmp_seq1 ttl64 time0.522 ms
^C--- 192.168.10.100 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.522/0.605/0.688/0.083 ms
rootubuntu22-24:~# ping 192.168.20.200 -c3
PING 192.168.20.200 (192.168.20.200): 56 data bytes
64 bytes from 192.168.20.200: icmp_seq0 ttl64 time0.431 ms
64 bytes from 192.168.20.200: icmp_seq1 ttl64 time0.499 ms
^C--- 192.168.20.200 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.431/0.465/0.499/0.034 msubuntu22-24 开启 IP转发功能
rootubuntu22-24:~# sysctl net.ipv4.ip_forward1
net.ipv4.ip_forward 1测试ubuntu22-25 经过 ubuntu22-24 IP转发与ipvlan l3 vepa子接口通讯 ubuntu22-25 可以Ping192.168.10.100 和 192.168.20.200 通 。说明ubuntu22-25 经 ubuntu22-24 IP转发再经 ipvlan l3 vepa 父接口三层转发可以与ipvlan l3 vepa 子接口通讯。
ubuntu22-25 -- ubuntu22-25 静态路由 -- ubuntu22-24 IP转发 -- ubuntu22-24 静态路由 -- ubuntu22-25 ipvlan父接口 -- ipvlan子接口
// 查看 ubuntu22-25 的IP
rootubuntu22-25:~# ip a show enp0s5
2: enp0s5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ffinet 10.211.55.25/24 brd 10.211.55.255 scope global enp0s5valid_lft forever preferred_lft foreverinet 192.168.10.254/24 scope global enp0s5valid_lft forever preferred_lft foreverinet 192.168.20.254/24 scope global enp0s5valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:21c:42ff:fe1c:d10f/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591806sec preferred_lft 604606secinet6 fe80::21c:42ff:fe1c:d10f/64 scope link valid_lft forever preferred_lft forever
// ubuntu22-25 Ping 192.168.20.200通
rootubuntu22-25:~# ping 192.168.20.200
PING 192.168.20.200 (192.168.20.200): 56 data bytes
92 bytes from 10.211.55.24: Redirect Host
64 bytes from 192.168.20.200: icmp_seq0 ttl64 time91.659 ms
64 bytes from 192.168.20.200: icmp_seq1 ttl64 time0.521 ms
64 bytes from 192.168.20.200: icmp_seq2 ttl64 time0.519 ms
64 bytes from 192.168.20.200: icmp_seq3 ttl64 time0.537 ms
64 bytes from 192.168.20.200: icmp_seq4 ttl64 time0.535 ms
64 bytes from 192.168.20.200: icmp_seq5 ttl64 time0.556 ms
^C92 bytes from 10.211.55.24: Redirect Host
--- 192.168.20.200 ping statistics ---
7 packets transmitted, 6 packets received, 14% packet loss
round-trip min/avg/max/stddev 0.519/15.721/91.659/33.960 ms
// ubuntu22-25 Ping 192.168.10.100通
rootubuntu22-25:~# ping 192.168.10.100
PING 192.168.10.100 (192.168.10.100): 56 data bytes
64 bytes from 192.168.10.100: icmp_seq0 ttl64 time0.436 ms
64 bytes from 192.168.10.100: icmp_seq1 ttl64 time0.509 ms
64 bytes from 192.168.10.100: icmp_seq2 ttl64 time0.603 ms
64 bytes from 192.168.10.100: icmp_seq3 ttl64 time0.591 ms
^C--- 192.168.10.100 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.436/0.535/0.603/0.068 ms测试ipvlan l3 vepa子接口间经 ubuntu22-24 IP转发互相通讯 ipvlan l3 vepa子接口: ipvl3_vepa1 192.168.10.100 可以Ping ipvlan l3 vepa子接口: ipvl3_vepa2 192.168.20.200通。说明ipvlan l3 vepa 子接口之间经过外部路由转发可以互相通讯。
ipvl3_vepa1 -- ubuntu22-25 ipvlan父接口 -- ubuntu22-25 静态路由 -- ubuntu22-24 IP转发 -- ubuntu22-24 静态路由 -- ubuntu22-25 ipvlan父接口 -- ipvl3_vepa2
rootubuntu22-25:~# ip netns exec ns1 ping 192.168.20.200
PING 192.168.20.200 (192.168.20.200): 56 data bytes
92 bytes from 10.211.55.24: Redirect Host
64 bytes from 192.168.20.200: icmp_seq0 ttl63 time0.795 ms
64 bytes from 192.168.20.200: icmp_seq1 ttl63 time0.761 ms
64 bytes from 192.168.20.200: icmp_seq2 ttl63 time0.725 ms
64 bytes from 192.168.20.200: icmp_seq3 ttl63 time0.730 ms
64 bytes from 192.168.20.200: icmp_seq4 ttl63 time0.749 ms
64 bytes from 192.168.20.200: icmp_seq5 ttl63 time0.770 ms
64 bytes from 192.168.20.200: icmp_seq6 ttl63 time0.761 ms
^C--- 192.168.20.200 ping statistics ---
7 packets transmitted, 7 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.725/0.756/0.795/0.000 ms2.3.4 IPVLAN l3 bridge子接口与父接口通讯情况
1.测试步骤
在宿主机ubuntu22-25上以enp0s510.211.55.25作为父接口创建两个ipvlan子接口iplv3_bridge1和iplv3_bridge2mode:l3、flags:bridge。子接口iplv3_bridge1加入到网络命名空间ns1配置IP192.168.10.100并启用子接口iplv2_bridge2与父接口在相同网络命名空间中配置与父接口同段IP10.211.55.254并启用在宿主机ubuntu22-24上配置访问ipvlan子接口iplv3_bridge1的静态路由测试子接口iplv2_bridge1与父接口enp0s5网络通讯情况如下图
2.测试结果IPVLAN l3 bridge模式下父接口与子接口网络隔离不能直接通讯但是子接口可以通过与父接口同网络命名空间下的其它子接口转发通讯
补充vepa与private模式下ipvlan子接口之间无法直接通讯无法实现中转 2.3.4.1 配置IPVLAN l3 bridge 子接口与父接口通讯环境
创建网络命名空间ns1
rootubuntu22-25:~# ip netns add ns1创建两个ipvlan子接口mode l3flags bridgeipvl3_br1 和 ipvl3_br2
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_br1 type ipvlan mode l3 bridge
rootubuntu22-25:~# ip link add link enp0s5 name ipvl3_br2 type ipvlan mode l3 bridge配置ipvan子接口ipvl3_br1 ipvan子接口ipvl3_br1加入ns1ipvl3_br1配置IP 192.168.10.100/24ns1 使用 ipvl3_br1 配置默认路由
// ipvl3_br1 加入 ns1
rootubuntu22-25:~# ip link set ipvl3_br1 netns ns1
// ipvl3_br1配置IP 192.168.10.100
rootubuntu22-25:~# ip netns exec ns1 ip addr add 192.168.10.100/24 dev ipvl3_br1
rootubuntu22-25:~# ip netns exec ns1 ip link set ipvl3_br1 up
rootubuntu22-25:~# ip netns exec ns1 ip link set lo up
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns1 ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3: ipvl3_br1if2: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 192.168.10.100/24 scope global ipvl3_br1valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:11c:d10f/64 scope link valid_lft forever preferred_lft forever
// 配置默认路由
rootubuntu22-25:~# ip netns exec ns1 ip route add default dev ipvl3_br1
// 查看配置结果
rootubuntu22-25:~# ip netns exec ns1 ip r
default dev ipvl3_br1 scope link
192.168.10.0/24 dev ipvl3_br1 proto kernel scope link src 192.168.10.100
rootubuntu22-25:~# 配置ipvlan子接口ipvl3_br2 ipvl3_br2配置IP 10.211.55.254/24与父接口enp0s5 10.211.55.25 同段
// ipvl3_br2 配置IP
rootubuntu22-25:~# ip addr add 10.211.55.254/24 dev ipvl3_br2
rootubuntu22-25:~# ip link set ipvl3_br2 up
// 查看配置结果
rootubuntu22-25:~# ip a
1: lo: LOOPBACK,UP,LOWER_UP mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: enp0s5: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ffinet 10.211.55.25/24 brd 10.211.55.255 scope global enp0s5valid_lft forever preferred_lft foreverinet6 fdb2:2c26:f4e4:0:21c:42ff:fe1c:d10f/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591934sec preferred_lft 604734secinet6 fe80::21c:42ff:fe1c:d10f/64 scope link valid_lft forever preferred_lft forever
4: ipvl3_br2enp0s5: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ffinet 10.211.55.254/24 scope global ipvl3_br2valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft forever配置宿主机ubuntu22-24 访问ipvlan子接口ipvl3_br1的静态路由 为了配合测试ipvlan子接口ipvl3_br1 与外部通讯
// 配置宿主机ubuntu22-24 的静态路由
rootubuntu22-24:~# ip route add 192.168.10.0/24 via 10.211.55.25 dev enp0s5
// 查看配置结果
rootubuntu22-24:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.24
192.168.10.0/24 via 10.211.55.25 dev enp0s5
rootubuntu22-24:~# 2.3.4.2 测试IPVLAN l3 bridge 子接口与父接口通讯
ns1中ipvlan子接口ipvl3_br1 Ping 10.211.55.24ubuntu22-24通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.24 -c3
PING 10.211.55.24 (10.211.55.24): 56 data bytes
64 bytes from 10.211.55.24: icmp_seq0 ttl64 time0.484 ms
64 bytes from 10.211.55.24: icmp_seq1 ttl64 time0.558 ms
64 bytes from 10.211.55.24: icmp_seq2 ttl64 time0.697 ms
--- 10.211.55.24 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.484/0.580/0.697/0.088 ms
rootubuntu22-25:~# ns1中ipvlan子接口ipvl3_br1 Ping 10.211.55.25ipvlan父接口不通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.25 -c3
PING 10.211.55.25 (10.211.55.25): 56 data bytes
--- 10.211.55.25 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss宿主机默认命名空间下配置访问192.168.10.100的静态路由 配置使用ipvlan子接口ipvl3_br2进行中转
rootubuntu22-25:~# ip route add 192.168.10.100 dev ipvl3_br2
rootubuntu22-25:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev ipvl3_br2 proto kernel scope link src 10.211.55.254
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.25
192.168.10.100 dev ipvl3_br2 scope link
rootubuntu22-25:~# ipvlan子接口ipvl3_br1经ipvl3_br2转发与父接口enp0s5直接通讯 ns1下ipvl3_br1192.168.10.100Ping 父接口10.211.55.25通
rootubuntu22-25:~# ip netns exec ns1 ping 10.211.55.25 -c3
PING 10.211.55.25 (10.211.55.25): 56 data bytes
64 bytes from 10.211.55.25: icmp_seq0 ttl64 time0.056 ms
64 bytes from 10.211.55.25: icmp_seq1 ttl64 time0.084 ms
64 bytes from 10.211.55.25: icmp_seq2 ttl64 time0.153 ms
--- 10.211.55.25 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.056/0.098/0.153/0.041 msipvlan子接口ipvl3_br2上抓包查看中转结果 tcpdump -nn -e -i ipvl3_br2 not host 10.211.55.2 not host 抓包排除ssh客户端 10.211.55.2 -nn 两个n 表示不解析域名和端口 -e 在输出行打印出数据链路层的头部信息 -i 抓指定接口(网卡)的数据包
rootubuntu22-25:~# tcpdump -nn -e -i ipvl3_br2 not host 10.211.55.2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ipvl3_br2, link-type EN10MB (Ethernet), snapshot length 262144 bytes
21:07:10.375827 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 192.168.10.100: ICMP echo reply, id 1525, seq 0, length 64
21:07:11.377578 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 192.168.10.100: ICMP echo reply, id 1525, seq 1, length 64
21:07:12.379822 00:1c:42:1c:d1:0f 00:1c:42:1c:d1:0f, ethertype IPv4 (0x0800), length 98: 10.211.55.25 192.168.10.100: ICMP echo reply, id 1525, seq 2, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
rootubuntu22-25:~# 2.4 测试 IPVLAN L3S
IPVLAN L3S 与 IPVLAN L3 功能非常类似本节内容不再重复测试只介绍两者区别IPVLAN L3S启用了iptables (conn-tracking)
2.4.1 IPVLAN l3s 使用IPtables状态过滤conntrack
1.测试步骤
宿主机ubuntu22-25 启用nf_conntrack模块。使用宿主机ubuntu22-25 网卡enp0s5 创建两个ipvlan子接口mode:l3s、flags:bridgel3s_bridge1和l3s_bridge2分别加入到网络命名空间ns_bridge1和ns_bridge2中。在命名空间中配置两个ipvlan子接口IPl3s_bridge1 192.168.10.100l3s_bridge2 172.16.1.200和默认路由。在宿主机ubuntu22-24 添加静态路由将访问ipvlan子接口IP的路由指向宿主机ubuntu22-25。宿主机ubuntu22-25 配置iptables conntrack过滤规则。测试ipvlan子接口网络通讯的iptables状态过滤情况。
注意l3s与l2不同的是子接口需要配置路由。详见下图
2.测试结果 IPVLAN l3s 模式下可以通过iptables conntrack管控网络通讯 2.4.1.1 配置IPVLAN L3S测试环境
宿主机ubuntu22-25 启用nf_conntrack模块
// 加载nf_conntrack模块
rootubuntu22-25:~# modprobe nf_conntrack// 检查内核是否加载nf_conntrack模块
rootubuntu22-25:~# lsmod | grep nf_conntrack
nf_conntrack_netlink 49152 0
nf_conntrack 172032 1 nf_conntrack_netlink
nf_defrag_ipv6 24576 1 nf_conntrack
nf_defrag_ipv4 16384 1 nf_conntrack
nfnetlink 20480 6 nf_conntrack_netlink
libcrc32c 16384 3 nf_conntrack,btrfs,raid456// 查看状态跟踪记录。当前没有记录。
rootubuntu22-25:~# conntrack -L -o extended
conntrack v1.4.6 (conntrack-tools): 0 flow entries have been shown.宿主机ubuntu22-25 配置网络命名空间ns_bridge1 和 ns_bridge2宿主机ubuntu22-25 配置IPVLAN l3s子接口l3s_bridge1 和 l3s_bridge2
// 创建两个网络命名空间ns_bridge1 和 ns_bridge2
rootubuntu22-25:~# ip netns add ns_bridge1
rootubuntu22-25:~# ip netns add ns_bridge2
// 创建两个ipvlan子接口mode l3sflags bridge
rootubuntu22-25:~# ip link add link enp0s5 name l3s_bridge1 type ipvlan mode l3s bridge
rootubuntu22-25:~# ip link add link enp0s5 name l3s_bridge2 type ipvlan mode l3s bridge将l3s_bridge1 加入ns_bridge1配置IP 192.168.10.100使用l3s_bridge1配置ns_bridge1默认路由
// 将ipvlan子接口l3s_bridge1 加入网络命名空间ns_bridge1
rootubuntu22-25:~# ip link set l3s_bridge1 netns ns_bridge1
// 配置IP 192.168.10.100 并启用
rootubuntu22-25:~# ip netns exec ns_bridge1 ip addr add 192.168.10.100/24 dev l3s_bridge1
rootubuntu22-25:~# ip netns exec ns_bridge1 ip link set l3s_bridge1 up
rootubuntu22-25:~# ip netns exec ns_bridge1 ip link set lo up
// 配置ns_bridge1的默认路由
rootubuntu22-25:~# ip netns exec ns_bridge1 ip route add default dev l3s_bridge1将l3s_bridge2 加入ns_bridge2配置IP 172.16.1.200使用l3s_bridge2配置ns_bridge2默认路由
// 将ipvlan子接口l3s_bridge2 加入网络命名空间ns_bridge2
rootubuntu22-25:~# ip link set l3s_bridge2 netns ns_bridge2
// 配置IP 172.16.1.200 并启用
rootubuntu22-25:~# ip netns exec ns_bridge2 ip addr add 172.16.1.200/24 dev l3s_bridge2
rootubuntu22-25:~# ip netns exec ns_bridge2 ip link set l3s_bridge2 up
rootubuntu22-25:~# ip netns exec ns_bridge2 ip link set lo up
// 配置ns_bridge2的默认路由
rootubuntu22-25:~# ip netns exec ns_bridge2 ip route add default dev l3s_bridge2配置宿主机ubuntu22-24 静态路由与ipvlan子接口通讯
// 添加访问ipvlan子接口IP的静态路由
rootubuntu22-24:~# ip route add 192.168.10.0/24 via 10.211.55.25 dev enp0s5
rootubuntu22-24:~# ip route add 172.16.1.0/24 via 10.211.55.25 dev enp0s5
// 查看添加结果
rootubuntu22-24:~# ip route
default via 10.211.55.1 dev enp0s5 proto static
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.24
172.16.1.0/24 via 10.211.55.25 dev enp0s5
192.168.10.0/24 via 10.211.55.25 dev enp0s5
rootubuntu22-24:~# 2.4.1.2 测试IPVLAN L3S 默认情况下网络通讯情况
查看ns_bridge1中 l3s_bridge1 的配置
rootubuntu22-25:~# ip netns exec ns_bridge1 ip a show l3s_bridge1
3: l3s_bridge1if2: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UP group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 192.168.10.100/24 scope global l3s_bridge1valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:11c:d10f/64 scope link valid_lft forever preferred_lft forever两个IPVLAN子接口间测试Ping ns_bridge1中 l3s_bridge1192.168.10.100 Ping ns_bridge2中 l3s_bridge2172.16.1.200通 注意当前模式是bridge子接口之间内部转发通讯。private和vepa模式请参考 “2.3 测试IPVLAN L3”
rootubuntu22-25:~# ip netns exec ns_bridge1 ping 172.16.1.200
PING 172.16.1.200 (172.16.1.200): 56 data bytes
64 bytes from 172.16.1.200: icmp_seq0 ttl64 time0.063 ms
64 bytes from 172.16.1.200: icmp_seq1 ttl64 time0.074 ms
64 bytes from 172.16.1.200: icmp_seq2 ttl64 time0.073 ms
^C--- 172.16.1.200 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.063/0.084/0.146/0.028 ms命名空间中IPVLAN子接口Ping 外部宿主机ubuntu22-24 IP 通 ns_bridge1中l3s_bridge1 IP 192.168.10.100宿主机ubuntu22-24 IP 10.211.55.24
// 命名空间ns_bridge1中IPVLAN子接口 l3s_bridge1 IP 192.168.10.100
// Ping 宿主机ubuntu22-24 IP 10.211.55.24通
rootubuntu22-25:~# ip netns exec ns_bridge1 ping 10.211.55.24 -c3
PING 10.211.55.24 (10.211.55.24): 56 data bytes
64 bytes from 10.211.55.24: icmp_seq0 ttl64 time0.424 ms
64 bytes from 10.211.55.24: icmp_seq1 ttl64 time0.584 ms
64 bytes from 10.211.55.24: icmp_seq2 ttl64 time0.403 ms
--- 10.211.55.24 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.403/0.470/0.584/0.081 ms外部宿主机ubuntu22-24 Ping IPVLAN子接口 通
// 宿主机ubuntu22-24 Ping IPVLAN子接口l3s_bridge2的IP 172.16.1.200通
rootubuntu22-24:~# ping 172.16.1.200 -c3
PING 172.16.1.200 (172.16.1.200): 56 data bytes
64 bytes from 172.16.1.200: icmp_seq0 ttl64 time0.449 ms
64 bytes from 172.16.1.200: icmp_seq1 ttl64 time0.547 ms
64 bytes from 172.16.1.200: icmp_seq2 ttl64 time0.309 ms
--- 172.16.1.200 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.309/0.435/0.547/0.098 ms2.4.1.3 配置iptables 状态过滤规则
添加iptables filter表允许连接规则 确保ssh和已建立的连接不断开
// 允许外部连接22端口
rootubuntu22-25:~# iptables -A INPUT -p tcp --dport 22 -j ACCEPT
// 允许保持已建立的连接
rootubuntu22-25:~# iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
// 允许保持已建立的连接
rootubuntu22-25:~# iptables -A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT修改iptables filter默认规则为DROP
// iptables filter表 INPUT 默认 DROP
rootubuntu22-25:~# iptables -P INPUT DROP
// iptables filter表 INPUT 默认 DROP
rootubuntu22-25:~# OUTPUT -P OUTPUT DROP
// iptables filter表 FORWARD 默认 DROP
rootubuntu22-25:~# iptables -P FORWARD DROP添加iptables filter 涉及ipvlan 连接状态的规则 添加外部宿主机ubuntu22-24 访问ipvlan子接口 的连接状态规则INPUT链ACCEPT添加ipvlan子接口 访问外部宿主机ubuntu22-24 的连接状态规则OUTPUT链ACCEPT
说明为了便于理解针对每种连接都创建了一个独立规则实际使用中可以合并简化。关于连接状态的概念请参见Iptables状态跟踪机制介绍和优化探讨
// 添加宿主机ubuntu22-24 10.211.55.24 访问ipvlan子接口 192.168.10.100ACCEPT
rootubuntu22-25:~# iptables -A INPUT -s 10.211.55.24 -d 192.168.10.100 -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j ACCEPT
// 添加宿主机ubuntu22-24 10.211.55.24 访问ipvlan子接口 172.16.1.200ACCEPT
rootubuntu22-25:~# iptables -A INPUT -s 10.211.55.24 -d 172.16.1.200 -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j ACCEPT
// 添加ipvlan子接口 192.168.10.100 访问宿主机ubuntu22-24 10.211.55.24ACCEPT
rootubuntu22-25:~# iptables -A OUTPUT -s 192.168.10.100 -d 10.211.55.24 -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j ACCEPT
// 添加ipvlan子接口 172.16.1.200 访问宿主机ubuntu22-24 10.211.55.24ACCEPT
rootubuntu22-25:~# iptables -A OUTPUT -s 172.16.1.200 -d 10.211.55.24 -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j ACCEPT查看iptables filter 表添加后过滤规则
rootubuntu22-25:~# iptables -L -nvv --line-number
// INPUT 链默认DROP
Chain INPUT (policy DROP 87 packets, 6498 bytes)
num pkts bytes target prot opt in out source destination
1 1 64 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
2 9230 6973K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
3 2 168 ACCEPT all -- * * 10.211.55.24 192.168.10.100 ctstate NEW,RELATED,ESTABLISHED
4 2 168 ACCEPT all -- * * 10.211.55.24 172.16.1.200 ctstate NEW,RELATED,ESTABLISHED// FORWARD 链默认DROP
Chain FORWARD (policy DROP 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination // OUTPUT 链默认DROP
Chain OUTPUT (policy DROP 470 packets, 33928 bytes)
num pkts bytes target prot opt in out source destination
1 1242 290K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
2 4 336 ACCEPT all -- * * 192.168.10.100 10.211.55.24 ctstate NEW,RELATED,ESTABLISHED
3 0 0 ACCEPT all -- * * 172.16.1.200 10.211.55.24 ctstate NEW,RELATED,ESTABLISHED
rootubuntu22-25:~# 2.4.1.4 测试IPVLAN L3S的iptables 连接状态过滤
测试外部宿主机ubuntu22-24 Ping ipvlan子接口 宿主机ubuntu22-24 Ping 192.168.10.100ipvlan子接口 l3s_bridge1宿主机ubuntu22-25 nf_conntrack创建对应连接记录10.211.55.24 -- 192.168.10.100iptables filter表依据INPUT链conntrack规则放行10.211.55.24 -- 192.168.10.100宿主机ubuntu22-24 Ping 192.168.10.100通
// 宿主机ubuntu22-24 Ping 192.168.10.100, iptables INPUT链规则放行通
rootubuntu22-24:~# ping 192.168.10.100 -c2
PING 192.168.10.100 (192.168.10.100): 56 data bytes
64 bytes from 192.168.10.100: icmp_seq0 ttl64 time0.710 ms
64 bytes from 192.168.10.100: icmp_seq1 ttl64 time0.350 ms
--- 192.168.10.100 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.350/0.530/0.710/0.180 ms// 宿主机ubuntu22-25上对应产生的10.211.55.24 -- 192.168.10.100 的conntrack记录
rootubuntu22-25:~# conntrack -L -o extended
ipv4 2 icmp 1 17 src10.211.55.24 dst192.168.10.100 type8 code0 id4832 src192.168.10.100 dst10.211.55.24 type0 code0 id4832 mark0 use1
ipv4 2 tcp 6 431999 ESTABLISHED src10.211.55.2 dst10.211.55.25 sport65462 dport22 src10.211.55.25 dst10.211.55.2 sport22 dport65462 [ASSURED] mark0 use1
conntrack v1.4.6 (conntrack-tools): 2 flow entries have been shown.测试Ipvlan子接口l3s_bridge2 Ping 外部宿主机ubuntu22-24 ipvlan子接口l3s_bridge2 Ping 10.211.55.24宿主机ubuntu22-24宿主机ubuntu22-25 nf_conntrack创建对应连接记录172.16.1.200 -- 10.211.55.24iptables filter表依据OUTPUT链conntrack规则放行172.16.1.200 -- 10.211.55.24ipvlan子接口l3s_bridge2 Ping 10.211.55.24通
// 宿主机ubuntu22-25 清空nf_conntrack 记录表
rootubuntu22-25:~# conntrack -F
conntrack v1.4.6 (conntrack-tools): connection tracking table has been emptied.// 查看ns_bridge2中l3s_bridge2的配置
rootubuntu22-25:~# ip netns exec ns_bridge2 ip a show l3s_bridge2
4: l3s_bridge2if2: BROADCAST,MULTICAST,NOARP,UP,LOWER_UP mtu 1500 qdisc noqueue state UP group default qlen 1000link/ether 00:1c:42:1c:d1:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 172.16.1.200/24 scope global l3s_bridge2valid_lft forever preferred_lft foreverinet6 fe80::1c:4200:21c:d10f/64 scope link valid_lft forever preferred_lft forever
// ns_bridge2中l3s_bridge2 ping 10.211.55.24iptables OUTPUT链规则放行通
rootubuntu22-25:~# ip netns exec ns_bridge2 ping 10.211.55.24 -c2
PING 10.211.55.24 (10.211.55.24): 56 data bytes
64 bytes from 10.211.55.24: icmp_seq0 ttl64 time0.403 ms
64 bytes from 10.211.55.24: icmp_seq1 ttl64 time0.586 ms
--- 10.211.55.24 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.403/0.494/0.586/0.092 ms// 宿主机ubuntu22-25上对应产生的172.16.1.200 -- 10.211.55.24 的conntrack记录
rootubuntu22-25:~# conntrack -L -o extended
ipv4 2 tcp 6 431999 ESTABLISHED src10.211.55.2 dst10.211.55.25 sport65462 dport22 src10.211.55.25 dst10.211.55.2 sport22 dport65462 [ASSURED] mark0 use1
ipv4 2 icmp 1 22 src172.16.1.200 dst10.211.55.24 type8 code0 id2744 src10.211.55.24 dst172.16.1.200 type0 code0 id2744 mark0 use1
conntrack v1.4.6 (conntrack-tools): 2 flow entries have been shown.iptables删除conntrack过滤放行规则后对比测试 删除外部宿主机ubuntu22-24 访问ipvlan子接口的连接状态规则INPUT链和OUTPUT链测试外部宿主机ubuntu22-24 Ping 192.168.10.100iptables INPUT链规则默认丢弃不通测试ipvlan子接口l3s_bridge2 Ping 10.211.55.24iptables OUTPUT链规则默认丢弃不通ipvlan子接口之间正常访问不在iptables 过滤范围不受限制。
// 删除后的iptables filter已经去除了与10.211.55.24相关的放行规则默认是DROP
rootubuntu22-25:~# iptables -L -nvv --line-number
Chain INPUT (policy DROP 87 packets, 6498 bytes)
num pkts bytes target prot opt in out source destination
1 10111 7057K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
2 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22Chain FORWARD (policy DROP 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination Chain OUTPUT (policy DROP 575 packets, 45228 bytes)
num pkts bytes target prot opt in out source destination
1 1731 372K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED// 宿主机ubuntu22-24 Ping 192.168.10.100iptables INPUT链规则丢弃,不通
rootubuntu22-24:~# ping 192.168.10.100 -c3
PING 192.168.10.100 (192.168.10.100): 56 data bytes
--- 192.168.10.100 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
rootubuntu22-24:~# // 测试ipvlan子接口l3s_bridge2 Ping 10.211.55.24iptables OUTPUT链规则丢弃不通
rootubuntu22-25:~# ip netns exec ns_bridge2 ping 10.211.55.24 -c3
PING 10.211.55.24 (10.211.55.24): 56 data bytes
--- 10.211.55.24 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
rootubuntu22-25:~# ipvlan子接口之间正常访问不在iptables默认DROP 过滤范围不受限制
// 注意 ipvlan子接口之间正常访问不在iptables 过滤范围不受限制
rootubuntu22-25:~# ip netns exec ns_bridge2 ping 192.168.10.100 -c2
PING 192.168.10.100 (192.168.10.100): 56 data bytes
64 bytes from 192.168.10.100: icmp_seq0 ttl64 time0.096 ms
64 bytes from 192.168.10.100: icmp_seq1 ttl64 time0.081 ms
--- 192.168.10.100 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev 0.081/0.088/0.096/0.000 ms3. 总结 Linux的ipvlan分l2层、l3和l3s三种modebridge、private、vepa三种flags在同一父接口上均不可混用。其中l2是2层l3和l3s是3层区别是l3s支持iptables conntrackbridge、private、vepa决定同父接口下子接口之间的通讯bridge允许直接通讯、private禁止通讯、vepa可以通过外部转发通讯。 ipvlan子接口均可以与外部网络通讯需要外部网络、路由支持bridge下的ipvlan子接口可以通过与父接口同命名空间下的其它子接口中转实现与父接口通讯。
