为了增加带宽或提升高可用性,需要用到bonding功能,openvswitch也可以和bonding很好的结合。将分两种应用场景分别讲述bonding的配置。

本实验使用的服务器和ip地址如下:

服务器

ip

dl165-1

192.168.33.231

dl165-2

192.168.33.232

rh2285-3

192.168.33.233

rh2285-4

192.168.33.234

这些服务器的业务口全部连接到同一台物理服务器上,隶属于同一个vlan(192.168.33.0/24)。

场景一

一台物理服务器(RH2285-4),两个物理网口(eth2/eth3)接入同一台物理交换机(huawei 5328)的不同网口(gi0/0/7和gi0/0/8)

服务器端设置

eth2和eth3构成bond1,ovsbr1桥接至bond1,以下是详细配置:

  • rh2285-4端口配置

    ifcfg-eth2
    DEVICE="eth2"
    BOOTPROTO="none"
    ONBOOT="yes"
    NM_CONTROLLED="no"
    USERCTL="no"
    ifcfg-eth3
    DEVICE="eth3"
    BOOTPROTO="none"
    ONBOOT="yes"
    NM_CONTROLLED="no"
    USERCTL="no"
    ifcfg-bond1
    DEVICE=bond1
    ONBOOT=yes
    DEVICETYPE=ovs
    TYPE=OVSBond
    OVS_BRIDGE=ovsbr1
    BOOTPROTO=none
    BOND_IFACES="eth2 eth3"
    OVS_OPTIONS="bond_mode=balance-tcp lacp=active other_config:lacp-time=fast"
    HOTPLUG=no
    ifcfg-ovsbr1
    DEVICE=ovsbr1
    ONBOOT=yes
    DEVICETYPE=ovs
    TYPE=OVSBridge
    BOOTPROTO=static
    IPADDR=192.168.33.232
    NETMASK=255.255.255.0
    HOTPLUG=no
  • 查看bond状态

    $ sudo ovs-appctl bond/show
    ---- bond1 ----
    bond_mode: balance-tcp
    bond may use recirculation: yes, Recirc-ID : 300
    bond-hash-basis: 0
    updelay: 0 ms
    downdelay: 0 ms
    next rebalance: 2546 ms
    lacp_status: negotiated
    active slave mac: 3c:d9:2b:fd:39:f0(eth2)
    
    slave eth2: enabled
            active slave
            may_enable: true
    
    slave eth3: enabled
            may_enable: true
  • 查看lacp状态

    $ sudo ovs-appctl lacp/show
    ---- bond1 ----
            status: active negotiated
            sys_id: 3c:d9:2b:fd:39:f0
            sys_priority: 65534
            aggregation key: 1
            lacp_time: fast
    
    slave: eth2: current attached
            port_id: 2
            port_priority: 65535
            may_enable: true
    
            actor sys_id: 3c:d9:2b:fd:39:f0
            actor sys_priority: 65534
            actor port_id: 2
            actor port_priority: 65535
            actor key: 1
            actor state: activity timeout aggregation synchronized collecting distributing
    
            partner sys_id: 54:89:98:73:bb:07
            partner sys_priority: 32768
            partner port_id: 7
            partner port_priority: 32768
            partner key: 305
            partner state: activity timeout aggregation synchronized collecting distributing
    
    slave: eth3: current attached
            port_id: 1
            port_priority: 65535
            may_enable: true
    
            actor sys_id: 3c:d9:2b:fd:39:f0
            actor sys_priority: 65534
            actor port_id: 1
            actor port_priority: 65535
            actor key: 1
            actor state: activity timeout aggregation synchronized collecting distributing
    
            partner sys_id: 54:89:98:73:bb:07
            partner sys_priority: 32768
            partner port_id: 8
            partner port_priority: 32768
            partner key: 305
            partner state: activity timeout aggregation synchronized collecting distributing

交换机配置

本次使用的是华为5328交换机。

interface Eth-Trunk1
 description to [RH2285-4]
 port link-type access
 port default vlan 55
 mode lacp-static
 lacp timeout fast
 ntdp enable
 ndp enable
 bpdu enable
interface GigabitEthernet0/0/7
 description [RH2285-4 eth2]
 eth-trunk 1
 undo ntdp enable
 undo ndp enable
interface GigabitEthernet0/0/8
 description to [RH2285-4 eth3]
 eth-trunk 1
 undo ntdp enable
 undo ndp enable

分别在rh2285-3、dl165-1、dl165-2中执行:

$ sudo ping rh2285-4
$ sudo paping rh2285-4 -p 5900

rh2285-4使用tcpdump来观察eth2和eth3的流量分布情况,命令如下:

$ sudo tcpdump -nvi eth2 icmp
$ sudo tcpdump -nvi eth3 icmp

以及

$ sudo tcpdump -nvi eth2 port 5900
$ sudo tcpdump -nvi eth3 port 5900

观察到的结果:流量呈现分布状态。

  • icmp

    从eth2进来,从eth3出去。

  • tcp:5900

    从eth2、eth3进来后又出去,流量比较混杂,还夹杂了一些incorrect的数据包。

关于incorrect

从rh2285-3 ssh rh2285-4,可以成功登陆。 同时在rh2285-4使用tcpdump -nvi eth2 port 22 and host 192.168.33.233,出现incorrect数据包,但似乎不影响ssh服务。

18:14:28.085592 IP (tos 0x10, ttl 64, id 32770, offset 0, flags [DF], proto TCP (6), length 404)
    192.168.33.232.ssh > 192.168.33.231.36134: Flags [P.], cksum 0xf2a6 (incorrect -> 0x3d3e), seq 433:785, ack 432, win 189, options [nop,nop,TS val 96361804 ecr 1815678915], length 352

可能的原因是开启了网卡的tcp checksum offload。实际上incorrect checksum是正常的这里有更详细的说明。

场景二(h3c 5800)

服务器配置

服务器的配置跟场景一的服务器配置一致。

交换机配置

rh2285-4的两个物理网口分别连接两台物理交换机(h3c 5800)。两台h3c 5800已经做了IRF堆叠,link-aggregation的配置如下:

interface Bridge-Aggregation20
 description "rh2285-4 bond1"
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 10 20 24 28 111
 link-aggregation mode dynamic
interface GigabitEthernet1/0/20
 port link-mode bridge
 description "rh2285-4:port2"
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 10 20 24 28 111
 port link-aggregation group 20
interface GigabitEthernet2/0/20
 port link-mode bridge
 description "rh2285-4:port3"
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 10 20 24 28 111
 port link-aggregation group 20

实验的结果跟场景一的结果一致。特别需要提醒的是,两台物理交换机之间必须做了堆叠。每个厂家在堆叠技术方面都有自己的私有协议,因而不同厂家之间的交换机不一定能够成功实现堆叠。