WeOSTwo Consist Networks with Two Redundant ETB Nodes HowTo
This document describes how to configure a consist with two consist networks managed by two redundant ETBNs. The consist is depicted in Figure 1.
The result of the inauguration will be a network where ETBN1 has the ETBN ID 1, the IP-address 10.128.0.1 (based on the ETBN ID and since the network is configured as TCMS). ETBN2 will attain ETBN-ID 2 and IP 10.128.1.2. CN1 will attain the network IP 10.128.64.0/18, and CN2 IP 10.128.128.0/18.
.---------------------------.
/ cst1 \
'-----------------------------'
.-------. .-------.
--+ +----+ +-- dir1 = (ethX5, ethX6)
<-dir1| ETBN1 | | ETBN2 |dir2-> dir2 = (ethX7, ethX8)
--+ +----+ +--
'-+--+--' '--+--+-'
| | | |
--+----------------+-- CN1 vlan10 (ethX3)
| |
---+----------+-------- CN2 vlan20 (ethX4)
Figure 1: A consist with two consist networks managed by two redundant ETB nodes.
Note
In order to get two ETB nodes to manage two consist networks, they
both need to have the exact same configuration with respect to uuid,
node and, in this setup, ecn configuration. Also, the ETBN must
have different local-id.
Configuration
Start with ETBN 1 in the consist and configure VLANs and the TTDP part. Both nodes also need to have VRRP configured and enabled.
etbn1:#> configure etbn1:/config/#> vlan 10 Creating new VLAN vid:10 with name: vlan10 etbn1:/config/vlan-10/#> untagged ethX3 etbn1:/config/vlan-10/#> end etbn1:/config/#> vlan 20 Creating new VLAN vid:20 with name: vlan20 etbn1:/config/vlan-20/#> untagged ethX4 etbn1:/config/vlan-20/#> end etbn1:/config/#> iface vlan10 etbn1:/config/iface-vlan10/#> no inet etbn1:/config/iface-vlan10/#> inet static 10.10.0.1/18 etbn1:/config/iface-vlan10/inet-static-10.10.0.1/#> end etbn1:/config/iface-vlan10/#> end etbn1:/config/#> iface vlan20 etbn1:/config/iface-vlan20/#> no inet etbn1:/config/iface-vlan20/#> inet static 10.20.0.1/18 etbn1:/config/iface-vlan20/inet-static-10.20.0.1/#> end etbn1:/config/iface-vlan20/#> end etbn1:/config/#> ttdp Activating TTDP with default settings. etbn1:/config/ttdp/#> uuid 11111111-1111-1111-1111-111111111111 etbn1:/config/ttdp/#> node 1 ecn 1,2 etbn1:/config/ttdp/#> node 2 ecn 1,2 etbn1:/config/ttdp/#> ecn 1 vlan10 etbn1:/config/ttdp/#> ecn 2 vlan20 etbn1:/config/ttdp/#> dir1 ethX5,ethX6 Remember to manually set mdi/mdi-x mode in port settings for all agg ports. etbn1:/config/ttdp/#> dir2 ethX7,ethX8 Remember to manually set mdi/mdi-x mode in port settings for all agg ports. etbn1:/config/ttdp/#> local-id 1 etbn1:/config/ttdp/#> end TTDP: Created VLAN 492. Note that this VLAN will not be deleted automatically if TTDP is deactivated. etbn1:/config/#> vlan 2 Creating new VLAN vid:2 with name: vlan2 etbn1:/config/vlan-2/#> no multicast-snooping etbn1:/config/vlan-2/#> untagged lag-dir1,lag-dir2 etbn1:/config/vlan-2/#> end
Continue by configuring the ports.
etbn1:/config/#> port ethX5 etbn1:/config/port-ethX5/#> speed-duplex 100-full etbn1:/config/port-ethX5/#> no auto-negotiate etbn1:/config/port-ethX5/#> fastlink etbn1:/config/port-ethX5/#> mdix-mode mdi etbn1:/config/port-ethX5/#> end etbn1:/config/#> port ethX6 etbn1:/config/port-ethX6/#> speed-duplex 100-full etbn1:/config/port-ethX6/#> no auto-negotiate etbn1:/config/port-ethX6/#> fastlink etbn1:/config/port-ethX6/#> mdix-mode mdi etbn1:/config/port-ethX6/#> end etbn1:/config/#> port ethX7 etbn1:/config/port-ethX7/#> speed-duplex 100-full etbn1:/config/port-ethX7/#> no auto-negotiate etbn1:/config/port-ethX7/#> fastlink etbn1:/config/port-ethX7/#> mdix-mode mdi etbn1:/config/port-ethX7/#> end etbn1:/config/#> port ethX8 etbn1:/config/port-ethX8/#> speed-duplex 100-full etbn1:/config/port-ethX8/#> no auto-negotiate etbn1:/config/port-ethX8/#> fastlink etbn1:/config/port-ethX8/#> mdix-mode mdi etbn1:/config/port-ethX8/#> end
Note
In this HowTo, it is assumed that the ETB uses crossover cabling. For this reason, all ETB ports on both ETBNs use the same MDI-X mode. If non-crossover cabling is used, these settings must be changed for two of the inward-facing ETB ports.
Next step is to configure one VRRP instance for each CN on each node. Start with the node that will have the MASTER role for both CN’s. In this example, ETBN1 is selected as MASTER (see VRRP).
First, configure the VRRP instance for CN1:
etbn1:/config/#> router etbn1:/config/router/#> vrrp cn1 Creating new VRRP instance: cn1 Invalid settings: Interface not set. etbn1:/config/router/vrrp-cn1/#> iface vlan10 Warning: No ports have been added to the configured VLAN. VRRP will not work as expected with this configuration. Invalid settings: Invalid IPv4 address. etbn1:/config/router/vrrp-cn1/#> address 10.10.0.100 Invalid settings: VRID invalid. Valid values: 1 to 255. etbn1:/config/router/vrrp-cn1/#> vrid 132 etbn1:/config/router/vrrp-cn1/#> version 3 etbn1:/config/router/vrrp-cn1/#> priority 150 etbn1:/config/router/vrrp-cn1/#> interval 1 etbn1:/config/router/vrrp-cn1/#> preempt delay 1 etbn1:/config/router/vrrp-cn1/#> end
Configure the VRRP instance for CN2:
etbn1:/config/router/#> vrrp cn2 Creating new VRRP instance: cn2 Invalid settings: Interface not set. etbn1:/config/router/vrrp-cn2/#> iface vlan20 Warning: No ports have been added to the configured VLAN. VRRP will not work as expected with this configuration. Invalid settings: Invalid IPv4 address. etbn1:/config/router/vrrp-cn2/#> address 10.20.0.100 Invalid settings: VRID invalid. Valid values: 1 to 255. etbn1:/config/router/vrrp-cn2/#> vrid 132 etbn1:/config/router/vrrp-cn2/#> version 3 etbn1:/config/router/vrrp-cn2/#> priority 150 etbn1:/config/router/vrrp-cn1/#> interval 1 etbn1:/config/router/vrrp-cn1/#> preempt delay 1 etbn1:/config/router/vrrp-cn2/#> end etbn1:/config/router/#> end etbn1:/config/#> end Applying configuration. Configuration activated. Remember "copy run start" to save to flash (NVRAM). etbn1:/#>
Now, repeat the exact steps above in the second ETB node. The only differences are the IP addresses on the CNs and the local id of the ETBN.
etbn2:#> configure etbn2:/config/#> vlan 10 Creating new VLAN vid:10 with name: vlan10 etbn2:/config/vlan-10/#> untagged ethX3 etbn2:/config/vlan-10/#> end etbn2:/config/#> vlan 20 Creating new VLAN vid:20 with name: vlan20 etbn2:/config/vlan-20/#> untagged ethX4 etbn2:/config/vlan-20/#> end etbn2:/config/#> iface vlan10 etbn2:/config/iface-vlan10/#> no inet etbn2:/config/iface-vlan10/#> inet static 10.10.0.2/18 etbn2:/config/iface-vlan10/inet-static-10.10.0.2/#> end etbn2:/config/iface-vlan10/#> end etbn2:/config/#> iface vlan20 etbn2:/config/iface-vlan20/#> no inet etbn2:/config/iface-vlan20/#> inet static 10.20.0.2/18 etbn2:/config/iface-vlan20/inet-static-10.20.0.2/#> end etbn2:/config/iface-vlan20/#> end etbn2:/config/#> ttdp Activating TTDP with default settings. etbn2:/config/ttdp/#> uuid 11111111-1111-1111-1111-111111111111 etbn2:/config/ttdp/#> node 1 ecn 1,2 etbn2:/config/ttdp/#> node 2 ecn 1,2 etbn2:/config/ttdp/#> ecn 1 vlan10 etbn2:/config/ttdp/#> ecn 2 vlan20 etbn2:/config/ttdp/#> dir1 ethX5,ethX6 Remember to manually set mdi/mdi-x mode in port settings for all agg ports. etbn2:/config/ttdp/#> dir2 ethX7,ethX8 Remember to manually set mdi/mdi-x mode in port settings for all agg ports. etbn2:/config/ttdp/#> local-id 2 etbn2:/config/ttdp/#> end TTDP: Created VLAN 492. Note that this VLAN will not be deleted automatically if TTDP is deactivated. etbn2:/config/#> vlan 2 Creating new VLAN vid:2 with name: vlan2 etbn2:/config/vlan-2/#> no multicast-snooping etbn2:/config/vlan-2/#> untagged lag-dir1,lag-dir2 etbn2:/config/vlan-2/#> end
Continue by configuring the ports.
etbn2:/config/#> port ethX5 etbn2:/config/port-ethX5/#> speed-duplex 100-full etbn2:/config/port-ethX5/#> no auto-negotiate etbn2:/config/port-ethX5/#> fastlink etbn2:/config/port-ethX5/#> mdix-mode mdi etbn2:/config/port-ethX5/#> end etbn2:/config/#> port ethX6 etbn2:/config/port-ethX6/#> speed-duplex 100-full etbn2:/config/port-ethX6/#> no auto-negotiate etbn2:/config/port-ethX6/#> fastlink etbn2:/config/port-ethX6/#> mdix-mode mdi etbn2:/config/port-ethX6/#> end etbn2:/config/#> port ethX7 etbn2:/config/port-ethX7/#> speed-duplex 100-full etbn2:/config/port-ethX7/#> no auto-negotiate etbn2:/config/port-ethX7/#> fastlink etbn2:/config/port-ethX7/#> mdix-mode mdi etbn2:/config/port-ethX7/#> end etbn2:/config/#> port ethX8 etbn2:/config/port-ethX8/#> speed-duplex 100-full etbn2:/config/port-ethX8/#> no auto-negotiate etbn2:/config/port-ethX8/#> fastlink etbn2:/config/port-ethX8/#> mdix-mode mdi etbn2:/config/port-ethX8/#> end
Repeat the same steps for ETBN2 with a lower priority to make it backup for both CN’s. In this example the priority for both instances on ETBN2 is set to 100.
First, configure the VRRP instance for CN1:
etbn2:/config/#> router
etbn2:/config/router/#> vrrp cn1
Creating new VRRP instance: cn1
Invalid settings: Interface not set.
etbn2:/config/router/vrrp-cn1/#> iface vlan10
Warning: No ports have been added to the configured VLAN.
VRRP will not work as expected with this configuration.
Invalid settings: Invalid IPv4 address.
etbn2:/config/router/vrrp-cn1/#> address 10.10.0.100
Invalid settings: VRID invalid. Valid values: 1 to 255.
etbn2:/config/router/vrrp-cn1/#> vrid 132
etbn2:/config/router/vrrp-cn1/#> version 3
etbn2:/config/router/vrrp-cn1/#> priority 100
etbn2:/config/router/vrrp-cn1/#> interval 1
etbn2:/config/router/vrrp-cn1/#> preempt delay 1
etbn2:/config/router/vrrp-cn1/#> end
Now configure the VRRP instance for CN2:
etbn2:/config/router/#> vrrp cn2
Creating new VRRP instance: cn2
Invalid settings: Interface not set.
etbn2:/config/router/vrrp-cn2/#> iface vlan20
Warning: No ports have been added to the configured VLAN.
VRRP will not work as expected with this configuration.
Invalid settings: Invalid IPv4 address.
etbn2:/config/router/vrrp-cn2/#> address 10.20.0.100
Invalid settings: VRID invalid. Valid values: 1 to 255.
etbn2:/config/router/vrrp-cn2/#> vrid 132
etbn2:/config/router/vrrp-cn2/#> version 3
etbn2:/config/router/vrrp-cn2/#> priority 100
etbn2:/config/router/vrrp-cn1/#> interval 1
etbn2:/config/router/vrrp-cn1/#> preempt delay 1
etbn2:/config/router/vrrp-cn2/#> end
etbn2:/config/router/#> end
etbn2:/config/#> end
Applying configuration.
Configuration activated.
Remember "copy run start" to save to flash (NVRAM).
etbn2:/#>
Inauguration Result
Using the show ttdp command we now verify that we have recevied the intended network configuration.
Highlighted in orange are the values that are of interest.
etbn1:/#> show ttdp TTDP NODE backbone-id : 0 (tcms) etbn-id : 1 etbn-ip : 10.128.0.1 local-id : 1 own-mac : 0c:65:8c:93:00:00 uuid : 11111111-1111-1111-1111-111111111111 connTableCnt : 0xb81af94 EtbTopoCnt : 0xde33f1c2 Inauguration : INAUGURATED Inhibition : Self: Off, Others: Off, Train: Off CST-NET IFACE IP CONNECTED ETBNs 1 vlan10 10.10.0.1/18 1,2 2 vlan20 10.20.0.1/18 1,2 NODE ID MAC ADDRESS ORIENTATION 1 0c:65:8c:93:00:00 Direct 2 0c:7a:40:ee:00:00 Direct CONSIST NODES DIRECTION NETWORKS 11111111-1111-1111-1111-111111111111 1..2 Direct 10.128.64.0/18 10.128.128.0/18 etbn1:/#>
etbn2:/#> show ttdp TTDP NODE backbone-id : 0 (tcms) etbn-id : 2 etbn-ip : 10.128.0.2 local-id : 2 own-mac : 0c:7a:40:ee:00:00 uuid : 11111111-1111-1111-1111-111111111111 connTableCnt : 0xb81af94 EtbTopoCnt : 0xde33f1c2 Inauguration : INAUGURATED Inhibition : Self: Off, Others: Off, Train: Off CST-NET IFACE IP CONNECTED ETBNs 1 vlan10 10.10.0.2/18 1,2 2 vlan20 10.20.0.2/18 1,2 NODE ID MAC ADDRESS ORIENTATION 1 0c:65:8c:93:00:00 Direct 2 0c:7a:40:ee:00:00 Direct CONSIST NODES DIRECTION NETWORKS 11111111-1111-1111-1111-111111111111 1..2 Direct 10.128.64.0/18 10.128.128.0/18 etbn2:/#>