Brigded Broadband

RHEL

RHEL

RHEL Repos

Ensure Register

Check register Status with these commands.  Output might no match exactly just general checks to see if anything looks weird.

sudo subscription-manager identity
system identity: 3634c02b-c3ce-4533-8151-8c7cc2d13318
name: RP-Host-03
org name: 7752782
org ID: 7752782

And

sudo subscription-manager status
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Registered
Content Access Mode is set to Simple Content Access. This host has access to content, regardless of subscription status.

Register if not

sudo subscription-manager register --org=7752782 --activationkey=KEY_HERE

Repos

Repos should be added automatically after registering.  Check to ensure the correct repos are enabled.

Ensure only these repos are enabled

sudo subscription-manager repos --list-enabled

Should show

+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   rhel-8-for-x86_64-baseos-rpms
Repo Name: Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel8/8.10/x86_64/baseos/os
Enabled:   1

Repo ID:   rhel-8-for-x86_64-appstream-rpms
Repo Name: Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel8/8.10/x86_64/appstream/os
Enabled:   1

Run updates

sudo dnf update -y

Failed EUS Repositories

Redhat adds Extended Update Support (EUS) repos to the list by default.  These can be disabled as they are only required for unsupported dependencies.

Fix for most cases

sudo subscription-manager repos \
  --disable=rhel-8-for-x86_64-baseos-eus-rpms \
  --disable=rhel-8-for-x86_64-appstream-eus-rpms
sudo dnf update -y

Manual Checking

Check the listed repos

sudo subscription-manager repos --list-enabled

Disable repos by name

sudo subscription-manager repos --disable=rhel-8-for-x86_64-baseos-eus-rpms

Run upgrades

sudo dnf update -y

Network

Overview

There are 4 network interfaces.
Each interface bonds 2 physical network cards.

The 4 main vlans used are

Exact mappings and vlan information is in the spreadsheet:
Network Interfaces and Vlans

This sheet is for the RCPT hosts.  The KCSW hosts are the exact same except the third IP field is +50.

RCPT Vlan


Vlan

172.18.2.0/24
Vlan Tag

2102


KCSW Vlan

Vlan

 172.18.52.0/24
Vlan Tag 2152

Bond configurations

This creates the bond interfaces

nmcli connection add ipv4.method disabled ipv6.method ignore type bond mode active-backup ifname bond1 con-name bond1
nmcli connection add type ethernet slave-type bond ifname ens6f0 master bond1 con-name bond1-slave0
nmcli connection add type ethernet slave-type bond ifname ens6f1 master bond1 con-name bond1-slave1

nmcli connection add ipv4.method disabled ipv6.method ignore type bond mode active-backup ifname bond2 con-name bond2
nmcli connection add type ethernet slave-type bond ifname eno12409np1 master bond2 con-name bond2-slave0
nmcli connection add type ethernet slave-type bond ifname eno12399np0 master bond2 con-name bond2-slave1

nmcli connection add ipv4.method disabled ipv6.method ignore type bond mode active-backup ifname bond3 con-name bond3
nmcli connection add type ethernet slave-type bond ifname eno12419np2 master bond3 con-name bond3-slave0
nmcli connection add type ethernet slave-type bond ifname eno12429np3 master bond3 con-name bond3-slave1

nmcli dev

Bridge configurations

This creates the bridges and adds them as slaves to the respective bond

nmcli con add type bridge ifname br1 con-name br1
nmcli con modify br1 ipv4.method disabled ipv6.method disabled
nmcli con modify bond1 master br1 slave-type bridge
nmcli con up br1

nmcli con add type bridge ifname br2 con-name br2
nmcli con modify br2 ipv4.method disabled ipv6.method disabled
nmcli con modify bond2 master br2 slave-type bridge
nmcli con up br2

nmcli con add type bridge ifname br3 con-name br3
nmcli con modify br3 ipv4.method disabled ipv6.method disabled
nmcli con modify bond3 master br3 slave-type bridge
nmcli con up br3

nmcli dev

VM ENV

This makes sure to use the system qemu instance instead of the userspace one when running virsh commands.

Run this before using any VM commands otherwise you will not find any VMs!

echo 'export LIBVIRT_DEFAULT_URI="qemu:///system"' >> ~/.bashrc
source ~/.bashrc

VM Nets

Use this script to add the bridges to the VMs

add_br_vm.sh

VM Bridge Assignments:

Client Bridge (br1) Vms:

NSP Internal Bridge (br2) Vms:

NSP Internal Bridge (br3) Vms:

BBB Server Outputs

HOST: RCPT-Host-03

[mdcadmin@RCPT-Host-03 ~]$ cat nmcli.log
bond0: connected to bond0
	"bond0"
	bond, 8C:84:74:FD:88:DE, sw, mtu 1500
	ip4 default
	inet4 172.18.0.23/24
	route4 default via 172.18.0.1 metric 300
	route4 172.18.0.0/24 metric 300
	inet6 fe80::8e84:74ff:fefd:88de/64
	route6 fe80::/64 metric 1024

virbr0: connected (externally) to virbr0
	"virbr0"
	bridge, 52:54:00:26:94:37, sw, mtu 1500
	inet4 192.168.122.1/24
	route4 192.168.122.0/24 metric 0

vnet37: connected (externally) to vnet37
	"vnet37"
	tun, FE:54:00:39:19:3B, sw, mtu 9216
	master br2
	inet6 fe80::fc54:ff:fe39:193b/64
	route6 fe80::/64 metric 256

vnet38: connected (externally) to vnet38
	"vnet38"
	tun, FE:54:00:DF:48:20, sw, mtu 9216
	master br2
	inet6 fe80::fc54:ff:fedf:4820/64
	route6 fe80::/64 metric 256

vnet39: connected (externally) to vnet39
	"vnet39"
	tun, FE:54:00:56:9B:F5, sw, mtu 9216
	master br3
	inet6 fe80::fc54:ff:fe56:9bf5/64
	route6 fe80::/64 metric 256

vnet42: connected (externally) to vnet42
	"vnet42"
	tun, FE:54:00:5D:E2:16, sw, mtu 9216
	master br2
	inet6 fe80::fc54:ff:fe5d:e216/64
	route6 fe80::/64 metric 256

vnet43: connected (externally) to vnet43
	"vnet43"
	tun, FE:54:00:6E:3B:20, sw, mtu 9216
	master br2
	inet6 fe80::fc54:ff:fe6e:3b20/64
	route6 fe80::/64 metric 256

vnet46: connected (externally) to vnet46
	"vnet46"
	tun, FE:54:00:F5:D4:68, sw, mtu 1500
	master br1
	inet6 fe80::fc54:ff:fef5:d468/64
	route6 fe80::/64 metric 256

vnet47: connected (externally) to vnet47
	"vnet47"
	tun, FE:54:00:B3:EE:80, sw, mtu 9216
	master br2
	inet6 fe80::fc54:ff:feb3:ee80/64
	route6 fe80::/64 metric 256

vnet48: connected (externally) to vnet48
	"vnet48"
	tun, FE:54:00:81:87:68, sw, mtu 1500
	master br1
	inet6 fe80::fc54:ff:fe81:8768/64
	route6 fe80::/64 metric 256

vnet49: connected (externally) to vnet49
	"vnet49"
	tun, FE:54:00:69:8F:A2, sw, mtu 9216
	master br3
	inet6 fe80::fc54:ff:fe69:8fa2/64
	route6 fe80::/64 metric 256

vnet50: connected (externally) to vnet50
	"vnet50"
	tun, FE:54:00:17:BC:71, sw, mtu 1500
	master br1
	inet6 fe80::fc54:ff:fe17:bc71/64
	route6 fe80::/64 metric 256

vnet51: connected (externally) to vnet51
	"vnet51"
	tun, FE:54:00:BF:6F:37, sw, mtu 1500
	master br1
	inet6 fe80::fc54:ff:febf:6f37/64
	route6 fe80::/64 metric 256

vnet52: connected (externally) to vnet52
	"vnet52"
	tun, FE:54:00:BD:64:34, sw, mtu 1500
	master br1
	inet6 fe80::fc54:ff:febd:6434/64
	route6 fe80::/64 metric 256

vnet53: connected (externally) to vnet53
	"vnet53"
	tun, FE:54:00:06:3A:3A, sw, mtu 9216
	master br2
	inet6 fe80::fc54:ff:fe06:3a3a/64
	route6 fe80::/64 metric 256

vnet54: connected (externally) to vnet54
	"vnet54"
	tun, FE:54:00:E6:4C:C4, sw, mtu 1500
	master br1
	inet6 fe80::fc54:ff:fee6:4cc4/64
	route6 fe80::/64 metric 256

bond1: connected to bond1
	"bond1"
	bond, 8C:84:74:FD:88:DC, sw, mtu 1500
	master br1

bond2: connected to bond2
	"bond2"
	bond, 8C:84:74:47:86:00, sw, mtu 9216
	master br2

bond3: connected to bond3
	"bond3"
	bond, 8C:84:74:47:86:02, sw, mtu 9216
	master br3

br1: connected to br1
	"br1"
	bridge, 8C:84:74:FD:88:DC, sw, mtu 1500

br2: connected to br2
	"br2"
	bridge, 8C:84:74:47:86:00, sw, mtu 9216

br3: connected to br3
	"br3"
	bridge, 8C:84:74:47:86:02, sw, mtu 9216

eno12399np0: connected to bond2-slave1
	"Broadcom and subsidiaries BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb"
	ethernet (bnxt_en), 8C:84:74:47:86:00, hw, mtu 9216
	master bond2

eno12409np1: connected to bond2-slave0
	"Broadcom and subsidiaries BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb"
	ethernet (bnxt_en), 8C:84:74:47:86:00, hw, mtu 9216
	master bond2

eno12419np2: connected to bond3-slave0
	"Broadcom and subsidiaries BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb"
	ethernet (bnxt_en), 8C:84:74:47:86:02, hw, mtu 9216
	master bond3

eno12429np3: connected to bond3-slave1
	"Broadcom and subsidiaries BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb"
	ethernet (bnxt_en), 8C:84:74:47:86:02, hw, mtu 9216
	master bond3

ens6f0: connected to bond1-slave0
	"Broadcom and subsidiaries NetXtreme BCM5719"
	ethernet (tg3), 8C:84:74:FD:88:DC, hw, mtu 1500
	master bond1

ens6f1: connected to bond1-slave1
	"Broadcom and subsidiaries NetXtreme BCM5719"
	ethernet (tg3), 8C:84:74:FD:88:DC, hw, mtu 1500
	master bond1

ens6f2: connected to bond0-slave1
	"Broadcom and subsidiaries NetXtreme BCM5719"
	ethernet (tg3), 8C:84:74:FD:88:DE, hw, mtu 1500
	master bond0

ens6f3: connected to bond0-slave0
	"Broadcom and subsidiaries NetXtreme BCM5719"
	ethernet (tg3), 8C:84:74:FD:88:DE, hw, mtu 1500
	master bond0

vsr_nrc_mgt: connected (externally) to vsr_nrc_mgt
	"vsr_nrc_mgt"
	tun, FE:54:00:70:E2:93, sw, mtu 1500
	master br1

lo: unmanaged
	"lo"
	loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

vsr_nrc_p1: unmanaged
	"vsr_nrc_p1"
	tun, FE:54:00:03:CB:0E, sw, mtu 1500

DNS configuration:
	servers: 8.8.8.8 8.8.4.4 1.1.1.1
	interface: bond0

Use "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.

Consult nmcli(1) and nmcli-examples(7) manual pages for complete usage details.

VM: RCPT-NFM-P-COLLO

[mdcadmin@rcpt-nfm-p-collo ~]$ nmcli
enp1s0: connected to enp1s0
        "Red Hat Virtio"
        ethernet (virtio_net), 52:54:00:DF:48:20, hw, mtu 9216
        ip4 default
        inet4 172.18.1.62/24
        route4 172.18.1.0/24 metric 100
        route4 default via 172.18.1.1 metric 100
        route4 172.18.51.0/24 via 172.18.1.4 metric 100

virbr0: connected (externally) to virbr0
        "virbr0"
        bridge, 52:54:00:AB:46:FE, sw, mtu 1500
        inet4 192.168.122.1/24
        route4 192.168.122.0/24 metric 0

enp7s0: connected to enp7s0
        "Red Hat Virtio"
        ethernet (virtio_net), 52:54:00:56:9B:F5, hw, mtu 9216
        inet4 172.18.2.59/24
        route4 172.18.2.0/24 metric 101
        route4 72.16.20.0/24 via 172.18.2.4 metric 0
        route4 172.23.0.0/16 via 172.18.2.4 metric 0
        route4 172.22.0.0/16 via 172.18.2.4 metric 0

enp8s0: disconnected
        "Red Hat Virtio"
        1 connection available
        ethernet (virtio_net), 52:54:00:F5:D4:68, hw, mtu 1500

lo: unmanaged
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
        servers: 8.8.8.8 8.8.4.4
        interface: enp1s0

VM: RCPT-NSP-07

[mdcadmin@rcpt-nsp-07 ~]$ nmcli
enp1s0: connected to enp1s0
        "Red Hat Virtio 1.0"
        ethernet (virtio_net), 52:54:00:E6:4C:C4, hw, mtu 1500
        ip4 default
        inet4 172.18.1.57/24
        route4 172.18.1.0/24 metric 105
        route4 default via 172.18.1.1 metric 105
        route4 172.18.51.0/24 via 172.18.1.4 metric 105

enp7s0: connected to enp7s0
        "Red Hat Virtio 1.0"
        ethernet (virtio_net), 52:54:00:06:3A:3A, hw, mtu 1500
        inet4 172.18.2.57/24
        route4 172.18.2.0/24 metric 103
        route4 172.16.20.0/24 via 172.18.2.4 metric 103
        route4 172.23.0.0/16 via 172.18.2.4 metric 103
        route4 172.22.0.0/16 via 172.18.2.4 metric 103

enp8s0: connecting (getting IP configuration) to Wired connection 1
        "Red Hat Virtio 1.0"
        ethernet (virtio_net), 52:54:00:BD:64:34, hw, mtu 1500

lo: connected (externally) to lo
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536
        inet4 127.0.0.1/8
        inet6 ::1/128
        route6 ::1/128 metric 256

[mdcadmin@rcpt-nsp-07 ~]$