Proxmox VE with Open vSwitch

pve_and_ovs

By default, Proxmox networking uses the Linux bridge. There is no issue with the Linux bridges and it actually works really well. I have been using my PVE for two years now with Linux bridges. I recently switched to OVS for several reasons.

  • OVS can be configured as layer 2 or layer 3 bridges. Linux bridge, as far as I know, is just layer 2
  • I want to simplify my bridge setup. With OVS, I only have one bridge and use the VLAN tag. With Linux bridge, I would have one bridge per VLAN
  • OVS supports more protocols such as VxLAN, GRE, STT, LLDP, etc. Linux bridge, on the other hand, does support some protocols, but not as much as OVS
  • As a network guy and in my personal opinion, I think, OVS has a cleaner logic than the Linux bridge

Figure 1 shows the Network settings when creating a VM.

Figure 1

Somehow the Open vSwitch is not installed by default at least from Proxmox version 6 and below. I kind of hope it would come with it already. To install the Open vSwitch, use the web UI shell or SSH into the Proxmox server. In the web UI, there are a couple of ways to get to the shell. See Figure 2 as a reference.

  • Click on the PVE node then click on the Shell button on the upper right corner right below the Create VM button
  • navigate to PVE node > Shell
Figure 2

If we don’t install the openvswitch-switch, the system will get an error message Open VSwitch is not installed (need package 'openvswitch-switch') (500). Therefore, once access to the shell has been established, enter the command below to install Open vSwitch:

apt install -y openvswitch-switch
apt install -y openvswitch-switch
apt install -y openvswitch-switch

Don’t forget to make a backup of the current interfaces just in case something goes wrong. While you are in the shell, enter the command below to backup the original config file.

cp /etc/network/interfaces /etc/network/interfaces.bak
cp /etc/network/interfaces /etc/network/interfaces.bak
cp /etc/network/interfaces /etc/network/interfaces.bak

After installing the Open vSwitch and making a backup of the /etc/network/interfaces, we would need to delete the default Linux bridge. To do so, navigate to :

  • Data Center > PVE Node > System > Network
    1. Select the vmbr0
    2. Click on Remove
Figure 3

Once the Linux bridge vmbr0 has been removed, we need to create OVS bridge

  • Data Center > PVE Node > System > Network
    1. Click on Create
    2. Select the OVS Bridge
      1. Enter the physical interface name in the Bridge ports field
      2. Click Create
Figure 4

After creating a bridge, we would need to create an in-band interface to access the Proxmox VE server. We could have also done this while creating the OVS bridge by filling out the IPv4/CIDR and Gateway (IPv4). However, I wanted to make my config clean for me, so in my case, I am going to create an OVS IntPort for in-band management.

  • Data Center > PVE Node > System > Network
    1. Click on Create
    2. Select OVS IntPort
      1. Enter a name for this inband IntPort in the Name field
      2. Enter the IP address and prefix in the IPv4/CIDR field
      3. Enter the gateway address in the Gateway field
      4. Click on Create
Figure 5

For VLAN tags, create another OVS IntPort for each VLAN ID.

  • Data Center > PVE Node > System > Network
    1. Click on Create
    2. Select OVS IntPort
      1. Enter a name for this IntPort in the Name field
      2. Enter the VLAN ID in the VLAN Tag field
      3. Click on Create
  • Repeat steps 1 through 2.3 for each VLAN
Figure 6

In addition, every OVS IntPort we create will be automatically be added to the OVS Bridge. Once done, reboot the Proxmox server for the network changes to become active. Now, that all the VLAN tags are active, assigning the VM or LXC to specific VLAN can be easily done by just specifying the VLAN ID in the Networking tab in VM or LXC during their creation as shown in Figure 1

When creating a VM or LXC, entering a VLAN ID in the VLAN Tag field the OVS will behave like an access port. Leaving the VLAN Tag field empty, OVS will behave like a trunk port; therefore, the guest VM can be configured VLAN tagging within the VM itself. In addition to leaving the VLAN Tag empty, if the guest VM is not tagging its interface, it would use the untagged or native VLAN.

I hope you will this helpful. Cheers!

Subscribe
guest


19 Comments
Oldest
K.C. Callis
K.C. Callis
1 year ago

This was very helpful because I knew that I needed to switch from Linux Bridges to OVS, but what not ready to make the plunge. Using your tutorial, I make the jump and on paper, everything looks good. I am getting ready to spin up pfSense with all of my VLANs and I am wondering how to trunk from OVS to my Cisco router. I reinstalled PM so that the management interface now has a 10.0.0.0/24, along with my switch with a primary 10.0.0.0/24 address. The reason for this is because on my pfSense box, I am using 192.168.0.0/24 for… Read more »

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

I seem to have an issue with making the inband. Should the IP address and gateway be the same as the host, or should I have used another IP address and gateway??? Every time I try to add an IP address on the VLAN, it gives me a fail with an error saying the address conflicts with the inband. Proxmox Server: 192.168.10.250 gw 192.168.10.254 inband: 192.168.10.250 gw 192.168.10.254 I have not be able to configure these, but it should be like this vlan10: 192.168.10.1 gw 192.168.10.254 vlan20: 192.168.20.1 gw 192.168.20.1 vlan30: 192.168.30.1 gw 192.168.20.1 etc. Now should I be using… Read more »

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

In one fell swoop, you clarified many things for me. I always assumed that when I installed proxmox, I just set the IP address as a host in my management subnet and never gave it any thought. If I am reading your message clearly, I should have set the host IP address to (for instance) 10.0.0.250/24 and under ovs create the inband to 192.168.10.250 (which would fall into my MGMT VLAN once I spin up pfsense. Furthermore, switch the Cisco back to a L3 and also create the VLAN02, which will be attached to port 10 which will connect the… Read more »

KC Callis
KC Callis
Reply to  KC Callis
1 year ago

Actually, re-reading your above answer, I got some clarity.

“On PVE Network | Your L3 switch or router
vlan10: no ip addr| gw 192.168.10.254
vlan20: no ip addr| gw 192.168.20.1
vlan30: no ip addr| gw 192.168.30.1”

Now it makes sense… So then I just setup pfsense with the interfaces and vlans and it will pass to ovs which will pass it on to the Cisco switch.

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

Also, should I have ip routing on or off? switch-01(config)# ip routing Also, I was trying to figure out my interface name to make sure that I used your example above: ” switchport trunk allowed vlan add 20,30,40 switchport trunk native vlan 10 ! interface gigabitethernet10 description WAN switchport mode access switchport access vlan 2 ! exit ” This is my current config: set system mode router file SSD indicator encrypted @ ssd-control-start ssd config ssd file passphrase control unrestricted no ssd file integrity control ssd-control-end cb0a3fdb1f3a1af4e4430033719968c0 ! vlan database vlan 2,10,20,30,40,50,60,70,80 exit voice vlan oui-table add 0001e3 Siemens_AG_phone________ voice… Read more »

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

First off, I am using a Cisco SG-300 switch. It would seem that some of the IOS commands have been removed or have a change in syntax. This as been an ordeal in trying to get this to work. I was working with one of the old Netgate pfSense APU unit which had three interfaces and more or less worked. The WAN interface on the pfSense plugged into the DSL gateway; The LAN interface went to the Cisco, but since I really didn’t make use of a LAN, it was no big bust; And the OPT interface also went into… Read more »

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

I have a couple of network diagrams in pdf format… How to do I get a copy to you?

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago
KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

I am about to give this a shot. There is one issue that I am not clear on. Wouldn’t I see need to create the VLAN 20, 30, and 40 on the switch because of the need to have those VLANs available for the WAP (which I have setup with VLAN10 up and VLAN20, VLAN30 and VLAN40 as tagged?

KC Callis
KC Callis
Reply to  Karlo Abaga
1 year ago

Also, and I am sure that I am kicking a dead horse, but… The host IP address for the proxmox host is 192.168.10.250. Should I set the IntPort for the management port to be 192.168.10.251???

WordPress Appliance - Powered by TurnKey Linux