3

I was playing around with vagrant-libvirt and wanted to bridge my virtual machines to my LAN so that they get a DHCP lease and are reachable from the network. Since vagrant-libvirt does apparently not create bridges for you, I created one:

brctl addbr br0

and added enp10s0 to it:

brctl addif br0 enp10s0

and learned an important lesson in linux networking:

Note: Adding an interface to a bridge will cause the interface to lose its existing IP address. If you're connected remotely via the interface you intend to add to the bridge, you will lose your connection. This problem can be worked around by scripting the bridge to be created at system startup.

I can understand that it loses its address because the kernel might be reinitializing the interface, but why does it not recover? There is a dhcp server on enp10s0's network, so I naively expected it would come back (I am used to work with Hyper-V's virtual switches, where you can create and delete them without cutting your connection to the host).

Is it because I assigned no ip address to the bridge before attachking enp10s0? Or because the bridge is not up yet? Or both?

2
  • 1
    under the hood: once an interface becomes a bridge port, it's barred from participating in layer 3 (routing). Only the implicit self port of the bridge (ie here br0) can now do routing. If I had actual references from where I know this I'd post it as an answer, but I don't. Commented Sep 29, 2019 at 13:15
  • Thanks for shedding some light! So if I would have upped the bridge (e.g. ip link set dev br0 up) before attaching enp10s0 it would have worked? I'd love to upvote your comment, but I don't have enough rep yet. Commented Sep 30, 2019 at 9:21

2 Answers 2

5

I don't know why bridge-utils has this known limitation that is biting you. Although some may prefer the familiar brctl tool, at least in your case I'd suggest trying the newer iproute2 tools documented in the same Arch wiki page you linked to. In your case:

ip link add name br0 type bridge
ip link set br0 up
ip link set enp10s0 up
ip link set enp10s0 master br0

I don't have deeper knowledge, but believe at least some of the iproute2 tools use newer kernel interfaces than older tools like bridge-utils. If the above works for you where brctl fails, I'd suspect that is the reason.

3
  • No, just my incompetence - but I'd like to understand what is going on under the hood. Commented Sep 28, 2019 at 9:00
  • @Benni: I wouldn't call it incompetence to choose one set of tools over another. AFAIK many people prefer the more traditional brctl tools to the sometimes-confusing ip tools' syntax. The ip tools are newer and I think at least some of them make use of newer kernel APIs to do their work. I don't have insight deeper than that as to why using brctl results in the problem you encountered. Commented Sep 28, 2019 at 10:24
  • 2
    Using iproute2 also seems to release the IP of the underlying interface. Commented May 27, 2020 at 15:56
4

TLDR: Think of a bridge interface as an Ethernet Switch. All your [virtual] hosts are attached to the switch, and the physical port is the uplink to another switch. It would not make sense to try to plug your computer and the uplink cable into the same port on a switch, and that is why you cannot keep using that physical port for the host machine.


When you add a bridge to the system, you change the nature of the physical port. Normally, the physical port has a 1:1 correspondence between its MAC address and its IP address. This is what the ARP protocol is expecting - so that it can determine which Ethernet address to send a packet to in order to reach a certain host. This is what is usually called a host interface.

When you add a bridge you want to change all that. Typically you intend to have more than one host on that computer (presumably virtual machines or containers) and ARP will need a separate MAC address for each real or virtual machine. So you create a bridge, and it captures the physical port[s]. Then when you add virtual machines, they add MAC addresses for themselves to that bridge, creating virtual ports, and then Ethernet networking works. The bridge also handles the Ethernet switching among the internal virtual ports/machines.

Allow a digression... You could in theory also assign multiple MAC addresses to a single host port, but this causes problems if you want the two machines at the same physical port to be able to talk to one another. So this is rarely done. Similarly, you could add more than one physical port to a bridge and then you could allow it to switch between ports or to combine them into link aggregation groups - which are more than one link (cable) used in concert as a single virtual link to gain capacity or reliability.

Finally, you can accomplish what you want to do by adding your host to the bridge you created. The following is an example from a server's /etc/netplan/01-netcfg.yaml file that sets up a bridge when the server boots up both for virtual machines to attach to as well as the host itself.

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
  ethernets:
    enp9s0: {}
  wifis: {}
  bridges:
    br0:
      dhcp4: true
      interfaces:
      - enp9s0

This will set up a bridge associated with enp9s0 and it will also add a MAC address to that bridge for the host to use with IPv4 (and IPv6 SLAAC). When this netplan is instantiated, you should see something like the following:

user@server:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
    link/ether 70:cd:60:aa:1b:29 brd ff:ff:ff:ff:ff:ff
3: enp10s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 70:cd:60:aa:4e:cf brd ff:ff:ff:ff:ff:ff
4: wls5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 04:0c:ce:1d:61:07 brd ff:ff:ff:ff:ff:ff
    altname wlp13s0
16: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether d6:34:90:36:5e:c2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.95/24 brd 192.168.1.255 scope global dynamic noprefixroute br0
       valid_lft 82942sec preferred_lft 82942sec
    inet6 1024:2048:4096:6530:f468:8db5:f14c:fa4d/64 scope global temporary dynamic 
       valid_lft 3439sec preferred_lft 3439sec
    inet6 1024:2048:4096:6530:d434:90ff:fe36:5ec2/64 scope global dynamic mngtmpaddr 
       valid_lft 3439sec preferred_lft 3439sec
    inet6 fe80::d434:90ff:fe36:5ec2/64 scope link 
       valid_lft forever preferred_lft forever
17: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:39:55:a0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe39:55a0/64 scope link 
       valid_lft forever preferred_lft forever

This output shows the physical bridge interface (interface 2, enp9s0) as well as the bridge itself (interface 16, br0). In this set-up the host has IP connectivity with address 192.168.1.95, and a qemu-kvm VM attaches to the bridge with (interface 17, vnet2). The VM has the MAC address ending in 55:a0, and the linux server has no idea what IP address the VM has leased on that interface unless it does an ARP to learn it. In this arrangement, the host and VM can connect to the LAN and to each other. Additional VMs with bridged interfaces would look similar to interface 17, vnet2, and typically qemu-kvm will create the virtual interface as needed when a VM is started.

You could get fancier by adding the other unused Ethernet port (enp10s0), or wlan port (wls5) to the bridge as well, but this shows the basics and is actually what I use on my host machine.

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.