libvirt kvm virtual routed network: cannot ping gateway itself or beyond

Question

I'm having trouble with libvirt kvm's routed networks where a VM inside a routed virtual network can ping every VM in my home subnet except the default gateway... or any gateway for that matter.

I believe my situation is not a duplicate of libvirt KVM virtualhost cannot ping router address as I am able to ping machines in the host's subnet (unlike in the aforementioned post where the author says no other hosts in the 192.168.2.0/24 network can be pinged)

The environment

my home subnet is 192.168.1.0/24
- The default gateway is your standard consumer router, located at 192.168.1.1. Its static route table is as follows:
  - dst: 192.168.100.0 - gw: 192.168.1.100 - genmask: 255.255.255.0
  - dst: 172.16.0.0 - gw: 192.168.1.4 - genmask: 255.240.0.0
- My machine (server-01, runs EndeavourOS) has the following IP address 192.168.1.100
I am running libvirt kvm, I have a virtual routed network for my VMs. As such server-01 is a router for the 192.168.100.0/24 subnet.
- On that subnet there is a Debian 12 VM vm-alpha (192.168.100.200)
A test subnet 172.20.200.0/24 exists (default gw 172.20.200.1). It can be reached from the home subnet via 192.168.1.4 (OPNsense)
- There's a lonely Debian 12 VM vm-beta whose IP is 172.20.200.2

The problem

I am unable to ping the default gw from vm-alpha. Hence vm-alpha has no "Internet connection".

Below is a table outlining the pings I have attempted, with empty cells representing those that I have not tried

from => to ↓	vm-alpha (192.168.100.200)	server-01 home net IP (192.168.1.100)	vm-beta (172.20.200.2)
vm-alpha (192.168.100.200)		YES	YES
server-01 gw (192.168.100.1)	YES	YES	YES
server-01 home net IP (192.168.1.100)	YES		YES
OPNsense home net (192.168.1.4)	YES	YES	YES
default gw home net (192.168.1.1)	no response	YES	YES
OPNsense other side (172.20.200.1)	no response	YES	YES
vm-beta (172.20.200.2)	no response	YES
Cloudflare (1.1.1.1)	no response	YES	YES

vm-alpha (192.168.100.200) to home net gateway (192.168.1.1) - no response
- packets are simply lost, there is no "destination host unreachable" or "network is unreachable" errors that I could see with wireshark running on server-01, listening on interface virbr1.

mtr

mtr from 192.168.100.200 to 172.20.200.2

                        My traceroute  [v0.95]
vm-alpha (192.168.100.200) -> 172.20.200.2025-03-30T00:00:24+0100
Keys:  Help   Display mode   Restart statistics   Order of fields   qui
t                             Packets               Pings
 Host                       Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.100.1            0.0%     7    0.2   0.2   0.1   0.3   0.1
 2. (waiting for reply)

^ when that mtr was done, the routing table of the host (server-01) was as follows:

Kernel IP routing table  
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface  
0.0.0.0         192.168.1.1     0.0.0.0         UG    100    0        0 enp9s0  
192.168.1.0     0.0.0.0         255.255.255.0   U     100    0        0 enp9s0  
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr1

mtr from 172.20.200.2 to 192.168.100.200

                     My traceroute  [v0.95]
vm-beta (172.20.200.2) -> 12025-03-29T22:59:07+0000
Keys:  Help   Display mode   Restart statistics   Order of field
s   quit               Packets               Pings
 Host                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 172.20.200.1      0.0%    16    0.5   0.6   0.3   1.1   0.2
 2. 192.168.1.1       0.0%    15    1.3   1.1   0.9   1.4   0.1
 3. 192.168.1.100     0.0%    15    1.1   1.2   0.7   1.9   0.3
 4. 192.168.100.200   0.0%    15    1.3   1.3   0.8   1.7   0.2

route -n for 172.20.200.2:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.20.200.1    0.0.0.0         UG    0      0        0 eth0
172.20.200.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0

What I've tried

disabling firewalld which was running - the problem remains and the results from the ping table above still stand.
changing the routes on server-01 - I thought maybe the 192.168.1.1 gw was the problem so I removed it completely. I manually set the route for the 172.20.200.0/24 subnet.

With the routing table of server-01 now being:

Kernel IP routing table  
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface  
172.16.0.0      192.168.1.4     255.240.0.0     UG    0      0        0 enp9s0  
192.168.1.0     0.0.0.0         255.255.255.0   U     100    0        0 enp9s0  
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr1

server-01 is able to ping 172.20.200.2. However, vm-alpha still can't. mtr shows the new hop:

                        My traceroute  [v0.95]
vm-alpha (192.168.100.200) -> 172.20.200.2025-03-30T00:15:08+0100
Keys:  Help   Display mode   Restart statistics   Order of fields   qui
t                             Packets               Pings
 Host                       Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.100.1            0.0%     4    0.1   0.2   0.1   0.2   0.1
 2. 192.168.1.4              0.0%     4    0.7   0.7   0.5   0.9   0.1
 3. (waiting for reply)

Other info

iptables -L:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
FWD-vm-alpha  all  --  anywhere             192.168.100.200     
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FWD-vm-alpha (1 references)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             192.168.100.200      tcp dpts:tcpmux:65535
ACCEPT     udp  --  anywhere             192.168.100.200      udp dpts:tcpmux:65535

cat /proc/sys/vm/ipv4/ip_forward returns 1.

ip -c a on server-01:

(...)
2: enp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 10:ff:e0:3d:ab:c5 brd ff:ff:ff:ff:ff:ff
    altname enx10ffe03dabc5
    inet 192.168.1.100/24 brd 192.168.1.255 scope global dynamic noprefixroute enp9s0
       valid_lft 83966sec preferred_lft 83966sec
    inet6 fe80::6f46:313c:4159:fde4/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
(...)
11: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000
    link/ether 52:54:00:0d:2c:0d brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.1/24 brd 192.168.100.255 scope global virbr1
       valid_lft forever preferred_lft forever

firewalld runs on the system. Some config info:

sudo firewall-cmd --get-active-zones 

libvirt
  interfaces: virbr0
libvirt-routed
  interfaces: virbr1
public (default)
  interfaces: enp9s0

sudo firewall-cmd --list-all --zone=libvirt-routed

libvirt-routed (active)
  target: default
  ingress-priority: 0
  egress-priority: 0
  icmp-block-inversion: no
  interfaces: virbr1
  sources: 
  services: 
  ports: 
  protocols: 
  forward: no
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

Virtual network configuration:

<network connections="1">
  <name>routed-network-100</name>
  <uuid>a4b18fac-87e8-4f6c-aebd-04ca31b8c7f7</uuid>
  <forward dev="enp9s0" mode="route">
    <interface dev="enp9s0"/>
  </forward>
  <bridge name="virbr1" stp="on" delay="0"/>
  <mac address="52:54:00:0d:2c:0d"/>
  <domain name="routed-network-100"/>
  <ip address="192.168.100.1" netmask="255.255.255.0"/>
</network>

Check if the router’s firewall is preventing this setup from working as intended. For the router, requests from vm-alpha (192.168.100.200/24) may appear as requests outside its network (192.168.1.0/24). — Man made of meat
– Man made of meat, Commented Mar 31 at 14:48

metablaster · Accepted Answer · 2025-03-30 01:48:49Z

1

On your Server-01:

# This may result in error for disconnected NIC
ip route add default 192.168.1.0/24 via 192.168.1.1

# When adding route for disconnected but configured interface
ip route add default via 192.168.1.1 dev virbr1 onlink

answered Mar 30 at 1:48

metablaster

8164 silver badges18 bronze badges

Add a comment |

Stack Exchange Network

libvirt kvm virtual routed network: cannot ping gateway itself or beyond

The environment

The problem

mtr

What I've tried

Other info

1 Answer 1

You must log in to answer this question.

Linked

Hot Network Questions

libvirt kvm virtual routed network: cannot ping gateway itself or beyond

The environment

The problem

mtr

What I've tried

Other info

1 Answer 1

You must log in to answer this question.

Linked

Related

Hot Network Questions