Skip to main content
Notice removed Draw attention by zwol
Bounty Ended with countermode's answer chosen by zwol
added 334 characters in body
Source Link
zwol
  • 7.5k
  • 2
  • 21
  • 33
  1. Putting the tunnel device directly in the namespace with

     # ip netns add tns0
     # ip link set dev tun0 netns tns0
     # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
    

    These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

  2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which, if true, takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge! [EDIT: This appears to be because only tap devices can be put into bridges. Unlike the inability to put arbitrary devices into a network namespace, that actually makes sense, what with bridges being an Ethernet-layer concept; unfortunately, my VPN provider does not support OpenVPN in tap mode, so I need a workaround.]

     # ip addr add dev tun0 local 0.0.0.0/0 scope link
     # ip link set tun0 up
     # ip link add name teo0 type veth peer name tei0
     # ip link set teo0 up
     # brctl addbr tbr0
     # brctl addif tbr0 teo0
     # brctl addif tbr0 tun0
     can't add tun0 to bridge tbr0: Invalid argument
    
  1. Putting the tunnel device directly in the namespace with

     # ip netns add tns0
     # ip link set dev tun0 netns tns0
     # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
    

    These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

  2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge!

     # ip addr add dev tun0 local 0.0.0.0/0 scope link
     # ip link set tun0 up
     # ip link add name teo0 type veth peer name tei0
     # ip link set teo0 up
     # brctl addbr tbr0
     # brctl addif tbr0 teo0
     # brctl addif tbr0 tun0
     can't add tun0 to bridge tbr0: Invalid argument
    
  1. Putting the tunnel device directly in the namespace with

     # ip netns add tns0
     # ip link set dev tun0 netns tns0
     # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
    

    These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

  2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which, if true, takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge! [EDIT: This appears to be because only tap devices can be put into bridges. Unlike the inability to put arbitrary devices into a network namespace, that actually makes sense, what with bridges being an Ethernet-layer concept; unfortunately, my VPN provider does not support OpenVPN in tap mode, so I need a workaround.]

     # ip addr add dev tun0 local 0.0.0.0/0 scope link
     # ip link set tun0 up
     # ip link add name teo0 type veth peer name tei0
     # ip link set teo0 up
     # brctl addbr tbr0
     # brctl addif tbr0 teo0
     # brctl addif tbr0 tun0
     can't add tun0 to bridge tbr0: Invalid argument
    
clarify modularity of kernel
Source Link
zwol
  • 7.5k
  • 2
  • 21
  • 33

(A redacted version of destination.ovpn is at the end of this question.) However, if I use the scripts I wrote that put the tunnel device in a namespace:

# openvpn --config destination.ovpn --auth-user-pass credentials.txt \
    --ifconfig-noexec --route-noexec --script-security 2 \
    --up netns-up.sh --down netns-down.sh

I'm stuck on the next step, writing scripts that restrict the tunnel device to namespaces. I have tried:

(scripts are also at the end of the question) then connectivity outside the namespace is fine (and does not go through the VPN) but traffic generated inside the namespace (e.g. with ip netns exec vpn0 traceroute -n 8.8.8.8) falls into a black hole. I presume that this means my scripts are incorrect in some way, and I am asking for help debugging them.

  1. Putting the tunnel device directly in the namespace with

     # ip netns add tns0
     # ip link set dev tun0 netns tns0
     # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
    

    These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

  2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge!

     # ip addr add dev tun0 local 0.0.0.0/0 scope link
     # ip link set tun0 up
     # ip link add name teo0 type veth peer name tei0
     # ip link set teo0 up
     # brctl addbr tbr0
     # brctl addif tbr0 teo0
     # brctl addif tbr0 tun0
     can't add tun0 to bridge tbr0: Invalid argument
    
# uname -a
Linux [REDACTED] 3.14.5-x86_64-linode42 #1 SMP Thu Jun 5 15:22:13 EDT 2014 x86_64 GNU/Linux
# openvpn --version
OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014

The scripts at the end of this question are for the veth approach. The scripts for the direct approach may be found in the edit history. Variables in the scripts that appear to be used without setting them first are set in the environment by the openvpn program -- yes, it's sloppy and uses lowercase names.

Further note: The only documentation I can find for network namespaces is a blog post from 2013 which, among other things, claimsPlease offer specific advice on how to get this to work. I'm painfully aware that "you can only assign virtual Ethernet (veth) interfaces to a network namespace", but I can't evenI'm programming by cargo cult here -- has createanyone virtual Ethernet interfaces onwritten comprehensive documentation for this machinestuff? I can't find any -- so general code review of the scripts is also appreciated.

In case it matters:

# ipuname link-srvm
Linux add3.14.5-x86_64-linode42 dev#1 veth0SMP typeThu vethJun peer5 name15:22:13 veth1EDT 2014 x86_64
Error:# argumentopenvpn "veth0"--version is| wrong:head Unknown-1
OpenVPN device2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014
# ip -V
ip utility, iproute2-ss140804
# brctl --version
bridge-utils, 1.5

ContrariwiseThe kernel was built by my virtual hosting provider (Linode) and, assigningalthough compiled with tun0CONFIG_MODULES=y, has no actual modules -- the only CONFIG_* variable set to a namespace doesm according to /proc/config.gz was CONFIG_XEN_TMEM, and I do not actually nothave bomb out;that module (the kernel is stored outside my filesystem; /lib/modules is empty, and /proc/modules indicates that it just then doesn't workwas not magically loaded somehow). Excerpts from /proc/config.gz provided on request, but I don't want to paste the entire thing here.

#! /bin/sh

mask2cidr () {
    local nbits dec
    nbits=0
    for dec in $(echo $1 | sed 's/\./ /g') ; do
        case "$dec" in
            (255) nbits=$(($nbits + 8)) ;;
            (254) nbits=$(($nbits + 7)) ;;
            (252) nbits=$(($nbits + 6)) ;;
            (248) nbits=$(($nbits + 5)) ;;
            (240) nbits=$(($nbits + 4)) ;;
            (224) nbits=$(($nbits + 3)) ;;
            (192) nbits=$(($nbits + 2)) ;;
            (128) nbits=$(($nbits + 1)) ;;
            (0)   ;;
            (*) echo "Error: $dec is not a valid netmask component" >&2
                exit 1
                ;;
        esac
    done
    echo "$nbits"
}

mask2network () {
    local host mask h m result
    host="$1."
    mask="$2."
    result=""
    while [ -n "$host" ]; do
        h="${host%%.*}"
        m="${mask%%.*}"
        host="${host#*.}"
        mask="${mask#*.}"
        result="$result.$(($h & $m))"
    done
    echo "${result#.}"
}

maybe_config_dns () {
    local n option servers
    n=1
    servers=""
    while [ $n -lt 100 ]; do
       eval option="\$foreign_option_$n"
       [ -n "$option" ] || break
       case "$option" in
           (*DNS*)
               set -- $option
               servers="$servers
nameserver $3"
               ;;
           (*) ;;
       esac
       n=$(($n + 1))
    done
    if [ -n "$servers" ]; then
        cat > /etc/netns/$tun_netns/resolv.conf <<EOF
# name servers for $tun_netns
$servers
EOF
    fi
}

config_inside_netns () {
    local ifconfig_cidr ifconfig_network

    ifconfig_cidr=$(mask2cidr $ifconfig_netmask)
    ifconfig_network=$(mask2network $ifconfig_local $ifconfig_netmask)

    ip link set dev lo up
    ip link set dev $dev mtu $tun_mtu
    ip link set dev $dev up

    ip addr add dev $dev$tun_vethI \
        local $ifconfig_local/$ifconfig_cidr \
        broadcast $ifconfig_broadcast \
        scope link
 
    #ipip route add $ifconfig_network/$ifconfig_cidrdefault via $route_vpn_gateway dev $dev$tun_vethI
    ip routelink addset defaultdev via$tun_vethI $route_vpn_gatewaymtu dev$tun_mtu $devup
}

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=vpn$# For no good reason, we can't just put the tunnel device in the
# subsidiary namespace; we have to create a "virtual Ethernet"
# device pair, put one of its ends in the subsidiary namespace,
# and put the other end in a "bridge" with the tunnel device.

tun_tundv=$dev
tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun}
tun_vethI=tei${dev#tun}
tun_vethO=teo${dev#tun}

case "$tun_netns" in
     (vpn[0tns[0-9] | vpn[0tns[0-9][0-9] | vpn[0tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

if [ $# -eq 1 ] && [ $1 = "INSIDE_NETNS" ]; then
    [ $(ip netns identify $$) = $tun_netns ] || exit 1
    config_inside_netns
else

    trap "rm -rf /etc/netns/$tun_netns ||:;
          ip netns del $tun_netns      ||:
          ip link del $tun_vethO       ||:
          ip link set $tun_tundv down  ||:
          brctl delbr $tun_bridg       ||:
         " 0 

    mkdir /etc/netns/$tun_netns
    maybe_config_dns

    ip addr add dev $tun_tundv local 0.0.0.0/0 scope link
    ip link set $tun_tundv mtu $tun_mtu up

    ip link add name $tun_vethO type veth peer name $tun_vethI
    ip link set $tun_vethO mtu $tun_mtu up

    brctl addbr $tun_bridg
    brctl setfd $tun_bridg 0
    #brctl sethello $tun_bridg 0
    brctl stp $tun_bridg off

    brctl addif $tun_bridg $tun_vethO
    brctl addif $tun_bridg $tun_tundv
    ip link set $tun_bridg up

    ip netns add $tun_netns
    ip link set dev $dev$tun_vethI netns $tun_netns
 
    ip netns exec $tun_netns $0 INSIDE_NETNS

    trap "" 0
fi
#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=vpn$tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun} 

case "$tun_netns" in
     (vpn[0tns[0-9] | vpn[0tns[0-9][0-9] | vpn[0tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

[ -d /etc/netns/$tun_netns ] || exit 1

pids=$(ip netns pids $tun_netns)
if [ -n "$pids" ]; then
    kill $pids
    sleep 5
    pids=$(ip netns pids $tun_netns)
    if [ -n "$pids" ]; then
        kill -9 $pids
    fi
fi

# this automatically cleans up the the routes and devicethe asveth welldevice pair
ip netns delete "$tun_netns"
 
rm -rf /etc/netns/$tun_netns

# the bridge and the tunnel device must be torn down separately
ip link set $dev down
brctl delbr $tun_bridg

(A redacted version of destination.ovpn is at the end of this question.) However, if I use the scripts I wrote that put the tunnel device in a namespace:

# openvpn --config destination.ovpn --auth-user-pass credentials.txt \
    --ifconfig-noexec --route-noexec --script-security 2 \
    --up netns-up.sh --down netns-down.sh

(scripts are also at the end of the question) then connectivity outside the namespace is fine (and does not go through the VPN) but traffic generated inside the namespace (e.g. with ip netns exec vpn0 traceroute -n 8.8.8.8) falls into a black hole. I presume that this means my scripts are incorrect in some way, and I am asking for help debugging them.

# uname -a
Linux [REDACTED] 3.14.5-x86_64-linode42 #1 SMP Thu Jun 5 15:22:13 EDT 2014 x86_64 GNU/Linux
# openvpn --version
OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014

Further note: The only documentation I can find for network namespaces is a blog post from 2013 which, among other things, claims that "you can only assign virtual Ethernet (veth) interfaces to a network namespace", but I can't even create virtual Ethernet interfaces on this machine:

# ip link add dev veth0 type veth peer name veth1
Error: argument "veth0" is wrong: Unknown device

Contrariwise, assigning tun0 to a namespace does not bomb out; it just then doesn't work.

#! /bin/sh

mask2cidr () {
    local nbits dec
    nbits=0
    for dec in $(echo $1 | sed 's/\./ /g') ; do
        case "$dec" in
            (255) nbits=$(($nbits + 8)) ;;
            (254) nbits=$(($nbits + 7)) ;;
            (252) nbits=$(($nbits + 6)) ;;
            (248) nbits=$(($nbits + 5)) ;;
            (240) nbits=$(($nbits + 4)) ;;
            (224) nbits=$(($nbits + 3)) ;;
            (192) nbits=$(($nbits + 2)) ;;
            (128) nbits=$(($nbits + 1)) ;;
            (0)   ;;
            (*) echo "Error: $dec is not a valid netmask component" >&2
                exit 1
                ;;
        esac
    done
    echo "$nbits"
}

mask2network () {
    local host mask h m result
    host="$1."
    mask="$2."
    result=""
    while [ -n "$host" ]; do
        h="${host%%.*}"
        m="${mask%%.*}"
        host="${host#*.}"
        mask="${mask#*.}"
        result="$result.$(($h & $m))"
    done
    echo "${result#.}"
}

maybe_config_dns () {
    local n option servers
    n=1
    servers=""
    while [ $n -lt 100 ]; do
       eval option="\$foreign_option_$n"
       [ -n "$option" ] || break
       case "$option" in
           (*DNS*)
               set -- $option
               servers="$servers
nameserver $3"
               ;;
           (*) ;;
       esac
       n=$(($n + 1))
    done
    if [ -n "$servers" ]; then
        cat > /etc/netns/$tun_netns/resolv.conf <<EOF
# name servers for $tun_netns
$servers
EOF
    fi
}

config_inside_netns () {
    local ifconfig_cidr ifconfig_network

    ifconfig_cidr=$(mask2cidr $ifconfig_netmask)
    ifconfig_network=$(mask2network $ifconfig_local $ifconfig_netmask)

    ip link set dev lo up
    ip link set dev $dev mtu $tun_mtu
    ip link set dev $dev up

    ip addr add dev $dev \
        local $ifconfig_local/$ifconfig_cidr \
        broadcast $ifconfig_broadcast \
        scope link
 
    #ip route add $ifconfig_network/$ifconfig_cidr dev $dev
    ip route add default via $route_vpn_gateway dev $dev
}

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=vpn${dev#tun}
case "$tun_netns" in
     (vpn[0-9] | vpn[0-9][0-9] | vpn[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

if [ $# -eq 1 ] && [ $1 = "INSIDE_NETNS" ]; then
    [ $(ip netns identify $$) = $tun_netns ] || exit 1
    config_inside_netns
else

    trap "rm -rf /etc/netns/$tun_netns ||:; ip netns del $tun_netns ||:" 0
    mkdir /etc/netns/$tun_netns
    maybe_config_dns

    ip netns add $tun_netns
    ip link set dev $dev netns $tun_netns
 
    ip netns exec $tun_netns $0 INSIDE_NETNS

    trap "" 0
fi
#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=vpn${dev#tun}
case "$tun_netns" in
     (vpn[0-9] | vpn[0-9][0-9] | vpn[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

[ -d /etc/netns/$tun_netns ] || exit 1

pids=$(ip netns pids $tun_netns)
if [ -n "$pids" ]; then
    kill $pids
    sleep 5
    pids=$(ip netns pids $tun_netns)
    if [ -n "$pids" ]; then
        kill -9 $pids
    fi
fi

# this automatically cleans up the the routes and device as well
ip netns delete "$tun_netns"
 
rm -rf /etc/netns/$tun_netns

(A redacted version of destination.ovpn is at the end of this question.)

I'm stuck on the next step, writing scripts that restrict the tunnel device to namespaces. I have tried:

  1. Putting the tunnel device directly in the namespace with

     # ip netns add tns0
     # ip link set dev tun0 netns tns0
     # ip netns exec tns0 ( ... commands to bring up tun0 as usual ... )
    

    These commands execute successfully, but traffic generated inside the namespace (e.g. with ip netns exec tns0 traceroute -n 8.8.8.8) falls into a black hole.

  2. On the assumption that "you can [still] only assign virtual Ethernet (veth) interfaces to a network namespace" (which takes this year's award for most ridiculously unnecessary API restriction), creating a veth pair and a bridge, and putting one end of the veth pair in the namespace. This doesn't even get as far as dropping traffic on the floor: it won't let me put the tunnel into the bridge!

     # ip addr add dev tun0 local 0.0.0.0/0 scope link
     # ip link set tun0 up
     # ip link add name teo0 type veth peer name tei0
     # ip link set teo0 up
     # brctl addbr tbr0
     # brctl addif tbr0 teo0
     # brctl addif tbr0 tun0
     can't add tun0 to bridge tbr0: Invalid argument
    

The scripts at the end of this question are for the veth approach. The scripts for the direct approach may be found in the edit history. Variables in the scripts that appear to be used without setting them first are set in the environment by the openvpn program -- yes, it's sloppy and uses lowercase names.

Please offer specific advice on how to get this to work. I'm painfully aware that I'm programming by cargo cult here -- has anyone written comprehensive documentation for this stuff? I can't find any -- so general code review of the scripts is also appreciated.

In case it matters:

# uname -srvm
Linux 3.14.5-x86_64-linode42 #1 SMP Thu Jun 5 15:22:13 EDT 2014 x86_64
# openvpn --version | head -1
OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014
# ip -V
ip utility, iproute2-ss140804
# brctl --version
bridge-utils, 1.5

The kernel was built by my virtual hosting provider (Linode) and, although compiled with CONFIG_MODULES=y, has no actual modules -- the only CONFIG_* variable set to m according to /proc/config.gz was CONFIG_XEN_TMEM, and I do not actually have that module (the kernel is stored outside my filesystem; /lib/modules is empty, and /proc/modules indicates that it was not magically loaded somehow). Excerpts from /proc/config.gz provided on request, but I don't want to paste the entire thing here.

#! /bin/sh

mask2cidr () {
    local nbits dec
    nbits=0
    for dec in $(echo $1 | sed 's/\./ /g') ; do
        case "$dec" in
            (255) nbits=$(($nbits + 8)) ;;
            (254) nbits=$(($nbits + 7)) ;;
            (252) nbits=$(($nbits + 6)) ;;
            (248) nbits=$(($nbits + 5)) ;;
            (240) nbits=$(($nbits + 4)) ;;
            (224) nbits=$(($nbits + 3)) ;;
            (192) nbits=$(($nbits + 2)) ;;
            (128) nbits=$(($nbits + 1)) ;;
            (0)   ;;
            (*) echo "Error: $dec is not a valid netmask component" >&2
                exit 1
                ;;
        esac
    done
    echo "$nbits"
}

mask2network () {
    local host mask h m result
    host="$1."
    mask="$2."
    result=""
    while [ -n "$host" ]; do
        h="${host%%.*}"
        m="${mask%%.*}"
        host="${host#*.}"
        mask="${mask#*.}"
        result="$result.$(($h & $m))"
    done
    echo "${result#.}"
}

maybe_config_dns () {
    local n option servers
    n=1
    servers=""
    while [ $n -lt 100 ]; do
       eval option="\$foreign_option_$n"
       [ -n "$option" ] || break
       case "$option" in
           (*DNS*)
               set -- $option
               servers="$servers
nameserver $3"
               ;;
           (*) ;;
       esac
       n=$(($n + 1))
    done
    if [ -n "$servers" ]; then
        cat > /etc/netns/$tun_netns/resolv.conf <<EOF
# name servers for $tun_netns
$servers
EOF
    fi
}

config_inside_netns () {
    local ifconfig_cidr ifconfig_network

    ifconfig_cidr=$(mask2cidr $ifconfig_netmask)
    ifconfig_network=$(mask2network $ifconfig_local $ifconfig_netmask)

    ip link set dev lo up

    ip addr add dev $tun_vethI \
        local $ifconfig_local/$ifconfig_cidr \
        broadcast $ifconfig_broadcast \
        scope link
    ip route add default via $route_vpn_gateway dev $tun_vethI
    ip link set dev $tun_vethI mtu $tun_mtu up
}

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

# For no good reason, we can't just put the tunnel device in the
# subsidiary namespace; we have to create a "virtual Ethernet"
# device pair, put one of its ends in the subsidiary namespace,
# and put the other end in a "bridge" with the tunnel device.

tun_tundv=$dev
tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun}
tun_vethI=tei${dev#tun}
tun_vethO=teo${dev#tun}

case "$tun_netns" in
     (tns[0-9] | tns[0-9][0-9] | tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

if [ $# -eq 1 ] && [ $1 = "INSIDE_NETNS" ]; then
    [ $(ip netns identify $$) = $tun_netns ] || exit 1
    config_inside_netns
else

    trap "rm -rf /etc/netns/$tun_netns ||:
          ip netns del $tun_netns      ||:
          ip link del $tun_vethO       ||:
          ip link set $tun_tundv down  ||:
          brctl delbr $tun_bridg       ||:
         " 0 

    mkdir /etc/netns/$tun_netns
    maybe_config_dns

    ip addr add dev $tun_tundv local 0.0.0.0/0 scope link
    ip link set $tun_tundv mtu $tun_mtu up

    ip link add name $tun_vethO type veth peer name $tun_vethI
    ip link set $tun_vethO mtu $tun_mtu up

    brctl addbr $tun_bridg
    brctl setfd $tun_bridg 0
    #brctl sethello $tun_bridg 0
    brctl stp $tun_bridg off

    brctl addif $tun_bridg $tun_vethO
    brctl addif $tun_bridg $tun_tundv
    ip link set $tun_bridg up

    ip netns add $tun_netns
    ip link set dev $tun_vethI netns $tun_netns
    ip netns exec $tun_netns $0 INSIDE_NETNS

    trap "" 0
fi
#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=tns${dev#tun}
tun_bridg=tbr${dev#tun} 

case "$tun_netns" in
     (tns[0-9] | tns[0-9][0-9] | tns[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

[ -d /etc/netns/$tun_netns ] || exit 1

pids=$(ip netns pids $tun_netns)
if [ -n "$pids" ]; then
    kill $pids
    sleep 5
    pids=$(ip netns pids $tun_netns)
    if [ -n "$pids" ]; then
        kill -9 $pids
    fi
fi

# this automatically cleans up the the routes and the veth device pair
ip netns delete "$tun_netns"
rm -rf /etc/netns/$tun_netns

# the bridge and the tunnel device must be torn down separately
ip link set $dev down
brctl delbr $tun_bridg
Tweeted twitter.com/#!/StackUnix/status/498912285877100544
Notice added Draw attention by zwol
Bounty Started worth 200 reputation by zwol
Source Link
zwol
  • 7.5k
  • 2
  • 21
  • 33

Feed all traffic through OpenVPN for a specific network namespace only

I am trying to set up a VPN (using OpenVPN) such that all of the traffic, and only the traffic, to/from specific processes goes through the VPN; other processes should continue to use the physical device directly. It is my understanding that the way to do this in Linux is with network namespaces.

If I use OpenVPN normally (i.e. funnelling all traffic from the client through the VPN), it works fine. Specifically, I start OpenVPN like this:

# openvpn --config destination.ovpn --auth-user-pass credentials.txt

(A redacted version of destination.ovpn is at the end of this question.) However, if I use the scripts I wrote that put the tunnel device in a namespace:

# openvpn --config destination.ovpn --auth-user-pass credentials.txt \
    --ifconfig-noexec --route-noexec --script-security 2 \
    --up netns-up.sh --down netns-down.sh

(scripts are also at the end of the question) then connectivity outside the namespace is fine (and does not go through the VPN) but traffic generated inside the namespace (e.g. with ip netns exec vpn0 traceroute -n 8.8.8.8) falls into a black hole. I presume that this means my scripts are incorrect in some way, and I am asking for help debugging them.

# uname -a
Linux [REDACTED] 3.14.5-x86_64-linode42 #1 SMP Thu Jun 5 15:22:13 EDT 2014 x86_64 GNU/Linux
# openvpn --version
OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Mar 17 2014

Further note: The only documentation I can find for network namespaces is a blog post from 2013 which, among other things, claims that "you can only assign virtual Ethernet (veth) interfaces to a network namespace", but I can't even create virtual Ethernet interfaces on this machine:

# ip link add dev veth0 type veth peer name veth1
Error: argument "veth0" is wrong: Unknown device

Contrariwise, assigning tun0 to a namespace does not bomb out; it just then doesn't work.

netns-up.sh

#! /bin/sh

mask2cidr () {
    local nbits dec
    nbits=0
    for dec in $(echo $1 | sed 's/\./ /g') ; do
        case "$dec" in
            (255) nbits=$(($nbits + 8)) ;;
            (254) nbits=$(($nbits + 7)) ;;
            (252) nbits=$(($nbits + 6)) ;;
            (248) nbits=$(($nbits + 5)) ;;
            (240) nbits=$(($nbits + 4)) ;;
            (224) nbits=$(($nbits + 3)) ;;
            (192) nbits=$(($nbits + 2)) ;;
            (128) nbits=$(($nbits + 1)) ;;
            (0)   ;;
            (*) echo "Error: $dec is not a valid netmask component" >&2
                exit 1
                ;;
        esac
    done
    echo "$nbits"
}

mask2network () {
    local host mask h m result
    host="$1."
    mask="$2."
    result=""
    while [ -n "$host" ]; do
        h="${host%%.*}"
        m="${mask%%.*}"
        host="${host#*.}"
        mask="${mask#*.}"
        result="$result.$(($h & $m))"
    done
    echo "${result#.}"
}

maybe_config_dns () {
    local n option servers
    n=1
    servers=""
    while [ $n -lt 100 ]; do
       eval option="\$foreign_option_$n"
       [ -n "$option" ] || break
       case "$option" in
           (*DNS*)
               set -- $option
               servers="$servers
nameserver $3"
               ;;
           (*) ;;
       esac
       n=$(($n + 1))
    done
    if [ -n "$servers" ]; then
        cat > /etc/netns/$tun_netns/resolv.conf <<EOF
# name servers for $tun_netns
$servers
EOF
    fi
}

config_inside_netns () {
    local ifconfig_cidr ifconfig_network

    ifconfig_cidr=$(mask2cidr $ifconfig_netmask)
    ifconfig_network=$(mask2network $ifconfig_local $ifconfig_netmask)

    ip link set dev lo up
    ip link set dev $dev mtu $tun_mtu
    ip link set dev $dev up

    ip addr add dev $dev \
        local $ifconfig_local/$ifconfig_cidr \
        broadcast $ifconfig_broadcast \
        scope link

    #ip route add $ifconfig_network/$ifconfig_cidr dev $dev
    ip route add default via $route_vpn_gateway dev $dev
}

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=vpn${dev#tun}
case "$tun_netns" in
     (vpn[0-9] | vpn[0-9][0-9] | vpn[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

if [ $# -eq 1 ] && [ $1 = "INSIDE_NETNS" ]; then
    [ $(ip netns identify $$) = $tun_netns ] || exit 1
    config_inside_netns
else

    trap "rm -rf /etc/netns/$tun_netns ||:; ip netns del $tun_netns ||:" 0
    mkdir /etc/netns/$tun_netns
    maybe_config_dns

    ip netns add $tun_netns
    ip link set dev $dev netns $tun_netns

    ip netns exec $tun_netns $0 INSIDE_NETNS

    trap "" 0
fi

netns-down.sh

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH

set -ex

tun_netns=vpn${dev#tun}
case "$tun_netns" in
     (vpn[0-9] | vpn[0-9][0-9] | vpn[0-9][0-9][0-9]) ;;
     (*) exit 1;;
esac

[ -d /etc/netns/$tun_netns ] || exit 1

pids=$(ip netns pids $tun_netns)
if [ -n "$pids" ]; then
    kill $pids
    sleep 5
    pids=$(ip netns pids $tun_netns)
    if [ -n "$pids" ]; then
        kill -9 $pids
    fi
fi

# this automatically cleans up the the routes and device as well
ip netns delete "$tun_netns"

rm -rf /etc/netns/$tun_netns

destination.ovpn

client
auth-user-pass
ping 5
dev tun
resolv-retry infinite
nobind
persist-key
persist-tun
ns-cert-type server
verb 3
route-metric 1
proto tcp
ping-exit 90
remote [REDACTED]
<ca>
[REDACTED]
</ca>
<cert>
[REDACTED]
</cert>
<key>
[REDACTED]
</key>