I'm running a Kubernetes cluster with RKE2 v1.30.5+rke2r1 on Linux nixos 6.6.56 amd64, using Cilium CNI.
Here's the setup:
I have two pods (yaml manifests at the bottom):
Pod A (xfrm-pod) is running in the default network namespace.
Pod B (charon-pod) is running in the host network namespace (hostNetwork: true).
On Pod A, I check the inode of its network namespace using:
readlink /proc/$$/ns/net
This gives the expected value, e.g., net:[4026532702]
.
Then i mount /var/run/netns
on pod B e.g. to /netns
and run ls -li /netns
, the inode for Pod A's network namespace is a strange value, like 53587.
Permission show this is the only file there is write access to. (I can delete it)
However, when I ls -li /var/run/netns
directly on the host, the inode and file name are what I expect: the correct namespace symlink and inode number.
Why is the inode different inside the host-network pod? And why does it appear writable, unlike other netns files?
Any idea why this happens, and how I can get consistent behavior inside host network pods?
Pod yaml manifests (fetched with kubectl get pod -o yaml since i create them in a controller in go): Pod A:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2025-04-24T14:57:55Z"
name: xfrm-pod
namespace: ims
resourceVersion: "7200524"
uid: dd08aa88-460f-4bdd-8019-82a433682825
spec:
containers:
- command:
- bash
- -c
- while true; do sleep 1000; done
image: ubuntu:latest
imagePullPolicy: Always
name: xfrm-container
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /netns
name: netns-dir
readOnly: true
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-cszxx
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: nixos
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
sysctls:
- name: net.ipv4.ip_forward
value: "1"
- name: net.ipv4.conf.all.rp_filter
value: "0"
- name: net.ipv4.conf.default.rp_filter
value: "0"
- name: net.ipv4.conf.all.arp_filter
value: "1"
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- hostPath:
path: /var/run/netns/
type: Directory
name: netns-dir
- name: kube-api-access-cszxx
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
Pod B:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2025-04-24T14:57:45Z"
labels:
ipserviced: "true"
name: charon-pod
namespace: ims
resourceVersion: "7200483"
uid: 1c5542ba-16c8-4105-9556-7519ea50edef
spec:
containers:
- image: someimagewithstrongswan
imagePullPolicy: IfNotPresent
name: charondaemon
resources: {}
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_ADMIN
- NET_RAW
- NET_BIND_SERVICE
drop:
- ALL
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/
name: charon-volume
- mountPath: /etc/swanctl
name: charon-conf
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-jjkpm
readOnly: true
- image: someimagewithswanctl
imagePullPolicy: Always
name: restctl
resources: {}
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_ADMIN
drop:
- ALL
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/
name: charon-volume
- mountPath: /etc/swanctl
name: charon-conf
- mountPath: /netns
name: netns-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-jjkpm
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
hostIPC: true
hostNetwork: true
hostPID: true
initContainers:
- command:
- sh
- -c
- "echo 'someconfig'
> /etc/swanctl/swanctl.conf"
image: busybox:latest
imagePullPolicy: Always
name: create-conf
resources: {}
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/swanctl
name: charon-conf
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-jjkpm
readOnly: true
nodeName: nixos
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- emptyDir: {}
name: charon-volume
- emptyDir: {}
name: charon-conf
- hostPath:
path: /var/run/netns/
type: Directory
name: netns-dir
- name: kube-api-access-jjkpm
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace