Skip to main content

Common issues

ClusterIP and DNS resolution failures​

If the vnode-manager logs dial tcp: lookup ... no such host and tenant pods fail with FailedCreatePodSandbox: no runtime for "vnode" is configured, the Platform host is set to a cluster-internal DNS name instead of the Service ClusterIP.

The vnode-manager enters the node's network namespace to configure containerd. From that namespace, the node's DNS resolver cannot resolve cluster-internal service names such as loft.vcluster-platform. Set config.platform.host to the Platform Service's ClusterIP instead.

See Configuration for the full explanation and the command to retrieve the ClusterIP.

AppArmor profile loading failures​

Symptoms: Pods using the vNode RuntimeClass remain stuck in ContainerCreating. Events on the pod show a message similar to:

Failed to create pod sandbox: ... apparmor: failed to apply profile "vnode": ...

Diagnose: Check which AppArmor profiles are currently loaded on the node:

aa-status

If the vnode profile is absent, AppArmor is active on the node but the profile was not loaded. The vnode-manager loads the profile at startup. If it fails, subsequent pod creation attempts fail with the error above.

Workaround:

  1. Check the vnode-manager logs for the profile load error:

    kubectl logs -n vnode-runtime -l app=vnode-runtime --prefix | grep -i apparmor

  2. Restart the vnode-manager DaemonSet pod on the affected node to trigger a fresh profile load:

    kubectl rollout restart daemonset/vnode-runtime -n vnode-runtime

  3. If the profile load continues to fail, check that the AppArmor kernel module is loaded (aa-enabled) and that the node's AppArmor policy files are not corrupted.

Node unavailability after DaemonSet startup​

On some nodes, kubelet stops when vNode restarts containerd and does not restart automatically. This makes the node unavailable until kubelet is restarted manually.

This is documented in Limitations, including the root cause (BindsTo=containerd.service in the kubelet systemd unit), which systems are affected, and the workaround.

Device permissions​

Devices mounted from the host into a vNode appear owned by nobody:nogroup inside the container. This is a consequence of Linux user namespace ID mapping.

Symptoms: A workload that reads from or writes to a host-mounted device fails with a permission error, even though the device is accessible on the host.

Fix: On the host, add read/write permissions for non-owners on the affected device:

chmod o+rw /dev/<device>

Replace <device> with the actual device name. This must be applied on each node where the device is present. The permission change persists until the node is rebooted or the device is re-initialized.

This behavior is documented in Limitations.