Can a GPU workload run inside a strongly isolated virtual machine without sacrificing the performance that makes expensive AI accelerators worthwhile? NAVER's engineering team has published a detailed case study of how it connected NVIDIA B200 GPUs, InfiniBand networking and NVMe storage to Kata Containers virtual machines, then corrected CPU placement and PCIe topology problems. The answer is nuanced: optimized inference throughput was only 0.4–7% below bare metal in its tests, but latency-sensitive training and cold starts still carried substantial overhead.
This is the fourth installment in NAVER's series on running workloads with container-style management and VM-style isolation. It builds on earlier discussions of Kata's Pod architecture, GPU passthrough, NVLink fabric partitions and Kubernetes Dynamic Resource Allocation (DRA). This analysis draws on the detailed Korean source text supplied for NAVER D2's fourth installment. The original page could not be independently opened during our verification session. Figures below are reported engineering measurements, not independently reproduced benchmarks.
Why GPU isolation is harder than ordinary container isolation
In a conventional Kubernetes GPU Pod, workloads often share a host kernel. Kata Containers places a lightweight VM between a tenant workload and that kernel. That provides a stronger isolation boundary, but GPUs are not self-contained devices: distributed training also needs high-speed network interfaces, local storage, accurate PCIe topology and predictable CPU scheduling. Simply passing through the accelerator does not reproduce the performance of a physical host.
NAVER's B200 servers have eight compute InfiniBand NICs paired with eight GPUs and two additional storage-network NICs. The team passed compute NIC physical functions directly into guests using VFIO. Because there were too few storage NICs to dedicate one per small VM, it used SR-IOV to create virtual functions, sharing physical NIC bandwidth between guests. The guest kernel was built with the necessary mlx5 and InfiniBand drivers.
RDMA and GPUDirect: preserving the shortest path
Distributed training repeatedly exchanges gradients across servers. Remote Direct Memory Access (RDMA) allows NICs to access registered memory without the conventional TCP data-copy path. GPUDirect RDMA can further let a NIC access GPU memory without first staging data in host RAM. But those benefits depend on the actual hardware route.
In NAVER's setup, each GPU and its matching compute NIC shared a PCIe switch. The Kata guest did not see that real switch topology: virtual root ports made all devices appear similarly distant. NCCL could therefore choose a NIC physically far from the GPU it served. NAVER addressed this by having its DRA driver discover GPU–NIC pairs on the host, publish a paired “railset” resource and provide a generated NCCL topology XML file through NCCL_TOPO_FILE. The guest could then make a placement decision based on the real physical pairing rather than its virtual PCIe tree.
The source reports that 16-GPU, two-server NCCL all-reduce bus bandwidth started at 158 GB/s. With the topology file and NVLSTree algorithm, it reached 613 GB/s; explicitly selecting eight compute NICs using NCCL_IB_HCA brought it to 620 GB/s. In subsequent 8 GiB message tests with GPUDirect enabled, it reached 697.6–712.4 GB/s, versus 241–266 GB/s with GPUDirect disabled. These measurements involve different test conditions and optimization stages; they should not be presented as a single controlled before/after comparison.
ACS isolation versus peer-to-peer performance
Access Control Services (ACS) can redirect PCIe peer-to-peer requests through the root complex, supporting isolation at the expense of a direct GPU–NIC path. NAVER selectively enabled Address Translation Services (ATS) for NICs sharing a switch with their GPUs and the Direct Translated P2P (DT) bit on the relevant switch ports. In the reported B200 measurements, NIC-to-GPU-memory write bandwidth per link improved from 32.0 to 43.9 GB/s.
This is not a universal recommendation to disable PCIe protections. The implementation relies on specific device behavior and trust assumptions, and the changes were deliberately restricted to the relevant hardware paths. Operators need their own IOMMU, PCIe and isolation validation before considering similar changes.
Making local NVMe fast and shareable
Direct NVMe passthrough delivered 10.0 GB/s sequential read in the team's initial comparison but required exclusive ownership of a physical drive. That was unsuitable for a B200 server partitioned into as many as eight one-GPU VMs sharing two guest-designated NVMe devices. Conventional virtio-fs and virtio-blk could share storage, but their measured sequential reads were 4.75 and 2.1 GB/s respectively.
NAVER instead used the Storage Performance Development Kit (SPDK) with vhost-user-blk. SPDK manages NVMe in userspace and polls I/O queues on dedicated reactor cores, avoiding per-request host filesystem and syscall overhead. Its logical volume store divides a storage pool into volumes, each exposed to a guest as a block device through the Kata CSI driver. The implementation used two NVMe devices in RAID 0 for its SPDK pool and selected devices by PCI address rather than potentially unstable Linux device names.
This design comes with operational costs. Reactor cores consume CPU continuously while polling. SPDK must map guest memory, so it becomes a highly trusted host component. RAID 0 is a performance configuration, not a redundancy or durability strategy: failure of a member can affect the entire pool.
Storage benchmark results
| Metric | Earlier virtio-fs path | Optimized SPDK path | Ratio |
|---|---|---|---|
| Sequential read | 4.75 GB/s | 19.8 GB/s | 4.17× |
| Sequential write | 3.74 GB/s | 14.1 GB/s | 3.77× |
| Random 4 KiB read | 87.8k IOPS | 1,154k IOPS | 13.14× |
Source: NAVER D2 fourth installment. The SPDK path uses two NVMe devices in RAID 0; these figures are architecture-level comparisons, not evidence that a single drive became 4–13 times faster. Random-read figures are medians over five runs.
A silent vCPU scheduling error that froze guests
One of the most transferable lessons is not about GPUs at all. NAVER found guests where some vCPUs ran at half speed despite spare host capacity, and occasionally stopped responding to kubectl exec even while their Pods remained in the Running state. Pausing QEMU released host CPU time and allowed blocked kernel work to complete, helping isolate the problem to CPU placement rather than defective hardware.
Some virtual CPUs had been pinned to the same host CPU; others competed with host-kernel work or SPDK reactor cores. The team combined kubelet's static CPU manager, Guaranteed QoS and integer CPU requests, strict reservation of system CPUs, one-to-one vCPU pinning and an additional internal Kata option to exclude reserved CPUs. It also matched the guest vCPU count to requested CPUs. Under full load, the guest reported 0.0% CPU steal in the shown test.
This matters to anyone running local inference services, vector databases or AI agents on shared hardware. An apparently idle server can still experience severe latency if the wrong threads compete for a particular core. But NAVER's fixes include internal Kata changes and are not all available as drop-in settings in standard distributions.
How close was the optimized VM to bare metal?
| Workload | Bare metal | Kata VM | Interpretation |
|---|---|---|---|
| LLM inference output throughput, three models ~27B–750B | Baseline | 0.4–7% lower | Relatively small penalty |
| TTFT, ~280B model, TP2, concurrency 1 | 142.1 ms | 181.4 ms | 27.7% higher latency |
| Training step, ~8B model, 16k tokens/step | 514 ms | 716 ms | 39.3% longer |
| Training step, ~8B model, 131k tokens/step | 956 ms | 998 ms | 4.4% longer |
| Cold start, ~280B model, TP8 | 537 s | 980 s | 82.5% longer |
These figures reveal why the phrase “near bare-metal performance” needs context. The optimized interconnect paths were similar in the two environments, and output-token throughput was close. But a CUDA kernel submission took 3.96 microseconds on bare metal versus 5.83 microseconds in the Kata guest. Frequent short operations magnify that overhead, while larger training steps amortize it. Cold starts also pay for VM boot, device attachment and model warm-up, with warm-up accounting for nearly half the additional time in the team's analysis.
What operators can actually deploy
NAVER's end-user interface remains a Kubernetes Pod specification. Administrators preconfigure GPU tiers combining a RuntimeClass with a ResourceClaimTemplate. Users select the tier, claim the GPU plus its paired InfiniBand NICs, request guest memory through 1 GiB hugepages and mount SPDK-backed persistent volumes. Pod networking—including IP, Service and DNS—remains familiar.
That simplicity is the result of substantial platform engineering. The implementation includes custom DRA allocation, guest topology generation, CSI changes, CPU placement fixes and VFIO integration. NAVER notes that its Kata and CSI versions remain internally modified. It also lists upstream contributions for DRA device attachment (PR #13323 and #13420) and hugepage-aware VM sizing (PR #13853). Their mention is not confirmation that every change has merged or shipped in a stable upstream release.
Implications for AI infrastructure and AI Search
For organizations hosting inference APIs, retrieval pipelines and agent systems, the case points to a three-way trade-off: isolation, resource utilization and latency. Shared accelerators can improve utilization, while VM isolation may limit the impact of a container escape. But the isolation boundary moves to the hypervisor, which still requires patching and security review. The source does not claim absolute protection against all cross-tenant attacks.
For SEO and AI Search infrastructure, the lesson is operational rather than algorithmic. A retrieval or content-analysis service can have excellent model quality yet deliver poor user experience if storage queues, NUMA placement or cold-start delays dominate request latency. No Google or Naver ranking change is implied by this engineering publication. It provides an evidence-based framework for measuring the systems behind AI products, not a new search visibility signal.
Practical evaluation checklist
- Define the threat model and whether VM isolation is required for untrusted tenants.
- Inventory physical GPU, NIC, PCIe switch and NUMA relationships before allocating devices.
- Benchmark collective communication with representative message sizes and GPUDirect enabled and disabled.
- Compare storage under equivalent drive counts, queue depths and workloads; distinguish throughput from durability.
- Reserve host and polling cores, then check vCPU pinning and CPU steal under load.
- Measure output throughput, TTFT, short and long training steps, and cold starts separately.
- Record internal patches and upstream version dependencies before reproducing the architecture.
- Evaluate total resource cost, including dedicated polling cores and idle VM memory.
FAQ
Did Kata Containers match bare-metal performance?
For the tested LLM inference throughput, it came close, at 0.4–7% lower. Other workloads had larger penalties, especially short training steps and cold starts.
Why did NCCL performance initially collapse?
The guest's virtual PCIe topology hid physical GPU–NIC proximity. A host-derived topology file and paired NIC allocation helped restore efficient communication.
Why did NAVER use SPDK?
To share fast NVMe storage among VMs while avoiding expensive per-request host I/O layers, using userspace polling and logical volumes.
Can this be reproduced on an ordinary desktop or small local AI server?
Not as-is. The reported system depends on B200-class servers, multiple InfiniBand NICs, specific PCIe topology and internal platform patches. The transferable principles are careful CPU affinity, realistic I/O benchmarking and explicit isolation-versus-latency trade-offs.
Is SR-IOV the same as giving each VM its own NIC?
No. Virtual functions have separate PCI functions but share resources and bandwidth of their parent physical function.
Is this a new Naver Search algorithm?
No. It is an engineering article about GPU virtualization and distributed AI infrastructure.
The bottom line
NAVER's engineering case demonstrates that strong VM isolation and high GPU utilization can coexist—but only when the entire system is designed around physical topology, network data paths, storage I/O and CPU scheduling. The result is close to bare metal for some throughput-oriented workloads, not a universal performance equivalence. For AI platform teams, that distinction is more useful than a headline promising “zero virtualization overhead.”
Primary material: NAVER D2, “컨테이너처럼 쓰고, VM처럼 격리하기 4편 - IB 네트워크와 로컬 NVMe, vCPU 배치, 베어메탈 대비 성능”, detailed Korean source text supplied with the editorial brief. Supplementary project references: NVIDIA NCCL environment variables; SPDK vhost documentation; Kata Containers. Benchmark figures were not independently reproduced. Barry Schwartz: NOT CHECKED.