GPU-Owned Remote I/O Speeds Graph ANNS

Graph-based approximate nearest neighbor search has a storage problem once vector indexes grow beyond the memory budget of a single GPU server. NVMe-oF offers a plausible disaggregated-storage path, but the usual remote I/O stack is still organized around the CPU: the host forms requests, advances transport, receives completions, and decides when GPU work can continue. That control path is a poor fit for graph ANNS, where the next access is discovered inside GPU traversal rather than in a host-side iterator. The paper introduces GORIO, a GPU-centered remote I/O design that keeps the search scheduler, page-miss generation, pending-query state, and resume logic resident on the GPU while leaving the CPU as a transport and completion proxy.

Core Contribution

GORIO’s main claim is architectural rather than algorithmic: remote storage should follow the execution locus of graph search. Prior GPU-storage systems showed that split-phase issue/consume semantics can improve overlap for irregular GPU workloads, mostly in local storage settings. GORIO extends that idea to NVMe-oF and specializes it for graph ANNS, where fabric latency makes pending and resume behavior a first-order scheduling concern.

The genuine shift is that a page miss does not force graph-search control back to the CPU. GPU-resident query state remains live, and the scheduler can make decisions using information the GPU already has: which query lanes are blocked, which page requests are outstanding, and which other traversal work can continue. That matters for vector databases and retrieval systems because the bottleneck is not only raw storage bandwidth. It is the control delay between discovering a graph edge, fetching the associated record, and resuming useful GPU work.

Technical Approach

The design has two layers. The first is a GPU-direct remote I/O path that converts local page-cache misses into split-phase remote operations. A miss discovered during traversal is published into GPU-visible state, forwarded through the CPU-side NVMe-oF transport path, and completed back into memory that the GPU scheduler can observe. The CPU still participates, but not as the owner of search progress.

The second layer is ANNS-specific scheduling. GORIO keeps pending-query state on the GPU and lets traversal proceed where possible while remote pages are being served. The paper’s discussion emphasizes three consequences: query state stays local to GPU computation, CPU cycles are spent on transport rather than graph scheduling, and the GPU scheduler directly sees lane and page-miss state. That visibility enables policies such as coalescing duplicate requests and selectively retrieving data for active lanes, although the paper presents some of these as future refinements rather than completed mechanisms.

Results and Analysis

The reported evaluation uses a DiskANN-style graph workload on SIFT1M over an RDMA NVMe-oF path. Against the authors’ remote-I/O reference path, GORIO is 1.31× faster at the same recall. Against a direct remote page-cache path, it is 4.89× faster. Those are meaningful gains for a systems paper because the comparison isolates the cost of where I/O ownership and scheduling live, not a new graph index or a different retrieval objective.

The result supports the paper’s central interpretation: for graph ANNS, remote storage performance depends on the interaction between traversal and I/O completion, not just on NVMe-oF latency or bandwidth. Moving page-miss generation and resume decisions to the GPU reduces control-path mismatch when the GPU is the component discovering the next access. The 4.89× gap versus direct remote page-cache access suggests that merely making remote pages available to the GPU is insufficient; the scheduler has to be aware of pending work and able to overlap traversal with page service.

The evidence is still narrower than the system ambition. The paper reports a concrete SIFT1M DiskANN-style experiment, but the discussion explicitly calls for larger indexes, more datasets, deeper graph traversal, different recall targets, and multi-GPU or multi-tenant deployments. That leaves open whether the same design remains favorable under heavier contention, different vector distributions, or production retrieval workloads with mixed query patterns.

Limitations

The authors are clear that several parts of the design need more development. A finer lane/page-miss scheduler could reveal more ready GPU work when only part of a query is blocked. Miss coalescing could merge duplicate page requests across lanes or queries before they reach the transport. Target-side selective materialization could return only records needed by active lanes, reducing useful-byte amplification for page-granularity reads. Query-aware multipath steering across multiple NVMe-oF paths is also left as future work. These limits do not undercut the main result, but they place it in the category of a focused system study rather than a complete production storage layer.