DeepSeek Cuts KV Cache Footprint Fourfold
A causal encoder-decoder MoE pairs cross-layer sparse-attention reuse with FP4 caching to reduce global context memory to 890 bytes per token.
Underlying Paper
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work has substantially reduced the cost of long-context computation, prefill remains computationally expensive, and large KV caches continue to strain HBM and SSD capacity and data-transfer bandwidth. Together, these compute, storage, and bandwidth demands constitute the primary bottleneck to further lowering deployment costs. To address this challenge, we introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. With its Causal Encoder-Decoder (CED) architecture, the model activates 16B parameters per token during decode but only 8B parameters during prefill, substantially improving cost efficiency for agentic workloads. To push the limits of KV cache compression, DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching. These designs reduce its global KV cache footprint (always in HBM) to 890 bytes per token, roughly 1/4 of the corresponding footprint of DeepSeek-V4-Flash. Further, through a dedicated deployment optimization known as SWA Bounded Replay, DeepSeek-V4.1-Flash reduces its persistent KV cache footprint (always on SSD or in host memory) to roughly 1/8 of that of DeepSeek-V4-Flash. Despite its much smaller KV cache footprint, the model delivers substantially better performance than the baseline. In addition, we streamline the DeepSeek-V4 architecture and introduce several efficient architectural extensions. We pretrain DeepSeek-V4.1-Flash on a multimodal corpus comprising 45T tokens and conduct comprehensive post-training, yielding strong performance across diverse text-based and multimodal agentic scenarios. Model checkpoints are available at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.
Long-horizon agents turn context handling into a deployment constraint rather than a secondary systems detail. Prefill must process growing histories, while key-value caches occupy accelerator memory, host memory, or SSD and must be moved across those tiers during inference. DeepSeek-V4.1-Flash addresses that pressure with a 552B-parameter multimodal Mixture-of-Experts model designed for contexts up to 1 million tokens. Its central wager is that aggressive cache compression and sparse retrieval can coexist with competitive agent performance.
Core Contribution
The paper combines a Causal Encoder-Decoder (CED) layout with Compressed Sparse Attention 2 (CSA2), FP4 KV caching, and deployment-oriented replay. During decoding, the model activates 16B parameters per token; during prefill it activates 8B. That asymmetry targets agent traces, where long inputs can dominate cost before a model produces much output.
The more consequential systems result is memory. The authors report a global KV-cache footprint of 890 bytes per token, about one quarter of DeepSeek-V4-Flash, and a persistent cache footprint roughly one eighth as large through SWA Bounded Replay. The distinction matters: global cache is retained in HBM, whereas persistent cache can reside on SSD or host memory. Reducing both addresses capacity and transfer bandwidth rather than shifting a single bottleneck elsewhere.
Technical Approach
The 40-layer network divides into a 20-layer causal encoder and a 20-layer decoder. Feed-forward blocks use DeepSeekMoE throughout. The first two encoder layers use sliding-window attention; later encoder layers use CSA2. In CSA2, a layer attends to a sparse set of context positions selected through an indexer rather than materializing and consuming full attention KV state at every layer.
Figure 3 lays out that encoder-decoder split and the placement of sliding-window attention, CSA2, Engram, DSpark, and the Hierarchical Sparse Indexer. The architecture is not simply a lower-precision cache: it reduces which KV states must be newly created, which can be reused across layers, and which positions are examined.
CSA2 has Full, Reindex, and Minimal modes. Full mode computes the main KV representation, indexer keys, and Top-K indices; Reindex recomputes selection while reusing earlier main KV and indexer keys; Minimal mode reuses all three except the current layer's query and sliding-window KV. The hierarchical indexer further constrains later selection: the decoder's first CSA2 Full layer chooses Top-512 indices and forms a shared candidate pool from selected blocks, while Reindex layers choose their Top-512 from that pool. This is a concrete way to make cache reuse compatible with changing queries across depth.
Results and Analysis
The authors pretrain on a 45T-token multimodal corpus and report post-training evaluations across text and multimodal agent settings. Figure 1 compares agentic benchmarks against named counterparts and plots the per-token global-cache reduction across DeepSeek generations. The reported approximately 4-fold reduction versus DeepSeek-V4-Flash and 437-fold reduction versus DeepSeek-V1 are large enough to matter operationally, particularly when serving many long-running sessions.
Figure 2 adds an important compute qualification. Its precision-weighted decode-FLOP plot shows nearly constant single-token decode cost as context length grows, using weights of 1 for BF16, 0.5 for FP8, and 0.25 for FP4 operations. That supports the paper's argument that the model attacks long-context inference at both the cache and decode-compute levels. The evidence is strongest for the authors' internal model comparison: the paper reports lower bits-per-byte on its held-out sets than both DeepSeek-V4-Flash-Base and DeepSeek-V4-Pro-Base, while agent curves improve as reinforcement-learning training scales and as maximal context reaches 1 million tokens.
The result is persuasive as a systems-and-model integration exercise, but the public evidence leaves several practical questions open. The headline memory ratios are relative to earlier DeepSeek models, not a cross-vendor serving study. The plots establish trends, yet the supplied material does not provide a cost-per-request analysis under production concurrency, cache misses, SSD traffic, or different hardware. The design should therefore be read as a substantial reduction in the authors' deployment footprint, not as a universal inference-cost ranking.
Evidence Box
moderateKey Claims
- •Cross-layer CSA2 reuse and FP4 caching reduce long-context KV storage
- •CED lowers active parameters during prefill relative to decode
- •Sparse retrieval preserves agentic and multimodal capability under cache compression
Key Results
- •890 bytes per token global KV cache, about 4× lower than DeepSeek-V4-Flash
- •Approximately 437× lower per-token global KV cache than DeepSeek-V1
- •16B active parameters per decode token versus 8B during prefill
- •1M-token maximum context and 45T-token multimodal pretraining corpus
Limitations & Caveats
- •Comparisons emphasize prior DeepSeek models rather than independent cross-vendor baselines
- •No supplied cost-per-request or production-concurrency evaluation
- •Persistent-cache savings depend on SWA Bounded Replay and SSD or host-memory deployment assumptions
- •Agent benchmark trends do not isolate the contribution of each compression component
