SlimPer Scales Recommendation Histories Without Quadratic Attention

Industrial recommenders increasingly borrow transformer-style sequence modeling, but their objective is different from language modeling: a ranking model scores a user-item pair rather than predicting every token. SlimPer starts from that mismatch. Instead of carrying an $N$-length hidden state through every layer, the paper represents the current user-item hypothesis as a fixed-size knowledge base and repeatedly queries raw user-side evidence from it.

The result is a recommendation architecture built around long user histories without making depth proportional to history length. The authors report deployment on Instagram Reels and Feed, with offline gains across engagement tasks and full-traffic launches in production.

Core Contribution

SlimPer’s main move is to separate evidence from state. User-side tokens remain available at every layer, but the intermediate representation is a compact $K \times d$ knowledge base rather than a full sequence tensor. That lets the model stack refinement layers while keeping memory tied to $K$, not to the number of historical events.

Figure 1 summarizes the three design choices: a slim fixed-size knowledge base, complete access to user-side tokens at each layer, and request-only optimization that shares user features across candidate items.

This is not simply a smaller transformer. The paper argues that recommendation needs selective, discriminative relevance matching, not token-level generation. SlimPer therefore spends capacity on user-item matching rather than on preserving pairwise user-token interactions that may not affect the final ranking score.

Technical Approach

Each SlimPer layer follows a select-match-refine loop. The current knowledge base generates queries; all user-side sparse and sequence tokens provide keys and values; the model selects relevant evidence by modality-aware attention. It then computes explicit dot-product relevance scores against learned templates, and uses those scores plus dense features to update the knowledge base for the next layer.

Figure 2 shows the multi-layer form of this loop: a compact knowledge base is refined layer by layer while repeatedly accessing the full user history.

The tokenization module brings sparse, dense, and sequential signals into one backbone. Sparse features are embedding-pooled, events are encoded from content, action type, temporal information, and context, and dense features are projected through an MLP. In production terms, the paper describes $N=O(10^3)$ to $O(10^4)$ user-side tokens, $K=64$ knowledge-base slots, and 5 to 7 SlimPer refinement layers for the main Feed and Reels settings.

The architecture also incorporates request-only optimization. Because a ranking request usually scores many candidate items for the same user, SlimPer computes user-side tokens once per request batch and shares them across candidates. That is a practical systems detail, but it matters: the paper attributes part of the memory reduction to avoiding duplicate user-side representations per item.

Results and Analysis

The offline evaluation compares SlimPer with Instagram’s existing late-fusion baseline, described as HSTU for variable-length user history plus Wukong for fixed-size sparse and dense features. Normalized Entropy is the main offline metric; lower is better, and the authors state that an improvement of about 0.03% is empirically meaningful.

On Reels, SlimPer with 2k events improves NE by -0.51% for reshare and gives an 11.0% QPS gain with 9.32% lower memory. With 5k events, the model improves NE further, including -0.80% reshare and -0.62% save, but incurs a 10.0% QPS regression and 4.59% higher memory. Feed shows the same trade-off: 1k events improves like NE by -0.54% with 12.5% higher QPS and 18.12% lower memory, while 4k events reaches -1.05% save NE with a 16.07% QPS regression and 2.04% higher memory.

The scaling result is the clearest evidence for the design. At sequence length 2,048, the paper estimates about 24 GFLOPs for the baseline HSTU encoder versus about 3 GFLOPs for SlimPer, an 8× reduction. At 4,096 tokens, the estimate is 74 versus 5 GFLOPs, or 16×. At 6,144 tokens, it is 150 versus 6 GFLOPs, or 25×. Figure 4 plots the quality-efficiency curve and shows SlimPer gaining more NE as history length grows while avoiding the steeper throughput cost of the baseline.

Ablations support the mechanism but also define its operating range. Reducing the knowledge base from $K=64$ to $K=4$ worsens NE by +1.2% while improving QPS by 23.2%; intermediate values recover much of the quality. Layer depth has a softer optimum: 7 layers is the reported baseline, 3 and 5 layers are worse by +0.32% and +0.14% NE, and 9 layers slightly improves NE by -0.12% but costs 7.16% QPS and 4.92% memory.

Limitations

The strongest caveat is comparability. The authors evaluate against a mature internal baseline rather than recent unified architectures such as OneTrans, HHFT, or RankMixer, because those systems were not directly comparable in Meta’s production stack. The experiments also use internal Instagram training and evaluation data, so exact reproduction outside Meta is not possible. The paper gives architectural details and ablations, but the central deployment evidence remains tied to one company’s recommender infrastructure.