Wolfgang Nejdl

Hybrid Mamba MoE Trades Dense Scale for Throughput

Soofi S activates 3B of 30B parameters per token and matches larger open models while reaching 4.8k TPS/GPU at 40K context.