Selective Activation Sparsity Might Be the Most Important LLM Paper You Haven't Read Yet

Every few months a research idea shows up that makes the cost curves bend, and this month's candidate is selective activation sparsity — a training method where the model learns to use only the parameters that are actually relevant to the task in front of it.

The headline result: models trained this way are performing comparably to models roughly three times their size on reasoning benchmarks. Read that again with your infrastructure hat on. Same quality, a fraction of the active compute, on both training and inference.

Why this one feels different

We've had sparsity tricks for years — mixture-of-experts being the famous one, where a router picks between big chunks of the network. Selective activation sparsity pushes the idea finer-grained: instead of routing between experts, the model learns which individual parameters matter per task and lets the rest go quiet. Less scaffolding, more surgical.

The implication I care about most is on-device AI. The gap between 'model that fits on a phone' and 'model that's actually smart' has been the defining constraint of edge AI. If a 7B-class model can reason like a 20B-class one because it activates intelligently, laptops, phones, and even my daily-shipping side projects change their assumptions. (I run LLMs on an iPhone in Project 1 of my portfolio — believe me, every activated parameter counts.)

Alongside it

Google's inference work this cycle rhymes with the same theme — speculative decoding upgraded with block verification and tree-structured drafting, accepting more tokens per step. Sparsity cuts what you compute; better decoding cuts how often you compute. Efficiency is compounding from both ends.

The era of 'just make it bigger' isn't over, but the era of 'make it smarter per watt' is clearly the more interesting race right now.