Little Models, Language and otherwise
« previous post | next post »
The last couple of months have seen some interesting publications about AI systems that are small by modern standards.
An idea that's been around for a while is to shrink standard "Large" models for deployment on devices of modest power. A related idea is that an array of specialized small models is the right solution for the "mass of applications in which language models perform a small number of specialized tasks repetitively and with little variation" — see "Small Language Models are the Future of Agentic AI", 6/2/2025.
Another recent publication argues that even general systems can be "natively designed—not adapted—for the unique constraints of local devices: weak computational power, limited memory, and slow storage": "SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment", 7/30/2025.
And yet another recent paper proposes HRM (= Hierarchical Reasoning Model), "a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency". HRM works well at modest computational scales with modest amounts of training – – "With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples": "Hierarchical Reasoning Model", 6/26/2025. (Code and data here.)
Such developments are encouraging for researchers who don't have long-term access to multi-billion-dollar computer farms. And they raise additional questions about the future ROI for those installations.