small language model

small language model (plural small language models)

1. (machine learning) A type of neural network specialized in language but is smaller scale, typically includes millions of parameters, instead of billions to trillions.

   Synonym: SLM

   Hypernyms: LM, language model < model

   Coordinate terms: LLM, large language model

   • 2025 April 13, Stephen Ornes, “Small Language Models Are the New Rage, Researchers Say. Larger models can pull off a wider variety of feats, but the reduced footprint of smaller models makes them attractive tools”, in Wired‎:

     Large language models work well because they’re so large. The latest models from OpenAI, Meta, and DeepSeek use hundreds of billions of “parameters” […] With more parameters, the models are better able to identify patterns and connections, which in turn makes them more powerful and accurate. But this power comes at a cost […] huge computational resources […] energy hogs […] In response, some researchers are now thinking small. IBM, Google, Microsoft, and OpenAI have all recently released small language models (SLMs) that use a few billion parameters—a fraction of their LLM counterparts. Small models are not used as general-purpose tools like their larger cousins. But they can excel on specific, more narrowly defined tasks, such as summarizing conversations, answering patient questions as a health care chatbot, and gathering data in smart devices. “For a lot of tasks, an 8 billion–parameter model is actually pretty good,” said Zico Kolter, a computer scientist at Carnegie Mellon University. They can also run on a laptop or cell phone, instead of a huge data center. (There’s no consensus on the exact definition of “small,” but the new models all max out around 10 billion parameters.) To optimize the training process for these small models, researchers use a few tricks. […]

• large language model

• small language model on Wikipedia.Wikipedia