Definition

What is a small language model (SLM)?

Sean Michael Kerner

A small language model (SLM) is a generative AI technology similar to a large language model (LLM) but with a significantly reduced size.

LLMs -- such as OpenAI's GPT-3 and GPT-4 -- are trained and optimized for many purposes, including general-purpose tool use. However, that wide range of capabilities has a downside: The sheer number of parameters and computational resources needed to train, fine-tune and operate LLMs is vast and costly.

In contrast, SLMs have a smaller model size, enabling LLM-type capabilities, including natural language processing, albeit with fewer parameters and required resources.

Small language models are commonly fine-tuned on domain-specific data sets. That specialization increases efficiency in targeted use cases such as specialized chatbots, summarization or information retrieval within particular industries. With their smaller size, these models are particularly effective on systems with limited computational resources, including mobile devices or edge computing environments.

Similar to their larger counterparts, SLMs are built on transformer model architectures and neural networks. SLM development commonly integrates techniques such as transfer learning from larger models and may incorporate advancements such as retrieval-augmented generation to optimize performance and expand the knowledge base.

The growing interest in SLMs transcends the need for more efficient artificial intelligence (AI) solutions in edge computing and mobile devices. For example, SLMs lower the environmental impact of training and running large AI models on high-performance graphics processing units. And many industries seek the more specialized and cost-effective AI solutions of an SLM.

Training small language models often involves techniques such as knowledge distillation, during which a smaller model learns to mimic a larger one. Fine-tuning typically uses domain-specific data sets and techniques, including few-shot learning, to adapt the model to specific tasks quickly.

SLMs range in parameter counts from a few million to several billion, whereas LLMs have hundreds of billions or even trillions of parameters. For example, GPT-3 has 175 billion parameters. Meanwhile, Microsoft's Phi-2, a small language model, has 2 billion.

Advantages of small language models

Small language models provide numerous benefits throughout an organization, including the following:

Cost-effectiveness. Smaller models are significantly less expensive to train and deploy compared to LLMs. The reduced computational requirements mean lower costs for hardware, energy and maintenance.
Energy efficiency. SLMs significantly reduce the carbon footprint associated with AI.
Rapid deployment capability. Due to their smaller size, small language models can be trained and deployed much faster than larger models.
More hardware options. SLMs run on significantly less powerful hardware than a typical LLM, with some capable of running on CPUs.
Customization. The smaller size of SLMs affords easier fine-tuning for specific tasks.
Security and privacy. Small language models deployed locally or within private cloud environments ensure sensitive information remains under organizational control.
Improved accuracy for specific tasks. SLMs fine-tuned for domain-specific tasks improve accuracy and reduce the risk of AI hallucinations or incorrect responses.
Lower latency. The smaller size potentially reduces delays when processing requests.

Limitations of small language models

While SLMs provide numerous advantages, they have limitations that, in certain scenarios, negatively impact performance or applicability, such as the following:

Scope. SLMs are designed for specific domains or tasks, which means they lack the broad capabilities of LLMs across various topics.
Limited capacity for complex understanding. Small language models have significantly fewer parameters than LLMs, restricting their ability to capture complex contextual dependencies and nuanced language patterns.
Data quality challenges. An SLM's effectiveness depends on the quality of its training data, which is typically less robust than an LLM's training set.
Scalability issues. While small language models are efficient for small- to medium-scale applications, they struggle to work effectively for large-scale deployments.
Technical expertise requirements. Customizing and fine-tuning SLMs to meet specific enterprise needs requires specialized expertise in data science and machine learning.

Small language models vs. large language models

SLMs and LLMs have unique strengths and weaknesses.

SLMs are ideal for specialized, resource-constrained applications, offering cost-effective and rapid deployment capabilities. In contrast, LLMs are well suited for complex tasks that require deep contextual understanding and broad generalization capabilities, typically at a higher cost with more resource requirements.

Feature	SLM	LLM
Parameter count	500 million to 20 billion	100 billion to over 1 trillion
Training data volume	Smaller, domain-specific data sets	Vast and diverse data sets
Training time	Hours to days	Weeks to months
Cost of training	Lower	Higher
Inference speed	Faster	Slower
Memory requirements	Lower (1-10 GB)	Higher (100 GB or more)
Performance on complex tasks	Moderate	High
Generalization capability	Limited	Strong
Deployment requirements	Less resource intensive	More resource intensive
Customization	Easier and more flexible	More complex and rigid
Suitability for domain-specific tasks	Highly suitable	Suitable, but often requires fine-tuning
Energy consumption	Lower	Higher
Environmental impact	Lower	Higher

Examples of small language models

The number of SLMs grows as data scientists and developers build and expand generative AI use cases.

Among the earliest and most common SLMs remain variants of the open source BERT language model. These variants feature customizable sizes for all manner of deployment. Large vendors -- Google, Microsoft and Meta among them -- develop SLMs as well.

A Lite BERT (ALBERT). First released in 2019 by Google Research, ALBERT reduces model size through parameter sharing and factorization techniques to offer a more efficient alternative to BERT.
DistilBERT. DistilBERT is a distilled version of BERT developed by Hugging Face. It claims to retain 97% of its language-understanding capabilities, while being 60% faster and 40% smaller. It is effective for tasks such as sentiment analysis, text classification and question answering.
MobileBERT. Developed by Google and specifically designed for mobile devices, MobileBERT is a compact version optimized for performance on resource-constrained hardware.
Phi-3-mini. Part of the Phi-3 family from Microsoft, the Phi-3-mini has applications in language processing, reasoning, coding and math.
Gemma 2. Part of Google's open Gemma family of models, Gemma 2 is a 2 billion-parameter model developed from the same foundation as the Google Gemini LLM.
H2O-Danube. This open source model from H2O.ai is designed for enterprise use cases. It performs well on tasks such as text generation and classification while being efficient enough to run on consumer-grade hardware.
Llama. Meta's Llama open models are generally considered LLMs. Still, the 8 billion-parameter version of Llama 3.1 is significantly smaller than Llama's 405 billion-parameter model.

Potential use cases for small language models

SLMs have a broad range of capabilities across multiple use cases:

Customer service chatbot. SLMs are trained to resolve customer inquiries and interactions. These chatbots automate responses to frequently asked questions and provide quick support on routine issues.
Sentiment analysis. Small language models tackle basic sentiment analysis of content, including customer reviews, social media comments and other feedback.
Point-of-sale systems. SLMs tailor functions to specific business operations.
Content generation from specified knowledge bases. Small language models create targeted content based on an organization's internal information.
Information retrieval from private internal documents. SLMs efficiently search and extract information from company-specific databases.
Data catalog enhancement. A small language model approach creates descriptions of different assets in a data catalog.
Data pipeline management. SLMs assist data engineers in building data pipelines, documenting environments and testing data quality.
Code assistance. Small language models show potential for basic code assistance, generating code snippets for developers, suggesting improvements and automating repetitive coding tasks.
Education. SLMs power intelligent tutoring systems, providing personalized learning experiences.
Finance. In the financial sector, small language models deliver fraud detection, risk assessment and personalized financial advice.
Healthcare. SLMs process electronic health records, assist with diagnoses and provide personalized health information.