Few-shot learning explained: What you should know

What is few-shot learning?

Few-shot learning is an ML approach that enables a model to classify new data using a limited set of training examples.

Supervised learning typically uses thousands or even hundreds of thousands of labeled data points to train and refine an ML system's classification and decision-making abilities. However, such detailed and extensive training is impractical when extensive volumes of training data can't feasibly be obtained or simply don't exist.

Few-shot learning builds on a pretrained ML model that already performs well in data identification and classification tasks. With few-shot learning, the pretrained ML model receives additional training to add new classifications using just a few data samples. While few-shot learning isn't intended for training an ML model from scratch, it's a valuable strategy for quickly and easily extending an existing model's capabilities.

There are three primary approaches to few-shot learning, each based on the type of prior knowledge the ML model possesses:

• Prior knowledge on similarity. This approach relies on learned patterns in training data that enable the ML model to separate or classify previously unlearned data.
• Prior knowledge on learning. This method involves using prior knowledge to tune the algorithm -- a process known as hyperparameter tuning -- so that it can operate effectively with few examples.
• Prior knowledge on data. In this approach, the ML model has an understanding of the variability and structure of data, which aids in building models from limited examples. An example is using pen stroke data as a foundation for handwriting analysis.

As a prerequisite to few-shot learning, the ML model must already possess some viable data. For example, consider a visual ML model trained to recognize bird species from thousands of diverse and accurately classified bird images.

If a new species of bird is discovered and only a few labeled images of it exist, few-shot learning can incorporate this new species into the model's training. Because the new data fits the underlying structures that the model has already learned, the model can learn to recognize the new species with only a handful of images.

N-shot learning

The practical challenges of training data availability and quality have spawned a general category of ML training called n-shot learning, where n represents some small number of samples.

There are three typical variations of n-shot learning:

• Few-shot learning, which uses a relatively small set of labeled data points to train an ML model.
• One-shot learning, a variant of few-shot learning that uses just one labeled sample for training.
• Zero-shot learning, an extreme approach that attempts to handle new data without any existing data samples.

When is few-shot learning appropriate?

While supervised learning is often the ideal approach for ML models, it isn't always desirable, practical or even possible in real-world scenarios. Few-shot learning can supplement supervised learning in various situations, including the following:

• Data costs too much. Most businesses do not possess enough data to adequately train an ML model, leading them to purchase or license additional data from outside data sources. If the costs of doing so are excessive, few-shot learning might be a more attractive option.
• Data isn't properly labeled. Training data requires accurate labels, but data quality problems and imperfect labeling are common. Often, labeling is performed by people with limited knowledge of the data. Poor labeling and other data quality issues might lead a business to use few-shot learning.
• Data is limited or doesn't exist. Ample data examples do not exist for every possible topic. For example, diagnosing rare diseases, identifying new species or analyzing unique samples might require few-shot learning due to the scarcity of training data.
• Compute resources are limited. Training an ML model using supervised learning can demand significant time and compute resources. A business that cannot allocate these resources might turn to few-shot learning.

Few-shot learning use cases

Numerous ML and AI fields can use few-shot learning, including the following:

• Computer vision. Few-shot learning can support tasks such as character recognition, image classification, object recognition, object tracking and object labeling. Few-shot learning can provide new classifications for similar but different data, as in the above example of adding a new bird species to a library of previously learned species.
• Robotics. Few-shot learning enables robots to learn tasks based on limited human demonstrations, such as how to move from one place to another or how to assemble certain parts.
• Audio processing. Few-shot learning can support tasks such as voice cloning, conversation and translation when few audio samples exist. For example, a speech recognition system could learn to accurately identify and transcribe a new speaker's voice with just a handful of voice samples.
• Natural language processing. Few-shot learning can assist with NLP tasks such as parsing, translation, sentence completion and sentiment analysis. For example, if an ML translation platform incorrectly translates a word, few-shot learning can correct the context, pronunciation or usage, or add new words to an existing translation capability.
• Healthcare. Few-shot learning is well proven in image analysis and can be a powerful addition to image processing and diagnostic capabilities. For example, it can help an image processing platform distinguish between a normal cell and various abnormal expressions of that cell, such as a cancerous version.
• Math and analytics. Few-shot learning can be applied in situations where data is limited or specific queries are not fully supported by the existing training data. It's useful for IoT analytics and mathematical tasks such as curve fitting and reasoning.

Advantages of few-shot learning

Few-shot learning offers several key advantages:

• Reduced data collection. Few-shot learning takes the focus off data collection, saving time, money and storage resources. Less data also means less labeling and classification work.
• Reduced computing resources. Few-shot learning requires significantly less computational power and time compared with supervised and other comprehensive ML model training approaches.
• Reduced data dependency. Few-shot learning reduces reliance on large data sets, enabling businesses to develop models and deliver meaningful platforms even when data is scarce or too expensive to acquire.
• Greater model flexibility. Models capable of supporting few-shot learning can adapt quickly to new data or situations using limited data sets, enhancing versatility in busy, rapidly changing environments.

Disadvantages of few-shot learning

Despite its advantages, few-shot learning has several potential drawbacks that business and project leaders should consider:

• Lower diversity. Comprehensive ML training benefits from high data diversity. For example, learning from a thousand images of bluebirds in varied poses, locations and plumages increases a model's probability of accurately identifying that species in the future. With only a few examples, the model's ability to succeed with new data might suffer.
• Limited support for complexity. ML models see relationships between data points, and complex tasks often require significant amounts of data to establish those patterns and relationships. Few-shot learning might not be suitable for complex tasks or determinations due to insufficient data samples.
• Exaggerated impact of bad data. Models trained with few-shot learning are severely affected by erroneous or incomplete data samples, known as noisy data. This makes data selection and usage critical in few-shot learning.
• Memorization instead of understanding. Limited data samples used in few-shot learning might cause the model to memorize the samples rather than analyze and assimilate data into useful patterns, a phenomenon known as overfitting. This causes the model to provide good results with test data, but poor results with live data.

Stephen J. Bigelow, senior technology editor at TechTarget, has more than 20 years of technical writing experience in the PC and technology industry.