Meta-Learning for Medium-shot Sparse Learning via Deep Kernels

Few-shot learning assumes that we have a very small dataset for each task and trains a model on the set of tasks. For real-world problems, however, the amount of available data is substantially much more; we call this a medium-shot setting, where the dataset often has several hundreds of data. Despite their high accuracy, deep neural networks have a drawback as they are black-box. Learning interpretable models has become more important over time. This study aims to obtain sample-based interpretability using the attention mechanism. The main idea is reducing the task training data into a small number of support vectors using sparse kernel methods, and the model then predicts the test data of the task based on these support vectors. We propose a sparse medium-shot learning algorithm based on a metric-based Bayesian meta-learning algorithm whose output is probabilistic. Sparsity, along with uncertainty, effectively plays a key role in interpreting the model's behavior. In our experiments, we show that the proposed method provides significant interpretability by selecting a small number of support vectors and, at the same time, has a competitive accuracy compared to other less interpretable methods.

Keywords:

Bayesian Meta-Learning , Medium-shot Learning , Sample-based Interpretability , Sparse Kernel , attention