FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

We are working up to a guest talk from Leslie Smith on his recent work https://arxiv.org/pdf/2006.09363.pdf which builds on FixNet.

First, we'll work our way through prior papers. I'd love help if anyone is willing to present!

----------------------------------------------------------------------------------------------
https://arxiv.org/pdf/2001.07685.pdf
FixMatch: Simplifying Semi-Supervised Learning with
Consistency and Confidence
Kihyuk Sohn∗ David Berthelot∗ Chun-Liang Li Zizhao Zhang Nicholas Carlini
Ekin D. Cubuk Alex Kurakin Han Zhang Colin Raffel
Google Research

Semi-supervised learning (SSL) provides an effective
means of leveraging unlabeled data to improve a model’s
performance. In this paper, we demonstrate the power of a
simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm,
FixMatch, first generates pseudo-labels using the model’s
predictions on weakly-augmented unlabeled images. For a
given image, the pseudo-label is only retained if the model
produces a high-confidence prediction. The model is then
trained to predict the pseudo-label when fed a stronglyaugmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10
with 250 labels and 88.61% accuracy with 40 – just 4 labels per class. Since FixMatch bears many similarities
to existing SSL methods that achieve worse performance,
we carry out an extensive ablation study to tease apart
the experimental factors that are most important to FixMatch’s success. We make our code available at https:
//github.com/google-research/fixmatch.

----------------------------------------------------------------------------------------------
https://arxiv.org/pdf/2004.04141.pdf
Empirical Perspectives on One-Shot Semi-supervised Learning

Leslie N. Smith
Adam Conovaloff
US Naval Research Laboratory

One of the greatest obstacles in the adoption of deep
neural networks for new applications is that training the
network typically requires a large number of manually labeled training samples. We empirically investigate the scenario where one has access to large amounts of unlabeled
data but require labeling only a single prototypical sample per class in order to train a deep network (i.e., oneshot semi-supervised learning). Specifically, we investigate
the recent results reported in FixMatch [13] for one-shot
semi-supervised learning to understand the factors that affect and impede high accuracies and reliability for Cifar-10.
For example, we discover that one barrier to one-shot semisupervised learning for image classification is the class accuracy unevenness of the training. These results point to
solutions that might enable more widespread adoption of
one-shot semi-supervised training methods for new applications.