Classification¶
Classify clothing images, and recognize inputs unlike the training data (handwritten digits). Dataset: Fashion-MNIST, MNIST.
Training¶
Make the model itself uncertainty-aware.
-
A Bayesian CNN: asking a network several times
Average a Bayesian CNN's repeated predictions, and use their disagreement to flag unfamiliar inputs.
Conv2DFlipout·DenseFlipout -
Detecting unfamiliar inputs with SNGP
Make a classifier's confidence drop on inputs unlike its training data, in a single forward pass.
SpectralNormalization·GaussianProcessClassificationHead·ResetCovarianceCallback
Post-processing¶
Add calibrated uncertainty to an already trained model, without retraining it.
-
Conformal prediction sets: "it's one of these"
Predict sets of classes that contain the true class 90% of the time, after training.
SplitConformalClassifier -
Fixing an overconfident classifier with temperature scaling
Fix an overconfident image classifier after training, with a single parameter.
TemperatureScaling