Issue #103 Theory Edwin Dong, Anurag Shinde
Deep Evidential Regression, Simply Explained
Free editionPRO edition
This week’s PRO issue includes a Colab notebook, where we’re going to build an evidential regressor from scratch with NumPy and PyTorch, and then perform some tests from the paper. Please do read the full article before attempting the code tutorial.
As always, the code in this issue can be executed on a normal home PC and does not require a large amount of VRAM.
The monthly interview question bank can be found here. These are industry standard questions that are tested across major tech companies, AI research firms, and more. The questions are sorted by difficulty and topic. Check back frequently for more questions!
AI systems are now integrated in high-stakes domains such as healthcare, security, autonomous vehicles, and finance, where incorrect predictions have severe consequences.
However, most machine learning models provide only a single prediction such as a class label or numerical value without conveying how confident the model actually is.
Feed models an input they have never seen before, and they will still output a number, with no indication of a bad answer.
This limitation has motivated the development of uncertainty quantification techniques, where ML/AI models are developed to be able to simultaneously compute confidence along with their predictions.
A couple weeks ago, we looked at conformal prediction, an uncertainty quantification technique that creates guaranteed prediction sets based on quantiles.
Today, we’ll look at another helpful method from this field.
Deep Evidential Regression (Amini et al., 2020), published at NeurIPS by researchers at MIT and Harvard, proposes another method to quantify model uncertainty.
It trains a standard neural network to output not just a prediction, but uncertainty descriptors driven by probability.
The result is a model that is genuinely uncertain about doubtful outputs, calibrated enough to use in safety-critical settings, and fast enough to run in real time.
Deep Evidential Regression has the capability to understand two different types of uncertainties, which puts it a step above a method we have seen before: Gaussian Process Regression.
This is considered important because it can provide a higher level of clarity about prediction confidence and, additionally, can also significantly benefit the training process.
This edition is for PRO members.
