Comes from the stochastic nature of the input and output in machine learning.

It cannot be reduced with more training data. It can however be reduced with additional features that provide more information about the input space.

It arises from ignorance about the model due to a lack of observations.

Either there is Insufficient data. Or the model is not adequate (not complex enough) for the problem.

Epsitemic_uncertainty.png

  • Parameter uncertainty: Limited training data (see the image above)
  • Model form uncertainty: Uncertainty from the model choice.

Bayesian Neural networks are an example that give as output the prediction and the uncertainty of the prediction (in the form of a mean and variance)

Closely related to Decomposing Model Error