Measure the accuracy of classified examples against their true labels:

Obtain the statistical uncertainty on this measure:

Compare the accuracy to a baseline (always predicting the most likely test-set class, "B" in this case):

Measure the error on the same data:

Check that error and accuracy sum to 1:

Measure the accuracy using a higher weight for the last two examples:

Create a training set and a test set on MNIST data:

Train a model on the training set:

Compute the accuracy of the model on the test set:

Compute the "top-3" accuracy of the model on the test set (classification is considered correct if the true class is among the three predicted classes with the highest probabilities):

Measure Cohen's kappa coefficient on classified examples against their true labels:

Measure Scott's pi on the same data:

Create a training set and a test set on MNIST data:

Train a model on the training set:

Create a classifier measurements object for this classifier on the test set:

Find the 10 test examples that have the worst classification (lowest probabiity for the correct class):

Compute their correct-class probability:

Find the 10 test examples that have the best classification:

Compute their correct-class probability:

Visualize the confusion matrix:

Extract the confusion matrix:

Extract the number of test examples of "3" classified as a "5":

Find the 5 most frequent class confusions:

Restrict the confusion matrix to the set of 3 classes that are the most confused with each other:

Restrict the confusion matrix to the set of 3 classes that are the least confused with each other:

Restrict the confusion matrix to classes "1", "4" and "9":

Extract the examples of "4" that are confused with a "9":

Load the Fisher's Irises dataset:

Create a training set and a test set:

Train a classifier to recognize the iris species from their attributes:

Measure the log-likelihood of the test set (total log-probabilities of correct classes):

Measure the mean cross-entropy:

The mean cross-entropy is the average negative log-likelihood:

Measure the geometric mean of the probabilities for the correct class:

This geometric mean is related to the mean cross-entropy:

Create a training set and a test set on MNIST data:

Train a random forest classifier for which the probability calibration is deactivated:

Visualize the reliability diagram (a.k.a calibration curve) of the classifier on the test set in a logit scale:

Visualize the same reliability diagram using a linear scaling:

Train a classifier where the calibration is activated:

Visualize its reliability diagram in logit scale:

Visualize its reliability diagram in linear scale:

Load the Fisher's Irises dataset:

Create a training set and a test set:

Train a classifier to recognize the iris species from their attributes:

Compute a ROC curve for each possible "positive" class:

Compute the area under these curves:

Measure the F₁ score for each possible "positive" class:

Measure the F₁ score if versicolor is the "positive" class:

Take the mean of all F₁ scores:

Weight each score according to the class prior:

Compute F₁ scores by joining true positive/true negative, etc. examples of all classes to give a unique measure:

Load the training set and test set of the "Satellite" dataset:

Train a classifier on the training set:

Train a classifier on the training set:

Visualize the confusion matrix obtained when the classifier has a different value of ClassPriors:

Perform the same operation by first generating a ClassifierMeasurementsObject:

Load the training set and test set of the "Titanic" dataset:

Train a classifier on the training set:

Visualize the confusion matrix of the classifier on the test set:

Visualize the confusion matrix obtained when the classifier has a different value of IndeterminateThreshold:

Measure the accuracy of the classifier on the test set for different values of IndeterminateThreshold:

Visualize the accuracy-versus-rejection curve:

Train a classifier using a neural network:

Measure the accuracy of the classifier on a test set for different setting of TargetDevice:

Load the training set and test set of the "Mushroom" dataset:

Train a classifier on a part of the training set:

Visualize the confusion matrix of the classifier on the test set:

Visualize the confusion matrix obtained when the classifier has a different value of UtilityFunction:

Perform the same operation by first generating a ClassifierMeasurementsObject:

Train a classifier that classifies movie review snippets as "positive" or "negative":

Generate a ClassifierMeasurements[…] object using a test set:

Obtain a measure of the accuracy along with its uncertainty:

Obtain a measure of other properties along with their uncertainties:

Create a classifier on a training set:

Generate a measurement object while specifying the weights that each test example has:

Compute the accuracy:

Weights can also be modified when using the measurement object:

Uncertainties are also affected by weights: