Detecting Erroneous Classifiers in Batch Distributed Inference

Distributed inference is a promising paradigm in machine learning, but errors from participants may corrupt the inference result. To mitigate this in binary-classification tasks, we study the problem of detecting erroneous classifiers. Each classifier in a distributed ensemble provides a batch of classification outputs to a central node, and the proposed detectors aim to find the erroneous ones among them. Two types of detectors are studied: 1) blind detectors that know nothing about the statistics of the classifiers, and 2) informed-statistics detectors that know the classifiers’ pair-wise agreement statistics. We develop analytical tools for evaluating the detection performance, and demonstrate the tools for ensembles following the Bernoulli-Mixture model. In addition, we provide empirical results to validate the improvement in classification accuracy on real neural-network based classifiers.