EMSS 2011 Proceeding

Effect of reject option on classifier performance

Authors:   Stephan Dreiseitl, Melanie Osl

Abstract

Binary classifier systems that provide class membership probabilities as outputs may be augmented by a reject op- tion to refuse classification for cases that either appear to be outliers, or for which the output probability is around 0.5. We investigated the effect of these two reject op- tions (called ?distance reject? and ?ambiguity reject?, re- spectively) on the calibration and discriminatory power of logistic regression models. Outliers were found us- ing one-class support vector machines. Discriminatory power was measured by the area under the ROC curve, and calibration by the Hosmer-Lemeshow goodness-of- fit test. Using an artificial data set and a real-world data set for diagnosing myocardial infarction, we found that ambiguity reject increased discriminatory power, while distance reject decreased it. We did not observe any in- fluence of either reject option on the calibration of the logistic regression models.

I3M  Scientific Sponsors

I3M  Industrial Sponsors

I3M  Media Sponsors