Title: Neyman-Pearson Classification

Abstract:

In many binary classification application in biomedical sciences, such as disease diagnosis, practitioners commonly face the need to control type I error (i.e., the conditional probability of misclassifying a class 0 observation as class1) so that it remains below a desired threshold. To address this need, the Neyman-Pearson (NP) classification paradigm is a natural choice; it minimizes type II error (i.e., the conditional probability of misclassifying a class 1 observation as class 0) while enforcing an upper bound, alpha, on the type I error. Although the NP paradigm has a century-long history in hypothesis testing, it has not been well recognized and implemented in classification schemes. Common practices that directly control the empirical type I error to no more than alpha do not satisfy the type I error control objective because the resulting classifiers are still likely to have type I errors much larger than alpha. As a result, the NP paradigm has not been properly implemented for many classification scenarios in practice. In this work, we develop the first umbrella algorithm that implements the NP paradigm for all scoring-type classification methods, such as logistic regression, support vector machines and random forests. Powered by this umbrella algorithm, we propose a novel graphical tool for NP classification methods: NP receiver operating characteristic (NP-ROC) bands, motivated by the popular receiver operating characteristic (ROC) curves. NP-ROC bands will help choose alpha in a data adaptive way and compare different NP classifiers. Implementation of the NP umbrella algorithm and NP-ROC bands is available in the R package nproc.

Bio:

Jingyi Jessica Li is an Associate Professor in the Department of Statistics (primary) and the Departments of Human Genetics and Biomathematics (secondary) at the University of California, Los Angeles (UCLA). She is also a faculty member in the Interdepartmental Ph.D. Program in Bioinformatics. Prior to joining UCLA in 2013, Jessica obtained her Ph.D. degree from the Interdepartmental Group in Biostatistics at the University of California, Berkeley, where she worked with Profs Peter J. Bickel and Haiyan Huang. Jessica received her B.S. (summa cum laude) from the Department of Biological Sciences and Technology at Tsinghua University, China in 2007. Jessica and her students focus on developing statistical and computational methods motivated by important questions in biomedical sciences and abundant information in big genomic and health related data. On the statistical methodology side, her research interests include association measures, high-dimensional variable selection, and classification metrics. On the biomedical application side, her research interests include bulk and single-cell RNA sequencing, comparative genomics, and information flow in the central dogma. Jessica is the recipient of the Hellman Fellowship (2015), the PhRMA Foundation Research Starter Grant in Informatics (2017), the Alfred P. Sloan Research Fellowship (2018), the Johnson & Johnson WiSTEM2D Math Scholar Award (2018), and the NSF CAREER Award (2019).

Website