Efficient, Noise-Tolerant, and Private Learning via Boosting
Mark Bun, Marco L Carmosino, Jessica Sorrell
Subject areas: Privacy, fairness, Excess risk bounds and generalization error bounds, PAC learning
Presented in: Session 4B, Session 4D
[Zoom link for poster in Session 4B], [Zoom link for poster in Session 4D]
Abstract:
We introduce a simple framework for designing private boosting algorithms. We give natural conditions under which these algorithms are differentially private, efficient, and noise-tolerant PAC learners. To demonstrate our framework, we use it to construct noise-tolerant and private PAC learners for large-margin halfspaces whose sample complexity does not depend on the dimension.\n\nWe give two sample complexity bounds for our large-margin halfspace learner. One bound is based only on differential privacy, and uses this guarantee as an asset for ensuring generalization. This first bound illustrates a general methodology for obtaining PAC learners from privacy, which may be of independent interest. The second bound uses standard techniques from the theory of large-margin classification (the fat-shattering dimension) to match the best known sample complexity for differentially private learning of large-margin halfspaces, while additionally tolerating random label noise.