Thinking Inside the Ball: Near-Optimal Minimization of the Maximal Loss
Yair Carmon , Arun Jambulapati , Yujia Jin , Aaron Sidford
Session: Optimization(A)
Session Chair: Simon Du
Poster: Poster Session 1
Abstract:
We characterize the complexity of minimizing the maximum of ๐ convex, Lipschitz functions. For non-smooth functions, existing methods require O(๐๐โปยฒ) queries to a first-order oracle to compute an ๐-suboptimal point and ร(๐๐โปยน) queries if the functions are O(๐โปยน)-smooth. We develop methods with improved complexity bounds ร(๐๐โปยฒ/ยณ + ๐โปโธ/ยณ) in the non-smooth case and ร(๐๐โปยฒ/ยณ + โ๐๐โปยน) in the O(๐โปยน)-smooth case. Our methods consist of a recently proposed ball optimization oracle acceleration algorithm (which we refine), combined with careful implementation of said oracle for the softmax function. We also prove an oracle complexity lower bound scaling as ๐บ(๐๐โปยฒ/ยณ), showing that our dependence on ๐ is optimal up to polylogarithmic factors.