A Closer Look at Small-loss Bounds for Bandits with Graph Feedback
Chung-Wei Lee, Haipeng Luo, Mengxiao Zhang
Subject areas: Bandit problems, Online learning
Presented in: Session 3B, Session 3D
[Zoom link for poster in Session 3B], [Zoom link for poster in Session 3D]
Abstract:
We study small-loss bounds for adversarial multi-armed bandits with graph feedback, that is, adaptive regret bounds that depend on the loss of the best arm or related quantities, instead of the total number of rounds. We derive the first small-loss bound for general strongly observable graphs, resolving an open problem of Lykouris et al. (2018). Specifically, we develop an algorithm with regret Õ((κL*)^(1/2)) where κ is the clique partition number and L*is the loss of the best arm, and for the special case of self-aware graphs where every arm has a self-loop, we improve the regret to Õ(min{(ɑT)^(1/2), (κL*)^(1/2)}) where ɑ ≤ κ is the independence number. Our results significantly improve and extend those by Lykouris et al. (2018) who only consider self-aware undirected graphs.\n\nFurthermore, we also take the first attempt at deriving small-loss bounds for weakly observable graphs. We first prove that no typical small-loss bounds are achievable in this case, and then propose algorithms with alternative small-loss bounds in terms of the loss of some specific subset of arms. A surprising side result is that Õ(T^(1/2)) regret is achievable even for weakly observable graphs as long as the best arm has a self-loop.\n\nOur algorithms are based on the Online Mirror Descent framework but require a suite of novel techniques that might be of independent interest. Moreover, all our algorithms can be made parameter-free without the knowledge of the environment.