Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Fast rates for prediction with limited expert advice

El Mehdi Saad 1, 2 Gilles Blanchard 2, 3
1 CELESTE - Statistique mathématique et apprentissage
Inria Saclay - Ile de France, LMO - Laboratoire de Mathématiques d'Orsay
3 DATASHAPE - Understanding the Shape of Data
CRISAM - Inria Sophia Antipolis - Méditerranée , Inria Saclay - Ile de France
Abstract : We investigate the problem of minimizing the excess generalization error with respect to the best expert prediction in a finite family in the stochastic setting, under limited access to information. We assume that the learner only has access to a limited number of expert advices per training round, as well as for prediction. Assuming that the loss function is Lipschitz and strongly convex, we show that if we are allowed to see the advice of only one expert per round for T rounds in the training phase, or to use the advice of only one expert for prediction in the test phase, the worst-case excess risk is Ω(1/ √ T) with probability lower bounded by a constant. However, if we are allowed to see at least two actively chosen expert advices per training round and use at least two experts for prediction, the fast rate O(1/T) can be achieved. We design novel algorithms achieving this rate in this setting, and in the setting where the learner has a budget constraint on the total number of observed expert advices, and give precise instance-dependent bounds on the number of training rounds and queries needed to achieve a given generalization error precision.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03405899
Contributor : El Mehdi Saad Connect in order to contact the contributor
Submitted on : Wednesday, October 27, 2021 - 2:46:33 PM
Last modification on : Friday, October 29, 2021 - 3:52:00 AM

Files

main.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03405899, version 1
  • ARXIV : 2110.14485

Citation

El Mehdi Saad, Gilles Blanchard. Fast rates for prediction with limited expert advice. 2021. ⟨hal-03405899⟩

Share

Metrics

Record views

63

Files downloads

20