Theory of adaptive estimation

Abstract

The paper is an introduction to the modern theory of adaptive estimation. We introduce a universal estimation procedure based on a random choice from collections of estimators satisfying a few very general assumptions. In the framework of an abstract statistical model, we present an upper bound for the risk of the proposed estimator ($\ell$-oracle inequality). The basic technical tools here are a commutativity property of some operators and upper functions for positive random functionals. Since the obtained result is not related to a particular observation scheme, many conclusions for various problems in different statistical models can be derived from the single $\ell$-oracle inequality.