for any unbiased estimator of , where is the Fisher Information. This gives a lower bound on the Variance of any unbiased estimator of a parameter of the distribution of a Random variable : no unbiased procedure can estimate more precisely. For independent and identically distributed observations the information adds, , so the bound becomes .
Derivation
Setting: has density , open, and is an estimator with (unbiased) and . Regularity assumptions: the support does not depend on , is differentiable, differentiation in commutes with the integrals below, and .
Define the score . Then , and
so . Similarly
Applying the Cauchy-Schwarz inequality to the centred variables and , with inner product ,
which is the bound.
Remarks
- Biased estimators: if , the same argument gives and hence .
- Equality holds if and only if is an affine function of , almost surely. Integrating in shows that this requires to be an exponential family with as sufficient statistic. An estimator attaining the bound is called efficient. Example: the sample mean of draws from with known has variance .
- The maximum likelihood estimator attains the bound asymptotically: under regularity, in distribution.
- Multiparameter version: in the positive semidefinite order, with the Fisher information matrix.
- The regularity assumptions matter: for the support depends on , and the unbiased estimator has variance , decaying as rather than .
- The quantum analogue replaces by the quantum Fisher information, maximised over measurements.