for any unbiased estimator of , where is the Fisher Information. This gives a lower bound on the Variance of any unbiased estimator of a parameter of the distribution of a Random variable : no unbiased procedure can estimate more precisely. For independent and identically distributed observations the information adds, , so the bound becomes .

Derivation

Setting: has density , open, and is an estimator with (unbiased) and . Regularity assumptions: the support does not depend on , is differentiable, differentiation in commutes with the integrals below, and .

Define the score . Then , and

so . Similarly

Applying the Cauchy-Schwarz inequality to the centred variables and , with inner product ,

which is the bound.

Remarks

  • Biased estimators: if , the same argument gives and hence .
  • Equality holds if and only if is an affine function of , almost surely. Integrating in shows that this requires to be an exponential family with as sufficient statistic. An estimator attaining the bound is called efficient. Example: the sample mean of draws from with known has variance .
  • The maximum likelihood estimator attains the bound asymptotically: under regularity, in distribution.
  • Multiparameter version: in the positive semidefinite order, with the Fisher information matrix.
  • The regularity assumptions matter: for the support depends on , and the unbiased estimator has variance , decaying as rather than .
  • The quantum analogue replaces by the quantum Fisher information, maximised over measurements.