RaoBlackwell theorem
From Academic Kids

In statistics, the RaoBlackwell theorem describes a technique that can transform an absurdly crude estimator into an estimator that is optimal by the meansquarederror criterion or any of a variety of similar criteria. (Pronunciation: Rao rhymes with "cow".)
Contents 
Some prerequisite definitions
 An estimator is an observable random variable (i.e. a statistic) used for estimating some unobservable quantity. For example, one may be unable to observe the average height of all male students at the University of X, but one may observe the heights of a random sample of 40 of them. The average height of those 40the "sample average"may be used as an estimator of the unobservable "population average".
 A sufficient statistic T(X) is an observable random variable such that the conditional probability distribution of all observable data X given T(X) does not depend on any of the unobservable quantities such as the mean or standard deviation of the whole population from which the data X was taken. In the most frequently cited examples, the "unobservable" quantities are parameters that parametrize a known family of probability distributions according to which the data are distributed.
 A RaoBlackwell estimator δ_{1}(X) of an unobservable quantity θ is the conditional expected value E(δ(X)  T(X)) of some estimator δ(X) given a sufficient statistic T(X). Call δ(X) the "original estimator" and δ_{1}(X) the "improved estimator". It is important that the improved estimator be observable, i.e., that it not depend on θ. Generally, the conditional expected value of one function of these data given another function of these data does depend on θ, but the very definition of sufficiency given above entails that this one does not.
 The mean squared error of an estimator is the expected value of the square of its deviation from the unobservable quantity being estimated.
The theorem
One case of RaoBlackwell theorem states:
 The mean squared error of the RaoBlackwell estimator does not exceed that of the original estimator.
In other words
 <math>E((\delta_1(X)\theta)^2)\leq E((\delta(X)\theta)^2).<math>
A more general version of the theorem will be mentioned below.
The essential tools of the proof besides the definition above are the law of total expectation and the fact that for any random variable Y, E(Y^{2}) cannot be less than [E(Y)]^{2}. That inequality is a case of Jensen's inequality, although in a statistics course it may be shown to follow instantly from the frequently mentioned fact that
 <math>0\leq\operatorname{var}(Y)=E((YE(Y))^2)=E(Y^2)(E(Y))^2.<math>
The more general version of the RaoBlackwell theorem speaks of the "expected loss"
 <math>E(L(\delta_1(X)))\leq E(L(\delta(X)))<math>
where the "loss function" L may be any convex function. For the proof of the more general version, Jensen's inequality cannot be dispensed with.
The improved estimator is unbiased if and only if the original estimator is unbiased, as may be seen at once by using the law of total expectation. The theorem holds regardless of whether biased or unbiased estimators are used.
The theorem seems very weak: it says only that the allegedly improved estimator is no worse than the original estimator. In practice, however, the improvement is often enormous, as an example can show.
Example
Phone calls arrive at a switchboard according to a Poisson process at an average rate of λ per minute. This rate is not observable, but the numbers of phone calls that arrived during n successive oneminute periods are observed. It is desired to estimate the probability e^{−λ} that the next oneminute period passes with no phone calls. The answer given by RaoBlackwell may perhaps be unexpected.
A extremely crude estimator of the desired probability is
 <math>\delta_0=\left\{\begin{matrix}1 & \mbox{if}\ X_1=0 \\
0 & \mbox{otherwise}\end{matrix}\right\},<math>
i.e., this estimates this probability to be 1 if no phone calls arrived in the first minute and zero otherwise.
The sum
 <math>X_1+\cdots+X_n<math>
can be readily shown to be a sufficient statistic for λ, i.e., the conditional distribution of the data X_{1}, ..., X_{n}, given this sum, does not depend on λ. Therefore, we find the RaoBlackwell estimator
 <math>\delta_1=E(\delta_0X_1+\cdots+X_n).<math>
After doing some algebra we have
 <math>\delta_1=\left(1{1 \over n}\right)^{X_1+\cdots+X_n}.<math>
Since the average number X_{1}+ ... + X_{n} of calls arriving during the first n minutes is nλ, one might not be surprised if this estimator has a fairly high probability (if n is big) of being close to
 <math>\left(1{1 \over n}\right)^{n\lambda}\approx e^{\lambda}.<math>
So δ_{1} is clearly a very much improved estimator of that last quantity.
Idempotence of the RaoBlackwell process
In case the sufficient statistic is also a complete statistic, i.e., one which "admits no unbiased estimator of zero", the RaoBlackwell process is idempotent, i.e., using it to improve the already improved estimator does not do so, but merely returns as its output the same improved estimator.
When is the RaoBlackwell estimator the best possible?
If the improved estimator is both unbiased and complete, then the LehmannScheffé theorem implies that it is the unique "best unbiased estimator".pl:Twierdzenie RaoBlackwella