Unbiased Estimates of Population Mean and Variance

Estimation

Estimation is a statistical method used to approximate the true value of a population parameter, such as the mean or variance, based on a sample.
Point estimation gives us a single best guess of the parameter value, while interval estimation provides us with a range of plausible values, often in form of a confidence interval.
An estimator is a statistic, a rule for calculating an estimate of a given quantity based on observed data.

An estimator is called unbiased if the mean of its sampling distribution is equal to the true value of the parameter being estimated. In other words, an estimator is unbiased if, on average, it hits the bull’s eye.
The sample mean (x̄) is an unbiased estimator of the population mean (μ).
To illustrate, if we have a sample of observations x₁,x₂,…,xₙ from a population, the sample mean is computed as x̄ = (x₁+x₂+…+xₙ)/n.
The x̄ provides an unbiased estimate of μ, that is, E(x̄) = μ, where E(x̄) represents expected value or mean of x̄.

However, the usual formula for sample variance s² (calculated as the average of the squared deviations from the sample mean) is actually biased when used to estimate the population variance (σ²).
Specifically s² = (Σ(xᵢ - x̄)²)/(n), where i ranges from 1 to n, is a biased estimator of σ².
To correct for this bias, the calculation is slightly adjusted: Instead of dividing by n, we divide by n-1. The corrected estimator, which is unbiased, is often denoted as s²: s² = (Σ(xᵢ - x̄)²) / (n-1).
This principle is known as Bessel’s correction. The unbiased estimator s² is also known as the sample variance.
Thus, while the sample mean gives an unbiased estimate of the population mean, the sample variance s² gives an unbiased estimate only if Bessel’s correction is applied.

Using unbiased estimators is fundamental in statistics, particularly in hypothesis testing and construction of confidence intervals, to ensure the valid and reliable deduction of population parameters based on sample statistics.
For instance, when constructing confidence intervals for population mean with a known standard deviation, the sample mean (an unbiased estimator) is used, along with the standard deviation, sample size, and z-distribution critical value.
When standard deviation is unknown, the unbiased sample variance (and hence standard deviation) plays a crucial role in forming a t-distribution based confidence interval or conducting a t-test.
While it is key to remember the definitions and computation methods, it is equally important to understand the significance and application of these estimators in real-world statistical analysis.