Sampling distributions Knowledge-Based Study

Study notes

1. Parameters vs. Statistics

   In inferential statistics, we distinguish between two fundamental concepts:

   • Parameter: A numerical descriptive measure of an entire population, such as the population mean $\mu$ or population standard deviation $\sigma$. Parameters are typically fixed but unknown.
   • Statistic: A numerical descriptive measure computed from sample data, such as the sample mean $\bar{x}$ or sample standard deviation $s$. Statistics vary from sample to sample.

   What is a Sampling Distribution?

   If you take multiple independent random samples of the same size $n$ from a population, each sample will yield a slightly different statistic. The probability distribution of all possible values of a statistic computed from samples of size $n$ is called the sampling distribution of that statistic.

   Properties of the Sample Mean $\bar{X}$

   Let $\bar{X}$ represent the sample mean of a simple random sample of size $n$ drawn from a population with mean $\mu$ and finite standard deviation $\sigma$.

   Center: Mean of the Sample Means

   The expected value of the sample mean equals the population mean:

   $$\mu_{\bar{X}} = \mu$$

   Because $\mu_{\bar{X}} = \mu$, the sample mean $\bar{X}$ is an unbiased estimator of the population mean $\mu$.

   Spread: Standard Error

   The standard deviation of the sampling distribution of $\bar{X}$ is called the standard error of the mean, denoted $\sigma_{\bar{X}}$:

   $$\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$$

   As the sample size $n$ increases, the standard error decreases. Larger samples produce sample means that are more tightly clustered around the true population mean.

   Shape of the Sampling Distribution

   The shape of the sampling distribution of $\bar{X}$ depends on the shape of the underlying population and the sample size $n$:

   • Normal Population: If the original population is normally distributed with mean $\mu$ and standard deviation $\sigma$, the sampling distribution of $\bar{X}$ is exactly normal for any sample size $n$:
     $\bar{X} \sim N\left(\mu, \frac{\sigma}{\sqrt{n}}\right)$.
   • Non-Normal Population: If the population is not normal (e.g., skewed or uniform), we rely on the Central Limit Theorem (CLT).

   Statement of the Central Limit Theorem

   As the sample size $n$ becomes sufficiently large, the sampling distribution of the sample mean $\bar{X}$ approaches a normal distribution with mean $\mu$ and standard deviation $\frac{\sigma}{\sqrt{n}}$, regardless of the shape of the parent population distribution.

   Rule of Thumb: In practice, a sample size of $n \ge 30$ is generally considered large enough for the Central Limit Theorem to provide a reliable normal approximation, provided the distribution does not have extreme outliers.

   Calculating Probabilities for $\bar{X}$

   When the sampling distribution of $\bar{X}$ is approximately normal (either by population normality or via the Central Limit Theorem), probabilities regarding the sample mean are evaluated using the standard normal distribution.

   The Z-Score Formula for Sample Means

   To convert a sample mean $\bar{X}$ into a standard normal variable $Z$, use:

   $$Z = \frac{\bar{X} - \mu_{\bar{X}}}{\sigma_{\bar{X}}} = \frac{\bar{X} - \mu}{\frac{\sigma}{\sqrt{n}}}$$

   Example Summary

   Suppose a machine fills beverage bottles with a mean of $\mu = 500$ mL and a standard deviation of $\sigma = 10$ mL. For a random sample of $n = 25$ bottles, the standard error is:

   $$\sigma_{\bar{X}} = \frac{10}{\sqrt{25}} = \frac{10}{5} = 2\text{ mL}$$

   To find the probability that the sample mean exceeds $503$ mL, compute the corresponding $Z$-score:

   $$Z = \frac{503 - 500}{2} = 1.5$$

   Using standard normal distribution tables, $P(\bar{X} > 503) = P(Z > 1.5) \approx 0.0668$.