Random variables Knowledge-Based Study

Study notes

1. What is a Random Variable?

   In probability theory, the outcome of a random process is often described using numerical values. A random variable is a rule or mathematical function that assigns a real number to each possible outcome in the sample space of a random experiment.

   By convention, random variables are denoted by capital letters, such as $X$, $Y$, or $Z$. The specific numerical values that these variables can take are represented by lowercase letters, such as $x$, $y$, or $z$.

   Example

   Consider an experiment where a fair coin is tossed twice. The sample space is:

   $$S = \{ (\text{H}, \text{H}), (\text{H}, \text{T}), (\text{T}, \text{H}), (\text{T}, \text{T}) \}$$

   Let the random variable $X$ denote the total number of heads obtained. The function $X$ assigns a real number to each outcome:

   • $X((\text{T}, \text{T})) = 0$
   • $X((\text{H}, \text{T})) = 1$
   • $X((\text{T}, \text{H})) = 1$
   • $X((\text{H}, \text{H})) = 2$

   Thus, the possible values that the random variable $X$ can take are $0$, $1$, and $2$.

   Discrete vs. Continuous Random Variables

   Random variables are classified into two broad categories based on the nature of their possible values:

   1. Discrete Random Variables

   A random variable is discrete if its set of possible values is finite or countably infinite. These values typically result from counting processes.

   • The number of defective items in a batch of 100 products.
   • The number of rolls required before getting a 6 on a fair die.
   • The number of customer arrivals at a service desk during a one-hour period.

   2. Continuous Random Variables

   A random variable is continuous if it can take any real value within a given interval or collection of intervals. Continuous random variables typically result from measurements.

   • The exact time a battery lasts before failing.
   • The height or weight of an individual selected at random.
   • The ambient temperature recorded at noon at a weather station.

   The Probability Mass Function (PMF)

   The probability distribution of a discrete random variable describes how the total probability of 1 is distributed among its possible values. For a discrete random variable $X$, this is defined by its probability mass function (PMF), denoted as $P(X = x)$ or $p(x)$.

   Fundamental Properties

   For any valid discrete probability mass function:

   1. Non-negativity: $0 \le P(X = x) \le 1$ for every possible value $x$.
   2. Total probability: The sum of the probabilities over all possible outcomes must equal 1: $$\sum_{x} P(X = x) = 1$$

   Example

   Returning to the two-coin toss where $X$ represents the number of heads, assuming all 4 outcomes are equally likely with probability $\frac{1}{4}$:

   • $P(X = 0) = P(\{(\text{T}, \text{T})\}) = \frac{1}{4}$
   • $P(X = 1) = P(\{(\text{H}, \text{T}), (\text{T}, \text{H})\}) = \frac{2}{4} = \frac{1}{2}$
   • $P(X = 2) = P(\{(\text{H}, \text{H})\}) = \frac{1}{4}$

   We verify the total probability:

   $$\frac{1}{4} + \frac{1}{2} + \frac{1}{4} = 1$$

   Summary Measures of a Discrete Random Variable

   Two key numerical measures characterize the behavior of a discrete random variable: its expected value and its variance.

   Expected Value (Mean)

   The expected value, denoted as $E[X]$ or $\mu$, represents the long-run theoretical average value of $X$. It is calculated as the probability-weighted sum of all possible values:

   $$E[X] = \mu = \sum_{x} x \cdot P(X = x)$$

   Variance and Standard Deviation

   The variance, denoted as $\operatorname{Var}(X)$ or $\sigma^2$, measures the dispersion or spread of the values of $X$ around the mean $\mu$:

   $$\operatorname{Var}(X) = \sum_{x} (x - \mu)^2 \cdot P(X = x) = E[X^2] - (E[X])^2$$

   The standard deviation $\sigma$ is the positive square root of the variance: $\sigma = \sqrt{\operatorname{Var}(X)}$.

   Example Calculation

   For the number of heads $X$ in two fair coin tosses:

   $$E[X] = (0 \cdot 0.25) + (1 \cdot 0.50) + (2 \cdot 0.25) = 0 + 0.50 + 0.50 = 1$$

   $$E[X^2] = (0^2 \cdot 0.25) + (1^2 \cdot 0.50) + (2^2 \cdot 0.25) = 0 + 0.50 + 1.00 = 1.50$$

   $$\operatorname{Var}(X) = E[X^2] - (E[X])^2 = 1.50 - 1.00^2 = 0.50$$

   On average, one head is expected per two tosses, with a variance of 0.5.