Probability Inequalities and Their Practical Applications

1. Markov's Inequality

Formula: \( P(X \geq a) \leq \frac{E[X]}{a} \)

Solved Example: Given \( E[X] = 4 \) and \( a = 16 \), we compute:

\( P(X \geq 16) \leq \frac{4}{16} = 0.25 \)

2. Chebyshev's Inequality

Formula: \( P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2} \)

Solved Example: For \( k = 2 \), we compute:

\( P(|X - 70| \geq 20) \leq \frac{1}{2^2} = 0.25 \)

3. Jensen's Inequality

Formula: \( f(E[X]) \leq E[f(X)] \)

Solved Example: Let \( X \) be income, with \( E[X] = 3 \) and \( X = \{2, 4\} \), we compute:

\( f(3) = 3^2 = 9, \quad E[f(X)] = \frac{2^2 + 4^2}{2} = 10 \)

4. Hoeffding's Inequality

Formula: \( P(|\bar{X} - \mu| \geq t) \leq 2\exp(-2nt^2) \)

Solved Example: If \( n = 1000, t = 0.1 \), then:

\( P(|\bar{X} - \mu| \geq 0.1) \leq 2\exp(-2 \times 1000 \times 0.1^2) \approx 0.00004 \)

7. Azuma-Hoeffding Inequality

Formula: \( P\left(\sum X_i - E[\sum X_i] \geq t\right) \leq \exp\left(\frac{-t^2}{2nc^2}\right) \)

Solved Example: If \( n = 100, t = 10, c = 1 \), then:

\( P(S_n - E[S_n] \geq 10) \leq \exp\left(-\frac{10^2}{2 \times 100 \times 1^2}\right) \approx 0.6065 \)