Law of Large Numbers

The Law of Large Numbers (LLN) is one of the foundational theorems of probability theory. It formalizes the intuitive idea that the more times you repeat a random experiment, the closer the average result gets to the expected value.

If you flip a fair coin 10 times, you might get 7 heads (70%). Flip it 1,000 times, and you'll likely get around 480-520 heads (48-52%). Flip it 1,000,000 times, and the proportion will be extremely close to 50%. The LLN guarantees this convergence mathematically.

There are two versions of the theorem:

• Weak Law of Large Numbers: The sample average converges to the expected value in probability: for any ε > 0, P(|X̄_n - μ| > ε) → 0 as n → ∞.
• Strong Law of Large Numbers: The sample average converges to the expected value almost surely: P(X̄_n → μ) = 1. This is a stronger guarantee — the convergence happens with probability 1, not just in probability.

Both require that the random variables are independent and identically distributed (i.i.d.) with a finite expected value.

The Law of Large Numbers is the mathematical foundation of systematic trading. Here's why:

Consider a high-frequency trading strategy that makes 50,000 trades per day. Each trade has a positive expected profit of $0.50, but the standard deviation per trade is $100. On any single trade, the outcome is essentially random — you can't distinguish the $0.50 edge from the $100 of noise.

But over 50,000 trades per day:

• Expected daily profit: 50,000 × $0.50 = $25,000
• Daily standard deviation: $100 × √50,000 = $22,360
• Daily Sharpe ratio: $25,000 / $22,360 = 1.12 (excellent)

The LLN ensures that the actual daily profit converges to $25,000 as the number of trades increases. Over a year (250 trading days), the total expected profit is $6.25 million, and by the CLT, the probability of having a losing year is essentially zero.

This is why scale matters in quant trading — more trades mean faster convergence to the expected value. It's also why market makers who trade millions of shares per day can have near-certain profitability, while retail traders with a few trades per week are dominated by noise.

The LLN is frequently misunderstood. Here are the most common misconceptions:

• The gambler's fallacy: "I've lost 5 times in a row, so I'm due for a win." Wrong. The LLN says the average converges over many trials, but it does NOT say individual outcomes become more predictable after a streak. Each coin flip is still 50/50 regardless of previous flips. The convergence happens by dilution (new results swamp old results), not correction.
• "It evens out": People sometimes believe losses will be "made up" by future gains. The LLN says the average converges, but the cumulative sum can continue to deviate. If you lose $1,000 in 100 fair bets, the LLN does not guarantee you'll win it back — it guarantees that your average win per bet will approach zero (the expected value of a fair bet).
• Short-run guarantees: The LLN is an asymptotic result — it tells you what happens as n → ∞. For finite n, you can still deviate significantly from the expected value. This is why even a strategy with a genuine positive edge can have losing months or even losing years.
• Independence requirement: The standard LLN requires independence. If trade outcomes are correlated (e.g., because they all depend on the same market factor), the convergence is slower and the effective sample size is smaller than the actual number of trades.

The LLN connects to two other key concepts in quant finance:

Central Limit Theorem: While the LLN tells you where the average converges (to μ), the CLT tells you how it gets there — the distribution of the sample average is approximately normal with spread σ/√n. Together, LLN and CLT give you both the target and the uncertainty band.

Kelly Criterion: The Kelly criterion tells you how much to bet on each trial to maximize long-run growth. The LLN ensures that the geometric average growth rate converges to the expected log-growth rate — which is precisely what Kelly maximizes. Without LLN, there would be no guarantee that Kelly-optimal betting actually leads to maximum wealth over time.

Monte Carlo: The accuracy of Monte Carlo simulation is a direct consequence of LLN — the average simulated payoff converges to the true expected payoff. The CLT then quantifies the estimation error.