Bayes' Theorem

Bayes' theorem is a formula that tells you how to update your beliefs when you receive new evidence. Named after the Reverend Thomas Bayes (1701-1761), it is one of the most important results in all of probability and statistics.

The theorem answers a fundamental question: if you observe some evidence B, how should you update your belief about whether hypothesis A is true? The formula is:

P(A|B) = P(B|A) × P(A) / P(B)

Where:

• P(A|B) = the posterior — the probability of A after observing B
• P(B|A) = the likelihood — the probability of observing B if A is true
• P(A) = the prior — the probability of A before observing B
• P(B) = the marginal likelihood — the total probability of observing B

The beauty of Bayes' theorem is that it lets you flip a conditional probability around. You often know P(B|A) — the probability of the data given the hypothesis — but you want P(A|B) — the probability of the hypothesis given the data. Bayes' theorem makes the conversion.

Example 1 — Medical Testing (The Standard Interview Problem):

A disease affects 1% of the population. A test for the disease is 99% accurate: it correctly identifies 99% of sick people (sensitivity) and correctly identifies 99% of healthy people (specificity). You test positive. What is the probability you actually have the disease?

Solution:

• P(Disease) = 0.01 (prior)
• P(Positive | Disease) = 0.99 (sensitivity)
• P(Positive | No Disease) = 0.01 (false positive rate)
• P(No Disease) = 0.99

P(Positive) = P(Positive|Disease) × P(Disease) + P(Positive|No Disease) × P(No Disease)

P(Positive) = 0.99 × 0.01 + 0.01 × 0.99 = 0.0099 + 0.0099 = 0.0198

P(Disease | Positive) = (0.99 × 0.01) / 0.0198 = 0.0099 / 0.0198 = 50%

Despite the 99% accurate test, a positive result only means a 50% chance of having the disease! The key insight: when the condition is rare (1% prevalence), even a very accurate test produces many false positives relative to true positives.

Example 2 — Unfair Coins:

You have two coins. Coin A is fair (50% heads). Coin B is biased (75% heads). You pick a coin at random and flip it 3 times, getting HHT. What is the probability you picked Coin B?

P(HHT | A) = 0.5³ = 0.125

P(HHT | B) = 0.75² × 0.25 = 0.140625

P(HHT) = 0.5 × 0.125 + 0.5 × 0.140625 = 0.1328

P(B | HHT) = (0.5 × 0.140625) / 0.1328 = 52.9%

Bayesian thinking is woven into quantitative trading at every level:

• Signal updating: A quant model might start with a prior belief that a stock is fairly valued. As new data arrives — earnings reports, order flow, macro data — the model updates its belief using Bayesian logic. The posterior becomes the new prior for the next update.
• Regime detection: Is the market in a "trending" or "mean-reverting" regime? A Bayesian model starts with prior probabilities for each regime and updates them as new returns are observed. This is formalized as a Hidden Markov Model — a direct application of Bayes' theorem.
• Parameter estimation: In Bayesian statistics, model parameters (like the mean and variance of returns) are treated as random variables with prior distributions. As data is observed, the posterior distribution becomes more precise. This is more robust than point estimates, especially with limited data.
• Market making: A market maker implicitly uses Bayesian reasoning when adjusting quotes. When large buy orders arrive, the maker updates the probability that informed traders are active (increasing the probability of an upward move) and widens the spread accordingly.
• Machine learning: Naive Bayes classifiers, Bayesian neural networks, and Bayesian optimization are all built on Bayes' theorem and are used in quant research for signal generation and model selection.

Bayesian reasoning is counterintuitive for many people. The most common mistakes are:

• Base rate neglect: Ignoring the prior probability and focusing only on the evidence. In the medical test example above, many people guess 99% instead of 50% because they ignore the 1% base rate. Always start with the base rate.
• Confusing P(A|B) with P(B|A): The probability of rain given dark clouds is not the same as the probability of dark clouds given rain. Bayes' theorem converts between them, but they are fundamentally different quantities.
• Forgetting to normalize: P(B) in the denominator ensures the posterior probabilities sum to 1. It's often computed by expanding P(B) = P(B|A)P(A) + P(B|not A)P(not A).

A useful framework for interview problems: When solving Bayes' theorem problems in interviews, use a probability tree or a frequency table. For the medical test problem, imagine 10,000 people: 100 have the disease (99 test positive), 9,900 are healthy (99 test positive by false alarm). So 99 + 99 = 198 test positive, and 99/198 = 50% actually have the disease. This natural frequency approach is much more intuitive than plugging into the formula directly.