Study notes I'm keeping as I learn to program. The ideas here come from Imran Ahmad's 50 Algorithms Every Programmer Should Know — I'm sharing them in my own words. The plain-language explanations, the examples, and any mistakes are mine.
The concept in plain language
Punchline first: Bayes’ rule is the arithmetic of changing your mind. You start with a belief, you meet a piece of evidence, and the rule tells you exactly how far to move. That’s the whole idea. Everything else is bookkeeping.
Three words carry it. The prior is what you believed before the evidence arrived — “about 15% of the email this address receives is spam,” or “this kind of patient has maybe a 15% chance of the disease.” The evidence is the clue you just observed. The posterior is your updated belief afterward. Bayes’ rule is the recipe that turns prior + evidence into posterior, and it has one non-negotiable insight baked in: how much a clue should move you depends not just on how often it shows up when the answer is yes, but on how often it also shows up when the answer is no. A clue that appears either way tells you nothing, no matter how dramatic it looks.
Now the “naive” part. A real email — or a real patient — presents many clues at once, and in truth the clues are tangled: the word “FREE” and three exclamation points travel together. Modeling all those tangles is combinatorially hopeless. Naive Bayes makes one cheerful simplification: pretend every clue is independent — treat each one as its own separate nudge and just multiply the nudges together. That assumption is almost always false. And here is the small miracle the algorithm is famous for: it works anyway. For classification you don’t need the exact probability, only for the right answer to come out ahead — and the independence shortcut usually preserves the ranking while making the math cheap enough to run on every email you’ll ever receive.
Where you’ve seen it in the real world
The canonical story is spam. In A Plan for Spam (2002), Paul Graham described ditching hand-written filter rules for a statistical approach: score every token by how often it appears in your spam versus your legitimate mail, then, for a new message, combine the fifteen most telling tokens with Bayes’ rule into a single spam probability. What’s worth dwelling on — and what makes this a deep-dive rather than a footnote — is why that beat the rule-writers. First, the filter is personal: it learns your mail, so the word “mortgage” can be damning in my inbox and routine in a loan officer’s. Second, it’s adversary-resistant in an honest way: to evade it, a spammer has to make spam look statistically like the mail you actually receive, which mostly defeats the purpose of the spam. Third, it degrades gracefully: no single token decides; fifteen weak clues vote. Graham reported his filter missing fewer than 5 spams per 1,000 with zero false positives — from math a first-year statistics student can follow.
The same pattern quietly runs elsewhere. When your bank texts you about a suspicious card charge, something Bayes-shaped is combining weak clues — unusual merchant, unusual hour, unusual amount, distance from your last purchase — into one fraud probability. No clue alone proves anything; the product of the nudges is what trips the alert.
The healthcare angle
Here is the part I find genuinely satisfying: this is the algorithm a hospitalist already runs mentally on every shift. Clinical epidemiology just uses different words. Pre-test probability is the prior. The likelihood ratio (LR) of a test is the strength of the nudge. Post-test probability is the posterior. “Pre-test probability × likelihood ratio → post-test probability” is Bayes’ rule — the Fagan nomogram that generations of residents learned from is nothing more than a paper Bayes calculator, published in the New England Journal of Medicine in 1975.
Work one familiar example. A young patient with pleuritic chest pain but a low pre-test probability of pulmonary embolism — say a few percent. A negative d-dimer is a strong negative nudge (a very small likelihood ratio), and a few percent multiplied down lands close to zero: that is the entire logical basis for using d-dimer to rule out PE in low-risk patients, and for not ordering it when the pre-test probability is high, where even a negative result can’t push the posterior low enough to act on. Troponin behaves the same way: the identical number means something different in a 30-year-old with a pulled muscle and an 80-year-old with crushing substernal pressure, because the priors differ. Steven McGee’s Simplifying Likelihood Ratios even gives clinicians the mental shortcut: LRs of 2, 5, and 10 raise the probability by roughly 15, 30, and 45 percentage points. That’s a bedside naive Bayes, run in your head, no computer required.
So when a physician meets the spam filter, nothing new is being learned — the two are the same math wearing different clothes. The filter’s “tokens” are your history, exam, and labs; its spam score is your post-test probability. (As always in this series: the clinical examples are illustrative and educational, not medical advice.)
Why it matters when you work with AI
First, small and auditable beats big and opaque more often than the hype admits. A naive Bayes triage model has parameters you can print on one page: this clue, this weight, learned from these counts. When it errs, you can see which clue misled it and fix the data. Cynthia Rudin’s argument in Nature Machine Intelligence is exactly this: for high-stakes decisions, stop explaining black boxes after the fact and use models that are interpretable to begin with — the accuracy sacrifice is frequently zero. For a great many sorting-and-flagging jobs (which message, which claim, which chart needs a human first?), the tiny probabilistic model you can audit is the better engineering choice, not the budget one.
Second, Bayesian thinking is how you should read any model’s output — including a large language model’s. A confident answer is evidence, not truth: it should update your belief in proportion to how often the system is right on cases like this one, starting from a sensible prior. And never forget the base rate: a 95%-accurate alarm hunting a condition present in 1 patient per 1,000 will be wrong in the overwhelming majority of its alerts, because the prior was tiny before the alarm ever fired. Clinicians internalize this about screening tests; the same discipline applies verbatim to AI alerts, AI diagnoses, and AI-generated citations. The habit this deep-dive is really selling isn’t an algorithm — it’s thinking in posteriors: hold beliefs in percentages, move them in proportion to the strength of the evidence, and be suspicious of any clue (or any model) that claims to settle the question alone.
Sources
- Graham, P., A Plan for Spam (2002) — paulgraham.com
- McGee, S., Simplifying Likelihood Ratios (2002) — Journal of General Internal Medicine (open full text, PMC)
- Fagan, T.J., Nomogram for Bayes theorem (letter, 1975) — New England Journal of Medicine
- Rudin, C., Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (2019) — Nature Machine Intelligence (preprint, arXiv)
— Jeremy Tabernero, MD · More in this series · Get in touch