Latent Space: The Map of Meaning

Study notes I'm keeping as I learn to program. The ideas here come from Imran Ahmad's 50 Algorithms Every Programmer Should Know — I'm sharing them in my own words. The plain-language explanations, the examples, and any mistakes are mine.

The concept in plain language

Punchline first: a modern AI model doesn’t store words — it stores addresses. Every word, sentence, image, or patient record the model handles gets converted into a long list of numbers, and that list is a location in a huge abstract space. Things that mean similar things get nearby addresses. That space is called latent space (latent because nobody designed it — it emerges from training), and one thing’s address in it is called its embedding.

The mental model is a map. On a city map, two restaurants that are close together are close in geography. In latent space, two points that are close together are close in meaning: “chest pain” and “angina” land near each other, “chest pain” and “parking invoice” land far apart. Distance on this map is something you can compute — usually the angle between the two number-lists, called cosine similarity — so “how alike are these two things?” stops being a judgment call and becomes arithmetic.

Two more ideas complete the picture. First, arithmetic with meaning. The word2vec paper (Mikolov and colleagues, 2013) showed that directions in this space can carry concepts: take the point for king, subtract man, add woman, and the nearest point is roughly queen. Honest caveat: that famous example is the show pony — it’s the one that works beautifully, many analogies come out mushier — but the underlying finding is real and it changed the field: meaning has geometry.

Second, the flattening lens. Real embedding spaces have hundreds or thousands of dimensions; humans can look at two or three. Tools like t-SNE (van der Maaten & Hinton, 2008) and UMAP (McInnes and colleagues, 2018) squash the space down to a 2-D picture that tries to keep neighbors as neighbors. The gotcha, spelled out in Distill’s How to Use t-SNE Effectively: these pictures preserve who is near whom, but cluster sizes and the distances between clusters are largely artifacts — two blobs looking far apart, or one blob looking tighter than another, is not evidence of anything. Read the picture like a subway map, not a satellite photo: connections are real, geography is stylized.

Where you’ve seen it in the real world

Embeddings are why search stopped needing the magic words. Classic search matched the letters you typed; semantic search embeds your query and returns the nearest neighbors in meaning — ask “why does my chest hurt when I climb stairs” and documents about exertional angina surface, even if they never use your phrasing. The same trick drives recommendations (“people who liked this sit near you in taste-space”) and RAG — retrieval-augmented generation — where an assistant embeds your question, fetches the closest passages from a document store, and answers from those. When a chatbot “looks something up,” latent-space nearest-neighbor search is almost always the looking.

This very lab runs on it too. Our marketing pipeline keeps an anchor: the embedding of a body of approved, on-brand writing. Every new draft gets embedded and measured against that anchor, and copy whose cosine distance drifts too far — hype-flavored text that doesn’t sound like us — is rejected before a human ever sees it. It’s a spell-checker for voice instead of spelling, and it’s nothing more exotic than “how far is this point from home base?”

The healthcare angle

The clinical version of “words that behave alike sit together” is patients who behave alike sit together. Embed each patient — labs, vitals, medications, diagnoses — as a point, and clusters become candidate phenotypes: subtypes nobody defined in advance. The flagship example is the sepsis work by Seymour and colleagues in JAMA (2019), which clustered 63,858 patients and found four reproducible sepsis phenotypes (α, β, γ, δ) with different organ-dysfunction patterns and different mortality — and, provocatively, re-analyses where the phenotype mix changed whether a trial’s treatment looked helpful. Patient-similarity search points the same direction: “show me patients whose trajectory looks like this one’s” is a nearest-neighbor query in a latent space.

The discipline to carry in: clusters are hypotheses, not diagnoses. An algorithm will always hand you clusters — that’s its job — whether or not they carve nature at the joints. The sepsis phenotypes earn attention because they were validated across cohorts and tied to outcomes, not because a colorful UMAP plot contained blobs. A cluster is where the science starts, never where it ends. (Illustrative and educational, as always — not medical advice.)

Why it matters when you work with AI

First, embeddings are how AI systems actually remember and retrieve. Behind nearly every “the assistant searched your documents” feature is the same loop: embed everything once, embed the query, return the nearest neighbors. If you understand that one loop, RAG, semantic search, recommendation, and agent “memory” stop being separate mysteries — they’re all the same map lookup.

Second, the distance threshold is a judgment call, and someone has to own it. How close must a retrieved passage be to count as relevant? How far can a draft drift before our brand gate rejects it? Nothing in the math answers that; a human picks the number, watches what it lets through and what it wrongly blocks, and tunes. It is a sensitivity/specificity trade-off — a dial every clinician already knows how to reason about.

Third, latent space is where bias hides. The map is learned from human text, so it inherits human associations: Bolukbasi and colleagues showed in 2016 that word embeddings trained on news text placed man near computer programmer and woman near homemaker, and proposed ways to measure and reduce such bias. A patient-similarity space learned from historical care data can encode historical inequities in exactly the same silent way. The geometry is powerful precisely because it absorbs the data’s structure — including the structure we’d rather it hadn’t learned. Audit the map before you navigate by it.

Sources

1. Mikolov, T. et al., Efficient Estimation of Word Representations in Vector Space (2013) — arXiv (word2vec)
2. van der Maaten, L. & Hinton, G., Visualizing Data using t-SNE (2008) — Journal of Machine Learning Research
3. McInnes, L., Healy, J. & Melville, J., UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2018) — arXiv
4. Wattenberg, M., Viégas, F. & Johnson, I., How to Use t-SNE Effectively (2016) — Distill
5. Seymour, C.W. et al., Derivation, Validation, and Potential Treatment Implications of Novel Clinical Phenotypes for Sepsis (2019) — JAMA (open full text, PMC)
6. Bolukbasi, T. et al., Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (2016) — arXiv

— Jeremy Tabernero, MD · More in this series · Get in touch