Foundations: What Makes an Algorithm Work

Study notes I'm keeping as I learn to program. The ideas here come from Imran Ahmad's 50 Algorithms Every Programmer Should Know — I'm sharing them in my own words. The plain-language explanations, the examples, and any mistakes are mine.

The concept in plain language

Punchline first: an algorithm is a recipe, and a recipe you never tested is just a guess with confidence. Everything below is about writing the recipe well and proving it works.

An algorithm is just a finite list of unambiguous steps that turns an input into an output — sort these names, find this patient’s record, route this truck. The steps are the algorithm; the code is one way of writing them down. Before code, we usually write pseudocode — the recipe in plain structured English — precisely so we can check the logic before a single line runs.

The first thing engineers ask about a recipe is how does it scale? That is Big-O notation (the “complexity” of an algorithm) — a shorthand for how the work grows as the input grows. Think of it clinically: a test that takes the same time whether you have 10 patients or 10,000 is O(1), constant. One that doubles its work when you double the patients is O(n), linear. One that compares every patient against every other is O(n²) and will crawl on a big ward. Big-O isn’t about seconds on your laptop; it’s about the shape of the growth curve, and a gentle curve beats a vicious one once the data gets large.

How work grows with input size. The flatter the curve, the better it scales — at large n, O(n²) is hopeless where O(log n) barely moves.

Recipes need containers — data structures — and the container matters as much as the steps:

A list is an ordered shelf you can grow and index into.
A tuple is the same shelf, but glued shut — fixed, unchangeable.
A dictionary (a “hash map”) is a labeled pill organizer: hand it a label (“metformin”), get the value back instantly, no scanning.
A set is a bag that refuses duplicates — perfect for “unique diagnoses seen today.”
A stack is a spring-loaded plate dispenser: last plate in is the first one out.
A queue is the triage line: first in, first out.

Two of the oldest chores are sorting and searching. Sorting methods range from the naive — bubble, insertion, selection sort, simple but O(n²) — to the fast divide-and-conquer methods, merge sort and quicksort, at roughly O(n log n). Searching splits the same way. Scanning every item top to bottom is linear search. Binary search is the doctor’s move: with a sorted list, check the middle, throw away half, repeat — O(log n), absurdly fast. But note the fine print, because it matters later: binary search only works if the input is already sorted. Feed it unsorted data and it confidently returns the wrong answer.

Binary search for 13 in a sorted array. Each row throws away half the remaining cells — 16 → 8 → 4 → 1 in four checks, not sixteen. That's O(log n).

Finally, algorithm design is a small toolbox of strategies: divide and conquer (split the problem, solve the pieces, combine); dynamic programming (break into overlapping subproblems and cache the answers so you never recompute the same thing); greedy (grab the best-looking option at each step and hope local wins add up to a global one); and linear programming (optimize a goal under constraints). Some problems resist all of them — the classic is the Traveling Salesman Problem: find the shortest route visiting every city once. Easy to state, brutally hard to solve exactly, so in the real world we settle for good enough, fast.

Where you’ve seen it in the real world

On August 1, 2012, the trading firm Knight Capital deployed new software to its order router without adequate validation. Per the SEC’s order, in the first 45 minutes of trading the router fired more than 4 million orders trying to fill just 212 customer orders, traded more than 397 million shares, and left Knight with a loss of more than $460 million. The firm nearly died and paid a $12 million settlement. The algorithm wasn’t exotic — it was an unvalidated deployment, the whole lesson of this section in one headline.

Now the happier side. Every time you sort a list in Python, or your Android phone orders your contacts, a hybrid algorithm called Timsort — a marriage of merge sort and insertion sort — is doing the work. Tim Peters wrote it for CPython in 2002, so it has been Python’s default sort for two decades, and it was later adopted to sort object arrays in Java SE 7 and on Android. And here is the twist that proves validation never ends: in 2015 a team of European researchers trying to formally prove Timsort correct instead found a real bug — a crash (an array-index-out-of-bounds error) on certain large inputs, in code that had shipped inside billions of devices. A widely trusted algorithm still hid a defect until someone checked it rigorously; the fix landed in Python, Java, and Android that same year.

Binary search is everywhere once you know its shape. When a bug creeps into a codebase, the git bisect command finds the exact commit that broke things by binary-searching your history — mark one “good” and one “bad” point, and it halves the suspects at each step.

And optimization pays rent daily. UPS’s route engine, ORION, crunches package and route data to shorten deliveries; per UPS and INFORMS, at full deployment it is expected to save about 100 million miles and 10 million gallons of fuel a year, for a projected $300–400 million in annual savings — the Traveling-Salesman family, tamed just enough to matter.

The healthcare angle

The sharpest lesson lands in our own hospitals. The Epic Sepsis Model is a proprietary early-warning algorithm built into one of the country’s most widely used electronic health records. Epic’s internal materials cited an accuracy (AUC, where 1.0 is perfect and 0.5 is a coin flip) of 0.76 to 0.83. Then Wong and colleagues did the thing Knight Capital didn’t — they externally validated it on real patients. Published in JAMA Internal Medicine in 2021 across 27,697 patients and 38,455 hospitalizations, the model scored an AUC of just 0.63 — and at its alerting threshold it missed 67% of the patients who actually had sepsis (1,709 of 2,552) while flooding clinicians with alerts. The math wasn’t fraudulent — it had simply never been proven on the population it was deployed against. Validation isn’t a formality; it’s the difference between a tool and a liability.

Dynamic programming shows up on the bench, too. The Needleman–Wunsch algorithm, published in 1970 in the Journal of Molecular Biology by Saul Needleman and Christian Wunsch, was one of the first applications of dynamic programming in biology — it aligns two DNA or protein sequences optimally by building a grid of best partial alignments and reusing them, and it still underlies how we compare a patient’s variant against a reference genome.

Why it matters when you work with AI

If you take one habit from these four chapters into your AI work, make it this: validate the algorithm on your data, not the vendor’s brochure. Every AI model you’re offered comes with a number measured on someone else’s patients; your job is to demand the external validation — the AUC, the sensitivity, the miss rate — measured on people like yours.

Three foundations transfer directly. First, complexity is destiny at scale: a model that’s fine on a pilot ward can choke on a health system, so ask how the cost grows, not just how it runs today. Second, garbage in, wrong out — confidently: binary search returns nonsense on unsorted input without complaining, and an AI model does the same with biased or mismatched data. Third, correct-looking is not correct: Timsort was trusted for years before a proof exposed the bug. The discipline that separates a $460-million mistake from a working system isn’t cleverness — it’s the boring, non-negotiable step of proving the thing works before you trust it with a life.

Sources

About this series. These are study notes I'm keeping as I learn to program — I'm sharing the ideas, not claiming them. The concepts belong to 50 Algorithms Every Programmer Should Know, 2nd edition, by Imran Ahmad, Ph.D. (Packt Publishing) and its companion code repository, github.com/cloudanum/50Algorithms; the plain-language explanations, the real-world examples, and any mistakes are mine. No text is reproduced from the book, and this series is not affiliated with or endorsed by the author or Packt. If these ideas are useful to you, buy the book — it goes far deeper, with working Python for every algorithm. Where a case study leans on a third-party explainer (for example, Jay Alammar's The Illustrated Transformer or Chris Olah's Understanding LSTM Networks), it is credited in that page's Sources.

— Jeremy Tabernero, MD · More in this series · Get in touch