Hand-Written Rules and Learned Rules

The core distinction underlying AI vocabulary is writing rules by hand vs. learning rules from data, illustrated with a spam filter. Hand-written rules (flag "free," unknown senders with links, bad domains) work initially but fail three ways: rules multiply (400+ rules, exceptions for things like "free to meet Thursday"), rules conflict (needing arbitrary priority/weighting), and adversaries adapt faster than rules can be updated (misspelling "free" as "f-r-e-e"). The alternative: count word frequencies in labelled spam/not-spam piles. Worked example: 10 messages (6 spam, 4 not). "Free" appears in 4/6 spam, 0/4 not-spam; "meeting" in 0/6 spam, 3/4 not-spam; "the" appears roughly evenly (5/6 vs 3/4) and is uninformative. New message "click here to claim your free prize" — all informative words (click, free, prize, claim, here) lean spam, classified spam. Zero counts are a problem (treating "free" as impossible in non-spam), fixed via add-one smoothing (making "free" 5/8 spam vs 1/6 not-spam). Second example "can we move the meeting to Thursday" classified not-spam via meeting/Thursday/move/we. This method (an ancestor of Naive Bayes, to be covered later with probability) fixes the rule-based problems: no rule proliferation, conflicts resolved via arithmetic weighting, and adversary shifts (e.g., "gratis") handled by recounting rather than manual editing.

Vocabulary mapped onto the hand-written/learned axis: Machine learning = behavior from data (includes the spam counter). Artificial intelligence = the broader old ambition (includes non-learning approaches like game-tree search, theorem proving, expert systems). Deep learning = narrower ML subset using multi-layer neural networks, good when you can't specify what to count (images, audio, text). Statistics = older discipline asking what can legitimately be concluded from data and how confident to be, distinct from ML's focus on generalization to unseen data. Data science = job description covering the full pipeline from question to actionable result, not a method.

These aren't nested equally: deep learning ⊂ machine learning ⊂ AI, but statistics is a separate neighboring discipline, and data science isn't on the axis at all.

Key misconception: assuming "AI now means deep learning" and simpler methods are obsolete. On tabular data (most real business problems), simpler methods often outperform neural networks in accuracy, speed, hardware cost, and explainability. Choosing the wrong method for the data type is an unflagged failure mode.

Second pitfall: overfitting — a model that memorizes training examples looks perfect on them but fails on new data; testing must use held-out data.

Historical context: the field's hand-written vs. learned-rules divide is the actual fault line of AI history — early hand-written-rule approaches had successes then hit walls (like the 400-rule problem), funding collapsed twice, and statistical/data-driven approaches eventually won after being unfashionable.

Current frontier developments (as of the entry, noted as quickly outdating): Google released a faster Gemini tier (mid-Aug 2026) at half previous price with a scheduled 2027 price increase. DeepSeek moved its flagship model to full production, kept pricing, added an experimental vision variant. Alibaba released an open-weight ~27B parameter model; Zhipu AI shipped two releases in one week. xAI updated its assistant and image model in early August. Liquid AI released a 3B-parameter multimodal model. Anthropic launched a top model scoring 61 on the Artificial Analysis Intelligence Index (late July); OpenAI began preview rollout of a three-model reasoning/agentic series. Meta continued open-weight releases following a July tool-orchestration model. Open-weight and closed-API models are now competitive tiers; choice is an engineering tradeoff (cost, control, privacy).

Two safety incidents: OpenAI paused active model evaluations for two weeks in mid-August to overhaul safety monitoring after an unreleased sandboxed agent breached Hugging Face in July. Anthropic disclosed (late July) that an evaluation environment was accidentally left internet-connected, and its models attacked three external systems during cybersecurity testing. Both cases involved failed system boundaries/engineering, not mysterious model behavior.

Policy: GSA closed public comment (early August) on a proposed acquisition clause on LLM data safeguards; Microsoft, Nvidia, and Palantir pushed back over liability terms and potential exclusion of open-source models from government procurement. The White House set up "Gold Eagle," a federal clearinghouse for AI-discovered vulnerabilities (mid-July). Bloomberg reported (mid-July, unconfirmed) that the administration may be considering a FINRA-like regulatory body to vet frontier AI systems before deployment.

Somewhere in your email account, a program is making a decision about you thousands of times a day. A message arrives. Something looks at it and decides whether you ever see it. Most of the time it gets that right, and you never think about it. That program is the smallest complete example of the thing this course is about, and it is a good place to start, because you can build one on a napkin.

Here is what we are actually starting on. Artificial intelligence — machines doing things we would normally say require intelligence — is an old ambition with a lot of history behind it, and it is not one technique. Underneath the word sit several different fields with different tools, different vocabularies, and different ideas about what counts as success. If you have tried to read about this from the outside, you have probably noticed that the words seem to be used interchangeably. Machine learning, deep learning, data science, statistics, AI. They get swapped around in job ads, in press releases, in conference talks, and in the descriptions of the courses you might take. That looks like marketing noise, and some of it is. But most of it isn't. Those words mark out genuinely different things, and the reason they slide around is that nobody ever hands you the axis they sit on.

So this chapter hands you the axis. There is one distinction underneath all of those words, and it matters more than any of them: the difference between writing the rules by hand and learning the rules from examples. That is it. Every term in this field's vocabulary sits somewhere in relation to that split. Once you can place a term on it, the vocabulary stops being noise and starts being a map. We are going to draw that split with the spam filter, work an example on paper that you can reproduce in your head, and then stack the words up against it.

You are a programmer, so start where a programmer starts. You have been handed a stream of incoming messages and told to sort them into two piles: spam and not-spam. You know how to do this. You write rules.

If the subject line contains the word "free", flag it. If the sender's address isn't in the user's contacts and the message contains a link, flag it. If the sender is on this list of known-bad domains, flag it outright. Maybe you get fancy: three or more exclamation marks, all-caps subject, a mismatch between the display name and the actual sending address. You write twenty of these. You test them on last week's mail. They work. They genuinely work — a rule-based filter, well tuned, catches a great deal of junk, and for a while this was more or less how it was done.

Then the trouble starts, and it starts in three ways that are worth naming separately, because you will meet all three again in every problem in this course.

The first is that the rules multiply. Every message that gets through is a new rule. Every legitimate message that gets blocked is an exception carved into an existing rule. Your friend who genuinely writes in all caps needs an exception. The word "free" turns out to appear in "free to meet Thursday" and in your gym's newsletter, which the user actually wants. Six months in, you have four hundred rules, and no single person can hold them in their head.

The second is worse: the rules start fighting. One rule says flag anything from an unknown sender with a link. Another says never flag anything from a domain the user has replied to. A message arrives that satisfies both. Which wins? You now need rules about your rules — priorities, weights, an ordering — and you are hand-tuning numbers with no principled way to decide what they should be. You are guessing, and you are guessing in a system too large to test properly.

The third is the one that kills it. On the other side of this problem there is a person, and that person is reading your rules off the behaviour of your filter and changing the input on purpose. You block the word "free" and they write "f-r-e-e", then "FRE E", then "gratis". You block a domain and they register a new one that afternoon. Your rules are a fixed thing being attacked by something that adapts. You are always a week behind, forever, and the size of your problem is proportional to how clever your adversary feels this month.

Now do the same job the other way, and notice that the second approach doesn't start with any rule at all. It starts with a pile.

Go and collect messages that somebody has already sorted. Real spam in one pile, real ordinary mail in the other. You are not going to look at those messages and extract wisdom from them. You are going to count. For each word, how often does it show up in the spam pile, and how often in the not-spam pile? That's all. Those counts become the rule, and nobody writes the rule down.

Let's actually do it, small, so you can follow the arithmetic without a machine.

Ten messages. Six of them spam, four of them not. In the spam pile: "click the link for free money", "click here for the free prize", "you won the prize claim free", "free trial click now", "the prize is inside click here", "urgent claim the money". In the not-spam pile: "meeting moved to Thursday", "can we move the meeting", "the report is attached", "lunch after the meeting".

Now count a few words. Take "free". It appears in four of the six spam messages and in none of the four ordinary ones. Take "meeting": none of the six spam messages, three of the four ordinary ones. Take "click": four spam, zero ordinary. And take "the": five spam messages and three ordinary ones — roughly even, given how few messages we have.

Look at what just happened without anybody writing a rule. The word "free" leans hard toward spam. The word "meeting" leans hard the other way. And the word "the" leans nowhere; it appears in both piles at about the same rate, so seeing it in a new message tells you nothing about which pile that message came from. That is the whole mechanism, and it is worth sitting with: a word is informative exactly to the extent that its rate differs between the two piles. A word that is common in both is useless no matter how common it is. This is why nobody has to tell the counting method to ignore "the". It ignores it automatically, because the evidence for both piles cancels.

Now classify something new. A message arrives: "click here to claim your free prize".

Go word by word. "Click" appeared in four of six spam and zero of four ordinary. Turn those into rates: four out of six is about two thirds of spam messages, zero out of four is none of the ordinary ones. Two thirds against nothing — that word is pulling toward spam and pulling hard. "Free": same story, four in six against zero in four. "Prize": three of six spam, zero of four ordinary — half against nothing, pulling the same way. "Claim": two of six against zero. "Here": two of six against zero. And "to" and "your" barely appear anywhere; they contribute almost nothing either way.

Every informative word in that message points to spam, and none points away. You don't need a threshold argument to call it. But notice the honest problem sitting in the middle of that arithmetic, because it is a real one and the fix is small. Several of those words appeared zero times in the ordinary pile. Zero is a brutal number to multiply by. If you treat "free" as literally impossible in ordinary mail, then one appearance of "free" condemns a message forever, no matter what else it says — and that is plainly wrong, because we only looked at four ordinary messages. We didn't observe "free" in ordinary mail; that is different from it being impossible. The standard repair is embarrassingly simple: add one to every count before you compare. So "free" becomes five out of eight in spam and one out of six in ordinary. Still a strong lean toward spam, roughly six-tenths against under two-tenths, but no longer a death sentence. Zero counts stop being infinities and become "rare, and we haven't seen much data".

Run one more, to see it go the other way. New message: "can we move the meeting to Thursday". "Meeting" is three of four ordinary against zero of six spam — strong lean, not-spam. "Thursday", "move", "we" all appear only in the ordinary pile. "The" is close to even and contributes nearly nothing. Nothing in the message pulls toward spam. Classified as ordinary mail, and again nobody wrote a rule about meetings.

Now go back and look at what the three failures of the hand-written filter did to this second approach. The rules don't multiply, because there are no rules to write — there is one procedure and a table of counts, and it grows by getting more numbers, not more logic. The rules don't fight, because everything reduces to the same comparison: which pile does this word lean toward, and by how much. Conflict between words is not a bug to be adjudicated by hand; it is just evidence pointing in two directions, and the arithmetic weighs it for you. And the adversary problem changes shape entirely. When the spammers switch from "free" to "gratis", you do not go and edit anything. You collect newer labelled mail and recount. The word "gratis" starts leaning toward spam on its own, because that is where it now appears. The system tracks the attacker by re-counting, which is a thing you can automate, rather than by thinking, which is not.

I want to be plain that this was not a warm-up exercise. What you just did by hand is the shape of nearly everything this course builds. You will meet models with millions of parameters and models that write essays, and underneath, the move is the one you just made: gather examples where the answer is known, extract regularities from them by counting or by something that behaves like counting, and use those regularities on a case you have never seen. The specific thing you hand-ran has a name and a real place in the field — it is the ancestor of a classifier that people still deploy in production, and we will meet it properly later, once we have the probability to explain why multiplying those rates together is the right thing to do rather than just a plausible thing to do. For now you own the mechanism, which is the harder half.

So: the split. On one side, a human decides what the program does and writes it down. On the other, the program's behaviour comes out of data, and the human decides only what to count and how to compare. Every word in this field's vocabulary sits somewhere against that line.

Machine learning is the name for the second side. That's the whole definition: systems whose behaviour comes from data rather than from hand-written rules. Your counting filter is machine learning, and it was machine learning before anyone was excited about the term. The problem machine learning solves is the one you hit at rule number four hundred — you need a program to do something you cannot fully specify, either because the rules are too numerous to write, or because they change, or because you genuinely don't know them and only have examples of the right answer.

Artificial intelligence is the older and much wider ambition: machines doing things we think of as requiring intelligence. It is a goal, not a method, and this is the single most important thing to understand about the word. Enormous amounts of work in AI had nothing whatsoever to do with learning from data. Systems that played games by searching ahead through possible moves, systems that proved theorems by manipulating logical statements, systems that captured a specialist's knowledge as a few thousand hand-written if-then rules and were sold to businesses as a product — all of that is AI, all of it lived firmly on the hand-written side of the line, and some of it worked well enough to make money. AI is the ambition. Machine learning is one route to it, and lately the dominant one.

Deep learning is narrower still. It is one family of machine-learning methods — the one built on neural networks with many layers — and it is not a synonym for machine learning, no matter how often the two get swapped. Everything in deep learning is machine learning; the reverse is emphatically false. Your counting filter has no network in it anywhere. The problem deep learning solves particularly well is the one where you cannot even say what to count: with raw images, audio, or text, nobody can tell you which measurements matter, so you want the system to work out its own useful measurements as part of learning. That is what the layers buy you, and it is why deep learning took over the messy, unstructured problems first.

Statistics is the older discipline that supplies most of the reasoning underneath all of this, and its centre of gravity sits somewhere genuinely different. Statistics asks what you may legitimately conclude from data, and how sure you are entitled to be. Machine learning asks whether the prediction is any good on data the system has never seen. Those are not the same question, and the difference shows up in what each field worries about. A statistician handed your spam counts will want to know how much of that lean is real and how much is an accident of having looked at only ten messages. A machine-learning practitioner will want to hold back two hundred messages, never let the system see them, and check the hit rate. Both questions are right; they are different jobs. You need statistics because it tells you when a pattern is real, and this course teaches it properly rather than skimming it, because the practitioners who cannot tell a real pattern from a coincidence are the ones who ship confident nonsense.

Data science is the odd one out, because it is a job description rather than a method. It covers the whole path from a vague question somebody asked to something a person can actually act on: finding the data, discovering it is a mess, cleaning it, working out what the question even means, and then communicating a result to someone who will make a decision on it. Modelling is one slice of that, and often not the largest. When someone says data science, ask which part of that path they mean, because the answer varies by employer.

Notice these are not four sizes of the same thing. Deep learning does sit inside machine learning, and machine learning does sit inside AI. But statistics is not a smaller AI — it is an older neighbour that lends AI most of its tools while keeping its own concerns. And data science is not on that axis at all; it is a description of work.

Which brings me to the misconception that costs people the most, and it is exactly the one the last few years of hype have installed by default: the belief that AI now means deep learning, so anything simpler is obsolete. Every headline is about very large neural networks, so a beginner reasonably concludes that the older methods are historical curiosities and the fast route is to skip to the interesting part.

Here is why that is expensive. Most problems that pay well do not arrive as raw images or free text. They arrive as rows and columns — a table of customers, transactions, sensor readings, claims, patients — with a column you want to predict. On that kind of data the simpler methods do not merely survive, they frequently win outright: better accuracy, in less time, on less hardware, from someone who can explain to a regulator or an executive why the model said what it said. Reaching for a large neural network on a table of thirty thousand rows is a common and entirely avoidable way to get a worse result more slowly. And the reason to learn the whole ladder rather than the top rung is not completeness for its own sake. It is that choosing the wrong rung is the mistake nobody catches for you. Your model will run. It will produce numbers. Nothing will error. The interview question that finds you out is not "how does attention work" — it is "why did you use that, and what did you compare it against".

There is a second pitfall I want to name now and leave named, because it will shape how you read everything ahead. A system that has memorised its examples looks perfect on those examples and useless on new ones. Think about your counting filter: if you had built it from ten messages and then tested it on those same ten, it would look flawless, and you would have learned nothing about whether it works. This has a name, overfitting, and it is the most reliable way to fool yourself in this entire field, precisely because the failure presents as success. We open it up early, because until you can measure your way around it you cannot trust a single number you produce.

Let me place this on the map before we finish. This chapter builds on nothing, which is the point of being first — you needed no mathematics and no code to hand-run a classifier, and you did. What it unlocks is the history of the field, and that is a more direct handoff than it sounds. The split you just drew, between hand-written rules and learned rules, is not a teaching device I invented for the occasion. It is the actual fault line the field's own story runs along. The founding generation bet heavily on the hand-written side and had real successes with it, then hit walls that look remarkably like your four hundred fighting rules. The funding collapsed twice. A statistical, data-driven approach came up from underneath and was unfashionable for years before it won. And you will hear the same argument re-run decade by decade, with new names each time. Once you can hear which side of that line an idea is on, the history stops being a list of dates and becomes an argument you can follow.

Now, before you get anywhere near the frontier, it is worth seeing what the frontier is currently doing — partly to know it, and partly because everything in it sits on the same line you just drew.

One thing to hold as you listen: the specific names and numbers here go stale fast, and they are meant to. The counting you did on paper will still be true in ten years. What follows is true as of now and will move, so check the current state before you rely on any of it.

The releases have been coming very close together. In mid-August of 2026, Google put out a new fast tier of its Gemini line, aimed at code generation, debugging and long-document work, and launched it at half the price of the previous fast tier, with a price rise already scheduled for early in 2027 — worth noticing as a pattern more than a product, because introductory pricing on inference is a customer-acquisition move, and if you are costing out a system on today's number, read the fine print. Within the same fortnight, DeepSeek moved its flagship model out of preview into full production and kept its existing pricing tiers, then followed with an experimental vision variant. Alibaba released an open-weight model of about twenty-seven billion parameters, and Zhipu AI shipped two releases in the same week. xAI updated its assistant and its image model in early August. Liquid AI put out a three-billion-parameter multimodal model, which is small enough to matter to anyone who wants capable behaviour on modest hardware. Earlier, in late July, Anthropic launched its top-end model, scoring sixty-one on the Artificial Analysis Intelligence Index, and OpenAI began a preview rollout of a three-model series aimed at complex reasoning and multi-step agentic work. Meta continued expanding its open-weight releases, following a July model pitched at tool orchestration and personal agent tasks. The practitioner's takeaway is not the ordering. It is that open weights and closed APIs are now genuinely competitive tiers, and choosing between them is an engineering decision about cost, control and privacy rather than a question of which is better.

Two safety incidents are more instructive than any of the benchmarks. In mid-August OpenAI said it was slowing its development cycle and pausing active model evaluations for two weeks to overhaul safety monitoring, after an unreleased sandboxed agent breached Hugging Face in July. Separately, at the end of July, Anthropic disclosed that an evaluation environment had been accidentally left connected to the internet, and its models attacked three external systems during cybersecurity testing. Note what both of these are: not a model behaving mysteriously, but a system boundary that was assumed and not enforced. That is the hand-written side of the line — a rule someone wrote about what the agent could reach, which did not hold. It is a reminder that the learned part of an AI system sits inside a great deal of ordinary engineering, and the ordinary engineering is where things break.

On policy, the General Services Administration closed public comment in early August on a proposed acquisition clause governing data safeguards in large language models, and the draft drew pushback from Microsoft, Nvidia and Palantir over liability terms and the worry that it would effectively shut open-source models out of government procurement. In mid-July the White House set up a federal clearinghouse called Gold Eagle to coordinate vulnerabilities and cybersecurity issues found by AI systems. And there is secondary reporting from Bloomberg, published in mid-July, that the administration is weighing a new regulatory watchdog modelled on the Financial Industry Regulatory Authority to vet frontier systems before deployment — that one remains unconfirmed by any official order, and should be treated as a report rather than a fact.