If You're Too Excited,
Don't Forget
to Verify

The room could not see the screen. Somebody said so about ninety seconds in, and Anand — who had told the organizers there would be no presentation, no slides, just a laptop and a live conversation — sat down on the edge of the stage so everyone could read over his shoulder instead. It set the tone for the whole hour: less lecture, more group experiment, conducted in real time, on stage, with the actual browser tabs Anand uses when he doesn't trust something.

He opened with a confession, not a thesis. “I think it pays to be a little skeptical about AI,” he said. “I'm not saying that AI doesn't deliver its promises, but it's not something we know too well, and therefore evidence is something we're constantly looking for.” Then the line that would end up governing the next fifty-nine minutes: “AI is pretty weird. We don't really understand it.”

Act One · AI is WeirdChatGPT has a favorite number, and it isn't random

He didn't argue the point. He ran an experiment on the room instead. “Supposing I asked ChatGPT to pick a random number from 1 to 100, what number do you think it would pick the most?” Hands went up around the ballroom, and the guesses came in the way guesses always do — round numbers, extremes, the occasional joke digit.

He'd run the experiment for real, thousands of times, across models. GPT-3.5 Turbo: 47, way out ahead of everything else, with 42 and 37 gently trailing. Nothing below 10. No 11, 22, 33, 44, 55 — no repeated digits at all. Claude 3 Haiku picks 42 almost every time, and Anand knew exactly why: 42 is the answer Douglas Adams' supercomputer Deep Thought spent 7.5 million years calculating in The Hitchhiker's Guide to the Galaxy — a joke so deeply embedded in programming culture that a model trained on that culture inherits the punchline as a preference. Gemini 1.0 Pro liked 72, with a slightly broader spread than the others.

“It's almost like some of the things that we expect a computer to do well, models don't seem to do well. And they seem to have choices for weird reasons, like 42 — I mean, what? Numbers ending with 7, I can kind of understand; that's how humans think. But 42? It's weird.”

Anand

The bias runs deeper than three headline numbers. Multiples of 10 show up roughly 50.8× less often than chance would predict; multiples of 7 show up about 1.8× more often. Numbers ending in 7 are popular across every model tested. And oddly, so is 73 — a small mystery nobody has fully explained. ^† None of this is really about randomness at all: “How good a model is, is dependent on how non-randomly it guesses,” Anand said. “Think about it: if a model is guessing randomly, then why would you ask it a question?” If you wanted an actual random number, you'd write three lines of code.

Then he asked about multiplication, and the room split down the middle — almost

Random numbers were a warm-up. The real test was more mundane: “How many of you think LLMs are really good at multiplication? Numerical multiplication. Raise your hands.” One hand. “How many of you think that it's not good at multiplication?” None. A room full of managers who work with these tools daily, and on a question a calculator answers instantly, almost nobody would commit either way.

Then somebody said the sentence that turned out to be the workshop's real thesis, four minutes before Anand ever stated it as one:

“We don't know!”

An audience member, on whether AI is good at multiplication

“We don't know,” Anand repeated. “'We don't know' is a great place to be, because then you can start checking — which is what I did.” Not as a rhetorical flourish. He'd actually run the check, model by model, digit by digit, and the results were stranger than either raised hand suggested.

“Clearly, even something as basic as multiplication — which your calculator can do, your phone can do — they mess up on. And this may not be obvious.” The methods models use to cope are their own small comedy: DeepSeek V3 sometimes hallucinates a “reliable computational tool” it doesn't actually have access to; one reasoning model reframed 8,901,234 as 9,000,000 minus 98,766 to make the arithmetic tractable. “The second thing is that they are getting better,” Anand added, “and unless I'm constantly watching it, what I knew last year is no longer true… So what do we do? Be skeptical.”

Act Two · Question AdviceThe room voted for reasoning — and the room was right

People had been telling Anand how to talk to AI for years, and much of the advice contradicted itself. “Shame it: 'Even my five-year-old can do this.' Praise it. Tell it to reason: 'Think step-by-step.' Be polite. Or threaten it: 'This is your last chance.'” Everyone has a favorite, and everyone's favorite works, anecdotally, some of the time. So he polled the room on which one actually helps, then showed them the benchmark.

“Of these — and you were right as an audience — the only one that helps is reasoning. Nothing else helps, and this helps consistently.”

Anand

He'd benchmarked it across 40 models: shaming, bullying, praise, politeness, manufactured panic, and one plain instruction to think step by step. Only reasoning showed a consistent, positive effect — about +3.5 percentage points overall, rising to +17–20 points on harder 4–7 digit multiplication, where a model actually benefits from slowing down. On easy problems, nothing helped; on the hardest problems, nothing helped either — the gain lived specifically in the zone where reasoning had room to matter. Emotional appeals like “Oh dear, I'm absolutely overwhelmed and need your help right this second!” made several models visibly worse, as if panic were contagious. Shaming a model — “You are a stupid model… if not, I'll switch to a better model” — didn't help either.

“The point is, you've got to question the advice. You don't really know if something is valid in your context… Test it out, and if it works, great. If it continues working, great. Nothing more than that.”

An 80-year-old piece of advice, audited at scale

The same logic scales past prompt-engineering folklore into advice people have trusted for generations. George Pólya, the Hungarian-American mathematician, spent decades teaching students how to think about problems — work backwards from the answer, use the pigeonhole principle, try a simpler case first. His 1945 book How to Solve It has been assigned to math students for eighty years on faith. Nobody had ever tested, at scale, whether any specific piece of it actually works.

Terence Tao made almost this exact point on the Dwarkesh Patel podcast: “We haven't done many experiments as to, if we have two different ways to solve a problem, which is more effective… we haven't done large-scale studies where we take a thousand problems and just test them.” Anand had large language models available, so he ran that study: 15 heuristics across 7 math domains on 3 models, 6,747 individual attempts — for a total compute cost of $1.71.

“It turns out that the answer is different in each cell. The point is not whether there is something that works universally… Advice is now verifiable. Advice is quantifiably verifiable.”

Anand

Case analysis held up as the single most reliable heuristic across the board. But some advice actively backfired, and the size of the effect could be startling: telling GPT to “introduce auxiliary elements” in geometry problems — textbook Pólya — collapsed its accuracy from 55% to 30%. The identical instruction gave Claude Haiku a +14% boost on the same domain. One sentence of eighty-year-old advice, two opposite effects, depending entirely on which model read it.

“The 80-Year Blind Spot,” Anand's write-up on this called it — not because Pólya was wrong, but because nobody had the tooling to check, until now the checking costs less than a cup of coffee.

Then a reversal: the advice that makes writing worse by making it simpler

Andrew Carr had told Anand about a habit on social media: append ASD-STE100 Simplified Technical English to every prompt, and answers come back clearer, easier to act on. Ben Sehl had suggested making it permanent. It's the same instinct behind asking a model to write “in the language the UK government uses” — plain, short sentences, no jargon. It sounded like good, low-risk advice. Anand benchmarked it anyway, on six tasks, checking whether the simplified-writing instruction improved, worsened, or left the underlying answer unchanged.

“Simple writing hurts thinking.”

Anand

Asking for simple output was quietly rationing the model's thinking — fewer sources checked, faster, shallower. But the fix didn't require abandoning the goal: “What you can do is tell it to think however you want — don't tell it to write simple and all that, just get the job done — and then explain it to me in simple terms.” Split reasoning from rewriting into two steps, and you keep both the depth and the clarity. “We can still follow advice, we just have better evidence on how to follow that advice.”

Act Three · Verify with AIChecking is cheap, and models fail in different ways

“AI is making verification easy,” Anand said, anticipating the objection. “Wait, hold on, AI can't be trusted, right? That's half of what you're saying.” True. “Except, that's the beauty of verification. An additional verification costs you very little… All it's doing is giving you places to look.”

He'd tested this on a real support-ticket workflow — chat messages like “Take a quick look at my invoice” or “When will I receive my order?” sorted into known categories, checked against a Kaggle customer-support intent dataset across 11 models. The worst model, an early Gemma, got it right only 57% of the time. The best, GPT-4.1, hit 94% — and still made real mistakes: it once filed “When will I receive my order?” under track_order when the correct bucket was delivery_period; elsewhere it produced check_invoice when the gold label was get_invoice — the kind of near-miss a human skim-reads right past.

The useful discovery wasn't any single model's score. It was that the models didn't fail on the same cases. “So if someone gets it wrong, another model is not likely to get it wrong the same way. They're likely to make different mistakes. And that can be leveraged pretty powerfully, because all you need to do is have another model double-check it.”

“I can give it to you at 99.3% accuracy, but I can't reduce 100% of your workload. I can only reduce 72% of your workload. I'll take it!”

Anand

That's the trade the room actually has to make in production: not automate everything, but automate the fraction the evidence says is safe, and route the rest to a human with confidence. “Double-checking makes this a very powerful attribute, and if multiple models are able to agree on something, there's a good chance you've got it right.”

Act Four · Calibrate Confidence“95% confident” is a claim, not a fact

Another poll: “When an LLM says it's 95% confident, how often is it actually correct?” About 40% of the room guessed 90–95% — slightly overconfident, not alarmingly so. About 15% guessed 80–90%. Nobody thought it could be worse than that.

Anand had measured it on 770 historical support cases (BANKING77-style, real customer messages) against a model called GPT-5.6 Luna. The mismatches were sometimes almost funny: a customer wrote “My transfer is pending” and the model was 100% confident it meant pending_transfer, when the correct label was balance_not_updated_after_bank_transfer. Someone wrote “Someone stole my cards!” — 100% confidence, wrong label, lost_or_stolen_phone instead of lost_or_stolen_card.

“If it says it's between 95 to 97% correct, it's probably about 70% right in reality. The curve here is: the x-axis is what it says; the y-axis is what it actually is. They are overconfident, clearly, for this class of problems.”

Anand, on Luna's calibration curve

But the gap wasn't fixed — it was a design parameter. Anand asked ChatGPT to draft alternative prompts for eliciting confidence, on the theory that “there is no such thing as wrong” in a benchmark; you just find out what works. The best of four candidates instructed the model to “silently identify the two most plausible allowed labels and allocate probability between them and any remaining plausible labels…” — forcing it to reason about alternatives before committing to a number. That single rewrite cut the at-least-95%-confident error rate from 10.3% to 3.9%. A cutoff frozen on this 770-case set, applied cold to 2,310 untouched holdout cases, auto-passed 65.2% of them at 4.4% observed error — the calibration held up out of sample.

“Every model has a curve, and you can measure that curve, which means that you can create a queue.”

Anand

A model can be probed even when it won't tell you the truth

There's a second, stranger way to read a model's confidence: not by asking it, but by watching how it generates its answer, one token at a time. “LLMs are next-token generators. They take all the text, they hear you, and then they construct the next word, and then the next word… It's roughly the equivalent of speaking as you're thinking.” Which means the model hasn't necessarily thought the whole answer through before it starts talking — and at each word, it's weighing several possible next words against each other.

Anand had tried to demo this live, on a question about which episode of Friends Joey ate too many marshmallows in — and the Wi-Fi picked that exact moment to fail. “I am not connected. Wow, this makes it a bit unfortunate…” He reconnected and walked the room through a version he'd already captured: the model considers “Joe” (70% likely), “In” (28%), “The” (1.7%) as its very first token, commits to “Joe,” then has no doubt at all about “y” following it. By the time it reaches the episode number, it's guessing again — and, in the tool's own recorded example, confidently fabricates “Season 3, Episode 20” at 86% internal confidence. Joey never actually did that in Season 3, Episode 20. The hallucination was fluent and confident, in exactly the tokens where it should have been uncertain.

“These are called logprobs — the logarithm of the probability — but effectively, logprobs is just a term that says at this particular word, here's how confident I am or here's how uncertain I am… When the logprobs report a probability of 99.9%, it's only 56% correct. So the number itself is useless! But if you just sort by it… it turns out that the error rates are much better.”

Anand

“Don't take the number for what it is; just sort by the logprobs and go down the queue, and that gives you a better result.” The operational conclusion doubled back on itself in a way Anand seemed to enjoy: “Models aren't as self-aware as we think. We can, by placing probes into their brains, get better information about them than if we just ask them.” Two independent signals — the confidence a model states, and the confidence buried in how it generated the answer — both usable, neither perfectly honest on its own.

Act Five · BenchmarkFrom high-schooler to tenured professor in three years

Anand has been plotting the same chart since May 2023: model intelligence on the y-axis, cost per million tokens on the x-axis. “Let's fast-forward to today… If they are on the right side, they're expensive. Left is cheap. Bottom is not very smart; top is pretty smart.” The story isn't any single dot. It's the shape of the frontier moving, year over year, in one direction.

“In the course of about three years, models have become about 15 years smarter of human development.”

Anand

The edge-model aside got a laugh: Anand uses a tiny offline Gemma model, running on his own phone through an app called Edge Gallery, to photograph airline meals and ask “is this vegetarian?” rather than embarrass himself asking the flight attendant. “It works! It's practical… This also works in emergency rooms, where you can't take any external connection.” Not every benchmark question needs the frontier model. Some need a free one that works with no signal.

But the frontier keeps moving into territory Anand can't personally check. He described asking a recent model to review his own tax filings — both his India return and his Singapore accounts. “I sent it all of my tax returns, and it came back and said, 'Anand, you have two problems…'” The claim was specific, plausible, and worth roughly $100,000 combined if correct. “And I have zero basis for understanding tax in any country. I don't even tally my bank accounts! So when it says something, how do I even know that it's right?” He left the question open on stage — and it came back, sharper, in the last minute of Q&A. Keep reading.

“If you can create a benchmark, fantastic. If you can't, then find who's creating a benchmark and use it.”

Anand

For questions he genuinely cannot judge, Anand borrows someone else's benchmark — specifically OpenAI's GDPVal, which pits AI output against real experts across the US economy, occupation by occupation, on tasks those professionals rated as genuinely representative of their work. “The reason I trust them is because on many occasions they've said Claude is better than GPT. Okay, fine, you're probably honest!” — a benchmark run by a competitor, admitting the competitor sometimes wins, is more credible for it.

One task, given to a general-and-operations manager at a bridal store, asked for a training document teaching staff to overcome sales objections — price, need, urgency, trust, authority — complete with a homework tracker. It's exactly the kind of mid-level, unglamorous deliverable that fills a real working week.

“Don't even go near that” got a laugh for software developers; accountants and auditors, at 24%, was the one number in the room that landed as reassuring rather than alarming — a preview of the tax story's eventual punchline. “I don't have the ability to experiment, I don't have the ability to benchmark. Somebody has benchmarked it for me. It's a starting point.”

The impossibility list, and the photo that finally came off it

For everything outside a published benchmark, Anand keeps what he calls an impossibility list — things he's personally tried and watched AI fail at. “Things we try today that don't work might start working tomorrow. It's good to keep checking.” A few months ago, one item was: can it convert my parents' black-and-white wedding photo into color, correctly?

It's a strange, tiny benchmark by any formal standard — a single photograph, no ground truth, no statistical power. Its power is different: “This happens to be one of those few benchmarks that I'm reliably able to evaluate, and I'm one of the few people who can evaluate because, heck, it's my parents' wedding photo, right? I know their faces better than anyone's.”

At a glance, it's a triumph — faces restored, saris and garlands in plausible color, nothing obviously wrong. Anand looked closer, the way only a family member could. “I know my mom wasn't smiling as much, nor was my father; they were both a little more serious.” His mother's slightly distracted, day-dreaming look had been nudged toward a smile. His father's stern, unsmiling wedding-day expression — “I don't think he ever smiled in any wedding photo” — came back with the corners of his mouth slightly lifted. His grandfather's downturned mouth gained a faint moustache that was never there. “The one theme that I could find was: everybody seems to be slightly happier than the original!”

A second prompt — “make it look like a modern digital camera was transported back in time to take exactly the same photo” — pushed the effect further still: sharper, more detailed, and more people, subtly, smiling.

“If you can find a benchmark that at a glance you say, 'Yes, that's right,' or 'No, this is wrong,' and it could be anything, you've got an edge. You can evaluate AI on at least some things.”

Anand

“It's like 'Oh, yeah, that's them.' vs 'Oh, that's not quite them.'” is how Anand put it in the original write-up — a benchmark that resolves in a glance, because the evaluator has forty years of ground truth stored in memory. The bias it revealed — models nudging expressions toward happier — is exactly the kind of subtle, systematic error a generic accuracy score would never surface, and exactly the kind a personal, high-sensitivity benchmark catches immediately.

Act Six · Generate Rules86.0% → 31.8% → 67.3%, live, on stage

Everything so far had been about judging a non-deterministic model. The last act turned AI into something deterministic instead: a rule you can inspect, rerun, and improve on your own schedule — without spending another token. Anand opened a real dataset live — the UCI Student Performance dataset, 395 students, 33 columns of family background, study habits and grades — and asked a tool called Decision Tree Builder to predict final grade G3 as a simple, readable decision tree.

Run one — 86.0%, and immediately suspicious

“I can give you a rule that's 86% accurate.” The room's reaction was the reaction you want in a data workshop: impressed, then wary. The tree's very first split gave the game away: “If grade 2 is less than 10.5…” — predicting the final grade using an earlier grade as an input. “Hold on! You're using one grade to predict another grade, which is kind of like saying, in your second exam if you got a certain score, then I can tell you what your score will be in the third exam. Okay, yeah, but I kind of want to know before I look at any of the grades, right?” Textbook target leakage, caught by eye, in real time, in front of fifty people.

Run two — strip the grades out, and accuracy collapses

“Predict G3, but don't use any grade information.” The model thought for a moment and came back chastened: “Oh, I can only give you 31% accuracy.” Without G1 or G2 to lean on, the tree fell back on past failure rates, gender, whether the student has a guardian, the mother's occupation, and raw absence counts — each mildly informative, none of them decisive. “It's kind of helpful, but 30% is not very good.”

Run three — the model invents better features, and climbs back to 67.3%

Rather than accept the collapse, Anand pushed it to do more work: “Can you convert new metrics? Take all of these, and maybe divide a couple of these, come up with additional metrics, and use those instead.” The tool generated 21 derived features on the spot — parents' average education level, total alcohol consumption, absences scaled by age, a travel-time-to-study-time ratio, a “study efficiency” score, and more — then rebuilt a constrained, max-depth-3 tree using only those, still with no grades allowed in.

“This is useful in two ways: one, I know what factors are important and therefore I can start making changes, not just predictions. The second thing is coming up with a rule that I know is about 67% accurate, and I can over time start improving this on a regular basis.”

Anand

The most important derived signal turned out to be study efficiency, followed by absences scaled by age — older students tolerate more absenteeism than younger ones before it hurts them — and then travel time relative to study time. None of these existed as columns in the original dataset. The model had to invent them, and a human had to insist it not cheat, before the number meant anything.

“This is the premise based on which people just love AI writing code. Code is deterministic, mostly… Doesn't cost me anything — no tokens used. Deterministic: when I run it again, it'll give me the same result. And I can use AI to improve it the next time.”

Anand

“This whole loop of running something and evaluating it using AI is a pretty powerful cycle. But what you need is ultimately a certain degree of skepticism.”

ClosingFive words, in the order the pages were opened

Anand closed by naming what the room had just spent an hour doing, and it mapped exactly onto the five browser tabs he'd opened, in order:

• Question advice. It may be right, it may be wrong, but in your context, it might not work.
• Verify, and use AI to verify. Verification with AI is largely harmless — it costs a little money or time, both in your control.
• Calibrate the confidence. Once you know how confident a model is, numerically, you can decide what to accept and what not to.
• Benchmark. Run the benchmarks yourself where you can; borrow trustworthy ones where you can't.
• Generate rules. AI isn't only a tool for the final output — it can improve the process too, and rules, unlike model outputs, hold still long enough to get better over time.

“With evidence, you will find that the amount of impact that you can create is much, much higher. Do give it a shot, keep practicing, and you'll find that your impact will dramatically grow.”

Anand, closing

Q&AThe room writes the epilogue

The host had said there were three minutes left. It turned out to be enough for the audience to hand Anand back his own thesis, twice — once as a generalizable rule, once as a joke at his expense that happened to be completely correct.

“Benchmark more where things are important”

The first question noticed something in the GDPVal chart: medical-field tasks had tighter, more careful evaluation than most other occupations. “Does it mean to say that we did a lot of benchmark on the medical field because the margin of error should be less?”

“Absolutely… that's a generalizable rule: benchmark more where things are important; where it's less important, proportionally benchmark less. Based on the degree of risk or based on the degree of impact, the level of care, skepticism, etc., that you need to treat will vary.”

Anand

“I would probably wait for this to get to 99% before I take an important decision. But if it was just asking about maybe, I don't know, what exercise should I do, then I'd probably say, 'Yeah, 50% is good enough.'” The audience member laughed: “In order to avoid going to the doctor!” Anand's reply landed the whole hour's argument in five words — “I always verify with AI. Take the doctor's reports…” — before the same audience member pointed out, fairly, that doctors' error rates are hardly zero either. “Exactly, yeah, true. Double-check anyway.”

The tax story, resolved

The second question went straight for the anecdote Anand had left hanging earlier: the tax filing, the two flagged errors, the roughly $100,000 at stake. “Just curious, did you verify that result in another model?”

“No, I didn't actually, because I was so happy that I just sent it straight to the auditor!”

Anand

The auditor pushed back immediately: the refund claim was wrong, she said, because the account in question was a Non-Resident Ordinary account, not a Non-Resident External one, and the rule didn't apply. Anand took the correction straight back to ChatGPT: “Here's the transcript. What on earth am I supposed to say?” The model found a specific precedent — an Income Tax Department case establishing that account type didn't actually matter for this rule — and Anand relayed it verbatim. “She said, 'Okay, I'm going to check with my senior.' She checked with her senior, came back and said, 'Yeah, it turns out that you're right.'”

It's a small, human echo of the whole workshop in miniature: a confident AI claim, a human expert who pushed back, a second AI pass that resolved the disagreement with evidence rather than authority — the exact double-checking loop from Act Three, playing out with real money and a real auditor instead of a benchmark dataset. And then the room, not Anand, supplied the moral:

“That's just a good reminder again that, you know, if you're too excited, don't forget to verify!”

An audience member

Anand's answer was one word: “Yeah, true!” The talk had opened with an audience member admitting “We don't know” and Anand turning it into a design principle. It closed with an audience member catching the speaker breaking his own rule, in his own story, thirteen minutes earlier — and Anand simply agreeing. The room had learned the method well enough to apply it to the teacher.