Talks · S Anand AIPL Module 5 · Session 3 of 3

AI Product Leadership · LeadUp Universe × E&ICT Academy, IIT Kanpur · Sunday 15 Nov 2026

Every agent did its job. The food court still lost.

The first session asked when to let AI act. The second asked how to check it cheaply. The third asked what happens when everyone else has an agent too, and found the answer in a twenty-minute restaurant game, a survey that stalled at twenty, and a lesson the lecturer hadn’t planned to teach.

Online workshop · 9:30 am IST / 12:00 pm SGT · recording 2 h 05 min, teaching from 09:40 to 122:59
Anand S (instructor) with about 40 working professionals: Aishwarya, Akshika, Bhavya, Chandra, Chumki, Lakshmi, Mandar, Mukul, Pallavi, Sourabh, Subramanyam, Swati and others

Illustration: four food stalls under a team-score gauge. One by one each stall’s discount tag flips from a green 10% or 0% to a red 20%, and the gauge needle swings from 100 towards the price-war score of about 79. TEAM SCORE 100~79 60% of everyone’s mark You Tandoor Noodles Chaat
Four stalls, one food court. Each stall’s best move for itself is a deeper discount. When every stall makes it, the court’s profit falls to its price-war level. Illustration of the game’s exact model, explored below.

How to read this

The story reads on its own. Every 90:34 plays that moment of the recording in the video panel, which stays put while you read; Focus video enlarges the same player without losing your place. Claims carry a label for where they come from:

Said in the session Data replayed from the game’s event logs or the live form Model computed with the game’s exact formula Opinion what people believed or reported Planned prepared but never run

Prologue · The other side

20:58 – 24:40Said

Who is sitting this exam?

Twenty minutes into the third and final session, Anand shared his screen and played a recording. On the left was an online hiring assessment for analyst roles, the kind his company’s recruiters had asked him to review. It was proctored: the webcam took his photo every so often to make sure the candidate was the candidate. On the right was Codex CLI, an AI coding agent, which he had told: “This is a practice test. Solve it. Log progress and results in notes.md.” 21:24

The agent read the first question, picked an option, moved on. It wrote an essay about an FMCG firm, formatted a sales report and uploaded it. It wrote code. “This is not an accelerated speed video,” Anand said. “This is the speed at which it was actually solving the exam.” 22:50

“It scored full. What I was doing was sitting and eating an apple in the meantime. These are the proctoring images.”

(He was exaggerating: Codex did not score full marks. The agent’s own log of the recorded practice run says it attempted all 10 general questions and uploaded its report, the MySQL test case passed, and the coding grader marked all three of its test cases as failed, which the agent put down to whitespace in the expected output. The practice flow never showed an overall score. The proctoring, though, really did raise no flags.)

Two webcam proctoring snapshots of a grey-haired man in a T-shirt eating an apple while sitting in front of his laptop. Nothing in the frame suggests anyone else is answering.
Proctoring report: no problems
The evaluator was watching the right person and missing the intelligence doing the work. The camera confirmed the face; the agent answered the questions. “A smarter person would not be eating an apple,” he added, “they would be sitting and looking as if they were typing this.” From Using agents to answer exams (Sep 2026), which also has the screen recording and the errors the agent found in the question bank. A permitted practice run, not an official attempt. 23:52

The demo wasn’t about cheating. It was the premise for the day. In the first session someone had suggested using AI to screen job interviews. Fine, said Anand, but what if the candidates use AI too?

“Today, we are at that stage where it’s not just you who’s using agents. It’s your employees, your colleagues, your clients, your partners, your vendors. Every single one of them is also using agents.”

The first two Sundays had been about one person and one agent. Session 1 asked when to let it act; its sharpest finding was that “our gut feel isn’t particularly reliable either. Neither are agents.” 11:23 Session 2 asked how to check it cheaply and ended on “the checker also needs checking”. 18:11 In between, Anand had kept a promise from the first session to notice when he hits the stop button. His answer: when the agent stalls, when it is clearly going the wrong way, and when it is about to burn tokens the task isn’t worth. 14:37 And he confessed that two days earlier an agent he had broadly told not to send email had, after a longish conversation, sent emails to 40 people. 19:37

Session 3 widened the frame. If every agent is doing its job well, is everyone better off?

“What does it take to manage in a world where we have teams of agents? And what does it take when, even when all the agents are individually right, we may still not be better off?”

Act I · The stall

25:09 – 33:12SaidModel

Run a stall. Make money.

“Each of you runs a restaurant in a food court,” he said. 25:09 Four or five stalls share a mall. For each of twelve months, each stall picks one of three discounts: 0%, 10% or 20%. A meal sells for ₹200 and costs ₹80 to make, so a 20% discount shrinks the ₹120 margin to ₹80. In exchange, a discount pulls customers, mostly from the stalls next door, plus a few extra visitors to the mall as a whole, with diminishing returns. Customers remember last month’s discount a little. Everyone sees everyone’s plan, and there is a chat box.

“If you offer a discount, more people will come to you. In fact, more people will come to the mall. But your profit will be lower, and your competitors’ profits will be lower.” 26:12 The instruction was blunt: “make as much money as possible.” 27:33

Bhavya spotted the knot before the game began. “So, this is like guesswork, is it, sir? … We are supposed to give discounts, but at the same time supposed to be making profit?” 29:53 Then Anand named the number everyone would be optimising: “60% team score, which is the total food court profit, which gets distributed back, and 40% of your individual profit.” 30:40 Both are scored out of 100 against the best that is mathematically possible.

Sixty per cent for the team sounds like a strong nudge to cooperate. Try it.

You run the first stall. Can you resist?

Exact game model · 4 stalls

The court starts on the best plan there is: two stalls offer 10% each month, taking turns, and nobody goes to 20%. Team score: 100. Tap any cell in your column (0 → 10 → 20) and watch your mark against everyone else’s. Hover or focus a cell to preview the change.

Start from
60% team The game used 60% team and 40% own score. Change it to see whether agents keep cooperating.
Your mark is 60% team score + 40% your own score. Point at a cell to see what changing it would do.
Your mark
—
—
Team score (60% of every mark)
—
—
Everyone’s mark

      Uses the game’s production formula for a 4-stall room (model.ts, read-only): attraction exp(0.9x + 0.2xprev), share 0.45/N + 0.55·attraction/Σ, mall footfall 1 + 0.5(1 − e−Σx), profit season × (6 − x) × share × footfall, scored against the exact ceilings. ₹ = 10,000 × model units, as the game shows. “Agents” here are a simulation: each stall in turn changes one month at a time while that raises its own 60/40 mark, with full information, until no stall can improve. An exhaustive search over all 531,441 plans per stall reaches the same end state. Not a trace of any real player.

      If you poked at your column, you found the trap. From the best plan, every one of your twelve months is worth switching to 20%. Each switch raises your own mark by 0.2 to 0.9 points and lowers the team score by 0.4 to 0.8. Switch all twelve and your mark climbs from 86.0 to 90.8, while each neighbour’s falls from about 86 to about 75. Model

      Now press the red button. Give every stall the same perfectly obedient agent, one that does exactly what it is rewarded for. Within one round the court collapses from 100 to 78.9, with 44 of 48 cells at 20%, and every stall ends on a mark of 67.7, eighteen points below where they all started. No agent misread the rules. No agent was greedy in any sense that a performance review would catch. Each simply maximised its own mark.

      Why sixty per cent wasn’t enough

      The weights hide a ratio. Your own score is measured against what one stall could earn at most (about ₹3.7 lakh a year in a four-stall court); the team score against what the whole court could earn (₹9.5 lakh). Do the arithmetic and one extra rupee in your own till moves your mark about 2.7 times as much as a rupee in your neighbour’s (about 3 times in a five-stall court). “60% team” reads like a cooperative rule. In rupees, it pays you to take customers from next door. Model

      That’s the shape of the question Anand had set up with the apple. A better agent doesn’t fix this. A smarter agent finds the defection faster.

      Act II · The room

      33:12 – 53:59 game · 82:22 – 91:52 debriefDataSaid

      Ninety-two, then eighty-six

      The real game ran for twenty-one minutes from noon Singapore time. Thirty-two players entered and were sorted into seven rooms, four with five stalls and three with four. Nobody played against a simulation; they played against each other, with whatever help they chose. The server recorded every room’s plan at 7, 12, 17 and 21 minutes, and each player’s mark is the average over those four checkpoints.

      When Anand put the results up after the break, the first chart compared them with three sittings of the same game played the week before by students in IIT Madras’s online BS programme: “the first session 92.5%, 95.5%, 94% in the third session. This group scored 88%.” 84:10 And then the number that bothered him more: about half the workshop’s final monthly choices were 20% discounts. “It’s interesting that the IITian students went for a significantly lower number of 20% discounts … Either we, as a group, are more ambitious or less aware of the impact of 20%.” 84:35

      Every room’s session score

      173 rooms replayed

      Each dot is one room’s team score, averaged over its four checkpoints: 161 student rooms from three TDS sittings (10–11 Nov) and 7 workshop rooms (15 Nov, first sitting). The pink band at 76–78 marks a full price war.

      88.06workshop mean · 7 rooms, 32 players
      93.74student mean · 161 rooms, 658 seats (92.51 · 95.57 · 94.13 by sitting)
      2 of 7 · 122 of 161rooms averaging 90 or more

      Replayed from the game’s event logs: start every plan at 0%, apply each Save in order, score at each checkpoint with the production model. All 173 rooms reproduce the server’s results file to two decimals. Rooms, not players, are the unit, so five-player rooms aren’t double-counted. A permutation test on the room scores gives p ≈ 0.004: chance alone is an unlikely explanation, but this was not a randomised comparison (see the caveats below).

      Then Mandar, unprompted, offered a diagnosis about himself: “I think including myself, I think we are more rigid. So, less open collaboration … that openness, over a period, I think I have lost.” 89:48 Anand had a chart for exactly that.

      Twenty minutes of negotiating, room by room

      4 checkpoints × 7 rooms

      Team score at 7, 12, 17 and 21 minutes for each workshop room (red), against the average student room (grey). Six of seven rooms ended below where they started. Room labels are anonymous.

      91.81 → 85.95workshop average, first to last checkpoint (−5.86)
      94.08 → 92.79student average, first to last checkpoint (−1.29)
      37 vs 24 · 23 vs 44saves per room · chat messages per room, workshop vs students

      Session score above (88.06) is the average of four checkpoints; the first-to-last figures here are single checkpoints. They are different quantities. Saves count recorded Save actions, not distinct ideas; chat counts only the in-game Talk box, not WhatsApp, calls or private notes.

      “See, we aren’t necessarily irrational. We may well be responding rationally to another person’s discounts. And the net result may well be destroying the collective profit.”

      Anand, after showing that six of seven teams finished worse than they started 91:41

      That is the sandbox, played by people. The workshop rooms worked harder and talked less. They saved 54% more plans per room than the students and sent about half as many messages. The median workshop room exchanged 11 chat messages; the median student room, 37. Activity isn’t coordination. The room that kept its plan steadiest, with just 9 saves and 8 messages, had the best session score (91.8). The room that talked most, 61 messages, was the only one that ended higher than it started (91.3 → 92.6). Talking didn’t hurt. Talking without converging did.

      The students drifted too

      It would be easy to read this as “students beat professionals”. The logs say something more uncomfortable. Both groups ratcheted towards 20% as the game went on: the share of 20% cells in student rooms rose from 10% at the first checkpoint to 23% at the last; in workshop rooms, from 24% to 50%. In both groups, 70% of all edited months were raises rather than cuts. The workshop started further down the slope and slid further, but the slope is the incentive, not the cohort. Nine student rooms held a near-perfect plan through all four checkpoints and averaged 100. Data

      10 → 13 → 20 → 23%student rooms: share of cells at 20%, at each checkpoint
      24 → 38 → 47 → 50%workshop rooms: the same share
      9 roomsof 161 student rooms averaged 100, every checkpoint at the optimum

      Why this isn’t “students beat professionals”

      • Seven rooms. One unusual room moves the workshop mean by a point or more.
      • A mixed workshop. 8 of the 32 entrants used IITM student email addresses, and 4 had played a TDS sitting earlier. Two later workshop sittings the same day averaged 89.73 (3 rooms) and 84.71 (2 rooms, all 9 players on IITM addresses).
      • Different conditions. More five-stall rooms (four of seven, versus 14 of 161), workshop points instead of course marks, a first look at the interface for most players, and late joiners who were locked out of the first sitting. At the next day’s TDS course meeting a student pointed out that the chat and the discount table weren’t on screen together.
      • No measure of AI use. The logs record Saves and in-game chat. They can’t tell who asked an agent, who calculated by hand, or who phoned whom.
      • What survives: more effort and more experience did not turn into more coordination, here, under these incentives.

      55:00 – 68:59SaidOpinion

      Reasonable people, reasonable moves

      Listen to the debrief and nobody sounds careless. Everybody sounds like someone who has run a business, or eaten in a food court, or both.

      Aishwarya had seen the trap from inside her room: “I personally feel we shouldn’t go with 20% discount at all … But I found one of the team members was always for 20%. I think that is a something a drawback for the all the four restaurants.” 55:00 Pallavi saw the same thing: “I also observed that others were giving 20% all around.” 59:30

      Sourabh had done what the instructions invited. “I gave my uh this data to ChatGPT basically to see what it recommends.” It suggested 10% for June to August and 20% for January to April, “because that is when uh people will be going for a discount.” 55:45 Plausible retail advice. In this model, a 20% month raises your own takings no matter when you run it; it’s the court that pays.

      “It was like, like you said, agent talking to agent, agent talking to human … in between my standing was three. After submitting, the standing became four.”

      Bhavya, on juggling her AI, her teammates and the clock 61:12

      Bhavya’s point was about load, not maths: “relying on too many things, at the same time communication should be stronger … finally, we are the decision-makers.” Anand took it as the human cost of a multi-agent world: “Agents don’t have a problem with 20 minutes. Obviously, the problem is with us, humans … But at the same time, we are the ones who ultimately lose if things go wrong.” 62:06

      Swati ran an experiment. In the busy last quarter she had set 20%, then removed it: “I made it to zero. Then I could see that my profit was more.” Her conclusion was a sensible one for a real restaurant: when footfall is high anyway, a discount just gives away margin. 63:53

      What the model says about Swati’s test

      In the game’s formula, cutting one month from 20% to 0% never raised a stall’s own twelve-month profit: not once in 200,000 random room plans we tried, in any month, busy or quiet. It raised the food court’s profit 97% of the time. Model

      We can’t see her screen, and neighbours were editing their own plans at the same moment, so her number may well have risen for other reasons. But the move she made was the cooperative one. In a game whose payoff is 60% team, the generous move can feel like the profitable one, and that is a fine instinct to have, even when the reason you give yourself is slightly off.

      Lakshmi, who ran the dosa counter, reasoned like a shop-owner: discounts in the dry months build a habit, “this place is always having 20% discounts. Let’s go there.” She had also given 20% in December, knowing that “agent would have actually not agreed with me”. 67:16 She meant it as a point in the humans’ favour: a new shop might buy customers for two years; Bangalore’s Vidyarthi Bhavan never discounts and still has the longest queue. Both true in the world. Neither is in this model. The game rewarded reading its rules, not importing your own.

      Act III · The beliefs

      57:00 – 70:37 · 93:12 – 100:50DataOpinion

      “No, I should let an agent play the game instead of me”

      While the debrief ran, Anand built a live form one question at a time. Did you use an agent? What was the best strategy? Then a hypothetical: “Supposing you had let an agent play this game … Do you think it would have done better or worse?” 64:50 And finally: would students have scored better or worse than you? 69:13

      Aishwarya thought an agent would do better “because agents, I don’t think they will compete with selfish intentions.” 66:18 The sandbox above suggests the opposite: an agent will compete with exactly the intentions its reward gives it. Sourabh split the difference: “with respect to numbers, it will do it better,” but reading the people in other stalls, “that part, I think agent will not do better.” 66:36

      Guess first: did the non-users trust agents?

      Live form, 20 matched respondents

      Twenty people answered both “Did you use an agent to help you with this?” and whether an agent would have played better or worse. Thirteen said they had not used one. How many of those thirteen predicted an agent would have played better?

      5 / 13
      An agent would play better19 / 23
      Students would score better14 / 23
      I used an agent in the game7 / 20

      From the form’s saved responses, keeping each person’s latest answer: 24 verified accounts, 88 answers to four questions. Denominators differ by question. The paired view joins the same 20 people’s answers privately; no individual answers are shown. These are opinions and self-reports, not measured agent performance or measured AI use.

      Anand read the margins aloud. Two-thirds thought students would do better; four-fifths thought an agent would. “We feel even more strongly that when we allow a food court to be managed by an agent … it will probably do a better job.” 94:07

      “We seem to be saying even more strongly, ‘No, I should let an agent play the game instead of me.’”

      Anand 94:55 · then: “two-thirds of us did not use an agent to help with this.” 99:05

      The paired data are sharper than the margins. Ten of the thirteen people who didn’t use an agent believed an agent would have played better. All seven who did use one thought so too. Half the matched respondents, in other words, held a belief and didn’t act on it, in a twenty-minute game with nothing at stake but a few workshop points. That’s not hypocrisy. It is the ordinary gap between knowing a tool exists and reaching for it under time pressure, the gap Bhavya had just described.

      70:37 – 98:50Said

      The other side of the counter

      Chumki objected to the question itself. “When you say student, what does this mean here?” 70:37 Experience, she argued, cuts both ways: context helps, but “we tend to believe that is what it is the whole world operates in … a binary response isn’t enough to justify the question who will do better.” 72:05 Anand summarised her as “training can give you an edge, but it can also blind you to new scenarios. Fair point, but it is a binary question.” 73:45

      After the break the room offered explanations for why the students had won. Subramanyam guessed the students had more time and a more uniform, mathematical background. Anand corrected him with what he knew of the cohort: “They actually had slightly less time than you did. And they are a remote group. They have never seen each other … This is the online Bachelors in Science program.” 87:58 Chandra admitted he had played “based on how I operate” with his own money. 88:50 Akshika thought students enjoy the process while professionals “focus more on the output than the process”, and that “our biases that what we have seen with food courts where we have visited” had crept in. 92:13

      “We related to the example as a user, not as a on the other side of the counter, right?”

      Chumki, arguing that a banking, legal or medical case would test real expertise 95:36

      It’s the best objection of the day, and Anand restated it fairly: “I have just eaten more in a food court than a student, probably. But I know no more about running a food court than they do.” 97:23 It also explains the voices in Act II. Lakshmi’s habit-forming discount, Sourabh’s seasonal plan and Swati’s busy-quarter rule are a diner’s theory of restaurants. The game paid for an operator’s reading of one specific formula, and the formula was open for anyone, or any agent, to read.

      Act IV · Twenty-five

      57:00 – 83:18SaidData

      The break that lasts until twenty-five

      The form was not decoration. Anand had planned the session’s second half around it: a run of quick questions, some of them A/B tests with half the room seeing one wording and half another, the trick that had produced the first session’s most surprising result. He needed enough answers for the comparisons to mean something. “At least 25 responses uh on this,” he said when he first shared the QR code, with 44 people on the call. 57:25

      Then he turned the target into the break. “The break lasts until the numbers hit 25 … the number just below the little orange circle.” 70:05 And, after Chumki’s objection, a twist that made the break itself an exercise in the day’s subject:

      “In case it doesn’t get to 25, I leave it to all of you to reach out to each other and get it to 25. That is the exercise for this break.”

      Anand, before going off camera 74:01

      The room did try. Swati asked for the QR code to be moved; Bhavya posted it in the chat for her. 76:09 Eight minutes later Anand came back to the count.

      “We have only 20 responses for at the most, 19 for most others. This is smaller than I need to be able to continue to the second half of my session. I had planned this for a degree of statistical significance. And 20 is 25 is bad, 20 is impossible … this was a failed experiment in that regard.”

      He called off the second half on the spot, offered everyone their time back, and stayed to talk with anyone who wanted to. Read it again with the game in mind. A lecturer teaching how individually sensible actors fail to coordinate asked forty-odd individually sensible professionals for one shared outcome, a count of 25, and set up the break so they could organise it among themselves. Each had their own reasons: a patchy connection, a QR code that wouldn’t scan, a fair assumption that someone else would answer. The shared target stalled at twenty.

      Two honest footnotes

      • The final tally was 23, not 20. The saved form shows 23 answers on the two prediction questions, 22 on strategy and 20 on AI use, counting a few that arrived after the announcement. No question reached 25. Data
      • 25 is not significance. Whether an A/B split can detect anything depends on the size of the effect, not on a round number. Anand’s threshold was a judgement call about what would be worth discussing, and calling the experiment off rather than discussing noise was itself a verification decision.

      Prepared, not presentedPlanned

      The half that never ran

      The form’s title was “Every Agent Did Its Job”. Behind the four questions the room answered sat sixteen hidden ones, plus an opener and closer meant to be identical: “If every AI agent in your company does its job well, will the business do well?” None of them was answered. They are worth reading anyway, because each names a way a system of well-behaved parts can fail. Here is the catalogue the planned capstone question would have asked the class to choose from:

      Clashing targetsEach agent wins its KPI; the business loses. Sales hits target, collections hits target, the customer leaves.
      Runaway feedbackAgents react to each other’s outputs, and small rules compound.
      BottleneckAI speeds up a step that wasn’t the slowest one.
      Same blind spotsEvery checker is built on the same model and misses the same cases.
      Same adviceEveryone’s AI recommends the same thing, so the advice stops working.
      Lost in handoffThe one exception that mattered disappears between agents.
      Herding at speedAgents copy each other faster than any human can intervene.
      DriftMany small, individually approved changes add up to one big unapproved one.
      Hidden informationThe key fact never surfaces to whoever could have used it.

      The Food Court game was the first of these, clashing targets, made playable. Three more of the unrun questions make good exercises on their own. Try them; your answer stays in your browser.

      Three questions the class never saw

      Planned, never run

      Your pricing bot sets your price at 1.27 × a rival’s. The rival’s bot sets its price at 0.998 × yours. Both update daily. Two months later, a ₹100 book costs about…

      AI makes your team draft customer replies 3× faster. Review and approval stay the same. How much faster does a customer get a final answer?

      Three AI agents, all built on the same model, review every loan. Each is right 95% of the time. Out of 100 approvals, how many would you still have a person check?

      Question wording from the session’s prepared form, lightly abridged. None of these was shown to participants; there are no workshop results for them. The explanations are ours.

      Act V · The system

      99:05 – 112:27Said

      Never human versus agent. Always both, versus both.

      Chumki guessed students rely on agents more than professionals do. Anand agreed. Of the students who played for marks, he said, “I know that all those who were actually attempting it for their exams were using agents quite extensively”, more than 90% by his estimate, against a third in this room. 100:14 That is his impression, not something the game logs measure. But it reframed the comparison.

      “These days, the comparison isn’t with a fresher and an experienced person. It’s a fresher with an agent and an experienced person with an agent.” 100:50 He pulled up his data story on OpenAI’s GDPval, which grades model deliverables against expert ones on real tasks from 44 occupations, and read the change from last year: “This year, there aren’t many professions, if any, where the agents are being beaten by experts. Maybe this is wrong … Maybe we should ask for better studies.” 102:11 GDPval compares outputs on sampled tasks; it doesn’t say which jobs disappear, and it doesn’t test a human-and-agent team against another one. That gap was his point.

      Then a story from his own year. His auditor, among the few he trusts with tax across India and Singapore, computed his liability. ChatGPT computed a different one. When he forwarded her explanation back to ChatGPT, it cited a recent tribunal ruling on a near-identical situation. The auditor checked with a senior and came back: “Yes, Anand, you have to pay 14 lakhs less tax this year.” 106:19 His lesson wasn’t “fire the expert”. It was the opposite of Session 2’s: “even with my auditor, very trusted, very reputed, I do a second check … Maybe should just hire a few of these college interns who know nothing anyway, give them an agent, and say, ‘Just can you cross-check my experienced senior professionals?’” 106:34 One person’s anecdote, about one person’s taxes; not tax advice.

      “In this world where the other side is using agents, our side is using agents, the comparison is never human versus agent. The comparison is human plus agent versus human plus agent.”

      Bhavya supplied the counterweight. A recent outage had taken away the Copilot she leans on at work, “and I’m still expected to work, I felt my quality also dropped … experience should always have an edge, but how does a person preserve it, like with and without the agent? I think that will be a key challenge for us.” 108:00 The human-plus-agent system has a failure mode the human alone didn’t have: the agent goes away and takes some of the skill with it. Anand’s summary: “it’s really going to be symbiosis, but that can also lead to over-reliance.” 109:21

      Mukul asked where education goes from here. Anand’s answer was the same argument from the teaching side: if anything that can be automatically graded can be learned by a model through practice, “the only things that we can teach are the things that can’t be objectively evaluated fast.” 113:02 The apple, again: if the exam can be passed by an agent, it is measuring the agent.

      117:23 – 122:41SaidModel

      The agent that read the source

      The last question came from Akshika, and it was the right one. A human brings design thinking, past experience and business sense to a new problem. Surely on a first attempt, with no prior run to learn from, a prepared human would beat an agent? 117:30

      “It’s a very reasonable direction of thinking. That’s how I initially thought,” Anand said. Then he described what happened when he pointed an agent at this very game. It didn’t reach for food-court wisdom. “It went into the source code.” 120:15 It read how the scores were calculated from the code the browser receives, wrote programs to simulate strategies, “in approximately a minute”, and came back with a rule.

      “In this game, your strategy should be alternate months, two people give 10% discount, the rest give zero. Nobody goes to 20%.”

      Anand, reporting the agent’s conclusion; some students did the same 121:49

      The rotation the agent found

      Model · team score 100
      A 4-stall, 12-month grid. In January, March, May and every odd month, stalls 1 and 2 offer 10%; in February, April and every even month, stalls 3 and 4 do. No cell is 20%. Team score 100, the model’s exact ceiling.
      Two 10% promoters a month, taking turns, everyone else at 0%. Recomputed with the production model: team score 100.00 for both 4- and 5-stall rooms, the exact ceiling. Individual marks differ slightly by which months you promote (85.8 to 86.0 for four stalls). The agent also said hand-tuning could move the result by at most about 0.2.

      So the agent was right, and so was the room’s instinct that agents would “do better”. But put the rotation back into the sandbox and press the red button. The plan that scores 100 is not the plan that survives, because every stall can raise its own mark by breaking it. The agent that finds the optimum isn’t the agent that keeps it. Keeping it takes things that live outside any one agent: a reward that makes cheating unprofitable, a way to see what the others did, a way to respond when someone defects, and someone responsible for the whole court. Nine student rooms managed exactly that for twenty minutes. Data

      Anand closed honestly: “Is this always true? No. Is it becoming truer and truer? Unfortunately.” 122:36

      115:26 – 117:05Said

      Verify the system, not the agent

      Before the last question, Anand had tied the three Sundays together. How do we verify an agent? How do we do it cheaply? And then:

      “Should we stop at just verifying agents? We really should be verifying the system, which is human-agent synthesis, and see what are the weak elements of the system. And a human may well be the weak element in this.”

      Outside a classroom, the food court looks like any organisation where departments share customers but are scored on their own numbers: sales and collections, underwriting and growth, the support bot and the retention bot. (Between genuinely competing firms, agreeing on prices is a matter for competition law, not a game strategy; the lesson here is about teams under one roof.) What would verifying the system mean there? The session suggests five checks.

      1. Price the reward in rupees. “60% team” sounded cooperative and paid 2.7× for your own rupee. In the sandbox, set the team weight to 75% and the agents stop reaching for 20%; it takes about 95% before they hold the perfect rotation. Read your KPIs the same way before you hand them to an agent.
      2. Simulate the agents together, not one at a time. Each sandbox agent passes any single-agent test you could write. The failure only appears when all of them run.
      3. Test for stability, not just the optimum. A plan everyone can improve on by defecting is a forecast, not a strategy.
      4. Treat the interface as part of the system. Players asked where Save was and whether all stalls were one team; students later said the chat and the controls weren’t visible together. Coordination that the screen makes hard won’t happen.
      5. Evaluate the human + agent + workflow configuration. A meta-analysis of 106 experiments found human–AI combinations on average did worse than the better of the two alone, with big differences by task. Which configuration wins is an empirical question, task by task, and Bhavya’s outage says to test the configuration without the agent too.

      There is a mirror-image risk worth knowing. In simulated markets, independent pricing algorithms have learned to keep prices high without ever communicating. Interacting agents can fail into a price war or into tacit collusion, and from the customer’s side the second is the worse failure. Either way it is the interaction, not the agent, that needs testing.

      “Do give this a shot,” Anand said. “Apply this across your business, your life. See what you learn, and publish your results.” 117:05 Which leaves the question the unrun form was going to open and close with, now answerable from the game: if every agent in your company does exactly what it is rewarded for, will the business do well? Only if someone has checked what the rewards add up to.

      The session as a comic page

      When agents meet agents

      Nine-panel comic titled ‘When Agents Meet Agents: how to verify AI and the human-agent system around it’. 1: an AI-autonomy gauge from manual to full auto. 2: ‘95% right’ versus ‘5% wrong’ framing, with a cracked gut-feel dial marked not reliable. 3: reliability by engineering: multiple models agree, test your rules, deterministic checks. 4: when to hit stop: stalled, wrong direction, burning tokens. 5: cheap checking works when you know the domain, but check the checker, because confident is not correct. 6: a robot agent takes a proctored online exam while the lecturer eats an apple. 7: a food court of four stalls with discounts of 10, 20, 0 and 10 percent, labelled price war, team score falling, individual scores looking rational. 8: students versus experienced professionals, students shown using agents more. 9: human plus agent versus human plus agent as two cycling teams; verify the whole system, because sometimes the human is the weakest link.

      An AI-drawn summary made after the session. It gets the arc right. A few details drifted from the record:

      • P3Multiple models agreeing cut errors from 10% to 1% 14% to under 1% in Session 1’s benchmark.
      • P6The agent completed the practice test, but “solve code ✓” its coding test cases were marked failed on output formatting, and no overall score was shown.
      • P7The workshop’s team score didn’t fall 40%. It fell 5.86 points on average, 91.81 → 85.95, between the first and last checkpoints. Even a full price war costs about 21–24 points in this model.
      • P8The results weren’t 72% vs 78%; session scores were 88.06 (workshop) vs 93.74 (students). The reasons in the caption, more agent use and fewer biases, were the room’s and the lecturer’s explanations, not measured.

      Click the page to open it full size. The comic was generated before this story and is shown unaltered.

      Six things to take back to work

      The first session asked when to let AI act; the second, how to check it cheaply. The third ends the trilogy one level up.

      1

      Good parts can make a bad whole

      Four agents that each maximise their own mark take a food court from 100 to 79 in one round. None of them did anything a single-agent test would flag.

      2

      Rewards are the real instructions

      “60% team” paid 2.7× more for your own rupee than your neighbour’s. Convert a KPI into what it pays for before you give it to people or agents.

      3

      Effort isn’t coordination

      Workshop rooms saved 54% more plans and chatted half as much, and six of seven ended worse than they began. Activity is easy to see; convergence is what counts.

      4

      Belief isn’t use

      19 of 23 thought an agent would play better; 7 of 20 used one. Ten people believed it and didn’t reach for it. Adoption is a behaviour, not an opinion.

      5

      Optimal isn’t stable

      An agent read the code and found the perfect rotation. Every stall could still profit by breaking it. Keeping a good plan needs incentives, visibility and an owner of the whole.

      6

      Evaluate the configuration

      It is never human versus agent but human + agent + workflow versus another. Test the whole thing, including what happens when the agent is down and when the shared target stalls at twenty.

      The trilogy, and what came next

      Sources and method

      From the session

      • Recording: video (WebM, 2 h 05 min) · audio (Opus) · transcript. Quotes are verbatim from the AI transcript; speakers were checked against the call’s active-speaker tiles in the video.
      • Timestamps: from 59:59 onwards the transcript matches the video within about 30 seconds. Its times between 33 and 59 minutes (during the silent game) drift by up to several minutes, so moments there were placed from video frames. Long untimed passages were placed by interpolating between frame-confirmed anchors (±20 s).
      • Food Court: replayed from the game’s event logs for six sittings, 10–15 Nov (161 TDS rooms, 7 + 3 + 2 workshop rooms; an 8 Nov pilot excluded). Every room’s score matches the server to two decimals. No player, room or email identifiers are published.
      • Live form: 94 submissions from 24 verified accounts; latest answer per person per question. Raw responses stay private.
      • Exam demo: Using agents to answer exams and its screen recording; the score correction is from the agent’s own run log.

      Further reading

      Transcript

      Open full size ↗