S Anand · Talks PyCon India · 21 Sep 2024
PyCon India 2024 · Track 1 · Saturday 21 September

He wanted an Irrfan Khan look-alike. The trick was a dot product.

S Anand asked a hall full of Python programmers which Bollywood star they'd want a double for. His own answer was Irrfan Khan. To find one, he didn't train a face model. He took vectors from a model Google had already trained and compared them, the same arithmetic that had sorted 150 movie quotes twenty minutes earlier. The answer made the room laugh. Then a question from the session host showed nobody in the room knew what “similar” was measuring.

Speaker
S Anand, introduced as “LLM psychologist at Straive”
Talk
Finding actor look-alikes with multi-modal LLMs · schedule: 14:45–15:15
Where
NIMHANS Convention Centre, Bengaluru
Recording
Python India on YouTube, 30:50 with Q&A, published March 2025
Models used
OpenAI text-embedding-3-small · Google multimodal embeddings · GPT-4o mini (for naming)
Listen instead · audio only, 30:50, Opus

The soundtrack of the same PyCon recording. Much of the talk is a screen demo, so the video below shows what the audio describes. Download the .opus file.

Eight-panel comic. 1: a speaker on a 2021 conference stage points at a board asking ‘Who looks like Irrfan Khan?’, beside labels ‘too expensive’ and ‘too unreliable’ and photos of Shah Rukh Khan, Deepika Padukone, Ranveer Singh and Alia Bhatt. 2: a robot with a fake moustache guesses ‘random’ numbers 42, 47 and 37, with a chart and a note that 10, 20, 11, 22 and 100 are missing. 3: a price-versus-quality chart where newer models move up and to the left along a Pareto frontier. 4: the word ‘apple’ passes through an embedding model into a short vector, plotted near ‘orange’ and away from Apple the company. 5: movie quotes go through K-means into coloured clusters with outliers and an automatic taxonomy. 6: a face photo and the word ‘smile’ enter one multimodal embedding model and land in a shared space between ‘smile’ and ‘frown’. 7: driver application photos are accepted or rejected for plain background, duplicates, a company logo and a tuxedo; a six-week plan is crossed out for ‘Day 6’. 8: a ‘Bollywood look-alike finder’ pairs Shah Rukh Khan with Varun Dhawan, Deepika Padukone with Priyanka Chopra, Ranveer Singh with Vicky Kaushal, Alia Bhatt with Katrina Kaif and Irrfan Khan with Nana Patekar; an audience member asks ‘Why not spouses?’ and the speaker replies ‘I have no clue!’
Artist's interpretation. A comic page drawn with ChatGPT after the talk, not a picture of the stage. The talk was at PyCon in 2024, not a 2021 conference, and the real 2024 embedding had about 1,536 numbers. The stage evidence is in the story below. Tap to enlarge.
Reel 1 · 26:18

“Nana Patekar. Maybe.”

Twenty-six minutes into his PyCon talk, S Anand typed a name into a search box he'd been saving for the end. He'd opened by asking the hall whose double they wanted to find. The room voted for Shah Rukh Khan. Anand had said his own choice was Irrfan Khan, the actor who died in 2020: Who can replace—I mean, no one can, that's another story—but I mean, facially, who might be the closest?

Now the screen showed four small face crops joined by grey lines. Two were photos of Irrfan. A third carried the filename Nana_Patekar.649.jpg.

Four face crops on a white screen joined by grey lines: two photos of Irrfan Khan, one labelled Nana_Patekar.649.jpg, and one unlabelled smiling face.

“But to answer the question that I had almost two years ago: who could replace Irrfan Khan? And I think one answer clearly matches: Nana Patekar. Maybe. Yeah, even as an actor, probably.”

S Anand · 26:22

The room laughed. The answer is fun, and it's the least important thing in the talk. What matters is what Anand didn't have to build to get it. He didn't collect a labelled face dataset or train a recognition network. He didn't hire anyone for computer vision. He took photos, passed them through a model someone else had trained, and asked which lists of numbers were close together.

A few years earlier the same question had been refused as a waste of money. To see what changed, rewind to minute three.

Read the result narrowly

The screen shows which photo in that collection sat closest to an Irrfan photo, for that model, above that slider setting. It isn't a verified look-alike, a facial-recognition match or a claim about either man. Anand's “maybe” was the right word.

Reel 2 · 00:33 – 03:31

A quiz, a 42 and a veto

The talk began as a show of hands. Ranveer Singh: One, two hands, three. Deepika Padukone did better. Then Shah Rukh Khan: Wow, he beats Deepika Padukone! I did not expect this. Alia Bhatt came last. Anand wanted Irrfan, and said he'd first wondered about it around 2021.

The early language models weren't much help. Ask one for a random number and it isn't random. Claude 3 Haiku said 42 almost consistently every time, and GPT-3.5 Turbo preferred 47. Both avoided round numbers and repeated digits, exactly the same kinds of biases that a human exhibits. That was enough to make him distrust them for careful measurement. (The random-number experiment has its own story: “The computer said it gets stressed”, from his TEDx talk six weeks earlier.)

Two heatmap strips of numbers 17 to 95. GPT-3.5 Turbo's strip peaks in red at 47 with yellow at 57; Claude 3 Haiku's peaks in red at 42 with yellow at 47 and 57.
“LLMs can't even guess unbiased.” The slide at 01:14. Original artwork from sanand0/actor-lookalikes; live version: llmrandom.

The second problem was money. His next slide priced an A100 GPU on AWS, Azure and Google at $3.40 to $4.10 an hour, about $2,500 a month on a hyperscaler. So he asked his co-founder, Naveen Gattu, for permission to play.

“Anand, we're in the middle of a fundraising initiative. $5,000, two months of your time to find out who's the actor closest to Irrfan Khan? Don't you think we have better things to do?”

The co-founder's reply, as Anand recalled it · 02:59

I said, ‘Okay, fine.’ It's a fair answer, and it's the setup for the whole talk. The question was cheap to ask but expensive to answer, and in 2021 an unreliable, costly model wasn't worth spending on a whim.

Reel 3 · 03:31 – 07:01

The frontier moves left

And I waited, and things have changed a little bit. Actually, two things have changed. First, we actually got acquired, which helps. The second was price. Anand opened LLM Pricing, a chart his team maintained. It plots each model's cost per million input tokens (log scale) against its Elo score on the LMSYS Chatbot Arena, where people vote between two anonymous chatbots. The screen said “Last updated: 20 Sep 2024”, the day before the talk.

Scatter plot of language models by cost per million input tokens on a log scale (x) and Arena Elo score (y). Green dots on the upper-left frontier include gpt-4o-mini-2024-07-18, deepseek-v2, meta-llama-3.1-8b-instruct and llama-3-8b-instruct; gpt-4o-2024-05-13 is green at the top. Red dots on the lower right include gpt-4-0314, gpt-4-0613, claude-1, claude-2.0, claude-2.1 and mistral-large-2402.
Better models sit higher; cheaper ones sit further left. A green model has no rival that's both better and cheaper. This repository snapshot dates from mid-2024; on stage Anand dragged a time slider from August 2023 to September 2024 (04:52).

He dragged a slider through time. In August 2023 the frontier was GPT-3.5 and early Claude. Then the green dots kept jumping up and to the left. He explained the colour code in one sentence: I would never pick a model that is not in green. In other words, something that has nothing both above it and to the left of it. He moved his teams each time the frontier moved, through Claude 3 Haiku, Llama 3 and Gemini 1.5 Flash, to GPT-4o mini, and then Gemini Flash moved the needle again, making it half the price of GPT-4o Mini. Crazy.

A slip worth catching

At 05:46 Anand says Claude 3.5 Haiku. The automatic captions hear the same thing, but the tooltip on screen reads claude-3-haiku-20240307, at $0.25 per million tokens. Claude 3.5 Haiku was announced on 22 October 2024, a month after the talk. He meant Claude 3 Haiku, the model his own price slide names.

Price comparison drawn as blocks of squares: a large red block for GPT-4 at $30 per million tokens, a smaller orange block for GPT-4 Turbo at $10, one green square for Claude 3 Haiku at 25 cents and a tiny blue square for Gemini 1.5 Flash at 7.5 cents.
“The difference between a $10K and a $4M budget.” September 2024 list prices per million input tokens 06:29. GPT-4 cost 400 times as much as Gemini 1.5 Flash. That ratio is what the budget line means, not a measured saving on any project.

The ratio, not the dollar figure, is the point. $30 against 7.5 cents is a factor of 400, about the gap between $10,000 and $4 million. A project that needs a board meeting at one end of the chart is pocket money at the other. I had no problem going back to Naveen and saying, ‘Yaar, this I can fund out of my own pocket.’

All prices on this page are the September 2024 list prices shown in the talk. They have changed since, and Elo is one crowd's rating, not a universal measure of quality.

Reel 4 · 07:01 – 08:47

Cheaper than talking

Then came the turn Anand actually cared about. Generating text wasn't the cheap part. Embeddings were. An embedding model doesn't write anything. It turns a piece of text into a fixed list of numbers, and two numbers that are close to each other are similar in meaning.

Squares comparing price per million tokens: a dot for open models BGE and E5 at under 1 cent, two squares for text-embedding-3-small at 2 cents, and 25 squares for Claude 3 Haiku at 25 cents.
“Embeddings are 10x cheaper than generation.” The slide title at 07:15. On the slide, text-embedding-3-small is 2 cents per million tokens; Claude 3 Haiku, itself a cheap generator, is 25.
2¢per million tokens for OpenAI's text-embedding-3-small, September 2024
1/12of the price of Claude 3 Haiku, already a cheap model (25¢)
~2%of exam answers where the “same” embedding came back different

He called them practically free, and he valued something else even more: they're quantitative. The same text mostly gives the same numbers, so a distance computed today can be recomputed tomorrow. He was candid about the word “mostly”.

“Heck, I teach a course at IIT Madras, I set an exam paper assuming that it would always come back with the same response, and I found that roughly in about 2% of the cases it does not, and therefore I had to give the students full marks for getting the wrong answer.”

S Anand · 07:48

So experiments are mostly reproducible unless somebody changes the model behind the scenes. Is the distance accurate? Interpretable? That's a second-order question. Hold on to that sentence. The last question of the day comes back to it.

Checked, two years later

For the explorers below, this page re-embedded the talk's 150 movie quotes with the same model on 16 October 2026. The nearest pairs and loneliest quotes match what the stage app showed in 2024. That's evidence the method is repeatable. It doesn't prove the numbers are identical: providers retire models, preprocessing changes and caches hide drift, which is why Anand's 2% surprised him.

Reel 5 · 08:47 – 11:22

Apple, in 1,536 numbers

To show what an embedding looks like, Anand didn't open a notebook. He pasted a prompt into a code-generating chat app and let Claude 3.5 Sonnet write the tool.

The prompt on the slide · 09:00

Create a pretty vanilla JS single page web app using only CDN libraries. Show a form with a textbox for PHRASE. On submit, fetch this CORS-enabled API and show the JSON response as a pre-formatted 2-space indented JSON. Use this code:

curl -X POST [internal LLM Foundry proxy]/openai/v1/embeddings -H "Content-Type: application/json" -d '{"model": "text-embedding-3-small", "input": "$PHRASE"}'

Verbatim from the published slide prompt, except the endpoint: it pointed at Straive's internal LLM Foundry gateway, which isn't a public API.

He typed apple and clicked the button. Out came a whole bunch of numbers. Lots and lots and lots of numbers, about 1,530 odd in this particular case. The model, text-embedding-3-small, returns 1,536 by default. And these numbers are what constitute the meaning of this—of this phrase, ‘apple’.

Screenshot of a web app titled Phrase Embedding Generator with ‘Apple’ in the text box and a JSON result listing the start of a long list of decimal numbers.
The generated app, live on stage 09:27. Each number alone means nothing. The meaning is in how the whole list relates to other lists.

Picture each list as a point in a space with 1,536 directions. “Apple” and “orange” land near each other. To measure how near, Anand named the usual tool: you probably calculate it as a cosine similarity, the cosine of the angle between the two arrows. Pointing the same way gives 1; unrelated gives about 0.

He took a short detour that's now a familiar refrain. Me, I have moved entirely towards LLM-based coding. The deck itself was a reveal.js page generated from a prompt that's in the repository. Beyond that, the only thing that I did was the last 15 minutes of ‘Oh, maybe we should do this, maybe we should do that.’ That is purely obsessive-compulsive disorder.

Reel 6 · 11:22 – 13:25

“It's showtime!” meets “It's clobberin' time!”

Next came about 150 lines from a public list of movie catchphrases, from Beetlejuice, Beetlejuice, Beetlejuice! to Bond, James Bond. He pasted them into a clustering app, clicked Cluster and slid the similarity cutoff up until only the closest pairs stayed joined.

Top of the list: ‘It's showtime!’ and ‘It's clobbering time!’ Yeah, very similar, absolutely. These two: ‘They're here!’ and ‘They're back!’ Okay, that's also fairly similar, fair enough. At the other end were quotes with no close neighbour at all. ‘Machete don't text’ and ‘There can be only one.’ These are practically outliers. His summary had two numbered lines: outlier detection now becomes a whole lot easier; cluster detection becomes a whole lot easier.

Look at why those pairs match. “Showtime” is Beetlejuice; “clobberin' time” is The Thing from Fantastic Four. They share a rhythm, “It's ___ time!”, not a meaning. Proximity is useful, not comprehension. Try it yourself.

Explorer 1 · the neighbourhood

Pick a quote. See what the model puts next to it.

These are the talk's 150 catchphrases, embedded with the same model it used (text-embedding-3-small, 1,536 numbers each) on 16 October 2026. Every similarity below is a real cosine computed from those full vectors. The map is only a flat sketch: 1,536 dimensions squashed into two, so trust the numbers, not the distance on the map.

Data: movie-quotes-catchphrases.csv from the 2024 repository, catchphrase column only. The map is a t-SNE projection of the vectors. Cosines are stored to ±0.002. The 2024 stage numbers weren't published, so the values here are a 2026 recomputation, not the original run.

Clustering app on stage: a ring of blue dots with a single short line joining two dots, labelled ‘It's showtime!’ and ‘It's clobberin' time!’ in a tooltip.
The same pair on stage 12:07. With the cutoff high, one link survives. Lower it and a network appears, as in the explorer above.
Reel 7 · 13:25 – 19:12

Name it, then overrule it

Then Anand asked the question that turns a toy into a tool. If we can do cluster detection, then maybe we can get a sense of what are the different types of movie quotes in this list of movie quotes without actually having to sit and read and classify all of these, right?

He asked for 15 topics. The app grouped the vectors with bisecting K-means (Why bisecting K-means? I don't know, I just read that it was slightly better), then sent five samples from each group to GPT-4o mini with a short prompt: Suggest 2-4 word topic names for each cluster. Capture the spirit of each cluster. Out came Philosophical Reflections, Legendary Heroic Phrases, Classic Film References, Inspirational Life Lessons and others. He admitted the number was a choice: I could have chosen maybe 16 clusters, 14 clusters, however many clusters—it's entirely arbitrary.

Not every name fit. Inspirational Life Lessons held Forrest Gump's box of chocolates next to ‘There can be only one,’ that's probably not inspirational. He changed the prompt to ask for Hindi names and got them. In other words, the prompting part of it is all yours.

Topic list on stage. ‘13. Philosophical Reflections, 8 distinct docs’ lists Well la-de-da, Hasta la vista baby, Frankly my dear I don't give a damn, A martini shaken not stirred and the Badges quote. ‘14. Inspirational Life Lessons, 4 distinct docs’ lists After all tomorrow is another day, the box of chocolates line, There can be only one (−1.7%) and Go ahead make my day.
Named by a language model, scored by geometry 14:25. The percentage after each quote is a silhouette score, roughly how much closer it sits to its own cluster than to the next one. Negative means it would fit better elsewhere, like “There can be only one”.

The next step is the one that matters most. He deleted the generic topics, I want only specific stuff, kept the rest as his taxonomy, and asked the app to file every quote under it. Beetlejuice's triple summons scored 37% for Creepy Movie Moments. ‘It's showtime!’ very strongly falls under High-energy Showdowns. No retraining was needed and nothing was re-embedded: the topics are just more vectors, and classification is one matrix multiplication.

“In case you're not familiar with what a dot product is, just think of it as one function which is surprisingly fast even without a GPU.”

S Anand · 18:25
  1. Encode onceEach quote becomes 1,536 numbers. Paid for once, cached.
  2. CompareDot products between every pair: neighbours and outliers.
  3. GroupK-means proposes clusters. The number of groups is your choice.
  4. LabelAn LLM reads samples and suggests names. This is the only generative step.
  5. EditA person deletes, merges and renames. The taxonomy is now yours.
  6. ClassifyTopic vectors × quote vectors. One matrix product.
arithmetic on stored vectorsgenerationhuman judgement
Explorer 2 · group, then overrule

Let the data propose groups. Then impose your own.

Same 150 quotes, same 2026 vectors. Propose replays the talk's bisecting K-means, computed in advance for every group count from 4 to 15 and five random seeds. Impose files each quote under whichever of your chosen topic names is nearest, using dot products computed live in your browser.

How many pairs of quotes this seed keeps together like seed 1 does (adjusted Rand index; 1 = identical, 0 = chance)

The vectors are unit length (every norm is 1.000 ± 0.001), so each dot product here equals the cosine similarity. A “close call” means the top two topics are within 0.01 of each other. K-means used scikit-learn's BisectingKMeans with k-means++ initialisation, as in the talk's notebook. On stage, the deleted topics were the generic ones, Iconic Movie Quotes and Memorable Catchphrases, and they start switched off here.

Two lessons fall out of the explorer, and neither came up in the talk. First, the groups are not stable. With 14 groups, the adjusted Rand index between seed 1 and the other four seeds is only 0.17 to 0.26, where 1 would mean identical groupings. K-means is a way to draft a taxonomy, not to discover one. Second, the filing reflects the names as much as the quotes. With the stage's twelve specific topics, Witty Comebacks takes about a third of the 150 quotes, Philosophical Reflections gets none, and about 35 quotes are close calls. A good taxonomy is a human decision, and so is the check that it works. The vectors only make it cheap to apply.

Dot product or cosine?

On stage Anand said the dot product would give you the distance. Strictly, it gives a similarity: bigger means closer. It equals the cosine only when the vectors have length 1, which OpenAI's do. For vectors of other lengths, normalise first, or a long vector will look similar to everything.

Reel 8 · 19:12 – 22:01

Words and photos in the same room

But the best part of it is we can apply this to images as well. Anand used Google's multimodal embeddings API on Vertex AI, and he was honest about why: this happened to be the first one that I found and it worked well enough, so I've never really dug deeper into this. Send it a sentence, a photo or a video clip and you get back a vector. The point is where the vectors land.

“…it embeds them all in the same space, which means that the distance between the photo of an apple and the picture of an apple is actually very low.”

S Anand · 19:59
One shared space versus two separate ones Left: a single multimodal model puts the word ‘smile’, the word ‘frown’ and two photos as arrows from one origin, so the angles between a word and a photo can be measured. Right: a text-only model and an image-only model each have their own axes; an arrow from one cannot be compared with an arrow from the other. “smile” (text) “frown” (text) photo A photo B text model image model One multimodal model: angles between words and photos mean something Two separate models: different axes ≠
Comparable only if trained together. A word vector and a photo vector can be compared only when one model was trained to place both in a shared space, as CLIP-style models are (Radford et al., 2021). Google's current documentation says its image and text vectors are in the same semantic space with the same dimensionality. A text vector from OpenAI and an image vector from Google don't share axes, so comparing them means nothing.

His prototype, ImageExplore, plotted about a hundred actors' photos against two words at once. Similarity to “smile” ran along the x-axis and similarity to “frown” along the y-axis. He introduced them as Bollywood actors, but the faces on screen were Hollywood stars. So what this is saying is this picture of Tom Holland is the closest to a smiling picture; Tom Holland's smiling the most. At the other extreme: Clint Eastwood is totally frowning. Yeah, we get that.

Scatter of about a hundred small actor headshots. Dropdowns at the top read Smile and Frown. Clint Eastwood's photo is high on the frown axis at upper left; smiling photos of Tom Holland and Ryan Reynolds are at the far right.
“Smile” on one axis, “frown” on the other. The image from the 2024 repository; the live version is at 20:32. The axes measure the photos: Clint Eastwood scores as “frowning” in this picture, not in general. ImageExplore's code isn't public.
The axis this page won't rebuild

At 21:27 Anand swapped the words for “African” and “Chinese” and read off which faces scored highest. It shows how easily a shared space will score a face against any word. It doesn't show anyone's ancestry, ethnicity or nationality. A score like that mixes skin tone, lighting, makeup, photo style and whatever stereotypes the training captions carried, and NIST's 2019 study found demographic differences in error rates across many face-recognition systems. That's why there's no word-against-face slider on this page.

Reel 9 · 22:01 – 23:35

Six months, six weeks, six days

Then the story turned from celebrities to payroll. A logistics client screened photos from drivers applying to join its fleet. Some were duplicates. Sometimes they don't send their photos—in fact, a whole bunch of people send the logo of our company as their photo. Sometimes they're wearing tuxedos, not a driver's uniform. They wanted no high-visibility vests and a plain background.

Anand's team did what any team would do: We will do training, we will do computer vision, blah, blah, blah, and budgeted about six months. The client wanted a six-week pilot. We looked at our hands and said, ‘Okay, maybe we can try with LLMs.’

“The sixth day, the project was done. All we did was this: took the embeddings, and we found that if we take the word, let's say, ‘high-visibility vest,’ it has a good match. ‘Plain background,’ it has a good match.”

S Anand · 22:50

The rest of the anecdote is a comedy of early delivery. Nobody knew what to do with the remaining five weeks, so the client asked for enhancements. We literally had to tell them, ‘Boss, these are once-in-a-lifetime occasions when the project actually gets done before the budgeted time. Let the team have a holiday.’ Then came the line that is the real thesis of the talk:

“I don't think the world has gotten used to the scenario where we don't have to train models anymore to get to the result that we want. The pre-training and some amount of prompt engineering works well enough.”

S Anand · 23:16

Notice what the trick is. Nobody trained a vest detector. They wrote the requirement down as words and measured each photo against it. Changing a rule means typing a new phrase, not labelling a new dataset.

What the anecdote does and doesn't establish

This is the speaker's account of a client pilot. The talk gives no dataset size, error rates, thresholds or comparison against a trained model, and doesn't say whether duplicate detection worked or what went to production. A responsible version would hold out labelled photos, set each threshold on a validation set, report false accepts and false rejects for every check, and route uncertain photos to a person. It would also need applicants' consent and a check for demographic differences in error. A photo-requirements checker isn't an identity checker, and a cosine is not proof of who someone is.

Reel 10 · 23:35 – 26:50

The audition

Finally, the question the room had voted on. ImageExplore compares every photo of a set of Bollywood actors with every other one. A slider sets the minimum similarity. Raise it, and only the closest photos stay joined. Search for a name and you see who stays linked to it.

Before the stars, Anand showed what the slider does at its extremes. The three most similar photos in the whole set were two photos of Riteish Deshmukh and one photo of Shreyas Talpade. The three loneliest were Manoj Bajpayee, Tusshar Kapoor and Kunal Khemu: Very atypical Bollywood faces.

Explorer 3 · what the screen showed

Pick a casting call. See the stage result, and what it can't tell you.

Each panel is a frame from the recording, cropped to the result. There are no scores here that weren't on the screen. ImageExplore's data and code weren't published, so this page can't rerun them.

Anand treated it as a game, and said so. So now as a casting director, I have at least one backup. But look at what kind of answer the tool gives. Every result is photo to photo. Deepika Padukone's nearest neighbour was another photo of herself. Ranveer Singh and Vicky Kaushal share similar beard, similar look, which is a fact about two pictures. In a different pair of photos, the beards and the result might not match.

Pose and cropTwo three-quarter profiles can sit closer than two photos of one person facing different ways.
Light and colourWarm stage lighting, black-and-white stills and saturation all shift the vector.
Expression and groomingA grin or a beard is a strong signal, and both change from film to film.
Who else is in the set“Nearest” only means nearest among these photos. Add a photo and the answer can change.
The model and its settingsAnother model, crop size or slider value is a different measuring instrument.
What the model learnedTraining on captioned web images may tie a face to a name, a film or a partner. That's the next question.
Reel 11 · 27:30 – 29:11

“I have no clue”

Anand closed with three adjectives, cheap, quantitative and multimodal (26:50), and the applause came. Then the session host, who had seen him at PyCon over the years, asked the best question of the afternoon.

“…using an image embedding model for this that also embeds text, so it's the same model—so I'm curious why it wouldn't rank spouses in similarity as well? Like, why is it just visual?”

The session host · 27:30

The question goes deeper than it sounds. A model that learned from captioned photos might place a star near their partner, not because the faces match but because the two names keep appearing together. Then “similar” would partly measure fame and association, not just appearance. Anand didn't bluff.

“I can give you a long answer to that, but I'll give you a summary of the answer. I have no clue. I have no idea how they're trained. I haven't even explored what similarities work and don't work. Great question.”

S Anand · 28:13

That's the real ending of the talk. After half an hour on how easy similarity is to compute, the speaker admits he doesn't know what property the number measures. A cosine is a measurement. Like any measurement, it needs a test of what it actually measures. Anand sketched that test himself, using Deepika Padukone and Ranveer Singh, who are married: Two people looking image-wise roughly equally similar, but one pair is husband and wife and another pair is not—are they going to come closer together? If so, how much closer? Worth exploring.

A test that could prove the look-alike finder wrong

Run this before trusting “nearest” as “looks alike”. Decide in advance what result would count against the method.

GroupExample designWhat it controls
A · CouplesReal spouses whom independent raters judge not to look alikeAssociation without resemblance
B · StrangersUnrelated people matched to group A on rated dissimilarityThe baseline
C · Look-alikesPairs that independent raters do judge alikeProves the tool can see resemblance at all
  1. Several photos per person, from different years, poses and lighting. Report how much the ranking changes between photos.
  2. Blind human ratings for every pair, collected before any model is run.
  3. Same crop, same model, same settings for everyone. Then repeat with a second, independently trained image model.
  4. Unknown faces as a control: consenting volunteers whose names can't be in any training data.
If couples ≈ strangersThe spouse worry fails this test, and the tool measured faces, not fame.
If couples sit reliably closerThe space mixes in association. “Nearest” isn't “looks alike”.
Reel 12 · 26:50 · 29:11 – 30:28

Cheap, quantitative, multimodal. And conditional.

“But the message that I wanted to leave you with is primarily that embeddings are really cheap. They are quantitative and therefore reproducible, perhaps even reliable, and they are multimodal. Just use them a bit more.”

S Anand · 26:50

Notice the hedge: perhaps even reliable. “Cheap” was true at September 2024 prices. “Quantitative” held up: two years later the same model still puts his quote neighbours next to each other. “Multimodal” worked on stage. What the talk shows is narrower and more useful than “no more training”. A pretrained model's vectors are a cheap first baseline, good enough to try an idea in an afternoon. Whether they're good enough to decide anything is a separate, measurable question.

Two last audience questions showed how that afternoon worked. LLM Foundry, the app behind half the demos, is Straive's internal gateway: Because it's an internal router, you don't get access to it by default. And the charts? All the visualizations are from some LLM where I just said, ‘Do this, this is what I want,’ and it generated it. The code got cheap too. That's how one person could build the demos, the slides and the look-alike search himself.

So the Irrfan answer stands, in its narrow form: in one 2024 photo collection, under one Google model and one slider setting, a Nana Patekar photo sat close to an Irrfan Khan photo. Whether that's resemblance or something the model learned about two famous character actors is the next experiment. Anand said it best: Worth exploring.

After the credits

Four things to take from a 2024 talk

01

Cheap changes the questions

A $5,000 whim was refused in 2021. At 2024 prices the same whim cost pocket money. Falling prices don't just save money; they decide which questions get asked. 02:59

02

Encode once, ask again

Stored vectors answered “what's near this?”, “what groups exist?” and “file these under my topics” without retraining, using a dot product. 18:25

03

Nearest is conditional

Every match depends on the model, the photo, the crop and who else is in the set. “Nana Patekar, maybe” is a result about photos, not people. 26:22

04

Ask what it measures

A cosine is easy to compute and hard to interpret. Decide what result would prove the tool wrong, then run that test. 28:13

Evidence ledger

What the talk said, and what the record shows

Quotes are from the recording. Stage details were checked against frames of the video. Statements published after 21 September 2024 are marked as later.

ClaimIn the talkRecordStatus
Date, place, slotPyCon India 2024Schedule: Track 1, 21 Sep, 14:45–15:15. YouTube description: “21st Sep 2024, NIMHANS Convention Centre, Bengaluru”. Uploaded March 2025Confirmed
When the question began“this was 2021” 01:14; “almost two years ago” 26:222021 to 2024 is about three yearsLoose recollection
Favourite random numbersClaude 3 Haiku 42, GPT-3.5 Turbo 47 01:14Slide image llm-random.webp; data in llmrandomMatches slide
$5,000 and two months refusedCo-founder's reply 02:59Slide: A100 at $3.40–$4.10/hour, “~$2,500 per month”. No budget documentSpeaker's recollection
“Claude 3.5 Haiku” moved the frontier05:46Screen tooltip: claude-3-haiku-20240307, $0.25/MTok. Claude 3.5 Haiku was announced 22 Oct 2024 (later)Verbal slip: Claude 3 Haiku
$10K versus $4M06:29Slide: GPT-4 $30 vs Gemini 1.5 Flash 7.5¢ per million input tokens, a 400× ratioIllustrative ratio, not a project cost
Embeddings cost ~1/10 of Claude 3 Haiku07:01Slide: text-embedding-3-small 2¢, 3-large 13¢, ada-002 10¢ per million tokens; Claude 3 Haiku 25¢Sep 2024 prices
Embeddings mostly repeat~2% changed in an IIT Madras exam 07:48No published audit. Recomputed Oct 2026: same nearest pairs and outliers as on stageAnecdote; consistent
“about 1,530 odd” numbers for “apple”09:27text-embedding-3-small returns 1,536 by defaultRounded
Closest quotes; outliersShowtime/clobberin', here/back; Machete, There can be only one 12:072026 recomputation, same model: top pairs 0.63 and 0.61; loneliest quotes Machete (0.26) and There can be only one (0.29)Reproduced
Dot product “will give you the distance”18:25It gives similarity (higher = closer); equals cosine for unit vectorsWording
Images and text in one spaceGoogle multimodal embeddings 19:12Google docs (current): image and text vectors share one space; default 1,408 dimensions; English text up to 32 tokensConfirmed for that model
“100 Bollywood actors” on smile/frown20:32The faces on screen are Hollywood actors (Tom Holland, Clint Eastwood…)Spoken slip
Driver-photo pilot done on day six22:50No client, data, error rates or deployment details publishedSpeaker's anecdote
Look-alikes: Varun, Priyanka, Vicky, Katrina, Nana24:54–26:22Frames show the linked photos and filenames. No ratings, no published data or code for ImageExploreAs displayed; photo-level
Do spouses embed closer?“I have no clue” 28:13Untested here. See the experiment aboveOpen question

Transcript

Open full size ↗