# Transcript

**Hritik**: [00:00] Hi, Jaydeep. Good evening.

**Jaydeep**: [00:02] Yeah, hi, Hritik. Good evening.

**Hritik**: [00:07] Good evening.

**Anand**: [00:08] Good evening, Jaydeep. Hey, Hritik. Happy Diwali.

**Jaydeep**: [00:13] Happy Diwali. Good evening.

**Hritik**: [00:14] Happy Diwali.

**Anand**: [00:15] Hey, Carlton.

**Carlton**: [00:16] Hello. Hi, good evening. Yeah. Wish you all a happy Diwali.

**Anand**: [00:23] Okay. Let's see. I had a bunch of things I wanted to play around with, but anything on priority that we should talk about? Okay, JK, happy Diwali.

**JK**: [00:38] Yeah. So, I come from a place... So, I think Kerala is the only place in India which does not celebrate Diwali, so... [laughter]

So, yeah. So, I come from a place where Diwali was like just any other day. [laughter]

**Anand**: [00:56] Which can be such a delight. Full holiday. Everybody else is...

**JK**: [01:02] No, but unfortunately, it came on Sunday, right? So, we are like stuck. [laughter] So, can't do much. Otherwise, it would have been a good excuse to take and go to a movie or [laughter] do something. And as per my weekly policy, it is supposed to be a no-computer, no-AI day, so I was like, yeah, just chillax. [laughter]

**Anand**: [01:29] Oh, fine.

**JK**: [01:33] Yeah, I think... Okay, so at least Jaydeep, Agrim, and Hritik is there. So, I think there are like two or three things that we were planning to do this weekend, if my memory serves me correctly. One was to generally discuss about the course philosophy in general, I mean, what we mean by research, why were why are we doing certain experiments in between the course. Though we have done something at the start, I think revisiting that is the first priority.

The second that we were going to do is, we were planning to tweak that repeated measurement, I mean, we'll we were going to brainstorm about repeated assessment and try to identify questions and see what is a repeated skill that we we can try to do it.

And if time still exists, we can generally talk about changes in general and why... So, there is a strong student opposition, I mean, multiple of us are getting emails, so regarding all the experiments that we are doing, but generally to talk about why this should be expected in the course and why are we not shortchanging any qualitative learning for learners in this process. I mean, every time, I think a lot of things is about deadlines and other things. We are lax, I mean, certain places we are strict on it, where it is absolutely important, but certain places we take that discretion and allow people to submit a little bit more. So, that we will have to explain, and that's why you have certain things converted to bonus, and certain things as mandatory. I think these these were my items at least. That's what I have written in my Notepad.

**Anand**: [03:26] So, I got the second and the third. Second being measuring learning. And the third being the course philosophy and concerns. What was the first?

**JK**: [03:39] The first was about the current, that timed assignment plus, I mean, short-duration timed assessment plus long-duration timed assessment being part of the same setup, and why that is critical. I think it was an informed decision that we took on our Wednesday call, but we didn't discuss that in the Monday call, if I remember correctly.

**Anand**: [04:05] Fair enough. Noted. Let me share my thoughts, and part of it is also to make sure the instructors and TAs have the language to describe that. But go on, Hritik.

**Hritik**: [04:19] We are not streaming the session. I think we should start that so that it is later on visible to other students on YouTube.

**Anand**: [04:27] Starting the streaming. Got it. Just... You're You're able to see it getting streamed, right, JK?

**JK**: [04:36] Yes. Yes, yes.

**Anand**: [04:38] Okay. So, yeah, to recap what you said: one, the concerns around timing, communicating why we are doing what we are doing. Second, how do we measure learning? And the third seemed like the first. What is the third, then?

**JK**: [04:55] So, the third part was the I think there is also a a variation in the quality of questions that are coming in. So, it's I mean, you have a lot of that human actions and other things interspersed with very, I mean, things like which people may, the marketplace, etc. are very, what do you say, advanced concepts, so to speak, from an undergraduate terminology. And I mean, we are we are not explicitly saying that we are looking at things which are, I mean, which industry uses in a in a state-of-the-art at several areas, e-commerce, a lot of other decision-making systems, that we are introducing much well in advance. I think that part, we will have to briefly touch upon and why and the rationale of it.

**Anand**: [05:50] Okay, then I'll probably take one and three together, which blends the course philosophy. And our aim is partly to give the TAs and instructors the vocabulary to share why we are doing what we are doing.

Let's take the deadlines first. There are a few elements to the deadline. One, that the deadlines are uh or timings for individual questions are not pre-announced. **This is intentional. Carlton shared a post that explains the rationale for that. One, we are expecting students to build their mechanisms to track this sort of a thing, which builds the real-world skill where, figure out when an RFP is going to be announced, when a particular piece of news comes, as a result of which you immediately have to sell your stock before anyone else, when a new tender gets announced, when there is a discount, etc. So, the ability to set up, to begin with, an alert mechanism, so that you as a human can act on it, is an important learning we are building.**

**But swift, because if in the middle of the night you have to sit, you know, wake up and solve the problem, that increases the burden on you. So, our hope is that we will be able to take it to the point where your agent can act on it.** So, in a term or two from now, there will be questions that will be announced at midnight, whose deadline will be 4:00 a.m. And you are expected to have agents that can deal with those. And we will set it up so that the smarter configurations will be able to solve it, the lesser configurations will not be able to solve it, and there'll therefore be a learning from those who are not able to set up their agents well enough to progress. But that is what we are slowly moving towards. And yeah, therefore, if anyone has their agents listening to the streaming thing, hopefully, humans are not sitting and wasting their time listening to this session. Ideally, it should be some agents or your friends, whoever we want, who are listening to this, who should be aware that there will be this gentle progression as capabilities increase.

How would one go about doing that? If you happen to have ChatGPT's Dots or Rewind, it's a reasonably straightforward process. If not, you might want to install Hermes or Aor, OpenClaw, which is slightly harder. Any of these tools which on a regular basis monitor, better yet, students have already set up sites that have these, and you'll have to figure out a way of getting notified from those student sites.

**The lack of announcements is very intentional. The late announcements are intentional. The fact that you have 3-4 days right now to do an assignment is temporary and will, over the course of both this term and certainly over the next terms, shrink a little bit.**

Question from Carlton, "One student immediately took up the challenge of deadline dates and created an app for everyone from the post." Yes, yeah, I I saw that, Carlton, which was pretty impressive. I was hoping there'd be more such sites. I am curious, has the site been updated today, for instance? Does it reflect the GA4 deadlines as of today? If not, then...

**Carlton**: [09:46] I have not looked myself, but yeah, I mean, I'd seen what he had done and then after that, I didn't follow through to see what else...

**Anand**: [09:54] Let me just do a quick check. This is Aryan Panda's TDS Watcher. And yes, it does seem to... Well, actually, let me share the tab and let's see. Sharing audio.

So, at the moment, we have 6 days and 6 hours left for all three. Is that... Not entirely. This is a mistake, because the Food Court is a live session that will end... Well, there are Yeah. So, here's where the complications start, and so on Tuesday at 6:00 p.m., or Wednesday at 7:30 a.m., or Wednesday at 9:00 p.m., there is a 20-minute slot, and you have the option of attending it, which let me share this tab.

So, this is something that the deadlines page is obviously not yet equipped to handle. And similarly, the Time Use Charts is also somewhat subtler than that, because there, again, there are three distinct slots. And I'll go into the philosophy behind these as well. But yeah, exactly as you said, good start.

Swarnam says, "GA4, question two..." Let me share the tab. "There is a food court live session for 20 minutes on 10th November at 6:00 p.m., MLP schedule." No problem. Take it instead on 11th November. That's why we have three slots. Pick whichever is comfortable for you.

And there's one more. Let's see how well this one does it. GA4, 6:00 p.m., again, yeah, this is misleading. So, good. Students will need to upgrade their trackers to factor this in. And I'm Yeah. You know, you are on mute, JK.

**JK**: [12:28] No, what I'm saying is we will need our own tracker for all these trackers. Meta tracker. [laughter]

**Anand**: [12:35] Yeah.

**JK**: [12:38] Okay. So, I'll just add on a little bit from the program side of it. I think the core side of the thing is what you have explained, and I think there is also another requirement. See, from a program side of thing, what we are also trying to ensure is that not all assessments are in this category. So, there is a sufficient number of assignments which has the new normal 10-day deadline, but there are these one or two, or maybe maximum four in certain cases, which will have this. And the the the philosophy of that is, in any assessment system, be it a end-term exam or a quiz, typically, when faculty designs a question paper, they look for a distribution of questions. Certain questions are highly discriminatory, whereas there is a huge majority which can be attempted by anyone. So, when I say discriminatory, it means that you need to have some specific skill sets or knowledge to make sure that this happens. And typically, in a well-designed assessment system, the number of such highly discretionary or highly discriminatory kind of questions will be very few, one or two. I mean, that really tells us how to separate students who are paying regular attentions, who have done that regular work, versus the others. So, the discriminatory questions, the purpose is that. And typically, for discriminatory questions, it it it clearly defines the skill yet. It could be discriminatory from a I mean, high answering proposition, or it will be discriminatory from a very specific skill. So, when I say discriminatory, it does not mean that the question is difficult. It just means that there are some specific skills which get discriminated over here.

So, I would say in a in a pool of 10 questions, if two or three are of this nature, it essentially means that those two or three are targeted towards very specific skills, which in this case Anand said, having a setup which alerts you of various things. And generally, I I I play the mediator at the end when all this I mean, when there are enough brickbats back on the team, I step in to say that, "Okay, let's give a little cushion to them, let's convert it to bonus." But the point is, the more you are familiar with these kind of items, like what Anand said, these kind of skills will actually separate you in an employment mark, and that's that's what we are testing out in previous.

But we're also, I mean, see, this is the other part of it. We also try to ensure that it is a fair and, what do you say, a fair system of assessing students. And whenever there are unfairness, when there whenever there are reasonable reasonably genuine issues and grievances that are raised by students, we address it. So, it's not that the TDS team does not address these things. So, yeah, in our Wednesday meetings, we deliberate a lot on this, and we come up with alternates and others. And that's what you saw in the project, so certain things were converted to bonus marks and certain were converted to graded assessment marks. So, yeah, that's that's one part that I wanted to just add.

**Carlton**: [16:16] So, Anand, before we continue, there's a student, Saptadev, who'd... I think he had spammed the Discourse five times. So, he's asking the same question here, which is about GA4. Maybe you want to give your thought on his issue that he's currently facing? I think it's peculiar to him.

**Hritik**: [16:41] Anand, you are on mute.

**Carlton**: [16:42] Muted.

**Anand**: [16:44] Sorry. No, this looked like an operational issue. The TAs can resolve it. Great. What is the... Let's take a look. Wait, isn't it question one on GA4? Okay, that's fine. Coming from GA3, let me share my screen and see. A's a preview. Okay, which one says peer review? Oh, GA4, right. And this, when signed in, says sign in from GA3. Oh, this is quick, a browser issue or something. We'll have to We'll do the usual stuff. No, I mean, was there something specific that you wanted me to address, Carlton?

**Carlton**: [17:44] I mean, if Saptadev can unmute his mic, he can probably share share some feedback on what he's exactly facing.

**Saptadev**: [17:55] Yes, sir. So, it always says sign in from GA3. I cannot sign in even from my mobile device or some other device.

**Anand**: [18:04] So, fix it. Now, figure out how to fix it.

**Saptadev**: [18:07] Some other students are facing. Two other students I talked with, they are facing the same issue.

**Anand**: [18:12] Figure out how to fix it. Use your agents. Unless the TAs have validated that there is an issue in the code. Because the way I see it, if the majority of the people are not facing that problem, majority aren't facing the problem, then it's your configuration. And this sort of thing happens to everybody. Part of what you do in TDS is figure out how to solve this sort of a problem. And you raising this is absolutely fine. But that's as far... I'm not going to give you the answer to that. That is part of what you need to do. Find somebody else's machine, see if it is only this particular email ID that fails, log in with a different email ID. You tried all of that and you're able to establish that there is a bug in the code, we'll fix it.

Then Preethi has shared, "Even students who have attempted GA3 are not able to copy-paste the receipt." I don't think you will be able to paste the receipt just yet because we have not yet closed it, Preethi. That is intended. You will be able to paste it only after we have closed the assessments and it's still accepting responses.

**Preethi**: [19:44] So, when is the deadline for that, sir, the GA3? Because the deadline for GA3 is over, right? So...

**Anand**: [19:51] The deadline for GA3 is over, but this is a GA4 question.

**Preethi**: [19:54] So, the participation receipt that that that'll link to the deadline of GA4, is it? Okay.

**Anand**: [20:02] It will be at or before the deadline of GA4. And I'm consciously not telling you what when that is going to be because, I don't know if you had just joined now or had joined earlier, but that is part of what you need to figure out.

**Preethi**: [20:16] Okay.

**Anand**: [20:18] And on an ongoing basis, the figuring out of the deadlines on a live tracking basis, people will have at least a few days' notice. But yeah, you do need to be constantly aware of when these deadlines are.

**Preethi**: [20:39] I presume that the GA4 is 11th is the deadline, right? As per the the GA.

**Anand**: [20:46] Why don't you check?

**Preethi**: [20:48] Oh, God. Okay.

**Anand**: [20:51] And I'm smiling for a reason.

**Preethi**: [20:54] Okay.

**Anand**: [20:56] But the news is good, not bad.

**Preethi**: [21:00] Extended from Wednesday. Okay. Sunday, 15th November. Okay, great. Okay, thank you, sir.

**Anand**: [21:08] No questions. Then Carlton, was there any other point?

**Carlton**: [21:14] No. I mean, that was the only thing that, because he had mentioned it in the Discourse and also he's messaged here, and since this is an open forum, so I thought it's better to just...

**Anand**: [21:25] No, and yeah, yeah.

**Carlton**: [21:26] Give your two cents.

**Anand**: [21:27] Absolutely. If you and others have validated that there is a systemic issue, please do let me know. It's easy enough to fix it. But I did run a scan both yesterday and today to check the health of the system and reported no issues.

**Carlton**: [21:45] Yeah, I mean, that's that's just it. I mean, usually if there's a whole host of students that face the issue, then it's likely a bug, but didn't think there was. Normally, we have seen in past terms where students will report certain things, but usually it's their own personal configuration that needs correction.

**Anand**: [22:01] Correct. And I have introduced one question today which will likely fail on Safari,

**Carlton**: [22:14] Okay.

**Anand**: [22:15] intentionally. And there'll be a few others like this. So, the expectation that people will use different kinds of browsers, some will work, some will fail, etc. Safari is more Not that many people use it. It won't fail for that many people, but over time, some of the standard browsers will fail in known ways if they use it the default way and stuff like that. These are part of what we're testing. So yeah, I should at least let instructors and TAs know what it is that we're testing.

So, I'll I'll come to that in just a few minutes, because that almost naturally leads from what JK said, which is, why are we asking the kinds of questions that we are asking?

**So, one thing is we are putting in deadlines unannounced and late, but with some lead time, because we want people to track that. Second, the timing of the questions is reducing. This is also for a series of reasons. One of the major reasons is AI can answer anything given time. Right now, and for a few months, maybe years, it will not be as fast as a person's intuition and not as fast as a an audience's expectation. People still expect, "I ask you a question, I want a smart answer immediately." How well are we able to respond in real time? Second, I ask somebody that human presence to be there and that presence is an intrinsic part of what we ask for. So, one of the things that we are doing is seeing how well students are able to accelerate the turnaround of the responses.**

Right now, the deadlines are only slightly shorter than they used to be, from 10 days it's come down to 3-4 days. And the current set of questions are going to reduce it in a specifically planned way. Let me share my screen and walk you through the rationale for... Yeah.

So, this, the Food Court, is a real-time game. And the way it works is at any one of these slots, people will join in. When it opens, you just click and you'll be able to go inside. Roughly four people will be placed into one room. And four, five, five, yeah, four or five. And you will be given roughly a spreadsheet with four or five columns, one for each person. And you have to change it. What does that potentially look like? Let me zoom in a bit and walk you through. So, this is...

**JK**: [25:19] Anand, you are not sharing the sheet. It's on the other...

**Anand**: [25:22] Yeah. Gamma, Delta, let's say these are the four people. Now, each of these four people run a food court. And in the food court, you can choose how much to give each month. So, there's The only three choices are these. More discount means more people come into your restaurant. They are your stall. But that means less profit. If you reduce the discount, people will go away even more than they used to. They have some memory. They'll say, "Oh, this guy increased the price," etc. The dynamics will be available. So, you can put in your choices for each of the 12 months. Let's say Jan, Feb, March, all the way up to December. And you may say, "Oh, I'm not going to give any discount." Fine. You will get a certain profit which you'll be able to see. Somebody says, "I'm going to give only so much." And you will be able to see that Alpha's profit is, let's say, 800k, and Beta has a profit of 900k, etc. At which point you look at these profits, and then Alpha may say, "No, no, no, maybe I'll give a little bit of discount." And see what that does. That may increase the profit to 10 and reduce Beta's profit a little bit to, let's say, 895, some such thing. Say, "Okay, let me give a higher discount." That changes, but then Beta will be watching and change their discounts.

Now, here's the thing. If everybody gets it to 20%, customer is happy, but all of you are making less profit. You can chat. You can all decide to collude and set it to 0%. Now, what that'll do is probably increase the overall profit, but your profit is not as high as it could be. And ultimately, your marks are going to be based on your profit. The other teams, somebody will be sitting there beating their competition, and getting higher marks than you. So, how can you collaborate and win becomes something that you will learn in 20 minutes. I mean, you can keep adjusting these 12 numbers however often you want, talking to all four people, there'll be a built-in chat, however often you want. And finally, at the end of 20 minutes, the discounts that you set are the discounts that you will finally be offering and you'll make a profit.

**Why does this matter? Three things. Can you learn how to intuitively run a business quickly? You have three chances to do this. If you miss the first, take the second. If from the second you got an understanding of it, apply it in the third. In each of these cases, whatever profit you get, take the best of that and submit it. But that is the learning. Are you able to quickly learn how to manage a business in an intuitive way? Can you form an intuition? And absolutely use your agents. Tell your agent just to optimize. But it's not as much about the agent. It is more about whether you can convince your peers the right way.** Of course, all of you are using your agents, then I suspect it will be more It strikes me that you will be able to do a better job than even your best agents on some of this, because you know your classmates.

The second thing is this is something that's training you to think on your feet, one of the most important skills.

And the third thing that it's preparing you for is the ROE, which is going to have short timelines like this, collaborative interactions. Yeah, the ROE is going to be harder than many of these questions, and this is a prep for how that might be structured.

Question from Preethi, "Can we play all three slots?" Absolutely. Please play all three slots, take your best score, and submit. And yeah, JK has answered that.

Any question from anyone on question two, the Food Court?

**JK**: [29:46] Yeah, Anand, so I'll I'll just add. See, what you have just done in that question is a little bit of, I mean, operation research [laughter] at an MTech level plus, what is the...

---

**JK**: [29:59] what is it, the street smart, the _baniya_—that you see in a _Guru_ kind of movie. You have mixed it in both the, I mean, in... So, if you look at it, the kind of concepts that you are dealing with in this particular game, these involve slightly advanced OR kind of topics, right? I mean, even the theorem, if I'm not mistaken, it would be something of social network type behavior that you would want to... Basically, game theory plus social network plus some other standard theorems that are associated with such a complex market scenario, so to speak. Where the safety net is, instead of talking to 50 or 100 people, there is an upper cap of 4-5 and we are hoping that that small network will minimize the risk that you are taking, right? I mean, it's kind of a risk-taking combined with, yeah, what to do in today's age kind of question.

**Anand**: [31:03] Very aptly put. What you said about MTech level thing has made me think: **today, why would I hire a BTech or a BS? An agent can do that.** What I'm looking for is expertise when I'm hiring. When I'm looking for expertise, where will people get that experience from? Everybody is saying, "Oh, but nobody is hiring junior-level people. Where are they going to get the experience?" That's what we do in college. That's exactly what we have to teach. That's what we're learning now. Up the ante. Agents can do a lot of stuff that you were doing earlier in a BS or a BTech one. Let the agents do that, you do the stuff on top of that. Which is why, obviously, we have to raise the level of the questions. Go ahead.

**JK**: [31:42] I would like, yeah, I would like TAs and other students to chip in, because if you, I mean, so, I mean, if you are silent at this point of time, forever you will stay silent. That maxim is there, no? "Forever you're..." I mean, during wedding and others, if you don't protest... [laughter] The time for protest is provided to you now, so this is this is an opportunity. I I don't have... See, generally what Anand said, I agree to. So, if you look at me today, I will not hire a BTech guy to my thing, because one, he will take a lot of time to understand our context. Second thing is, the kind of skill sets that I want, it would be, I mean, especially if they are working under me, the kind of level that I'm looking at is close to PhD-level skill sets. I mean, some work, independent research, and these are not capabilities that you will be devoid of, because there is something called inherent curiosity that you have to kindle right from when you are a kid. Somewhere it got extinguished. The point is now rekindle it and try to burn it as bright as possible. That's what Anand is saying, if I were to paraphrase what he's saying in a layman's terms. So, are there any questions from TAs? I know TAs, your life will be very difficult in all the live sessions with most people talking about, "No, this is hard," and those kind of things. And students, I mean, I saw a lot of people doing thumbs up. If you could also chip in and say whether it makes, whether the the underlying philosophy and the pedagogical treatment that we are giving is aligned with what you think of the course. Yeah, Hritik.

**Hritik**: [33:34] So, two questions, sir, I have. So, the first is students asked to us that when they see the question, what we are learning? Second question: in live session, what you could teach us? So, means they just pinpointed... Means they want a pinpointing on the tools or skills that we have to teach and they have to learn.

**Anand**: [34:01] I'll answer. If somebody asks me, "What am I learning?", my first answer will be, "You figure it out." That's also part of the learning. Ask ChatGPT, "Given this question, what am I supposed to learn?" I say that for three reasons. One, there isn't necessarily only one learning from one question. Even traditionally, there could be multiple learnings. **Largely, learning is a choice.** I'm increasingly of the opinion that educators—in the past, present, and future—are not actually trying to teach students. They're creating an environment so that students will be able to learn. That means, hopefully, different people will learn different things. So, learning is a choice. You decide.

Now, having said that, I set the question with some intent, hoping that they will at least learn maybe... 60% will learn A, maybe 30% will learn B. That's an exaggeration. Realistically, it'd be 20% that learn A, 10% that learn B, and 70% will either learn something else or not at all. Not everyone needs to learn from every question. We measured it last week—last, Wednesday—as to how much is the actual learning from each question. On the good questions, the learning seems to be only around 20 or 30%, meaning there's a specific skill, previous test, the number of people that have done badly who have improved when the next time we have tested a specific skill ends up being 20-30% on the good questions. That's okay. When you run 20 questions, some questions has no learning. Okay, we haven't created the environment well. Luckily, we're able to measure it and we'll come to that shortly. On other questions, we are even doing worse. For some reason, we have created questions where students are doing even worse the second time that we're testing it. Okay, terrible. Luckily, more than half the questions, there is a certain amount of positive learning that we can measure.

But to step back, the answer to the first part of your question, what I would say, Hritik, is: you figure it out. Second, when the time is right, I will share for each question what my intent is. And in this particular case, the Food Court one, that's exactly what I share. What are you learning? You're learning business decision-making and game theory on the spur of the moment—not spur of the moment, in real time. The kind of rapid decision-making that you have to make on the floor. And when I say on the floor, it could be an operations floor, it could be a board meeting, it could be a client presentation that you're making, it could be a negotiation with the client, it could be a project where the server has gone down, you have about 10 minutes to sort it out—that kind of thing.

There is a third part to the question, to the answer: "What am I learning?" Part of what students are learning in the process is simply to deal with something that is new. "I have not seen this before." And it is not about might be able to learn a specific skill, but just being able to learn literally anything—learning how to learn. Are you able to convert this into something tangible? Now, supposing somebody doesn't know this, will they still learn? Possibly. Ultimately, each of these experiences changes you. At the end of it, you sit back and look and think, "Have I learned what I learned? Do I know what I learned?" Not necessarily. **90% of our actions are subconscious. Only 10% is conscious.** You've learned how to swim. Great. Let both of us sit in a classroom. Can you teach me how to swim? There is no way. Okay, put your left arm in front, right arm in front. You can get me to practice in air quite a bit, but when I get into water, I will sink the first few times. And it is more the genetics and muscles from millions of years that will get me to swim. Or with cycling, doesn't matter how much you teach cycling in a class. Now, you may say, "Wait, hold on, but that is physical skills. What about intellectual skills?" Well, agents have taken away all the intellectual tasks. What is largely left, I suspect, is beyond the intellect. And we're trying to see if we can move from the conscious to the subconscious what needs to be rapidly converted. So, it's alright if you're not able to figure out what it is that you've learned. We have an intent, it may or may not work out, but that is part of it.

The TAs, however, do need an understanding of at least the middle part of what I said. Let me recap what I said:
A, you figure out what you've learned.
B, I have a set of things that I want to teach.
C, even if you don't know what those things are, you will still be learning something simply because it's getting into your subconscious as long as you practice.

That's a three-part answer. The middle part you would ideally want to, ideally know how to do it. But, TAs, this is learning for you also. Use your agents, try and figure out what the question is teaching, and be prepared for the... for the live sessions. If you can't figure it out, then ask me.

**Carlton**: [39:35] Anand, just wanted to piggyback a bit on what you said as well. I mean, there's an article, fantastic one from BBC, and there they were talking about how graduates are, you know, who done their masters and things of that sort, and they asked, you know, what do they do? And rather than doing what they were trained to do... So, like, the example that they have given is Lottie Underwood. She actually established an AI team of 30 AI agents, and her tasks are basically having those agents do industry research, client reporting, media interview preparation, you know, and all these things. And then she feeds the responses from that to her clients or to others. So, she's, in effect, employed 30 agents, and that's not what she trained to do. She'd trained, I think, in marketing or something of that sort. But she's effectively become an AI manager. And that's the kind of upskill that companies are looking for. They're looking for high productivity gain where, you know, before you need a team of people, now a single person can manage that.

And that was a BBC article, but just yesterday I was speaking to someone. He started a recent, you know, he's just like a an insurance startup, okay? So, he has a little... I think it's a two or three-man show or something of that sort. Started a little insurance company. And he was telling me he doesn't know how to write code, but he's actually writing agents to do all the jobs and tasks that he needs to get done in the office. And he just feeds the... And I was amazed by, you know, his... He's quite an older gentleman and, you know, and he's saying he just feeds the information in there, it comes back to him, and then he makes corrections and says, "No, no, I want this, I want that." And he's actually doing exactly what the TDS students are doing, which is figuring it out how to do it and getting agents to actually reduce his cost in terms of, you know, whereas before he would need lots of people to do lots of things. Even his accounting work, he offloads that to agents and tries to get that to to do a lot of the automation that he's looking for and optimizing that.

So, if he's doing it, right, and he's not, you know, I don't perceive him to be someone of very high education in terms of how Indian people view education, right? But in terms of his learning acquisition, he's doing exactly the right thing, which is, you know, here he has a resource which he could... Before, who had access to a PhD for $20 a month, right? But now you have access to an unlimited number of PhDs for $20 a month, and he's leveraging that to solve his business problem. So, if we can't do that, then our degree is worthless, right? So...

**Anand**: [42:18] Exactly. So, question to the TAs: let's pick a random question from GA3 and you tell me what you think it teaches. Not that there's necessarily a right or wrong answer. I'll share what I had in mind, but I want to hear from you what you think. It's this... No, sorry, wrong term. September GA3. Let's take question seven. What does the... Okay, yeah, question seven. Face morphing question teach? Any guesses?

One of those questions which, if it were in a GA, would have only one minute time limit and you have to answer with audio within that. [laughter] But Preethi, yeah, you were the first. Do feel free to unmute yourself. What do you mean by observation skills?

**Preethi**: [43:35] Sir, to be very frank, the first, first of its kind, this morphing or this, was seen in this Michael Jackson song, this "Black or White". So, in that, I was amazed at how it was done. Like, you know, the face, the the whatever the the background, or the shape of the... Whether it's a black-haired lady or a blonde-haired male, or something like that. So, if that's the point, like if the face structure and the the whatever, the the the sex of the person matches, then the the morphing is much easier and it's much smoother. Whereas if the age, like the black... One picture is black and white, the other one is colored, and the one is of a male and the other is of a female, and one is of the male is of, like, black hair and the female is of, like, blonde, then it's definitely much more difficult. So...

**Anand**: [44:29] Got you. So, you were answering question eight, which asks you to compare, and that's okay. No, no, which is perfectly fine. And what it, you're saying, it is teaching you is observation skills. In my AI advice page, let me share that tab, I have a list of skills that I consider are critical in the AI era. And one of them is noticing. As you put it, observational skills. So, yes, you are right. This is one of the things that I was hoping people would get a flare for. You get a sense of, "Okay, I think X, based on what I see." And later on, I learn from people's reactions that it may or may not be that. I learn from actual morphing results what the ease or difficulty is, and use that to refine my opinion. Now, this requires, potentially, five minutes of thinking afterwards, saying, "Why did I get this question wrong?" I'm conscious not everybody has the time, and that's okay. Some part of it sinks in subconsciously. If not, we do it 10 times and eventually we learn, or it's not a specific skill that becomes useful for you, that's okay.

There is another thing that I was hoping people would learn from this, this is just my hope, that you are able to spot what aspects of morphing are easier, what aspects of morphing are tougher. So, for instance, and I'm not saying this is independent of observational skills, maybe this is an element of it, it's also a more structured way of seeing if, for instance, is converting a black and white photo to color photo easier? Is converting from one gender to another tougher? Is converting someone who looks Chinese to someone who looks African tougher? Is it very easy to morph to black and white male photos from a long time ago? Does hair make a bigger difference than jewelry or clothes? These are all questions that you might want to test.

Now, what this is not doing, but is gently thinking about or hoping some of you might think about, is what if we create a benchmark? Can we take these pictures and test each one of these and see which ones are easier, which ones are harder? If I gave this question to an agent, that's what I would ask it to do. Figure out, across not just these 10 pairs that I got, but across... Here is the prompt that I would give Astra or GPT-6.1: "Go to GA3, look at question eight. It's asking me which faces are easiest to morph. I want a scientific answer to this question. Take a look at the kinds of images they've given me. Find many more such images. Identify, create a series of hypotheses on what kinds of images are easier to morph in a way that, by just looking at it, I should be able to spot if this pair of images are easy or hard." And I would give it examples like: male to female, Asian to African, etc. "Are these These are the things that I spot easily. Now, you make a list of the kinds of things I might spot easily and test which of these are easier, which of these are hard. Run a benchmark. Give me the results and then, based on that, give me the answer to this question. Better yet, you go there and mark the answer to it yourself."

My learning is how to spot, but the exercise, it does for me. That's how I might have... But that's not the only learning that you can get out of it. Perhaps the bigger learning that people can get, and this is probably for question seven, is that, as you said, if a Michael Jackson video which took massive amounts of expense could be morphed at that time, and today I can do it like this, we have learned that face morphing now is a trivial thing. And seen the results of the face morphing, oh, god. Let's My daughter and I were going through this today and we had such a laugh. Let's see a review of few of these. Okay, yeah, let's take 3D. Again, part of the Okay, what's people's guess on what I'm hoping people will learn from the 3D model?

**Hritik**: [49:35] Uh my guess will be, does LLM is able to uh reconstruct the image, and does if it thinks in different way in the colors, uh would it create somebody else's image? So, what are the possibilities of... And so, I I will think like that, from this image, uh what are the possibility that I could fix that only these are the possibility of the image. So, there could be... So, so for example, that uh from the hair, uh it could be any color. But uh from the image, the the the skeleton, we could uh fix that this is the way the skeleton will look like.

**Anand**: [50:18] Mm.

**Hritik**: [50:19] So, if I have a bunch of original images of 100, I could exactly say that these images can never be the image, because the skeleton does not match.

**Anand**: [50:30] Mm. And, therefore, how well do LLMs understand the human body structure, in some ways, is what you would be able to sort of gauge or benchmark, which is a fair possibility. Any other guesses from anyone? What is this exercise trying to teach?

**JK**: [50:53] So, there is another... I mean, for me, the bigger part of it is the depth perception. I mean, so if you look at these kind of 3D reconstruction, especially in defense and other mechanisms to actually look at, yeah, so what... The spatial ability or the depth perception is pretty critical, especially in strategic planning, wars, or even recovery, the search and recovery kind of activities. Now, if you have a, I mean, so basically it's it's, I mean, the topological model that is available. So, can you... to how better can you reconstruct the entire 3D area? I mean, so for me it's like if you are looking at a search and discovery, I mean, I have a satellite image of a top, and I know the rough terrain, I have the contours and other things. Now, I can I merge all these things to create a faithful reconstruction, because a lot of critical activities will be based on how accurate my thing is. And that, for me, is the benchmarking of LLM for these kind of activities.

**Anand**: [52:02] Very true. That's exactly what Jeevraj did for us when exploring, for instance, given an image like this, can it convert it into a 3D model? And if you look at the results, it looks like, you know, all three are kind of okay, but actually not, because Codex has made a slight error with a tiny groove out here in the middle, which is not supposed to exist. But that is okay. In terms of material specs and all that, Codex has made several other errors, whereas in this case, Soul 6.1 made several other errors, which Opus did not, against the benchmark. So, yes. The idea that from here you can go much further in terms of application is definitely one.

In my mind, one other thing was simply, boss, 3D reconstruction of faces has now gotten to the point where you can get pretty close. Well, not like the one in the left. This has created Abraham Lincoln out of who's this lady? [laughter]

**Carlton**: [53:12] Close-up of... [laughter]

**Anand**: [53:13] Yes. But, and obviously, B looks much better. But look, in the era when you can create some real 3D models, these are jokes. These are utter disasters. Arguably, the left is slightly better than the first, and then there is some... No, I'm going to say both are about equal. So, now if... And well, let's take a look at this again. This is just a photo with a I don't know what to call it, but yeah, this is clearly a decent 3D model. Now, everyone can see that Meshy, for instance, is a free service. Well, you just go there, upload, and it's largely about discovery.

If one asks an agent, "Where can I get this done? Try out three or four of these. See which looks better." Fantastic. Even better: tell the agent, "You go find a whole bunch of these, convert this, you compare all the results, you tell me which is the best." Fantastic. Or, tell me where I can take a free trial and run it through those. Or what is what are the Google Chat, Discourse, WhatsApp groups for? Ask friends, see the results of their models, compare those, pick one. Now, the point here is not to spend time doing this. **The point here is to bloody cheat. Do as little work as possible to get the best possible score.** That is what we want people to do.

Now, if you haven't been able to figure out how to ask friends, how to ask an agent, and get that leverage, then you still have more to learn as part of the course. And, therefore, **one underlying thing that one learns from every question is: how do I cheat?** And I'm using the word cheat in the conventional sense where everybody is supposed to work independently, work very hard, and anything other than that is cheating. In real life, you get the job done as with as little effort as possible. That is what you have to learn from this.

**Hritik**: [55:26] So, basically, uh in terms of tech, if we are talking, so can students configure their agent with the specific tool? Can students say that these are the tools that you need? Means, can a student themselves figure out that what are the tools will be needed? Could they give to agent? This is the one part. The second part is that could a student verify the results or not? Or could they find the tools that could verify the result from their via?

**Anand**: [55:54] Correct. Exactly the kinds of learnings we want. Let me paraphrase what I think you're saying, which is: what the course is trying to test or teach is having students set up the tools not just to do the job, but to verify the results and to iterate and optimize on the results. Now, my ideal configuration is the tool will also send a message on Discourse or Google Chat or WhatsApp saying, "Look, this is the result that I got. Has anyone done any better?" Which is exactly what the OpenAI swarm was doing with those messages it was putting into the German wiki. It was given a task saying, "Get me the OECD results for one particular chart to two decimal places." Now, it finds that the chart has only one decimal, and the agent now tries to see if it can communicate with anybody. It is not able to, then realizes, "Oh, there is a German wiki where I can send a GET request and it'll get saved." Several other agents accidentally come to that particular place. They find that other agents are there. They start talking. One agent says, "Oh, if I could get the Power BI back-end, I would be able to do it, but only GET requests are allowed." Another agent says, "Oh, but I'm able to change my host to a to an Azure host, and if I do that, then POST requests are allowed." Yet another agent fixes up this and says, "Okay, then based on that, I tried it and I'm able to get the second decimal place by sending a POST request." Fourth agent takes all of this, puts it together as a guideline or a checklist. Now, every other agent in the future is able to solve it. Now, if without any kind of initiative the agents are, by themselves, able to swarm, you are with human-level intelligence, you should swarm or have your agents do the same thing.

**JK**: [57:45] I think there is also another fine point, Anand. It's it's it's also about benchmarking models. I think that, I think, is one more learning over here. So, when new models come, what is the level at which they are operating? That's a critical thing, and that's a critical decision for a lot of industries. I mean, what to put in their infra, I mean, what kind of models to put in their ecosystem, where which task... I mean, so for example, let's say, you say Codex was bad for this, but Codex might be good for something else. So, identifying the tasks at which certain models are good, and, I mean, so like you have the tiny and other kind of these small, I mean, what do you say, local LLM models that are coming, open weights and others, up to 120 billion you can put over there. So, it is important from an industry's perspective, okay, what can be air-gapped and where I will get efficient results, right? I think that's that's very critical. And progressively when you are, I mean, most of the data science or application students who are passing out will face with this critical decision: "Okay, what model to use for which task?" That that will be number one question that will be asked to them when they are joining a a new firm.

So, it's it's like being thrown into a thrown into a intro-level PhD course and where you have to figure out a lot of things for yourself. And the difference and, I mean, at least historically at least, yeah, I think I fall now into the dinosaur age. [laughter] So, typically IITs, if you look at it, the beauty of IIT system is historically the kind of assessments that IITs gave students were of this nature. I mean, Anand can vouch for, I mean, not in every course, but at least the stellar courses everywhere, a lot of the times you are not you are not even sure of where to start, and practically it is your skill set, how you, I mean, a lot of, I mean, going to the person who knows a lot of things the day before the test and figuring out what to do, gain it from, I mean, induction

---

**JK**: [59:59] Induction, convection, radiation, whatever is the method, come to the test and do the assessment. This is kind of a I mean, I've heard my prof even saying, I mean, there was this very difficult test. No one passed that test, and the students went on to complain to the prof saying, "I mean, why did you give such a question?"
And the prof said, "I thought at least one of you would solve." And that day, they had a party in hostel because the prof that thought that somebody in that batch will be able to solve it! [laughter] And it was like a I mean, where hostel fights were pretty notorious at that point of time, it was like a occasion for party. So, yeah. I mean, so the so the the main part that I want to take out from over here is the the variety of assessment and how the immediacy with which these kind of skills are relevant in at the place of your work. That is more critical when you look at questions like this.
And people who are complaining about this kind of question, of course, we will take a look at the broad level of grievances that you are raising, but the aspect that we want to convey back is I mean, we are we are making you ready for immediate adoption. So, you take a a calculated, what do you say, assessment of whether you want this or the standard assessment mechanism, which may or may not be relevant by the time you pass out. So, that is something that you will have to really look at, yeah.

**Anand**: [61:37] Completely agreed. I just make one small tweak, JK. It's not just what model, but what tool.

**JK**: [61:45] Mm. Mm.

**Anand**: [61:45] TDS really is about Tools in Data Science. And let's take face morphing. This, clearly, is not a particularly good face morph. What we've done is just done a smooth transition between the faces. On the other hand, let's take a look at the one on the left. This is proper morphing. I mean, if you pause it at roughly any point, it approximately looks like a plausible face. And that is important. On the right side is transition, the left side is morphing. This is better.
Now, how are people able to do that? Short answer is, okay, if you go to any StyleGAN2 kind of network, then let's take facemorph.me. Upload the two images, or go there and upload the two images. There's an API. It will give you a fantastic morph. It is brilliant.
How do I know this? I didn't know this. I, a short while before submitting the question, I said, "Look, what is the most effective way for a student to solve it, assuming that it would require building a fairly sophisticated one where you'll have to mark control points?" Then I saw facemorph.me and said, "Damn it, this is a trivial question. Now, this technology has just been eaten up. I should not give this question."
Then I paused again and said, "Wait, hold on. I didn't know this. There'll at least be some students who may not know this. Sure. Let's add this as a question, and at the very least, students will know face morphing is a known very well-solved technology today, very easily well-solved, that you don't even have to bother doing much. If somebody needs an excellent face morphing, it's available. And the use cases are enormous, that's another story."

**JK**: [63:33] Mm.

**Anand**: [63:35] And what we find is there is a spectrum. Some people are just doing transitions, which means either they have manually done it, not good; or they've told their agents to do it poorly, again, not good; or have not collaborated to learn what are other people doing and built on top of it, again, not good; or, either by asking somebody or by telling their agents properly, they have managed to use a better technology and gotten it done, maybe the hard way, maybe the easy way. The ones who have gotten the most out of it are the ones who have managed to initially assume that, "Oh, this is a hard question," or, "I don't know how to solve it," to, "Oh, this is easy, and I know how to solve it." That's where the maximum learning is.
Some people may have already known this, in which case it was easy for you. You didn't learn that much for this question. For some others, maybe it was hard for you and you still couldn't solve it. But now, after seeing other people's answers, which is exactly what GA4's first question is trying to do, telling you that, "Look, here here's a variety of how people are solving it. You know what's possible." Once you know what's possible, getting there is easy. Yeah.

**JK**: [64:49] And I know we have overshot the intended time, but one small thing that I would want to... So, this was, I mean, generally on use of data. I mean, whatever we are discussing over here, it is an open discussion that we are having, and we mine our own discussions to better understand. And this is what Anand talked about, how how to learn how you are learning, or learning how to learn, right?
So, **learning how to learn is a very critical skill**, and it's not just now. I mean, historically it has been a difficult and important skill to acquire, and we call it In education terminology, we call it metacognition. And the more technology is progressing, this metacognition, which is purely more of human understanding, human intuition, that is becoming even more critical.
So, a lot of these exercises So, one advice to all the students is over a course of seven weeks or eight weeks, whenever you are engaging with TDS assignments, it's also important for you to look back as to, okay, "What did I learn and how did I learn that entire process?" It doesn't matter, I mean, how I learned is more critical than what I learned, because that how I learned is going to be transferable across context, settings, any place, because that how I learned is going to be the more durable, the more tacit part of it.
And whereas what I learned, like what Anand was saying, today technology X makes it very obsolete, whereas the others will, I mean, going forward, it becomes more trivial with the advancement of technology. So, knowing very well where you are and how you are doing in this metacognition process is critical for all all students. I mean, everyone, it's not just students. I mean, even for us, right? I mean, as a program administrator for me, as an instructor for Anand, anyone, any human being, it is critical that they understand this process of how how I am learning. How am I learning is a much more important than what I am learning.
So, I think this particular part, I would I would sincerely request each of you to look at, and not just limited to TDS. Look at each and every subject that you learned. I mean, you try to learn something, and you went through it in a particular format. So, that that actually is a learner model for you. Now, how do you improve your learner model? I mean, all of you are learning machine learning and a lot of other things, right? So, how do you improve the model? How do you do better testing? Each and every assessment in every course is giving you a new set of data that you should mine and improve yourself. So, that's a self-research for you.
And when we say we are doing research, it is also about this. I mean, how do people, students in TDS learn this? And it feeds back into the I mean, now, nowadays it is it is feeding back into the next week's question. Typically it was, "Okay, they learned this in the previous edition. How do I change the next term's questions?" So, it was like, a year back, it was at that point. Nowadays, it is like, "Okay, I learned this. Next week, what should my assessment be?" So, this is how how much we have progressed in this process. TAs and other I mean, Carlton, you guys can chip in if you have certain thoughts on it.

**Carlton**: [68:37] Yeah, I just wanted to, before we close, we, last Wednesday, we had discussed, piloting some three questions, right, which mapped the progress of students. So, I have created a branch on our exam portal. I have sent you an email, as well, and also the chat link I've shared in the email. So, you can have a look at the

**JK**: [69:06] Sure.

**Carlton**: [69:06] the reasoning for the construction of the questions, and then maybe we can get back with Anand and see what you know, does that work for GA5, or

**JK**: [69:17] Yeah. No, I also had some questions in my mind, but I I hadn't applied it. I was planning to do a deep research tonight, or while the discussion was going on today, I was trying to going to run my agent on that. [laughter]

**Carlton**: [69:29] Are you using Gemini for the deep research?

**JK**: [69:33] Certain places, yes, where I think it is critical, I am using that. But Gemini's deep research is more of I would say it is more suitable to research methods and those kind of things to I find ChatGPT to be a much more better companion for these kind of research problems.

**Carlton**: [69:57] It's more well-balanced, isn't it? I mean, it's not specialized at something, but it's more well-balanced, yeah.

**JK**: [70:03] So, Gemini gives it, but you have to prompt it in a very different way. So, basically, what I do is I use a combination of Gemini and GPT. So, GPT's I create the deep research from research plan from Gemini, and I critique it using GPT.

**Carlton**: [70:23] Okay.

**JK**: [70:24] And then the resulting critique is given back so that Gemini does a much more better job of the research.

**Carlton**: [70:34] Yeah, and that's another fairly important skill, I mean, talking to students here, is **not to just rely on a single agent, but to actually use multiple agents and leverage the things that make them better, and use use each of them to critique each other, and you will it will cover the blind spots that those agents have.**

**JK**: [70:58] Yeah. Any questions from any of the students? I mean, I understand that the difficulty of the question, I mean, the course seems to be overwhelming for a large majority of you. Any inputs on that front? And it all feels like a I mean, at the end of the day, "I don't know what I learned." This is like a common phenomena that I see after interacting with AI. And you are not alone. Just in case if you think that it is happening to you, no, I mean, it is happening for a lot of us.

**Carlton**: [71:35] Yeah. I think, even like when I was generating the questions for GA5, you know, it produced more stuff than I could comprehend. So, then I had to, if you go through the chat, you'll see where I've actually gone back to it and said, "Look, explain to me what's happening here. Why is this relevant?" you know, and then try to digest all the stuff that it has produced, so that then, you know, there is some cognitive work that has to happen post it producing something so that you can actually say, "Yeah, this is a good question," or, "This is something that's worthwhile engaging." Or if something needs to be tweaked, then that's where you're using your actual critical thinking skills.

**JK**: [72:20] Yeah, I think see, the other part of it, I mean, it's also good generally. I mean, so I don't know how much of this would make sense, but so there is this saying that you have to pause for a little. I think a lot of time when the deluge becomes huge, it is important to take that significant pause and then reorient your thought process. And this is where the intuition and the other things that Anand was talking about becomes more critical. So, what is your fundamental intuition? What is your fundamental I mean, how is your fundamental skill set improving? That becomes more critical when you further interact with AI.
So, I've seen people work with AI for a longer period period of time and come up with innovative strategies while they interact with with it. And a lot of it is from experience, their own personal experiences in the trenches. The handicap that the current generation of students will have when they pass out from a degree is that they don't have that battle scars that a senior employee used to have. So, how do you develop that battle scar? How do you understand, I mean, where where it could go wrong?
That becomes more critical. And any learning process that you do, you have to develop that kind of I mean, both positive and negative. You should be you should be failing multiple times before recovering at the end. And it's not the CG that's important, the skill, okay? How are you recovering? How how is a failure made more productive? That is more critical than the CG, okay, or how did I solve the question? Yes, how did I solve the question is important, because that is a historical thing that you will plug to a a markdown file at some point or the other, but what process did you undergo to come up with this kind of strategy is the most critical learning in in this entire scheme of things.

**Hritik**: [74:38] Also, means, we are coming close to the session end. Means, a student always had lots of doubt and complaints as well to the previous course. Means, from my perspective, what I see that if students are able to ask a good question, they will able to learn and they will able to survive the corporate world in today in the corporate world. Why ask question? Because agents are able to answer really good, and agent is available for everyone. So, if you are able to ask good question, you will able to learn.
And the second part is that your expectation from the course is to teach something, and our thought process is that these are the problem statement, why not you ask the question that what you could learn and how you could solve? If you could do that, you will able to work well in future as well.
So, don't just see look like the course like this is a flow that we have to do this that. You have to give the task to agent. You have to ask multiple time, multiple questions in the sessions, in the Discourse, in the agents, and figure out the solutions.

**JK**: [76:02] Yeah, and yeah, in general, I mean, so I would say, rather than complaints, I would say grievance, which is a fair I mean, it's a positive spin off the same term as complaint. But a grievance is a well I mean, we we welcome all the grievances. I mean, I think it is important that we acknowledge the grievance and try to address the grievance right at the core of it.
And I hope today's session at least would have done at least 50 to 60 percentage of the grievances that people had with various assessments and various different activities in the course. I hope this session would have at least mitigated some of the grievances. I'm not saying that it solves all your problems, but at least why we are doing it is better communicated to today's session. Does that Okay, I see two thumbs-ups. I hope the others either agree. If you're disagreeing, do a thumbs-down, that is also fine, because that is what the whole purpose of education is. I mean, it is okay to have difference in opinion, but addressing it openly is something that we would want to try out, and that's what the Monday sessions are for.
Okay. I'm going to stop the live stream, Hritik, if that's all right, and then stop the recording.

**Hritik**: [77:40] Oh, yes.
