WEBVTT 00:00:00.000 --> 00:00:04.000 h what else? But 00:00:04.000 --> 00:00:08.000 still you can hear that's 00:00:08.000 --> 00:00:12.000 all. Okay, Anand we'll get started because people are 00:00:12.000 --> 00:00:16.000 Okay, cool. 00:00:16.000 --> 00:00:20.000 Let's dive in then. And like before, it'll be good 00:00:20.000 --> 00:00:24.000 if you otherwise good if you interrupt with questions as 00:00:24.000 --> 00:00:28.000 much as you can because it's one way of staying 00:00:28.000 --> 00:00:32.000 awake. 00:00:32.000 --> 00:00:36.000 inform but I'll also stay in the stages so that if anyone has any 00:00:36.000 --> 00:00:40.000 question I will ask what that question is and I will relay on the computer to 00:00:40.000 --> 00:00:44.000 Sure. Thank 00:00:44.000 --> 00:00:48.000 you. 00:00:48.000 --> 00:00:52.000 awake and listening 00:00:52.000 --> 00:00:56.000 No, not for me to stay awake uh or me to get 00:00:56.000 --> 00:01:00.000 feedback but I had a a 00:01:00.000 --> 00:01:04.000 classmate senior classmate who was doing his PhD uh while 00:01:04.000 --> 00:01:08.000 I was doing my MBA. uh he was at that time the finance secretary of 00:01:08.000 --> 00:01:12.000 the current government and uh he would periodically 00:01:12.000 --> 00:01:16.000 ask questions and of course class participation is 00:01:16.000 --> 00:01:20.000 rewarded literally with marks but in his case 00:01:20.000 --> 00:01:24.000 he said I ask questions more to stay awake in class because if 00:01:24.000 --> 00:01:28.000 I don't ask questions I will fall 00:01:28.000 --> 00:01:32.000 asleep 00:01:32.000 --> 00:01:36.000 Okay. One second. 00:01:36.000 --> 00:01:40.000 Sure. Yeah. 00:01:48.000 --> 00:01:52.000 uh Bluetooth connection. 00:01:52.000 --> 00:01:56.000 Yeah. 00:02:00.000 --> 00:02:04.000 about rocker 00:02:12.000 --> 00:02:16.000 strength. 00:02:16.000 --> 00:02:20.000 Rocker rocker trans is this one. 00:02:32.000 --> 00:02:36.000 Huh? 00:02:36.000 --> 00:02:40.000 I can still hear you. But 00:02:40.000 --> 00:02:44.000 if I'm speaking, am I better 00:02:44.000 --> 00:02:48.000 audible? Okay, 00:02:48.000 --> 00:02:52.000 great. Wonder. 00:02:52.000 --> 00:02:56.000 Cool. Can I just uh briefly recap which is please 00:02:56.000 --> 00:03:00.000 ask questions if nothing else the class will go in the direction that you want 00:03:00.000 --> 00:03:04.000 rather than some random direction that I will be taking it in. 00:03:04.000 --> 00:03:08.000 But there are four broad themes that I'm planning to 00:03:08.000 --> 00:03:12.000 talk about today which will relate to some of the material that you have seen 00:03:12.000 --> 00:03:16.000 before. The first is that tools 00:03:16.000 --> 00:03:20.000 are going to be shaping visualizations like they used to 00:03:20.000 --> 00:03:24.000 uh when William player was creating his 00:03:24.000 --> 00:03:28.000 visualizations. They were more handdrawn. Uh and that meant that there 00:03:28.000 --> 00:03:32.000 was more variety, more flexibility, fewer people could create it. Then 00:03:32.000 --> 00:03:36.000 we had the Excel era where standardized charts became more the 00:03:36.000 --> 00:03:40.000 norm and uh therefore more people could do stuff 00:03:40.000 --> 00:03:44.000 but there was lesser variety in charts 00:03:44.000 --> 00:03:48.000 that used to come out with AI 00:03:48.000 --> 00:03:52.000 creating more visual formats. These days we have a lot more 00:03:52.000 --> 00:03:56.000 of the kinds of visual variety that 00:03:56.000 --> 00:04:00.000 we used to see before and can probably start seeing more. 00:04:00.000 --> 00:04:04.000 I'll show a few of the visualizations that I saw recently that 00:04:04.000 --> 00:04:08.000 I thought were interesting and 00:04:08.000 --> 00:04:12.000 were created by people with varied levels of visualization experience 00:04:12.000 --> 00:04:16.000 which I think is the main point. I'll start 00:04:16.000 --> 00:04:20.000 with a visualization called Jev's 00:04:20.000 --> 00:04:24.000 keep. Some of you may have heard of this model called 00:04:24.000 --> 00:04:28.000 Jev J V. a lot of 00:04:28.000 --> 00:04:32.000 buzz around it in recent times. It's a relatively inexpensive 00:04:32.000 --> 00:04:36.000 model unlike say the likes of chart GPT etc which generate 00:04:36.000 --> 00:04:40.000 text. It can only say yes or no or choose between 00:04:40.000 --> 00:04:44.000 categories or generate a number but it's pretty 00:04:44.000 --> 00:04:48.000 fast. So one of my colleagues uh who is fresh 00:04:48.000 --> 00:04:52.000 out of college um he took Jeb 00:04:52.000 --> 00:04:56.000 and started doing uh security 00:04:56.000 --> 00:05:00.000 screening that is when a request comes to a 00:05:00.000 --> 00:05:04.000 website. Can we check if it is an SQL injection 00:05:04.000 --> 00:05:08.000 attack? Can we check if it is a path traversal kind of 00:05:08.000 --> 00:05:12.000 okay not able to presume it makes it worse. uh but on 00:05:12.000 --> 00:05:16.000 the right side you may be able to see uh 00:05:16.000 --> 00:05:20.000 the different categories of attacks. the way 00:05:20.000 --> 00:05:24.000 he implemented it. I have no idea if it was uh intentional 00:05:24.000 --> 00:05:28.000 or just a a model choice but I think there 00:05:28.000 --> 00:05:32.000 must have been some intention behind it where he 00:05:32.000 --> 00:05:36.000 encoded each of these as little dots that is every request that's coming in as a little dot 00:05:36.000 --> 00:05:40.000 going through a casle and as the requests come in 00:05:40.000 --> 00:05:44.000 if they are a security act they get added to the 00:05:44.000 --> 00:05:48.000 tally board on top effectively creating a 00:05:48.000 --> 00:05:52.000 dynamic bar chart trace. Now 00:05:52.000 --> 00:05:56.000 a he's not familiar with bar chart 00:05:56.000 --> 00:06:00.000 traces. It emerged as part of the discussion that he was having with the 00:06:00.000 --> 00:06:04.000 model. The whole concept of a castle again uh just 00:06:04.000 --> 00:06:08.000 emerged and the dynamic uh streaming 00:06:08.000 --> 00:06:12.000 of course this I can and should 00:06:12.000 --> 00:06:16.000 accelerate. Let me just play it at 5x speed. 00:06:16.000 --> 00:06:20.000 uh came in because of a discussion that we had saying look let's run this as 00:06:20.000 --> 00:06:24.000 a simulation so that people can see what's happening 00:06:24.000 --> 00:06:28.000 pretty interesting now a lot of people have in the 00:06:28.000 --> 00:06:32.000 recent times asked me oh what is this kind of animated visualization that 00:06:32.000 --> 00:06:36.000 you use and I think that's this was recently in a trip 00:06:36.000 --> 00:06:40.000 to Manila where uh they were not as 00:06:40.000 --> 00:06:44.000 familiar with animated visualizations as they were with uh 00:06:44.000 --> 00:06:48.000 Excel visualizations Today that's so much 00:06:48.000 --> 00:06:52.000 more a possibility than it used to be that people are just using it left, right 00:06:52.000 --> 00:06:56.000 and center. Why not? Why does visualization have to be 00:06:56.000 --> 00:07:00.000 static when the medium that we are using these days 00:07:00.000 --> 00:07:04.000 can support dynamic formats? Let's 00:07:04.000 --> 00:07:08.000 take another visualization. This uh was created 00:07:08.000 --> 00:07:12.000 by a Japanese student in his first year of uh college from 00:07:12.000 --> 00:07:16.000 city university of Tokyo. The interesting part of the 00:07:16.000 --> 00:07:20.000 visualization this is a broader story 00:07:20.000 --> 00:07:24.000 but this was for uh 00:07:24.000 --> 00:07:28.000 I forget which football series this is but he 00:07:28.000 --> 00:07:32.000 effectively created a heat map of where the goals 00:07:32.000 --> 00:07:36.000 were successfully scored from 00:07:36.000 --> 00:07:40.000 and bunch of variations around this to see 00:07:40.000 --> 00:07:44.000 where individual uh specific spots from where 00:07:44.000 --> 00:07:48.000 each individual goal was stored. Um, with the ability 00:07:48.000 --> 00:07:52.000 to look specifically at at what time who scored which 00:07:52.000 --> 00:07:56.000 goal and animated right down to the point of 00:07:56.000 --> 00:08:00.000 the individual uh goal position 00:08:00.000 --> 00:08:04.000 and each one of these goals step by step walking 00:08:04.000 --> 00:08:08.000 through what exactly happened. So if 00:08:08.000 --> 00:08:12.000 then this goal so this was potentially the origin they don't 00:08:12.000 --> 00:08:16.000 have the origin data but this was probably the origin this is the record short 00:08:16.000 --> 00:08:20.000 position and it moved in here. 00:08:20.000 --> 00:08:24.000 Oh, I need you saying 00:08:24.000 --> 00:08:28.000 something. Okay, maybe not. 00:08:28.000 --> 00:08:32.000 Uh the idea here is that uh 00:08:32.000 --> 00:08:36.000 uh heat map of football goals is not 00:08:36.000 --> 00:08:40.000 something that 00:08:40.000 --> 00:08:44.000 uh Oh. Oh, 00:08:44.000 --> 00:08:48.000 okay. I don't think I've changed anything in the mic, but I'll get a 00:08:48.000 --> 00:08:52.000 little closer in any case. Is it okay 00:08:52.000 --> 00:08:56.000 now? Okay, fine. I'll just continue 00:08:56.000 --> 00:09:00.000 this way then. Now, it's not that uh 00:09:00.000 --> 00:09:04.000 heat map kind of a visualization is particularly 00:09:04.000 --> 00:09:08.000 difficult if you knew D3, but this is a student who knows no 00:09:08.000 --> 00:09:12.000 program and I think that's becoming the second kind of 00:09:12.000 --> 00:09:16.000 uh capability. uh unleash 00:09:16.000 --> 00:09:20.000 that's happened. What that means is that some 00:09:20.000 --> 00:09:24.000 of those who are artistically inclined or have a 00:09:24.000 --> 00:09:28.000 strong domain sense, they probably have a bit of 00:09:28.000 --> 00:09:32.000 an edge in creating visualizations 00:09:32.000 --> 00:09:36.000 because this student obviously knows football and says, 00:09:36.000 --> 00:09:40.000 "Oh, look, this is how I want to see it." And 00:09:40.000 --> 00:09:44.000 the tool ceases to be a constraint because now you just tell it to do something and it can do 00:09:44.000 --> 00:09:48.000 it. It almost becomes the question now 00:09:48.000 --> 00:09:52.000 becomes then do you have a good enough story to tell and do 00:09:52.000 --> 00:09:56.000 you have an interesting way of telling those stories? 00:09:56.000 --> 00:10:00.000 Some of these stories emerge just from the domain itself. For instance, 00:10:00.000 --> 00:10:04.000 u one that uh we've been sharing with 00:10:04.000 --> 00:10:08.000 uh the uh times of India is 00:10:08.000 --> 00:10:12.000 on environmental change insights. Uh I'm not sure if I 00:10:12.000 --> 00:10:16.000 showed this last time but if so it'll be great if someone could just let me 00:10:16.000 --> 00:10:20.000 know and I'll go a little faster. But what we did 00:10:20.000 --> 00:10:24.000 was looked at satellite imagery and zoomed 00:10:24.000 --> 00:10:28.000 in into specific uh areas comparing. 00:10:28.000 --> 00:10:32.000 So let's take Chennai. I'm going to 00:10:32.000 --> 00:10:36.000 u zoom out of Chennai a little 00:10:36.000 --> 00:10:40.000 bit. And what you have here 00:10:40.000 --> 00:10:44.000 in these little grids is how 00:10:44.000 --> 00:10:48.000 much water is retreating or drying. 00:10:48.000 --> 00:10:52.000 So green means water coverage has increased, red means 00:10:52.000 --> 00:10:56.000 water coverage has decreased between 2015 to 00:10:56.000 --> 00:11:00.000 2025. So effectively if I take the map as 00:11:00.000 --> 00:11:04.000 it was in 2015 compare it with the map as it is in 00:11:04.000 --> 00:11:08.000 2025 for each of those little I think 100 m by 00:11:08.000 --> 00:11:12.000 100 m grids we used a vision model 00:11:12.000 --> 00:11:16.000 specifically a geospatial vision model called ALMO 00:11:16.000 --> 00:11:20.000 earth to see what is the water coverage before what is the water coverage 00:11:20.000 --> 00:11:24.000 now and create a heat map of where in 00:11:24.000 --> 00:11:28.000 Chennai water has water bodies have grown and where they 00:11:28.000 --> 00:11:32.000 have shrunk. So, for instance, uh let's look at one of 00:11:32.000 --> 00:11:36.000 these hotspots here. Zooming 00:11:36.000 --> 00:11:40.000 in. Oh, you aren't even seeing the screen. 00:11:40.000 --> 00:11:44.000 Sorry. Let me do this again. 00:11:44.000 --> 00:11:48.000 Uh apologies. I will start 00:11:48.000 --> 00:11:52.000 again. So, that's Chennai's water coverage uh or 00:11:52.000 --> 00:11:56.000 water difference. So this is Chennai in 2015 00:11:56.000 --> 00:12:00.000 and this is 00:12:00.000 --> 00:12:04.000 uh Chennai satellite imagery. Now if we take 00:12:04.000 --> 00:12:08.000 Almo Earth's evaluation of in each 100 m 00:12:08.000 --> 00:12:12.000 by 100 m grid how much water coverage there was 00:12:12.000 --> 00:12:16.000 before versus now. The greens are where there is more coverage. 00:12:16.000 --> 00:12:20.000 Let me zoom in into one of these spots which should be 00:12:20.000 --> 00:12:24.000 pretty easily recognizable. But is it I mean uh 00:12:24.000 --> 00:12:28.000 is can anyone name what this bright 00:12:28.000 --> 00:12:32.000 green spot is likely to 00:12:32.000 --> 00:12:36.000 bright green spot. What is it likely to 00:12:36.000 --> 00:12:40.000 be? It's 00:12:40.000 --> 00:12:44.000 water. But where is it? 00:12:44.000 --> 00:12:48.000 Oh, no. I'm asking if anyone 00:12:48.000 --> 00:12:52.000 could guess what that green strip 00:12:52.000 --> 00:12:56.000 ah green they said greenery but yeah in the discussion it 00:12:56.000 --> 00:13:00.000 was. Huh? Correct. 00:13:00.000 --> 00:13:04.000 But let me put it this way. Does anyone know Chennai well enough to know where this 00:13:04.000 --> 00:13:08.000 water ah where where this could be the 00:13:08.000 --> 00:13:12.000 is exactly? 00:13:12.000 --> 00:13:16.000 Yeah. river. Correct. And if we uh just take a 00:13:16.000 --> 00:13:20.000 look at uh this 00:13:20.000 --> 00:13:24.000 is the current situation and uh this is 00:13:24.000 --> 00:13:28.000 what it used to be. Now uh the fadedness 00:13:28.000 --> 00:13:32.000 of the image is not what it's using as a metric. The amount of water 00:13:32.000 --> 00:13:36.000 itself which you may be able to see from the uh edges has grown 00:13:36.000 --> 00:13:40.000 more in this area. But of course you could also say it's because there is more 00:13:40.000 --> 00:13:44.000 water for it to grow. But still the Adar River has certainly become 00:13:44.000 --> 00:13:48.000 a lot uh wetter. Uh and 00:13:48.000 --> 00:13:52.000 there are other spots like let's just randomly pick one 00:13:52.000 --> 00:13:56.000 of these um where 00:13:56.000 --> 00:14:00.000 okay there is more water coverage probably 00:14:00.000 --> 00:14:04.000 some northern part of kuam or whatever uh which 00:14:04.000 --> 00:14:08.000 used to be a lot drier uh no not ku 00:14:08.000 --> 00:14:12.000 probably the northern part of mingham canal of some such thing u 00:14:12.000 --> 00:14:16.000 that is a lot better and similarly we can look at this from a 00:14:16.000 --> 00:14:20.000 vegetation perspective effective as well. uh or greenery 00:14:20.000 --> 00:14:24.000 as you looked at to see where greenery is growing, where greenery is 00:14:24.000 --> 00:14:28.000 shrinking very simple heat map visualization but the underlying data is 00:14:28.000 --> 00:14:32.000 changing yet another kind of uh visual 00:14:32.000 --> 00:14:36.000 representation that I'm finding a lot more of these days is uh 00:14:36.000 --> 00:14:40.000 sands so uh this is a research 00:14:40.000 --> 00:14:44.000 report that uh is based on McKenzie's uh global 00:14:44.000 --> 00:14:48.000 energy perspective report they've been publishing about 00:14:48.000 --> 00:14:52.000 27 of these very detailed reports 00:14:52.000 --> 00:14:56.000 every here and those reports 00:14:56.000 --> 00:15:00.000 what uh we've done was taken every single fact in 00:15:00.000 --> 00:15:04.000 it and broken it down saying that look there are 00:15:04.000 --> 00:15:08.000 a total of about 2,200 odd 00:15:08.000 --> 00:15:12.000 facts each of these is one such so 00:15:12.000 --> 00:15:16.000 here one of the facts that or claims let's say that they have put in 00:15:16.000 --> 00:15:20.000 is power sector gas demand in 00:15:20.000 --> 00:15:24.000 2035 will be less than 100 BCM because renewables are going to be 00:15:24.000 --> 00:15:28.000 more competitive. Some statement of fact and each one of these 00:15:28.000 --> 00:15:32.000 is one. The nice part is that it was 00:15:32.000 --> 00:15:36.000 easy for the audience to see uh how 00:15:36.000 --> 00:15:40.000 many facts did we get by year? What types 00:15:40.000 --> 00:15:44.000 of facts do we have? So lots of forecasts, relatively fewer 00:15:44.000 --> 00:15:48.000 estimates, sub 230 odd facts, some 00:15:48.000 --> 00:15:52.000 169 causalities, etc. Which ones should we watch 00:15:52.000 --> 00:15:56.000 more of? Which ones should we watch less of? What are the 00:15:56.000 --> 00:16:00.000 kinds of reports that these facts are presented? So getting 00:16:00.000 --> 00:16:04.000 a map of what is there and where it is 00:16:04.000 --> 00:16:08.000 becomes easier and that sort of a thing can be used in a variety 00:16:08.000 --> 00:16:12.000 of different areas. For instance, in product management. 00:16:12.000 --> 00:16:16.000 So uh this is uh for 00:16:16.000 --> 00:16:20.000 a company called Optum. Optum is 00:16:20.000 --> 00:16:24.000 a media 00:16:24.000 --> 00:16:28.000 we still looking at the uh the water map or is there a different map that 00:16:28.000 --> 00:16:32.000 research. I'm sorry. Uh I will start again. 00:16:32.000 --> 00:16:36.000 My apologies. I will yeah consciously 00:16:36.000 --> 00:16:40.000 switch tab to tab and seeing different screens. So let me do 00:16:40.000 --> 00:16:44.000 this again. Uh 00:16:44.000 --> 00:16:48.000 this correct 00:16:48.000 --> 00:16:52.000 exactly. So the McKenzie uh global energy 00:16:52.000 --> 00:16:56.000 perspective report has about 27 of these 00:16:56.000 --> 00:17:00.000 publications and each publication was broken 00:17:00.000 --> 00:17:04.000 down into about 2,200 claims. So one of 00:17:04.000 --> 00:17:08.000 these is a claim that says under accelerated renewables 00:17:08.000 --> 00:17:12.000 uh etc are projected to plateau after 2020. 00:17:12.000 --> 00:17:16.000 They're making a forecast of this kind like this. There 00:17:16.000 --> 00:17:20.000 are more than 2,000 of these which it 00:17:20.000 --> 00:17:24.000 becomes easier for people to get a sense of 00:17:24.000 --> 00:17:28.000 uh you know get a map off so to speak by 00:17:28.000 --> 00:17:32.000 saying by year we seem to have a lot more of these claims in 00:17:32.000 --> 00:17:36.000 2024. Um there are many more forecasts 00:17:36.000 --> 00:17:40.000 than estimates than facts than 00:17:40.000 --> 00:17:44.000 causalities. The ones that we need to watch for high probability or 00:17:44.000 --> 00:17:48.000 I importance there are about 1250 of those where 00:17:48.000 --> 00:17:52.000 they occur across the reports. And of course if you say look 00:17:52.000 --> 00:17:56.000 just tell me in which report are there things that I should watch and sort 00:17:56.000 --> 00:18:00.000 by that um or even let's say color by that 00:18:00.000 --> 00:18:04.000 it becomes easy to track groups of claims 00:18:04.000 --> 00:18:08.000 across different categorizations. This is a characteristic of 00:18:08.000 --> 00:18:12.000 the sandance kind of uh visual animations. We 00:18:12.000 --> 00:18:16.000 have seen that these are not well Excel 00:18:16.000 --> 00:18:20.000 creatable charts at the very least. Certainly D3 creatable 00:18:20.000 --> 00:18:24.000 charts which is limited to a smaller audience but almost all of these 00:18:24.000 --> 00:18:28.000 were created by people who were not able to create visualizations 00:18:28.000 --> 00:18:32.000 but knew the domain. Let me switch to another one. Share the 00:18:32.000 --> 00:18:36.000 tab before I forget. Um another visualization 00:18:36.000 --> 00:18:40.000 was created for uh company 00:18:40.000 --> 00:18:44.000 Optum to show their product portfolio. Again a very simple uh 00:18:44.000 --> 00:18:48.000 sandance kind of a visualization that said look you have about 00:18:48.000 --> 00:18:52.000 uh 28 products uh 00:18:52.000 --> 00:18:56.000 that are documented and they belong to a whole bunch of these 00:18:56.000 --> 00:19:00.000 categories and they fit into these 00:19:00.000 --> 00:19:04.000 workflows. So rather than think of a 00:19:04.000 --> 00:19:08.000 product being a monolithic whole, think of it as it is 00:19:08.000 --> 00:19:12.000 delivering some capabilities. These capabilities are focused 00:19:12.000 --> 00:19:16.000 in specific areas. What you need to do is maybe rationalize it in a 00:19:16.000 --> 00:19:20.000 slightly different way. If you look at what are the key common 00:19:20.000 --> 00:19:24.000 areas that you should be focusing on. There are some products that are emerging 00:19:24.000 --> 00:19:28.000 right in the middle of these capabilities. Therefore, maybe you should focus on 00:19:28.000 --> 00:19:32.000 rationalizing those. Here are some external dependencies where 00:19:32.000 --> 00:19:36.000 you're dependent on a competitor. So maybe watch out either buy that 00:19:36.000 --> 00:19:40.000 competitor out or remove that dependency are examples 00:19:40.000 --> 00:19:44.000 of somewhat more novel visualizations that I've seen 00:19:44.000 --> 00:19:48.000 in the last month or so. Again, almost all 00:19:48.000 --> 00:19:52.000 created by people with a strong understanding of the 00:19:52.000 --> 00:19:56.000 domain and a relatively weak understanding of 00:19:56.000 --> 00:20:00.000 visualization or technology. what am 00:20:00.000 --> 00:20:04.000 I therefore leading towards? Um you've 00:20:04.000 --> 00:20:08.000 seen in this course that the over history 00:20:08.000 --> 00:20:12.000 uh at different points when technologies became 00:20:12.000 --> 00:20:16.000 available the shape of visual representations changed. 00:20:16.000 --> 00:20:20.000 Today, today over the course of the last few years 00:20:20.000 --> 00:20:24.000 and the next few years, we are at another one of those inflection points. 00:20:24.000 --> 00:20:28.000 to treat what you're studying 00:20:28.000 --> 00:20:32.000 from a tools perspective with a little bit of skepticism. The 00:20:32.000 --> 00:20:36.000 tooling is changing rapidly and what you're learning from a 00:20:36.000 --> 00:20:40.000 technique and a history perspective with a little higher 00:20:40.000 --> 00:20:44.000 importance. The lessons both of history and the principles that 00:20:44.000 --> 00:20:48.000 we learn might last a little longer. And 00:20:48.000 --> 00:20:52.000 one of the principles that we are talking about 00:20:52.000 --> 00:20:56.000 is how well do models see 00:20:56.000 --> 00:21:00.000 things or sorry how well do people see things. graphical perception. We 00:21:00.000 --> 00:21:04.000 have our own set of biases. 00:21:04.000 --> 00:21:08.000 We are able to spot differences in uh color, 00:21:08.000 --> 00:21:12.000 even subtle shades. When they're right next to each other, but 00:21:12.000 --> 00:21:16.000 move them a little further away, surround them by other colors, we totally get 00:21:16.000 --> 00:21:20.000 confused. Length perception is pretty good. 00:21:20.000 --> 00:21:24.000 Area perception is not uh perhaps as good. Angular perception 00:21:24.000 --> 00:21:28.000 is pretty bad. When we start things near each other, 00:21:28.000 --> 00:21:32.000 we are able to see smaller differences further apart. It 00:21:32.000 --> 00:21:36.000 becomes trickier. These are the kinds of mistakes we 00:21:36.000 --> 00:21:40.000 make. But now with 00:21:40.000 --> 00:21:44.000 agents coming in, agents are increasingly a 00:21:44.000 --> 00:21:48.000 consumer of our visualizations as well. What I mean by that 00:21:48.000 --> 00:21:52.000 is okay half the time when somebody sends 00:21:52.000 --> 00:21:56.000 me a presentation which probably has a bunch of charts I just 00:21:56.000 --> 00:22:00.000 uploaded to chart GPT tell me what they're seeing 00:22:00.000 --> 00:22:04.000 now that will contain a chart ultimately chart GPT is that's going to be 00:22:04.000 --> 00:22:08.000 reading the chart if you are going to be creating visualization you may as well make 00:22:08.000 --> 00:22:12.000 sure that cloud or charge GPT or Pacity or whatever is going to be consuming the 00:22:12.000 --> 00:22:16.000 chart for probably half the audience can read it as 00:22:16.000 --> 00:22:20.000 Of course, you could put it in the text, too. That is a good thing. But 00:22:20.000 --> 00:22:24.000 there are likely a set of things that 00:22:24.000 --> 00:22:28.000 LLM are better able to 00:22:28.000 --> 00:22:32.000 visualize uh when shown as a picture or is just almost impossible 00:22:32.000 --> 00:22:36.000 to convey as images, right? So, 00:22:36.000 --> 00:22:40.000 what how do LLM see and what are their 00:22:40.000 --> 00:22:44.000 biases? That's something that's probably worth 00:22:44.000 --> 00:22:48.000 exploring as well. So, let's take a look. We were 00:22:48.000 --> 00:22:52.000 playing around with a few benchmarks. I'm shifting to a 00:22:52.000 --> 00:22:56.000 bookshelf benchmark. One of my colleagues, again, fresh out of 00:22:56.000 --> 00:23:00.000 college. He uh took a bunch of 00:23:00.000 --> 00:23:04.000 photos and uh here is an example 00:23:04.000 --> 00:23:08.000 of one of the photos he took. It's of a bookshelf. 00:23:08.000 --> 00:23:12.000 and asked a series of models, can you tell me 00:23:12.000 --> 00:23:16.000 all the books that are there in this particular bookshelf? 00:23:16.000 --> 00:23:20.000 A, B, C, etc. Manually classified all of those. Some of those, like the 00:23:20.000 --> 00:23:24.000 first two are relatively easy. again. see the 00:23:24.000 --> 00:23:28.000 titles clearly and completely. Some of them are harder like the last two 00:23:28.000 --> 00:23:32.000 where they're both messy, not very visible, 00:23:32.000 --> 00:23:36.000 etc. and created a benchmark. 00:23:36.000 --> 00:23:40.000 The benchmark was the prompt was fairly 00:23:40.000 --> 00:23:44.000 straightforward. It is list all the books and he compared the books against 00:23:44.000 --> 00:23:48.000 the results and the results are 00:23:48.000 --> 00:23:52.000 interesting. There were a couple of uh uh runs that 00:23:52.000 --> 00:23:56.000 he made and looking at the overall 00:23:56.000 --> 00:24:00.000 F1 score. What we're finding is couple 00:24:00.000 --> 00:24:04.000 of things. Uh the newer models especially the anthropic 00:24:04.000 --> 00:24:08.000 models Opus 5.5 and Sonnet 5.5 is a 00:24:08.000 --> 00:24:12.000 way ahead of the latest Gemini models which is not not 00:24:12.000 --> 00:24:16.000 way bit ahead. um 86% 00:24:16.000 --> 00:24:20.000 is what opus 5.5 scored with high effort as compared to 00:24:20.000 --> 00:24:24.000 sonnet 5.5 uh which was at 80% and Gemini 3.8 00:24:24.000 --> 00:24:28.000 flash scored 70%. In other words able to 00:24:28.000 --> 00:24:32.000 get more than half the books right but not 00:24:32.000 --> 00:24:36.000 uh none of the models could get all of the books 00:24:36.000 --> 00:24:40.000 right. Fair enough. Now that tells us a couple of things. one 00:24:40.000 --> 00:24:44.000 that there are models that can see better that are there are models that 00:24:44.000 --> 00:24:48.000 can see worse. The second in each of these cases he tried 00:24:48.000 --> 00:24:52.000 it with low effort as well as with high effort and the models have the 00:24:52.000 --> 00:24:56.000 tunable parameters. So you can literally tell them look a little closer. 00:24:56.000 --> 00:25:00.000 She'll just tell me what you see. And in many cases 00:25:00.000 --> 00:25:04.000 the uh high effort outperformed the low 00:25:04.000 --> 00:25:08.000 effort. Not always. For let's say Luna, if you actually asked 00:25:08.000 --> 00:25:12.000 it to look harder, it did worse. Happens. 00:25:12.000 --> 00:25:16.000 Um if you tell Gemini to look harder, 00:25:16.000 --> 00:25:20.000 it doesn't really have much of an improvement. But with Sonet, there was a 00:25:20.000 --> 00:25:24.000 decent improvement. The other factor is how much is this going to 00:25:24.000 --> 00:25:28.000 cost. Now on this chart, the x-axis 00:25:28.000 --> 00:25:32.000 is how much it cost. Yaxis is 00:25:32.000 --> 00:25:36.000 how accurate it was. You can see that the more expensive models tend to be 00:25:36.000 --> 00:25:40.000 more accurate or vice versa. You pay for a certain degree 00:25:40.000 --> 00:25:44.000 of accuracy. Put another way, 00:25:44.000 --> 00:25:48.000 unlike humans where you say I have an audience, 00:25:48.000 --> 00:25:52.000 the audience has to read it. You now have to start factoring 00:25:52.000 --> 00:25:56.000 in does the person whom I'm going to 00:25:56.000 --> 00:26:00.000 send a visualization to have a model that's going to read 00:26:00.000 --> 00:26:04.000 it? If so, do they have a better model? Do they have a worse 00:26:04.000 --> 00:26:08.000 model? How is that model or what are 00:26:08.000 --> 00:26:12.000 the biases that that model has that I should factor in 00:26:12.000 --> 00:26:16.000 start becoming a question? Incidentally, 00:26:16.000 --> 00:26:20.000 uh the person who built it, Atarva, he uh chose to use 00:26:20.000 --> 00:26:24.000 an interesting visual representation for the uncertainty 00:26:24.000 --> 00:26:28.000 in the cost. Uh you see that the 00:26:28.000 --> 00:26:32.000 cost has a horizontal bar. The reason is when I 00:26:32.000 --> 00:26:36.000 asked him look what is the cost of uh each of these he 00:26:36.000 --> 00:26:40.000 said oh I didn't measure it okay you have the chats 00:26:40.000 --> 00:26:44.000 right go back and uh do a measurement 00:26:44.000 --> 00:26:48.000 he said the charts are there but I don't have 00:26:48.000 --> 00:26:52.000 the uh reasoning uh text 00:26:52.000 --> 00:26:56.000 that is hidden by uh chart GPT so I don't actually have 00:26:56.000 --> 00:27:00.000 the full reasoning tokens I can probably do an estimate 00:27:00.000 --> 00:27:04.000 You could create user model, have it go through the 00:27:04.000 --> 00:27:08.000 transcript and run similar uh 00:27:08.000 --> 00:27:12.000 uh run a similar process again, see if you can measure it 00:27:12.000 --> 00:27:16.000 and put in an uncertainty estimate. So this 00:27:16.000 --> 00:27:20.000 is an uncertainty estimate by a 00:27:20.000 --> 00:27:24.000 model of a model's cost and the 00:27:24.000 --> 00:27:28.000 range varies. So for instance for set the range is relatively 00:27:28.000 --> 00:27:32.000 narrower for Luna the range of cost estimates is much wider because we have far 00:27:32.000 --> 00:27:36.000 less uh understanding of how much it thinks and we'll 00:27:36.000 --> 00:27:40.000 come to this uncertainty bit and how we handle it in a short 00:27:40.000 --> 00:27:44.000 life but where I want to go with this is that 00:27:44.000 --> 00:27:48.000 models are reading charts 00:27:48.000 --> 00:27:52.000 models have different abilities to read 00:27:52.000 --> 00:27:56.000 charts and there may also be a certain 00:27:56.000 --> 00:28:00.000 amount of bias in doing 00:28:00.000 --> 00:28:04.000 this. Let's look at what we know about 00:28:04.000 --> 00:28:08.000 the models biases. There is a 00:28:08.000 --> 00:28:12.000 benchmark called chat, sorry, chart QA 00:28:12.000 --> 00:28:16.000 pro. It's on GitHub. And what the team 00:28:16.000 --> 00:28:20.000 that put this together did was they took a series of questions. 00:28:20.000 --> 00:28:24.000 Let me open one of 00:28:32.000 --> 00:28:36.000 these. Okay, just going to make this 00:28:36.000 --> 00:28:40.000 table here. 00:28:40.000 --> 00:28:44.000 So for instance uh the first uh 00:28:44.000 --> 00:28:48.000 chart of the eight that you see here is an 00:28:48.000 --> 00:28:52.000 image with a question calculate the total percentage of deals made by 00:28:52.000 --> 00:28:56.000 buyers from USA, Japan and Singapore combined with an 00:28:56.000 --> 00:29:00.000 answer 17. Our charts are 00:29:00.000 --> 00:29:04.000 models able to figure this out is what chart QA was 00:29:04.000 --> 00:29:08.000 evaluating and the evaluation results are in 00:29:08.000 --> 00:29:12.000 here. Um, not sure how well you can hold 00:29:12.000 --> 00:29:16.000 on. Okay. Yeah, 00:29:16.000 --> 00:29:20.000 this probably about as big as I can make it, but I'll just summarize 00:29:20.000 --> 00:29:24.000 by saying there are some models that do a good job. some models that do a poor 00:29:24.000 --> 00:29:28.000 job. But what is perhaps more interesting 00:29:28.000 --> 00:29:32.000 is is there a bias? Meaning, do some 00:29:32.000 --> 00:29:36.000 models um really get the question right, 00:29:36.000 --> 00:29:40.000 really get uh some questions wrong, etc. 00:29:40.000 --> 00:29:44.000 Nobody seems to have published a paper based on that. So I said I told 00:29:44.000 --> 00:29:48.000 Chad GPT about an hour ago, look um what I want 00:29:48.000 --> 00:29:52.000 you to do is download that data 00:29:52.000 --> 00:29:56.000 set that yeah I want you to download 00:29:56.000 --> 00:30:00.000 uh chart QA pro and answer the question what are the 00:30:00.000 --> 00:30:04.000 different kinds of charts or questions that models typically get 00:30:04.000 --> 00:30:08.000 wrong? What does that tell us about their perception? Do different models 00:30:08.000 --> 00:30:12.000 have different kinds of weaknesses and strengths? Why don't you do the 00:30:12.000 --> 00:30:16.000 research and give me a story about it? And here is the story 00:30:16.000 --> 00:30:20.000 that it's given me which I'm looking at for the first time. I haven't uh seen this 00:30:20.000 --> 00:30:24.000 so far. And it's saying a model can 00:30:24.000 --> 00:30:28.000 know what calculation to do but still misread the 00:30:28.000 --> 00:30:32.000 picture that it is calculating from. So here 00:30:32.000 --> 00:30:36.000 is one question for this picture. When does the wind 00:30:36.000 --> 00:30:40.000 capacity first exceed 100 gaw? Okay, I'm 00:30:40.000 --> 00:30:44.000 not sure if you can answer that question, but let me try. Um, I have to 00:30:44.000 --> 00:30:48.000 look at this green thickness and see when it um 00:30:48.000 --> 00:30:52.000 exceeds 100 W. And I think my answer is never. 00:30:52.000 --> 00:30:56.000 U but it 00:30:56.000 --> 00:31:00.000 says, okay, the thickness I'm just measuring by hand. My thought 00:31:00.000 --> 00:31:04.000 would have been never but the 00:31:04.000 --> 00:31:08.000 okay ground proof is 00:31:08.000 --> 00:31:12.000 203738. Oh, okay. Fine. I should have looked at it cumulatively. Then all of that 00:31:12.000 --> 00:31:16.000 was ruined. I misread that chart. And 20 37 38 00:31:16.000 --> 00:31:20.000 is yes. When it hits the 100 mark 00:31:20.000 --> 00:31:24.000 and 2.5 said 00:31:24.000 --> 00:31:28.000 2024 or 5 2.5, a larger 00:31:28.000 --> 00:31:32.000 model said 2035. And yet another model said 00:31:32.000 --> 00:31:36.000 20334. At least the larger models seem to be roughly in the 00:31:36.000 --> 00:31:40.000 ballpark. Fair enough. 00:31:40.000 --> 00:31:44.000 What? 00:31:44.000 --> 00:31:48.000 Huh? 00:31:48.000 --> 00:31:52.000 Huh? 00:31:52.000 --> 00:31:56.000 H 00:31:56.000 --> 00:32:00.000 exactly. And 00:32:00.000 --> 00:32:04.000 I assume that the thickness or the height of the green 00:32:04.000 --> 00:32:08.000 is the wind. But it turns out that 00:32:08.000 --> 00:32:12.000 it's asking for the height from the bottom, not just the green part, 00:32:12.000 --> 00:32:16.000 which is my mistake. But 00:32:16.000 --> 00:32:20.000 uh yeah, the model 00:32:20.000 --> 00:32:24.000 seem to have done better than me in any case for 00:32:24.000 --> 00:32:28.000 here and I'm just trying to see uh okay 00:32:28.000 --> 00:32:32.000 this what are the kinds of mistakes that they are 00:32:36.000 --> 00:32:40.000 making the cleanest perception signal is 00:32:44.000 --> 00:32:48.000 approximation I see 00:32:48.000 --> 00:32:52.000 okay so uh if the 00:32:52.000 --> 00:32:56.000 question contains words like approximate, estimate 00:32:56.000 --> 00:33:00.000 or roughly then the models 00:33:00.000 --> 00:33:04.000 need to infer a number rather than just a copy 00:33:04.000 --> 00:33:08.000 of the printed label and that kind of 00:33:08.000 --> 00:33:12.000 inference they seem to be doing much worse at so 00:33:12.000 --> 00:33:16.000 if what this is saying is look at least if you look at chart 00:33:16.000 --> 00:33:20.000 QA one of the principles is don't ask the 00:33:20.000 --> 00:33:24.000 reader the model in this case to infer something from the 00:33:24.000 --> 00:33:28.000 chart print the number there directly otherwise they do a bad job 00:33:28.000 --> 00:33:32.000 of frankly this is something that I tell all of 00:33:32.000 --> 00:33:36.000 my team members anyway never have the user 00:33:36.000 --> 00:33:40.000 infer a number just show it there that's reflected for 00:33:40.000 --> 00:33:44.000 models as well 00:33:44.000 --> 00:33:48.000 visual density uh 00:33:48.000 --> 00:33:52.000 so what it's saying is okay the denser the 00:33:56.000 --> 00:34:00.000 picture. Okay, I'm 00:34:00.000 --> 00:34:04.000 not even able to figure out what it's saying. 00:34:04.000 --> 00:34:08.000 Uh h okay, this is a good 00:34:08.000 --> 00:34:12.000 comparison of how models compare with 00:34:12.000 --> 00:34:16.000 humans. And we know the Cleveland and you probably remember the Cleveland and 00:34:16.000 --> 00:34:20.000 McGill uh experiments where people were looking at how well 00:34:20.000 --> 00:34:24.000 uh humans do on comparing 00:34:24.000 --> 00:34:28.000 positions uh which are in a common scale non-aligned 00:34:28.000 --> 00:34:32.000 etc. 00:34:32.000 --> 00:34:36.000 And it's interesting that the models seem to have a 00:34:36.000 --> 00:34:40.000 similar perceptual gap. I'm going to go through 00:34:40.000 --> 00:34:44.000 this and probably come back with a more detailed understanding. 00:34:44.000 --> 00:34:48.000 But uh the two things that I wanted to share are 00:34:48.000 --> 00:34:52.000 a models have their own biases. They may be similar, 00:34:52.000 --> 00:34:56.000 they may be different, but they 00:34:56.000 --> 00:35:00.000 also are part of the audience now and we may as well get 00:35:00.000 --> 00:35:04.000 familiar with the kinds of mistakes they make. That's the second 00:35:04.000 --> 00:35:08.000 uh point. A 00:35:08.000 --> 00:35:12.000 model AI is able to create visualizations. That's going to change the shape 00:35:12.000 --> 00:35:16.000 of the visualizations. Therefore, focus more on technique than 00:35:16.000 --> 00:35:20.000 tools. Two, AI is going to be the audience for your visualizations. 00:35:20.000 --> 00:35:24.000 So understand them not just as a tool but as an audience that will consume your 00:35:24.000 --> 00:35:28.000 visualizations as well. That's going to make a 00:35:28.000 --> 00:35:32.000 difference. These were the first two of the four things that I'm planning to 00:35:32.000 --> 00:35:36.000 cover. But let me pause here. Any comments, 00:35:36.000 --> 00:35:40.000 Any questions, 00:35:44.000 --> 00:35:48.000 comments? If no question, we 00:35:48.000 --> 00:35:52.000 questions. Okay, then let me ask a question. Open 00:35:52.000 --> 00:35:56.000 question. What does this mean? As a result of this, what are you going to do about 00:36:00.000 --> 00:36:04.000 it? 00:36:04.000 --> 00:36:08.000 Now, what does this mean for you? What we have discussed so far, what are you going to 00:36:08.000 --> 00:36:12.000 do about 00:36:12.000 --> 00:36:16.000 it? No. No. What does this mean? Me 00:36:16.000 --> 00:36:20.000 E. Sorry. What does this 00:36:20.000 --> 00:36:24.000 imply? 00:36:24.000 --> 00:36:28.000 Yeah. No, no, not meaning as uh 00:36:28.000 --> 00:36:32.000 What? Okay. Got it. Got it. What does it mean for you? Okay. 00:36:32.000 --> 00:36:36.000 Okay. What are you going to do about this now that I've randomly said a couple of 00:36:40.000 --> 00:36:44.000 things? What will you do with this 00:36:44.000 --> 00:36:48.000 information? 00:36:48.000 --> 00:36:52.000 including uh going and looking at some benchmark information 00:36:52.000 --> 00:36:56.000 and generating these observations. Right. 00:36:56.000 --> 00:37:00.000 So how do you reflect upon that? that 00:37:00.000 --> 00:37:04.000 may not be the correct word. Uh how you use 00:37:04.000 --> 00:37:08.000 it or how might this reflect upon your work that I 00:37:08.000 --> 00:37:12.000 Yeah, 00:37:12.000 --> 00:37:16.000 exactly. What are the Exactly. What is the point of 00:37:16.000 --> 00:37:20.000 learning if you're not going to do something about 00:37:20.000 --> 00:37:24.000 it? 00:37:24.000 --> 00:37:28.000 that throughout the course. We have been asking this question again and again. So 00:37:28.000 --> 00:37:32.000 the question was 00:37:32.000 --> 00:37:36.000 that you know once again you're able to 00:37:36.000 --> 00:37:40.000 No, if you could summarize that would be 00:37:40.000 --> 00:37:44.000 great. Thank 00:37:44.000 --> 00:37:48.000 you. 00:37:48.000 --> 00:37:52.000 thing which you brought out was that it is becoming easy for each of us to kind of use the 00:37:52.000 --> 00:37:56.000 LLM and visualize in the way we want. That 00:37:56.000 --> 00:38:00.000 was the uh first uh you know thing which you 00:38:00.000 --> 00:38:04.000 kind of tried to convey to us. But the second thing is that 00:38:04.000 --> 00:38:08.000 whatever LLM will rather generate for each of us. Are we going to bank upon the same 00:38:08.000 --> 00:38:12.000 thing or are we going to analyze further? And that analysis can only 00:38:12.000 --> 00:38:16.000 be done if if and only if uh we know how 00:38:16.000 --> 00:38:20.000 are they rather trying to see the material 00:38:20.000 --> 00:38:24.000 or generate the charts in some sense. If I'm on the right 00:38:24.000 --> 00:38:28.000 um in 00:38:28.000 --> 00:38:32.000 uh got you interestingly I would frame that 00:38:32.000 --> 00:38:36.000 differently the second part and we'll come back to the first the second is 00:38:36.000 --> 00:38:40.000 saying when you create a 00:38:40.000 --> 00:38:44.000 visualization and you send it to somebody They are not 00:38:44.000 --> 00:38:48.000 going to open it. They're going to update upload it into their 00:38:48.000 --> 00:38:52.000 chart GPT and their chart GP is going to see the 00:38:52.000 --> 00:38:56.000 Okay. So, one model is rather creating and the other model is 00:38:56.000 --> 00:39:00.000 being uh evaluating the output of the first 00:39:00.000 --> 00:39:04.000 visualization. Exactly. So the 00:39:04.000 --> 00:39:08.000 model has to understand the visualization and you've 00:39:08.000 --> 00:39:12.000 aptly summarized what I've said. My question 00:39:12.000 --> 00:39:16.000 is and you're welcome to answer it or anyone else is welcome to join 00:39:16.000 --> 00:39:20.000 you but so what are you going to do about 00:39:20.000 --> 00:39:24.000 It's totally a 00:39:24.000 --> 00:39:28.000 subjective because now everybody wants so much of stuff in so little 00:39:28.000 --> 00:39:32.000 time. We would rather try to compete with our 00:39:32.000 --> 00:39:36.000 own uh people who are there around us and try to match what 00:39:36.000 --> 00:39:40.000 is going on in the world. The first is the survival whether 00:39:40.000 --> 00:39:44.000 we are able to survive or not and after we are very certain that 00:39:44.000 --> 00:39:48.000 we'll survive the competition. Then comes what are we really going 00:39:48.000 --> 00:39:52.000 to you know kind of uh add from our 00:39:52.000 --> 00:39:56.000 perspective. So as of now what I'm saying is that we are just trying to survive that is the 00:39:56.000 --> 00:40:00.000 which is 00:40:00.000 --> 00:40:04.000 doing it I will do it but to an extent till the time you know 00:40:04.000 --> 00:40:08.000 I'm stable. So once I'm table then I'll start asking next question 00:40:08.000 --> 00:40:12.000 like what it is going to give to me is really good is it really 00:40:12.000 --> 00:40:16.000 important. 00:40:16.000 --> 00:40:20.000 Got you. Any 00:40:20.000 --> 00:40:24.000 anyone else any thoughts based on what we've 00:40:24.000 --> 00:40:28.000 covered? How are you going to apply 00:40:36.000 --> 00:40:40.000 Any other 00:40:40.000 --> 00:40:44.000 it? 00:40:44.000 --> 00:40:48.000 Uh, no additional 00:40:48.000 --> 00:40:52.000 I'll 00:40:52.000 --> 00:40:56.000 wait. Sometimes it's worth 00:40:56.000 --> 00:41:00.000 he's he's unlike me. He says I will 00:41:04.000 --> 00:41:08.000 pushing. 00:41:08.000 --> 00:41:12.000 used. I mean there's a beautiful stuff that he showed in the first right that the 00:41:12.000 --> 00:41:16.000 G keep uh I don't know what is the 00:41:16.000 --> 00:41:20.000 underlying model that goes behind it but it is like real time monitoring 00:41:20.000 --> 00:41:24.000 so real time the the dynamic chart also keeps changing. uh 00:41:24.000 --> 00:41:28.000 this is one of the major stuff for instance if you're doing IoT any 00:41:28.000 --> 00:41:32.000 manufacturing stuff it's not about static data it's about there is 00:41:32.000 --> 00:41:36.000 components are coming in machines are running and you'll have to monitor this on an 00:41:36.000 --> 00:41:40.000 online stuff real time okay or marketing analytics for 00:41:40.000 --> 00:41:44.000 instance is real time so many of these things stock market many of these things are 00:41:44.000 --> 00:41:48.000 actually you want real time graph and not static graphs so 00:41:48.000 --> 00:41:52.000 I thought that was I I don't know the model I just saw that you keep 00:41:52.000 --> 00:41:56.000 first time any show today. So to me I think that's one 00:41:56.000 --> 00:42:00.000 thing that but then that was just the visualization part 00:42:00.000 --> 00:42:04.000 but uh if we use the interpretation 00:42:04.000 --> 00:42:08.000 engine you know that is one application that the whatever the 00:42:08.000 --> 00:42:12.000 last discussion might be relevant to me in that 00:42:12.000 --> 00:42:16.000 completely now. Got your take away. Let's also hear from 00:42:16.000 --> 00:42:20.000 some 00:42:28.000 --> 00:42:32.000 but right so 00:42:32.000 --> 00:42:36.000 what are we going to do right so he's sort of 00:42:36.000 --> 00:42:40.000 students. I got the question. 00:42:40.000 --> 00:42:44.000 Yeah, correct. 00:42:44.000 --> 00:42:48.000 That is my question to you. 00:42:48.000 --> 00:42:52.000 Paraphrasing exactly the question. 00:42:56.000 --> 00:43:00.000 Correct. No, 00:43:00.000 --> 00:43:04.000 no. Fair 00:43:04.000 --> 00:43:08.000 point. Okay. Okay, tell us tell us more, 00:43:08.000 --> 00:43:12.000 please. Go on. Go on. 00:43:12.000 --> 00:43:16.000 Maybe you could just come over to the mic and I'll hear 00:43:16.000 --> 00:43:20.000 Okay. So, he's asking that's it. He says that's 00:43:20.000 --> 00:43:24.000 it. That's all his point is accuracy. 00:43:24.000 --> 00:43:28.000 better. Got you. Which is a fair point, right? That is a practical 00:43:28.000 --> 00:43:32.000 implication. Any 00:43:32.000 --> 00:43:36.000 others? 00:43:36.000 --> 00:43:40.000 right. 00:43:44.000 --> 00:43:48.000 Please and I 00:43:48.000 --> 00:43:52.000 introduce my own bias when I say what you have. So 00:43:52.000 --> 00:43:56.000 yeah, I mean what I understood from all this is that uh it can save 00:43:56.000 --> 00:44:00.000 our time. I can give all the visualization and everything but at the end what we are going to do is 00:44:00.000 --> 00:44:04.000 we are the ones who will be deciding on whether to uh completely rely on it 00:44:04.000 --> 00:44:08.000 or like how we go about the 00:44:08.000 --> 00:44:12.000 True. And because of that, 00:44:12.000 --> 00:44:16.000 what do you think you would want to do 00:44:16.000 --> 00:44:20.000 differently? Yeah. So, what will you do 00:44:20.000 --> 00:44:24.000 differently? 00:44:24.000 --> 00:44:28.000 Um I mean uh maybe assessing 00:44:28.000 --> 00:44:32.000 the answers which it gave or yeah basically 00:44:32.000 --> 00:44:36.000 optimizing. 00:44:36.000 --> 00:44:40.000 Got you. Thank you. I'm learning something 00:44:40.000 --> 00:44:44.000 uh from this class actually. Whole bunch of things. 00:44:48.000 --> 00:44:52.000 Uh, one of my major takeaways 00:44:52.000 --> 00:44:56.000 is and now that I reflect on my classroom and my friends 00:44:56.000 --> 00:45:00.000 was a student sitting in a lecture as well, 00:45:00.000 --> 00:45:04.000 the mode of learning uh for 00:45:04.000 --> 00:45:08.000 many of us. Oh actually I I should frame 00:45:08.000 --> 00:45:12.000 this differently. For many years, I 00:45:12.000 --> 00:45:16.000 have been learning stuff because I wanted to do something. I have a 00:45:16.000 --> 00:45:20.000 problem. I will go learn how to solve the 00:45:20.000 --> 00:45:24.000 problem. That makes sense. In your case, you have a problem. 00:45:24.000 --> 00:45:28.000 You have an exam to crack. So, you're kind of using one 00:45:28.000 --> 00:45:32.000 technique which is sitting in the classroom and figuring out 00:45:32.000 --> 00:45:36.000 how to solve it. Obviously, you don't really know what of what I'm going to say is going to come 00:45:36.000 --> 00:45:40.000 in what exam or whatever. I find 00:45:40.000 --> 00:45:44.000 that half the time, most of the time when 00:45:44.000 --> 00:45:48.000 I don't know why I'm learning something, I learn a lot 00:45:52.000 --> 00:45:56.000 less. So, I usually just walk out of 00:45:56.000 --> 00:46:00.000 uh any place where I don't know why I'm in that 00:46:00.000 --> 00:46:04.000 room. You are in less fortunate 00:46:04.000 --> 00:46:08.000 situation. You probably at least for attendance got to have to sit in 00:46:08.000 --> 00:46:12.000 there. But I have one degree of 00:46:12.000 --> 00:46:16.000 freedom which is the at least the second half of this 00:46:16.000 --> 00:46:20.000 class I can choose 00:46:20.000 --> 00:46:24.000 to answer what you want to know rather 00:46:24.000 --> 00:46:28.000 than me telling you what I think you should know because you won't understand why I 00:46:28.000 --> 00:46:32.000 think you should know that and it's hard for me to communicate 00:46:32.000 --> 00:46:36.000 but if you say look I want to know this I can 00:46:36.000 --> 00:46:40.000 answer I'll Sir, that at least will be a 00:46:40.000 --> 00:46:44.000 more how they put it. At least 00:46:44.000 --> 00:46:48.000 you will get the answer to something that you want to know rather 00:46:48.000 --> 00:46:52.000 than knowing something that you don't even know you want to 00:46:52.000 --> 00:46:56.000 know. So, let's flip this around at least 15 00:46:56.000 --> 00:47:00.000 minutes pure Q&A. What would 00:47:00.000 --> 00:47:04.000 you like to know about? And we'll talk about that. I'll share from what I've 00:47:04.000 --> 00:47:08.000 learned. 00:47:08.000 --> 00:47:12.000 And in case the audio wasn't clear or I was just 00:47:12.000 --> 00:47:16.000 rambling summary at next 15 00:47:16.000 --> 00:47:20.000 minutes any questions related to visualization and AI that 00:47:20.000 --> 00:47:24.000 you 00:47:24.000 --> 00:47:28.000 He's he's flipping the set setup now. So 00:47:28.000 --> 00:47:32.000 he delivered some stuff. He asked you questions. 00:47:32.000 --> 00:47:36.000 Now I don't know from our end we may have an opinion 00:47:36.000 --> 00:47:40.000 about it. So what he says is he's turning the tables now and then he 00:47:40.000 --> 00:47:44.000 says you first raise the questions and accordingly I will 00:47:44.000 --> 00:47:48.000 set up the class. That's what he 00:47:48.000 --> 00:47:52.000 says. This is like absolutely 00:47:52.000 --> 00:47:56.000 open right it can be any question on visualization. It need not be what he covered 00:47:56.000 --> 00:48:00.000 today. It could be something that we have seen in the last six seven weeks or it 00:48:00.000 --> 00:48:04.000 could be any anything in that sense related to 00:48:04.000 --> 00:48:08.000 visualization or uh um 00:48:08.000 --> 00:48:12.000 AI or in the intersection of AI and 00:48:12.000 --> 00:48:16.000 and think about how it will help you. Please be 00:48:16.000 --> 00:48:20.000 selfish in your questions. No point asking random I 00:48:20.000 --> 00:48:24.000 mean don't feel pressurized to ask a question and it could 00:48:24.000 --> 00:48:28.000 be why should I care is a valid question. 00:48:40.000 --> 00:48:44.000 couldn't hear. 00:48:44.000 --> 00:48:48.000 Yeah, I just asked him to come over. Yeah. Is 00:48:48.000 --> 00:48:52.000 it compulsory that we need to entirely know the problem definition 00:48:52.000 --> 00:48:56.000 Oh, great question. 00:48:56.000 --> 00:49:00.000 Uh let me rephrase that. Do I need to know the problem 00:49:00.000 --> 00:49:04.000 before I start visualizing something? Depends 00:49:04.000 --> 00:49:08.000 very often on who needs a 00:49:08.000 --> 00:49:12.000 visualization. In practice, every 00:49:12.000 --> 00:49:16.000 dashboard that I've seen is created by someone who 00:49:16.000 --> 00:49:20.000 has no idea what they trying to visualize. They don't know the 00:49:20.000 --> 00:49:24.000 problem. It is also requested by someone who doesn't know the problem. 00:49:24.000 --> 00:49:28.000 Here is a typical sequence. There is 00:49:28.000 --> 00:49:32.000 a business owner 00:49:32.000 --> 00:49:36.000 typically senior leader who says I want to understand and 00:49:36.000 --> 00:49:40.000 he will give a list of why is my uh operations 00:49:40.000 --> 00:49:44.000 falling which of my machines are likely to fail in the next 3 months. 00:49:44.000 --> 00:49:48.000 He'll give 10 questions like this. Who 00:49:48.000 --> 00:49:52.000 is project manager? The project manager will 00:49:52.000 --> 00:49:56.000 understand 20% of those questions, write it 00:49:56.000 --> 00:50:00.000 down and then give it to a vendor. He'll say I will 00:50:00.000 --> 00:50:04.000 create an RFP and the RFP will say I want to be 00:50:04.000 --> 00:50:08.000 able to answer 10 such questions and 30 more 00:50:08.000 --> 00:50:12.000 of any kind. This will be picked 00:50:12.000 --> 00:50:16.000 up by somebody on the sales team 00:50:16.000 --> 00:50:20.000 of some company. Let's say the group uh uh who knows 00:50:20.000 --> 00:50:24.000 nothing about operations, who knows nothing about 00:50:24.000 --> 00:50:28.000 data visualization no but has a 00:50:28.000 --> 00:50:32.000 kota they will write a proposal saying here is how we will deliver the 00:50:32.000 --> 00:50:36.000 dashboard which will be reviewed by the project manager executed 00:50:36.000 --> 00:50:40.000 by a developer who doesn't understand operations who does not necessarily 00:50:40.000 --> 00:50:44.000 understand data visualization but knows powerba or tablet or 00:50:44.000 --> 00:50:48.000 excel or whatever and after 00:50:48.000 --> 00:50:52.000 a whole series of rounds of QA they will produce 20 charts 00:50:52.000 --> 00:50:56.000 the business user will say I don't understand head or tail of it. What 00:50:56.000 --> 00:51:00.000 rubbish project manager will say no they don't understand it. The developer will say okay I'll 00:51:00.000 --> 00:51:04.000 create 30 more charts. Okay look if I take this part of 00:51:04.000 --> 00:51:08.000 this chart and I put in this part from here 00:51:08.000 --> 00:51:12.000 then I might be able to answer at least one out of my 10 00:51:12.000 --> 00:51:16.000 questions but and then the project the business owner's 00:51:16.000 --> 00:51:20.000 boss will say you spent half a million dollars already right are 00:51:20.000 --> 00:51:24.000 you getting value for it? What is the business owner going to 00:51:24.000 --> 00:51:28.000 say? Yeah, yeah, yeah. I've got the answer to at least one of my questions. Okay, 00:51:28.000 --> 00:51:32.000 project has been a success. Now, in this 00:51:32.000 --> 00:51:36.000 chain, most of the people did not know the 00:51:36.000 --> 00:51:40.000 problem definition before they started visualizing 00:51:40.000 --> 00:51:44.000 something. They were given a statement, they tried to adhere to 00:51:44.000 --> 00:51:48.000 it. In contrast, some people are creating 00:51:48.000 --> 00:51:52.000 visualizations that they want to see or are testing 00:51:52.000 --> 00:51:56.000 against users. For example, the newsroom of 00:51:56.000 --> 00:52:00.000 uh of the New York Times or 00:52:00.000 --> 00:52:04.000 the editor has an excellent idea of what the 00:52:04.000 --> 00:52:08.000 general public will understand. They will 00:52:08.000 --> 00:52:12.000 tell the uh graphics team, create it like this. Make this 00:52:12.000 --> 00:52:16.000 change. Iterate, iterate, iterate. sitting literally on their head because the 00:52:16.000 --> 00:52:20.000 timeline is what a few hours before they 00:52:20.000 --> 00:52:24.000 publish something. In that case, it's 00:52:24.000 --> 00:52:28.000 still the graphics designer does not need to 00:52:28.000 --> 00:52:32.000 know what the problem is, but the editor is sitting right next to them. The 00:52:32.000 --> 00:52:36.000 editor knows and or the journalist, whoever. 00:52:36.000 --> 00:52:40.000 But that is a slightly different kind of situation. Same thing happens 00:52:40.000 --> 00:52:44.000 with let's say scientific American or springer 00:52:44.000 --> 00:52:48.000 nature some of the popular scientific publications or the 00:52:48.000 --> 00:52:52.000 economist where again they uh the person creating the visualization 00:52:52.000 --> 00:52:56.000 is sitting very close to the person who understands what the audience 00:52:56.000 --> 00:53:00.000 needs so still works. The third is the 00:53:00.000 --> 00:53:04.000 citizen data scientist or the analyst or 00:53:04.000 --> 00:53:08.000 whoever who says look I have a problem. I want to buy a 00:53:08.000 --> 00:53:12.000 laptop. I need the maximum features and the lowest 00:53:12.000 --> 00:53:16.000 cost. I will plot it. I will see which is on the top left or top right and buy 00:53:16.000 --> 00:53:20.000 it. There you know exactly what you want. Or last time when I 00:53:20.000 --> 00:53:24.000 showed you my internet movie database chart, right? Rating. things versus 00:53:24.000 --> 00:53:28.000 votes. I know exactly what I want. Put another 00:53:28.000 --> 00:53:32.000 way, most of the time if you're creating stuff for 00:53:32.000 --> 00:53:36.000 yourself, it's important that you should know what you want. But even if you 00:53:36.000 --> 00:53:40.000 don't, you can iterate on it. Otherwise, have somebody close to you who knows the 00:53:40.000 --> 00:53:44.000 problem definition. Well, in reality, 90% of 00:53:44.000 --> 00:53:48.000 visualizations today are created by people who don't know the uh the problem definition. You 00:53:48.000 --> 00:53:52.000 can still make a living out of it. Did that answer the question? 00:53:52.000 --> 00:53:56.000 I will assume 00:53:56.000 --> 00:54:00.000 closer to the 00:54:00.000 --> 00:54:04.000 concerned yeah he he's coming so for any 00:54:04.000 --> 00:54:08.000 data visualization to take place the first assumption is that there is some 00:54:08.000 --> 00:54:12.000 data and there is going to be a question that is going to be asked to him or he can himself 00:54:12.000 --> 00:54:16.000 mic. Um, 00:54:16.000 --> 00:54:20.000 that there should be data usually true. that there should be a 00:54:20.000 --> 00:54:24.000 question usually not true most dashboards 00:54:24.000 --> 00:54:28.000 are basically saying I don't know what question can be asked you should be able to answer any 00:54:28.000 --> 00:54:32.000 question I'm not saying there shouldn't be a question and you're saying in 00:54:32.000 --> 00:54:36.000 practice most data visualizations are created not to answer 00:54:36.000 --> 00:54:40.000 a specific question but to satisfy 00:54:40.000 --> 00:54:44.000 somebody but I'm still I I suspect I've still not answered your question so go on 00:54:44.000 --> 00:54:48.000 push 00:54:48.000 --> 00:54:52.000 what are you really asking 00:54:52.000 --> 00:54:56.000 uh the problem is also in trying to understand what 00:54:56.000 --> 00:55:00.000 kind of data are you going to visualize because there is so varied 00:55:00.000 --> 00:55:04.000 kinds of data as of today like you showed that the text can also 00:55:04.000 --> 00:55:08.000 be kind of visualized the satellite imagery can be 00:55:08.000 --> 00:55:12.000 visualized and you know the normal data the numbers as such can be visualized 00:55:12.000 --> 00:55:16.000 So like how do you really hold on to a very 00:55:16.000 --> 00:55:20.000 got 00:55:20.000 --> 00:55:24.000 you which is a slightly different question. I'll reframe it this 00:55:24.000 --> 00:55:28.000 way. I don't exactly know what I want but I am the 00:55:28.000 --> 00:55:32.000 audience. I have some data. There are many ways of visualizing it. 00:55:32.000 --> 00:55:36.000 How would I go about it? That's part of what you're learning in 00:55:36.000 --> 00:55:40.000 this course. Uh 00:55:40.000 --> 00:55:44.000 supposing you see 50 different 00:55:44.000 --> 00:55:48.000 kinds of visualizations and you take a closer look at 00:55:48.000 --> 00:55:52.000 them. You get a sense of okay, tomorrow I can use this. Very 00:55:52.000 --> 00:55:56.000 similar to cooking. You've 00:55:56.000 --> 00:56:00.000 tasted 50 dishes and you say, "Oh, I 00:56:00.000 --> 00:56:04.000 kind of like this. I kind of like that." At least you'll be able to 00:56:04.000 --> 00:56:08.000 pick dishes next time. Or if you've prepared a 00:56:08.000 --> 00:56:12.000 dish, you know that adding this ingredient makes it taste 00:56:12.000 --> 00:56:16.000 slightly this way. The next time you'll be able to 00:56:16.000 --> 00:56:20.000 cook much easier. So, how does one go 00:56:20.000 --> 00:56:24.000 about knowing um how we 00:56:24.000 --> 00:56:28.000 visualize data and what problem uh can be 00:56:28.000 --> 00:56:32.000 solved to trying out a variety of different techniques. You don't have to be the one 00:56:32.000 --> 00:56:36.000 who does all of it. Meaning to be a good cook, you don't 00:56:36.000 --> 00:56:40.000 have to cook every dish. You certainly ought to taste a lot of dishes 00:56:40.000 --> 00:56:44.000 and cook a few. So see as many 00:56:44.000 --> 00:56:48.000 visualizations as you can of different types. 00:56:48.000 --> 00:56:52.000 And by see I don't mean you necessarily need to spend 00:56:52.000 --> 00:56:56.000 more time looking for visualizations. 00:56:56.000 --> 00:57:00.000 The current time that you spend looking for visualizations look a little 00:57:00.000 --> 00:57:04.000 carefully and ask the question how can I use 00:57:04.000 --> 00:57:08.000 this? Which is exactly the question I was asking earlier. I showed a bunch of 00:57:08.000 --> 00:57:12.000 stuff. If you ask how can I use this? 00:57:12.000 --> 00:57:16.000 The time that you spend looking at the visualizations will help you 00:57:16.000 --> 00:57:20.000 more. That to me is probably the easiest. 00:57:20.000 --> 00:57:24.000 So I'll summarize my answer as 00:57:24.000 --> 00:57:28.000 um when you spend time 00:57:28.000 --> 00:57:32.000 with creating or seeing visualizations, 00:57:32.000 --> 00:57:36.000 ask yourself how is this helping me or how will it help me 00:57:36.000 --> 00:57:40.000 in the future? So that when you create your own, you will find it 00:57:40.000 --> 00:57:44.000 easier. 00:57:44.000 --> 00:57:48.000 Wait 00:57:48.000 --> 00:57:52.000 for internet issue to get 00:58:08.000 --> 00:58:12.000 resolved. 00:59:36.000 --> 00:59:40.000 Yes, I can. Am I 00:59:40.000 --> 00:59:44.000 us? Yes. Yes. So, I'm sorry the the 00:59:44.000 --> 00:59:48.000 wireless dropped. You are on data right now, but uh let me just make sure. Yeah. 00:59:48.000 --> 00:59:52.000 audible? No, is where did uh students lose 00:59:52.000 --> 00:59:56.000 me? No, 00:59:56.000 --> 01:00:00.000 sorry. Uh, what was I last saying that they 01:00:04.000 --> 01:00:08.000 a 01:00:08.000 --> 01:00:12.000 heard? 01:00:12.000 --> 01:00:16.000 Kumar is trying to log in. 01:00:16.000 --> 01:00:20.000 Santos, is it 01:00:20.000 --> 01:00:24.000 you? I'm sorry. Okay. 01:00:24.000 --> 01:00:28.000 Uh what was he trying to answer and uh where did we got 01:00:28.000 --> 01:00:32.000 Yeah. What is the last that you that 01:00:32.000 --> 01:00:36.000 Sorry was trying to enter the same meeting. So I 01:00:36.000 --> 01:00:40.000 any more the marrier? Let 01:00:40.000 --> 01:00:44.000 them join. What do we 01:00:44.000 --> 01:00:48.000 lose? Whatever it is, let them 01:00:52.000 --> 01:00:56.000 Yeah. What was he answering? He he was 01:00:56.000 --> 01:01:00.000 answering his question on you know should I know that what type of 01:01:00.000 --> 01:01:04.000 data to you connect and what what what is to be plotted and all that. He 01:01:04.000 --> 01:01:08.000 said something like a cookie choice and all that. Today you could you 01:01:08.000 --> 01:01:12.000 will be able to generate graphs. That that is where we sort of got disconnected 01:01:12.000 --> 01:01:16.000 join. Okay. Okay. Uh oh. Then I have to re 01:01:16.000 --> 01:01:20.000 which is probably about so the internet hung 01:01:20.000 --> 01:01:24.000 for couple of minutes. Got it. Fine. I'll just 01:01:24.000 --> 01:01:28.000 summarize then by saying that when you are looking at any 01:01:28.000 --> 01:01:32.000 visualization uh think about how you can 01:01:32.000 --> 01:01:36.000 apply whatever you are seeing. That 01:01:36.000 --> 01:01:40.000 way when in the future you want to create a visualization 01:01:40.000 --> 01:01:44.000 you will know what works, what doesn't work, what techniques might 01:01:44.000 --> 01:01:48.000 be relevant and so on. You don't have to spend 01:01:48.000 --> 01:01:52.000 more time studying visualizations but you have to 01:01:52.000 --> 01:01:56.000 spend the the time that you spent on visualizations more 01:01:56.000 --> 01:02:00.000 uh towards learning how to use it rather 01:02:00.000 --> 01:02:04.000 than just looking at it. 01:02:08.000 --> 01:02:12.000 other question. Yes. Yeah. 01:02:12.000 --> 01:02:16.000 Yeah. Come 01:02:16.000 --> 01:02:20.000 come. Hello sir. Um so my question is about 01:02:20.000 --> 01:02:24.000 LLM can generate a visualization or some plots 01:02:24.000 --> 01:02:28.000 and then it can interpret also but it may or may not correct 01:02:28.000 --> 01:02:32.000 uh the interpretation s may or may not correct. So is there 01:02:32.000 --> 01:02:36.000 any validation technique is there because it can give the plot it can give the 01:02:36.000 --> 01:02:40.000 trends but how can I believe it's accuracy accuracy of 01:02:40.000 --> 01:02:44.000 Thank you. Now I will go to the third part 01:02:44.000 --> 01:02:48.000 of what I was going to cover because this is said but 01:02:48.000 --> 01:02:52.000 question to you. Uh I will share something. How will 01:02:52.000 --> 01:02:56.000 you use 01:02:56.000 --> 01:03:00.000 the data to the ll and try to uh 01:03:00.000 --> 01:03:04.000 it? 01:03:04.000 --> 01:03:08.000 interpret the plot. So this is the way I can 01:03:08.000 --> 01:03:12.000 H okay. And how will that help you? That's the you're saying this is what I will 01:03:12.000 --> 01:03:16.000 do. Okay. What is the benefit of knowing um the answer to 01:03:16.000 --> 01:03:20.000 your question which is 01:03:20.000 --> 01:03:24.000 uh the data and what kind of decision I can make from 01:03:24.000 --> 01:03:28.000 the interpretation. So this is the one way I can do 01:03:28.000 --> 01:03:32.000 uh still weak. But that's I mean not a 01:03:32.000 --> 01:03:36.000 judgment. What I'm saying is that uh if you had 01:03:36.000 --> 01:03:40.000 a clearer idea of your question, 01:03:40.000 --> 01:03:44.000 the answer will land more strongly. But I'm still going to go ahead and give you 01:03:44.000 --> 01:03:48.000 an answer to a related question perhaps. 01:03:48.000 --> 01:03:52.000 Let's see where that goes. But I would love for you and 01:03:52.000 --> 01:03:56.000 anyone else uh to answer a specific question which 01:03:56.000 --> 01:04:00.000 is so how are you going to use what I'm just sharing. 01:04:00.000 --> 01:04:04.000 Um let's start maybe 01:04:04.000 --> 01:04:08.000 with a few well let's 01:04:08.000 --> 01:04:12.000 start my screen should be uh visible 01:04:12.000 --> 01:04:16.000 right okay um so one of the things that I 01:04:16.000 --> 01:04:20.000 did a short while ago was have an 01:04:20.000 --> 01:04:24.000 LLM take an engineering 01:04:24.000 --> 01:04:28.000 drawing and asked it now can you create a CAD diagram out of 01:04:28.000 --> 01:04:32.000 It 01:04:32.000 --> 01:04:36.000 tried. This was given to Codeex. The only 01:04:36.000 --> 01:04:40.000 information that it was given was this 01:04:40.000 --> 01:04:44.000 uh drawing. Frecad is the software that 01:04:44.000 --> 01:04:48.000 it had access to and it uh had 01:04:48.000 --> 01:04:52.000 the ability to run a render command. So first it did all kinds of things. 01:04:52.000 --> 01:04:56.000 It said I will look at this uh image closer. I will 01:04:56.000 --> 01:05:00.000 uh do some cropping of the image. I 01:05:00.000 --> 01:05:04.000 will try out other tools etc. Then it gave 01:05:04.000 --> 01:05:08.000 up and then next attempt it said okay I'm going to 01:05:08.000 --> 01:05:12.000 generate this image. Now this actually 01:05:12.000 --> 01:05:16.000 looks uh pretty much like the drawing. The interesting 01:05:16.000 --> 01:05:20.000 thing is that it passed all of its own tests. 01:05:20.000 --> 01:05:24.000 It said I will write a series of tests to make sure that it is same as the 01:05:24.000 --> 01:05:28.000 drawing. I will check if what I've drawn passes those 01:05:28.000 --> 01:05:32.000 tests and if it does then agreed. If not, I will keep iterating 01:05:32.000 --> 01:05:36.000 until it does. And it built a path 01:05:36.000 --> 01:05:40.000 that against the 01:05:40.000 --> 01:05:44.000 drawing. If I were to look at it, I would probably say, "Huh, this kind of looks like 01:05:44.000 --> 01:05:48.000 that. I'm not sure I know much 01:05:48.000 --> 01:05:52.000 better." But it looked at it closer and said, "No, 01:05:52.000 --> 01:05:56.000 I'm not happy with this." Um, it said the support 01:05:56.000 --> 01:06:00.000 uh is a 48 mm feature and it 01:06:00.000 --> 01:06:04.000 should be 35 mm. So, I'm going to redraw it. So, 01:06:04.000 --> 01:06:08.000 this protrusion that you see on the left is wider 01:06:08.000 --> 01:06:12.000 than the protrusion that you see on the left. It inferred that from the 01:06:12.000 --> 01:06:16.000 diagram, went through that and said, "Now, this is the final 01:06:16.000 --> 01:06:20.000 thing. I can't find any mistakes in my 01:06:20.000 --> 01:06:24.000 operations. It looks like the right kind of 01:06:24.000 --> 01:06:28.000 bracket. This is what it was supposed to produce on the 01:06:28.000 --> 01:06:32.000 left. The image on the left was not given to 01:06:32.000 --> 01:06:36.000 Codex. What it actually produced was 01:06:36.000 --> 01:06:40.000 the one on the right. And you can see the 01:06:40.000 --> 01:06:44.000 difference. Firstly, there is an entire bottom 01:06:44.000 --> 01:06:48.000 portion that is missing. it should be 01:06:48.000 --> 01:06:52.000 uh curved rather than rectangular at the bottom 01:06:52.000 --> 01:06:56.000 left and the bottom right. At least those are the two 01:06:56.000 --> 01:07:00.000 major errors. They're very similar but they are 01:07:00.000 --> 01:07:04.000 not the same object. And the 01:07:04.000 --> 01:07:08.000 interesting thing is that if you look at the uh metrics 01:07:08.000 --> 01:07:12.000 um okay I'm going to skip all of this uh if you look at 01:07:12.000 --> 01:07:16.000 how it decided to construct it the construction methods are 01:07:16.000 --> 01:07:20.000 similar the original reference the object that was to have been created 01:07:20.000 --> 01:07:24.000 started with an I boss it then we're supposed to 01:07:24.000 --> 01:07:28.000 add a main body profile then the lower work profile and 01:07:28.000 --> 01:07:32.000 then cut holes it cut holes codeex also cut holes 01:07:32.000 --> 01:07:36.000 at at the end, but it started with a base roughly the equivalent of 01:07:36.000 --> 01:07:40.000 the main body profile. Uh, and in the third 01:07:40.000 --> 01:07:44.000 step, it cut the hole instead of right 01:07:44.000 --> 01:07:48.000 up front. It added the circular right at the second 01:07:48.000 --> 01:07:52.000 iteration. But what it missed was this lower fork 01:07:52.000 --> 01:07:56.000 profile which it could not see clearly enough. But what 01:07:56.000 --> 01:08:00.000 is more interesting is that the tests for 01:08:00.000 --> 01:08:04.000 this that it created were insufficient as 01:08:04.000 --> 01:08:08.000 well. And uh the final tests 01:08:08.000 --> 01:08:12.000 I'm trying to see if I have the list of Oh yeah, further 01:08:12.000 --> 01:08:16.000 up, right? No, where did the tests 01:08:16.000 --> 01:08:20.000 go? Maybe this version doesn't have the 01:08:20.000 --> 01:08:24.000 tests, but the test cases ware uh 01:08:24.000 --> 01:08:28.000 generated by a CAD benchmark called FRECAD 01:08:28.000 --> 01:08:32.000 where uh it provides not just the drawing but also 01:08:32.000 --> 01:08:36.000 what is the volume of these what are 01:08:36.000 --> 01:08:40.000 uh some of the finer details of 01:08:40.000 --> 01:08:44.000 the shape about 40 50 parameters that you can very 01:08:44.000 --> 01:08:48.000 granularly test along with the specification and only if it matches all of 01:08:48.000 --> 01:08:52.000 those will we be able to say that yes this matches 01:08:52.000 --> 01:08:56.000 What does that imply? 01:08:56.000 --> 01:09:00.000 What I take away from these things 01:09:00.000 --> 01:09:04.000 is number one if I give a picture to a 01:09:04.000 --> 01:09:08.000 model and say produce something these days it is able to 01:09:08.000 --> 01:09:12.000 produce something not only produce something it is 01:09:12.000 --> 01:09:16.000 also able to correct mistakes in what it produces. 01:09:16.000 --> 01:09:20.000 Second thing that I'm learning is I'm not 01:09:20.000 --> 01:09:24.000 able to tell the difference. You already saw me reading a 01:09:24.000 --> 01:09:28.000 chart worse than even a basic model. I've been 01:09:28.000 --> 01:09:32.000 reading charts for decades. 01:09:32.000 --> 01:09:36.000 This kind of a mistake is just too 01:09:36.000 --> 01:09:40.000 easy for me to make. So I'm not very good at spotting mistakes. 01:09:40.000 --> 01:09:44.000 Third that there are benchmarks out 01:09:44.000 --> 01:09:48.000 there and there are ways of specifying 01:09:48.000 --> 01:09:52.000 uh correctness out there like the CAD 01:09:52.000 --> 01:09:56.000 bench that I can use as a format to say I will 01:09:56.000 --> 01:10:00.000 consider this correct only if it meets all of these criteria and I 01:10:00.000 --> 01:10:04.000 have to sit and create those criteria. 01:10:04.000 --> 01:10:08.000 Those are three takeaways for 01:10:08.000 --> 01:10:12.000 me in terms of how does one go about 01:10:12.000 --> 01:10:16.000 verifying having an LM generate a visualization. In 01:10:16.000 --> 01:10:20.000 this case, wasn't quite a visualization but close enough and I'll show you some visualization 01:10:20.000 --> 01:10:24.000 examples. And how do you verify those? But what are your 01:10:28.000 --> 01:10:32.000 takeaways? 01:10:36.000 --> 01:10:40.000 now. So Anand actually we are sort of done with the 01:10:40.000 --> 01:10:44.000 the class time. It's it's 3:15 here. We 01:10:44.000 --> 01:10:48.000 No, it was in any case the last thing 01:10:48.000 --> 01:10:52.000 I was going to cover. So, 01:10:52.000 --> 01:10:56.000 yeah. 01:10:56.000 --> 01:11:00.000 And uh for the next 01:11:00.000 --> 01:11:04.000 session uh I would like I can assign 01:11:04.000 --> 01:11:08.000 homework. I'll drop you an email 01:11:08.000 --> 01:11:12.000 with some 01:11:12.000 --> 01:11:16.000 homework. Cool. Thanks 01:11:16.000 --> 01:11:20.000 everyone. 01:11:20.000 --> 01:11:24.000 Okay. Just just one thing before 01:11:24.000 --> 01:11:28.000 you leave right see he's 01:11:28.000 --> 01:11:32.000 one person today if you're going to like pick up 01:11:32.000 --> 01:11:36.000 uh you know top three to five people on data visualization who has been 01:11:36.000 --> 01:11:40.000 there done hands- on stuff run a company uh sold 01:11:40.000 --> 01:11:44.000 a company as chief of innovation at some other place and all 01:11:44.000 --> 01:11:48.000 that it is that guy okay and many things that whatever he's 01:11:48.000 --> 01:11:52.000 showing including the mahabharata stuff is most likely 01:11:52.000 --> 01:11:56.000 handcoded by him. Okay. Of course, he has a team and all that. He is 01:11:56.000 --> 01:12:00.000 extremely extremely hands-off. Extremely hands-on, right? And whatever 01:12:00.000 --> 01:12:04.000 he showed probably generated after discussion with me just 01:12:04.000 --> 01:12:08.000 about an hour ago or hour and a half ago. Okay? So it is 01:12:08.000 --> 01:12:12.000 important for you to to ask the right questions or at least 01:12:12.000 --> 01:12:16.000 questions when such resources are being uh that could be the 01:12:16.000 --> 01:12:20.000 biggest difference between someone who comes here and physically 01:12:20.000 --> 01:12:24.000 talks and someone whom we bring from industry that that's the biggest 01:12:24.000 --> 01:12:28.000 difference that they could bring right because they see like lot more challenges for instance 01:12:28.000 --> 01:12:32.000 that one example that he showed like often times people who do dashboards 01:12:32.000 --> 01:12:36.000 are someone who doesn't know anything on the application front any of the software that you're 01:12:36.000 --> 01:12:40.000 talking about is Why banking software took about 20 years 01:12:40.000 --> 01:12:44.000 for them to what they are today is because it was built by someone who didn't 01:12:44.000 --> 01:12:48.000 appreciate what finance side was anyway. So that the point is not just in 01:12:48.000 --> 01:12:52.000 his class right in any other class where outside resources like this has 01:12:52.000 --> 01:12:56.000 been broad you should leverage that but you are actually 01:12:56.000 --> 01:13:00.000 going in the recursive side you know you're not even regressive side 01:13:00.000 --> 01:13:04.000 meaning you're not you're not asking any questions and 01:13:04.000 --> 01:13:08.000 which is not a bad thing, but at least get some 01:13:08.000 --> 01:13:12.000 sleep during class in that case. 01:13:12.000 --> 01:13:16.000 the I thought we got out of the call and I was giving 01:13:16.000 --> 01:13:20.000 Okay. All right. Anyway, I didn't say 01:13:20.000 --> 01:13:24.000 anything. I suddenly when you 01:13:24.000 --> 01:13:28.000 Or 01:13:28.000 --> 01:13:32.000 right. Yeah. Okay. Thank you. Thank you all. 01:13:32.000 --> 01:13:36.000 Sure.