# Transcript - Manufacturing (IoT performance)

There's actually only one message that I want to deliver as part of this talk and it will become fairly obvious as we go along. But you've probably seen, and one of our clients certainly saw, that line performance was a pretty major issue for them. They said, "Look, we have our equipment stopping periodically. It drifts out of spec, the parameters are not tuned well enough. We're not really sure why. What we'd like you to do is maybe set up a team. We definitely want someone who understands manufacturing. We want someone who understands analysis to come together and solve this problem. And see if you can solve it before our next leadership meeting which is coming up the next quarter," which is, yeah, plenty of time away.

In fact, one of them drafted an email which literally said, "Look, that's the data that we have. We have all of the IoT line-level parameters, both input and output. Just give us the top three drivers of the low throughput and the yield loss that we are facing with all of these details."

Now, the insight as far as we are concerned, when we tried solving this problem, we did it the usual way. We passed it to an analyst, to a person who's an expert in machinery. And after about three weeks of them trying to figure out what's happening and so on, we realized something, which is that LLMs can write code pretty well. They're their understanding of the domain is solid. They write code. They're agentic, which means that once they've written something then they've made a mistake, they can correct themselves. They can look at their work, go back, and redo the whole thing. So, is it actually possible to pass our client's email to an application which uses LLMs to solve the problem end to end? What does it take?

We did that, and the session roughly went like this. We took our application, uploaded the data set. To give you a sense of what this IoT data looks like, it has for every few minutes over the course of about 40,000 rows, five months of data. For each of those lines and machines in different shifts, we know what product it was producing, what was the planned versus actual cycle time, machine parameters, vibration, etc. We know the overall efficiency, we know the energy consumed, all of the usual parameters that a system captures.

Now, we then said, "Let's pretty much give it the email." So, I'm going to read this out. "Our production line throughput is a problem, and what we want to do is figure out how to improve the top three drivers of, well, first find out what are the top three drivers of low throughput and yield loss, and then based on that recommend the highest impact fixes. What you have as data here is six months of cycle level IoT data from three different lines. I'd like you to create, come up with domain-specific creative hypotheses and make sure that you do robust statistical analysis and give me the actions in simple business language. What I'd like you to do is basically act as an analytics team for me. In fact, wow me with your analysis." And go.

Now, what we're finding is that, and this is something that you could try with say, ChatGPT or pretty much any of the agentic systems. Of course, a certain amount of support in terms of execution is required. But if we start giving it a few tools, one of them being for instance, the ability to not just write code, but execute code as well, particularly to be able to do data analysis and you have a certain amount of power - by power I mean reasonably large RAM, reasonably powerful GPUs, and so on - they can do some wonderful stuff.

So out here for instance, it's thinking, exploring how it might analyze the data. It's written the code to explore this data set, and it has some understanding of what are the features in the data set. So it knows that there are cycle time, temperature, etc. because it ran the code and saw what the columns are. It's then writing code to do some analysis on it, perhaps finding correlations between these. Yeah, it looks like it is finding that the correlation for one of these is not a number or some such thing. Goes on, goes on, goes on.

The quality of an agentic system or even an LLM increasingly is being measured by how long they can work autonomously. In the era of ChatGPT, they could work for probably a few seconds by themselves, and then you had to intervene, give it guidance. These days they can work for several minutes autonomously and come up with results that are usually correct, which is powerful.

So now what we have is, and it's been working for about half a minute, at the end of half a minute, it's coded its way into an analytical solution that says, "Look, minimize micro-stops." And it's the sensor debounce and the conveyor speed that you need to worry about. So to do that, deploy stop alerts for operators so that they can instantly correct. Dynamic calibration: make sure that roughly every 1000 cycles, the drift seems to go beyond 0.05 millimeters, that's when you need the alerts. And stabilize the seal bar temperature.

I am not a domain expert. I don't understand some of this. I'm fairly sure that many of the people that we show this to won't necessarily know that these are the things to do. We actually took this solution to a few experts and asked them two things: One, was this agent correct? Two, can you think of anything else? And the answer to the first question was, "Every single step is correct." Two, "I can't think of anything else."

This was an engagement planned for a quarter that ended up getting executed in 15 minutes. Quarter of a year to a quarter of an hour. And this is not just not this is not just one area. This happens in so many areas. Let me give you another example. Another client said, "What we want to do is have our team walk around the shop floor, take pictures of equipment that they see might have potential safety risk and flag those off. How long might it take?" The first estimate was three people, four months. And he reached out and said, "Is there a way we can shorten this?" And we said, "Let's give it a shot."

So I'm going to try this. "Build me a web application that takes pictures from the webcam, sends it to an LLM, identifies the objects in the picture and mentions whether there is any safety risk or violation." This is a very, very crude prompt. But let's run this on Bedrock Claude 3.5 Sonnet. The first thing that we need is an agent that can translate very crude requirements to something that a developer can develop. And let's call this a business analyst agent, so to speak. So the business analyst agent's done its job. It says, "Okay, that's how you might go about building the application." Then the coding agent takes over and starts writing the code out here, saying, yeah, whatever, "Here's how we will send the image. Here's how we'll analyze it." And once it's done, it passes it over to a deployment agent, which will place the application somewhere where we can run it. So that's been built. Let me maybe do something like this. Make it look dangerous. Capture it, analyze it. I could have probably tried bending over or something. But yeah, hopefully it's okay. It's saying, "There is a trip hazard. The yellow..." Oh, okay. It's... Okay, I did not expect that to be a safety risk. I'm blocking an exit path. Oh, okay, so it got that wrong. It thought that was an exit path. And the chairs are stacked against the wall, so they do not... Okay, verify that they do not... Okay, it's saying warning. It's more a set of recommendations. But here's the thing. That's a one person-year effort estimate, say, a year ago, which now comes down to one minute. It's probably what it took.

Increasingly, we're seeing that because LLMs are able to actually solve problems by writing code, and this code can be written live, the power of the kinds of solutions that we are able to build has dramatically increased. Now, it doesn't always work. There are several problems where it will fail on. But because this is something that a non-expert can do, the sheer number of such applications that people can build is exploding. So internally, we've been trying this. We'd pass this to our team who in the course of the last six months or so have built what, about 2000 odd applications all by themselves. And these are mostly non-developers, just building tools.

Having a queue of such tools, of such use cases that we can build where it's okay for it to fail, but if we wanted to do it properly, it would have taken too long. That's a fairly large space. And increasingly, I believe that managers will have a growing need to discover such use cases, to start pushing them towards these models because these have voracious appetite. We built, we solved two real problems in the course of a 10-minute talk. And 100 people could solve 100 of these in the course of a few days. So now we have a strange problem scarcity rather than a solution scarcity. This hasn't really spread, meaning not enough people are aware of it, but once it spreads, the nature of problems will shift dramatically.

But I have only one request of all of you. Just give this a shot. Have LLMs write code, run them, and put them in a loop, an agentic loop effectively, to solve problems. You'd be really surprised at what they can do. All the best and thank you.

# Transcript - Supply Chain (Fleet utilization)

There's only one message that I want to deliver, and that will become obvious as we go through. But the message is simply that with LLMs, you can replace a significant chunk of your data analyst or data science teams. It's increasingly becoming possible.

Now, one of the things that we've done for our clients is take a classic fleet utilization problem. They said, we have a fairly low utilization on our trucks. There's a cost leakage, that's a problem with working capital, that's a problem with service levels, that's a problem with emissions. Now, what we'd like to do, they said, is set up a team and bring in domain experts. We'd like to bring in technological experts. We'd like to bring in analytical experts, put them together, and maybe in a month, we can create something, give us a dashboard that helps us make decisions on what we need to do. And that's the norm, that's how we approach it today.

In fact, one of the managers reached out to their team members, a logistics analyst, and said almost exactly this, this was roughly the email. He said our fleet utilization and service KPIs, they're being reviewed by the leadership.

"Here's six months of dispatch level data across some of our fleet lines. Now, could you pinpoint what is driving the empty kilometers that we have? Why do we have a low load factor? And what are the top fixes that we can make on this?"

And he didn't explicitly say this, but paraphrasing him, he said, "look, I want you to come up with creative hypotheses that are really grounded in the domain, your understanding of the data. And I want to make sure that the results that you get are verifiable and robust. I want to make sure that they're statistically significant. But also, please don't use your analyst language. Give it to me in terms that I can take to the leadership. Give me concrete, actionable recommendations."

And it was a large dataset. I anonymized some of that, and it has about, let's see, yeah, about 15,000 rows of dispatch level data where we have this was the kind of payload, this was the trip plan, this was the actual trip distance, how long, how many stops it made, over what window, when the dispatch arrived. A whole bunch of things on efficiency. Data was fairly well consolidated and put together.

With some details around what each of these columns are. Now, option one, have the team work, and the team did in fact give an estimate of a month and deliver it.

Option two, why don't we put this into an agent? What if LLMs could be the analytics team? And this is where I'm getting to the one message that I want to deliver, which is that LLMs have the ability to code. They have a reasonable amount of domain expertise. They're about as good as someone with maybe three, four years experience today. And they have the ability to correct themselves if they go wrong.

So, I'm going to actually take this, literally this email, and put it into the Strive Intelligence tool, which is an agent. I've already uploaded the data set out here. It's large, but it should be able to handle it. I'm going to give it some of the context on what these columns are. And yeah, that was the entirety of the email. And let's have it run it.

Now, it functions very similar to how ChatGPT works. It first thinks a little bit. It says, let's expand this. The user wants to analyze the data. They've shared some parameters. Now I'll start by loading and analyzing the dispatch levels. It has access to a container, meaning it has access to code, both the ability to write code, as well as the ability to run code.

So, it's read the file that we've uploaded. It's understood that these are the kinds of columns. So it knows by itself that it's dealing with a data set that contains various columns related to fleet operations, payload, capacity, etc. And it goes through, looks at a few of the parameters, and starts finding the correlation between the features and the load factor, the empty kilometers. Effectively, we've asked it to figure out why do we have empty kilometers? So it's looking at the correlation. And it's seeing that some of these are highly correlated, some of these are not so highly correlated. Looking at the correlations, and it's going on and on and on. And it's made a mistake here. It's correcting itself. It's going on and on and on.

Effectively doing the work that an analyst would do, that a logistics expert would do, both in terms of hypothesizing, as well as in terms of analyzing, creating the statistical analysis. And let's ignore all the thinking that is being doing behind the scenes. But after about half a minute, it has come up with a set of conclusions. Firstly, that there's poor consolidation.

That there is, every time we improve the consolidation score, that adds about 5 kilometers of empty running. There's a lack of backhaul. If we have loads that don't have a return trip, that's 36 kilometers more of empty kilometers on average. That's useful to know. And there are traffic delays that are causing it. And these are your top three factors.

This is a solid starting point because in half a minute, what we have is a full-fledged analysis of what are the biggest drivers of empty kilometers, as well as of low load factors.

And it goes on to explain in business terminology, look, what you need to do is consolidate. Raise the cutoff times and start pooling the orders. Expand the backhaul operations. Make sure that you outsource the return loads. Partner with third-party logistics. Right-size your fleets. You want smaller deploy vans. You want dynamic routing, and you want weekend scheduling to be consolidated. Effectively, five clean, actionable recommendations based on the data at one shot.

Now, if we have this kind of capability, the ability to have a first point of support with an analytics team is replaced by the first point of with the LLM or an agent like this becoming the first point of contact.

What we're seeing though is that it's not just analytics teams that LLMs are replacing, it's full-fledged development teams that are getting replaced as well.

One of our clients said, look, we would like a team to come in and build an application where people on our frontline can walk through the shop floor, take a picture, and find out if there is a safety violation that's happening on the fly. What's your estimate of how long it would take to do this? Because our team, they said, had come up with an estimate of about three months, four people to get this done. Is that how long it will take? Is it possible to get it done with higher quality?

So we said, let's try a slightly different approach because we do find that LLMs do a great job of writing code. So could we, in fact, let me put this in, build a web application that lets the user take a picture from their webcam, send it to an LLM, identify all the objects in the picture, and find out if there's a safety violation. I'm sending this to Bedrock's Claude 3.5 Sonnet model.

Now, what that does is, firstly, bear with me, that was not a. Oh, okay. Vertex. Any one of these models should work fine, give it a shot. Yeah.

Now, what it does is, first, acts as a business analyst. The first agent is someone who understands the requirements, translates that into technical language. And the second agent is a coder that goes through, sits and writes what the application code should look like.

Once the coder is done their job, the third agent, which will get triggered in a short while, is a deployer agent. And that deployer agent creates the application, pushes it onto a deployment server, and we have this. Let me maybe put that out here, take a picture. And with that picture, it's probably already sent it to an LLM. And yeah, it doesn't seem to be any visible protective equipment, such as a hard hat or safety glasses, which are generally blah blah blah. So it appears and it appears to be an indoor office, so maybe this is not a violation. And specifically, if it's controlled, then I can analyze for more safety violations.

What we have here is effectively 12 person-months of effort estimate, condensed into approximately what took a minute.

This doesn't always work. What we're finding though is sometimes when it works, it's magical. And a big part of the change in behavior we're seeing in organizations is to start discovering these low-hanging fruits. Creating a queue of tasks that can be done by LLMs, and when they can be done by LLMs, it offers a huge boost.

Lowers the cost to the point where people are now starting to think, what if I could have this particular application built? Where earlier it was prohibitive, I would have had to interact with a development team, I would have had to hire a vendor, I'd have to do all kinds of stuff. Now I can just give it a shot. Possibly even by myself, because all it takes is this kind of a description. So it doesn't even need a full-fledged development team or even for me to interact with anyone, and get it deployed.

We've done this internally at Strive. We've had non-developers start building applications like this, and as of this moment, there are about 2,100 applications that mostly non-developers have built as utilities all by themselves.

I mentioned that there's only one message that I wanted you to take away. It is primarily that increasingly analytics teams and development teams can be replaced by agents. We are now at that stage where that's possible. We are seeing it happen. A big part of the transformation effort is figuring out what kind of work can we give to these, because now you can give a thousand tasks, and they can get done instantly. And how do you start integrating it into the organization?

Do reach out if you'd like inputs on how we're seeing this used in practice. But do try it out. This is real fun. Thank you.