Artificial intelligence is quickly making its way onto the manufacturing floor. But when it comes to safety, the stakes are different. In this episode of Manufacturing Matters, Winn Hardin and Aaron Hand talk with Erik Reynolds, founder of Reynolds & Moore, about what it really takes to bring AI into safety-critical systems.

For decades, safety engineering has relied on deterministic, fully explainable systems. AI challenges that foundation, introducing models that learn, adapt, and sometimes behave in ways we don’t fully understand. So why use AI at all? Reynolds explains where it actually adds value — from enabling smarter human-robot collaboration to the possibility of systems that improve safety over time, not degrade. But with that potential comes a new challenge: proving that these systems can be trusted.

Winn Hardin: [00:00:02] Hello everybody, and welcome to another episode of Manufacturing Matters, where we talk about the technology and the trends affecting the global manufacturing industry. My name is Winn Hardin, host here with you today. And I’m lucky enough to be with my co-host Aaron Hand. Hey, say hey, Aaron.

Aaron Hand: [00:00:17] Hey.

Winn Hardin: [00:00:18] As you both know, we do day work over at Tech B2B Marketing, was our sponsor here at Manufacturing Matters, and we’re lucky enough to be here today with Erik Reynolds, founder of Reynolds and Moore. Erik, thanks for taking some time to join us today.

Erik Reynolds: [00:00:30] Yeah, glad to be here.

Winn Hardin: [00:00:31] Cool. So if we could start by telling us a little bit about Reynolds & Moore and where you fit in manufacturing and automation.

Erik Reynolds: [00:00:38] Sure. So Reynolds & Moore is functional safety engineering firm. And they’re kind of specialized. They focus on the safety engineering aspects of automation. So that includes, of course, supply chain automation, industrial robotics. Reynolds & Moore also does work in the energy sector as well, with, for instance, energy storage systems and even the process sector. But probably the bulk of the work they do is in robotics safety. That includes robotics implementation and also ground-up robotics design as well.

Winn Hardin: [00:01:14] Like full integration services or just from the safety element?

Erik Reynolds: [00:01:17] Oh, just the safety elements. Yeah, I should clarify that. Yeah. So R&M’s not a systems integrator, but they help especially with things that don’t quite fit into a typical safety engineering box yet because they’re an emerging technology, and they help to find ways to deploy those systems safely and at scale.

Winn Hardin: [00:01:38] And you’re not new to this game either if I’m not mistaken.

Erik Reynolds: [00:01:41] No, not really. I founded the company 10 years ago. And we were joking a little before the podcast recording started. For the first nine years, I was the founder and CEO. And thankfully for the last year, I’ve been the founder. We have a professional CEO now. His name is Geoffrey Athey, who I think you both met at maybe the A3 conference a while back.

Winn Hardin: [00:02:04] Yes, sir.

Erik Reynolds: [00:02:04] Yeah, yeah. So he’s done a great job of really developing and deploying the systems to allow that company to really scale well.

Winn Hardin: [00:02:13] Awesome.

Aaron Hand: [00:02:15] So today we really wanted to get into the artificial intelligence side of safety systems, which historically AI hasn’t really been allowed into safety-critical systems. So has that changed? Are people starting to take AI safety seriously?

Erik Reynolds: [00:02:33] Absolutely. It has changed and it is changing and it has not yet changed — all at the same time. And it depends on what sector you’re talking about, whom you’re talking to. So I will say it is imminent, especially in what’s being called physical AI or embedded AI. It depends on what industry you’re in. And you’re correct. The previous safety standards explicitly prohibit the use of artificial intelligence or machine learning in safety functions. Now somebody who’s a safety professional might quibble with me a bit. They say “not recommended.” And when it’s in a standard and it says “not recommended,” then effectively it does not permit it. And there are a lot of different reasons for that that we could get into. But the good news is there are new standards being written. The ones that are probably most relevant to manufacturing is one called ISO/IEC 22440. And the title — it says on the can what it does: artificial intelligence for functional safety. So it’s meant to be the foundational standard that says: If you’re going to deploy AI into a safety application, then here is the framework that you need to use to make sure it’s deployed safely and responsibly. That’s under development. The drafts are out and circulating, and people are voting and doing all that sort of stuff, but we’re expecting that it’s going to be officially published maybe around January of 27. So we’re looking at maybe nine months out.

Winn Hardin: [00:04:09] Okay. Very cool. Now, just so for those who may not understand, the reason why AI was not a preferred solution for safety is because most safety systems need to be deterministic, repeatable, verifiable, right? And when you put an AI model into there, in many cases it’s a probability matrix on different outcomes and with all kinds of different inputs. Repeating the exact same performance scenario from a given model and then being able to guarantee that it will always perform that way, that’s where the gray area is. And I think that’s what led to the IEC 22440 creation draft. Is that a fair statement?

Erik Reynolds: [00:04:49] Yeah. You’ve explained it well. So I think the main two things I’ll talk about are transparency and explainability. And the entirety of previous functional safety systems, ones that are not AI enabled, is based on those two principles — that no matter how complicated it is, you may have thousands of lines of an FMEA to explain or a very complicated architecture diagram or something. No matter how complicated it is, a team of people with enough time can trace out all the particular failure modes that might happen and can design diagnostics around that so that they can see when it goes off track and they can understand when they’ve left into the area of unsafe behavior. The difficulty with AI systems is they are, by and large, not explainable or transparent in that way. Their results, as you said, are not deterministic. You don’t always get the same result when you give it the same input. And this is the old Jeff Goldblum chaos theory thing from “Jurassic Park.” You know, you drop the drop of water on the same spot on your hand, but there’s all these very small, minor things that might affect the actual result in the end. That’s what makes trustworthiness of AI systems difficult under that framework. So really 22440 has tried to take an approach to say: How can we improve explainability of AI systems? How could we improve transparency of AI systems? Or how can we supervise them or govern them in a way that we can have something else determine when it’s moving outside of its boundary of safe behavior? Or is there a way to actually bound that probabilistic behavior and constrain that? So it sets out a framework for that. I will say it doesn’t set out all the solutions for it. It just says: Here’s a framework that we’re going to all agree upon. And then further work has to be done for certain areas and certain applications.

Winn Hardin: [00:06:55] If I could real quick — I know Aaron wants to dig in deeper into that validation step, which is I think the heart and soul of, a big part of, what we’re discussing here. But tell us, Erik, why would people want to use AI for safety systems? What are the benefits? Let’s give them the payoff before we set up the punchline.

Erik Reynolds: [00:07:11] Yeah, that’s a really good thing. I can tell you work in manufacturing because it’s like, okay, there’s a new fancy piece of equipment. Why in the world should I spend money on it? What makes sense? So like 10 years ago, everybody wanted to do blockchain on their inventory systems. And then you realized, wait a second, database works just fine. We don’t need all that extra stuff. The same is going to be true in AI. There are some things that are going to be much more straightforward and economical to deploy with what I’ll call classical safety systems. So those won’t be going away, the deterministic ones. Examples of that might be object detection, interlock systems, things like that that are already in place. Where AI is helpful: number one is it allows you to do some specific tailoring for a collaborative environment. So right now, if you have a safety system deployed that can do object detection, it doesn’t know if that object is a human, a dog, a parrot, another robot, a forklift. It just knows there’s an object there, right? So it has to always take the most conservative stance and assume that’s a human there and then behave that way. You might want to be able to say: Oh, I know that’s a robot or at least I know it’s not a human.

Erik Reynolds: [00:08:27] So I do have to change my behavior, but I don’t have to change it in such a way that I’m afraid of hurting or harming a human. So it gives you some more operational space, trade space to deal with there. So that’s reason number one. The second one and the one that made me get more excited and actually dedicated to helping with this is: In theory an AI-enabled safety system can get safer over time. Because of the data collection required to actually train an AI system to work, if you continually train and then update, it can learn about its environment in a way that engineers sitting behind imagining it before design happens or before deployment happens can’t. So you get a closer feedback loop. So I think that’s very exciting that over time safety systems could get safer. Right now, a system’s never safer than the first day you deploy it. It only degrades after that. In the future it could get better. The flip side of it is, and what we have to guard against is, it could also get worse, right? And so that’s where the validation piece comes in.

Winn Hardin: [00:09:36] Right on.

Aaron Hand: [00:09:37] So explain that validation piece, because part of what I was interested in understanding is: Just what does this look like on an actual manufacturing floor? And you’ve done a great job of explaining some of that, but explain it from a validation point of view. What does safe AI actually mean?

Erik Reynolds: [00:09:57] Yeah, if I answer this, I think I will win the equivalent of a Nobel Prize. What does safe AI mean? Again, different things in different contexts. So I like to talk about trustworthy AI, because as much as we like to believe it, nothing is actually ever safe. There’s always a chance, and all of our current safety framework is based on minimizing that chance. We want to make it to where you’re as likely to win the lottery as you are to get hurt on this manufacturing floor, right? It’s not impossible. But, man, it’s about as close as you can get, right? So safe AI: I like to talk about trustworthy AI, which says its behavior in the intended environment is predictable and its mal-behavior or misbehavior is also predictable. And that’s predictable by demonstrated evidence. Again, we can’t crack open the convolutional neural network or whatever it is and look inside and see exactly how all the inner workings go. Again, some research scientists may tell me, “I have a way to do that,” and my argument to him would be, “Kind of. You might have a way to do that. You don’t have it the way we need.” And so what it means is: Can we develop methods and deploy them in such a way that we can quantify the actual behavior of the AI model in its environment, in the intended environment – so the exact workspace that it’s intended to work in – and then can we measure what a failure rate might be? Just like we would measure a failure rate for a relay or something like that. Can we say: In an industrial workplace, in an industrial environment, a work cell moving boxes around, with people in its vicinity, this type of environmental condition, its likelihood of misidentifying that human is such and such times 10 to the such and such per hour or per year or what have you. And that kind of fits it in nicely with how we do safety quantification today. It’s all based on a probability of failure on demand or something like that. Of course, the magic is: How do you get from here to there? How do you get from an AI model that’s been trained and deployed to actual metrics about what its failure rates are?

Winn Hardin: [00:12:23] And the answer to that, my brother, would be?

Erik Reynolds: [00:12:30] That’s been something that I’ve been personally working on with the research team at Reynolds & Moore for about the last four years, and we’ve been developing and deploying. And basically the approach that we’re using is, number one you have the prevention of systematic errors that would come from ISO/IEC 22440. The way you prep your data, the way you define your operational environment, the way you deploy it, the way training happens — all those things are meant to prevent the introduction of systematic errors into the models themselves. And then the second one is on the flip side: Okay, now we need to go and validate. And I’ve got a little bit of a bone to pick with the safety community with this with traditional safety systems now, because if you’re going to do an acceptance test on a safety system, usually you have a punch list of items that you have to go through. Jump for this. If it faults, good. If it doesn’t fault, bad. Fail, right? So you’re kind of demonstrating it can do it once. And that is okay, I suppose, in a deterministic world, but in a probabilistic world it’s not. You may have just gotten lucky that time that you did it, right? So the real key is — of course the holy grail would be tested in the real environment, in all possible conditions. Sweep across everything and determine how many times it fails in that. And a friend of mine and I did a calculation once. If every data point that you were going to collect data on to satisfy that type of reliability were one particle in the universe, you would need three universes worth of particles to get enough data to say that. So does that make sense? Like the amount of data collection you would need is huge, right?

Winn Hardin: [00:14:17] Is that getting towards that 100% safe scenario? Is that what you’re defining in that scenario?

Erik Reynolds: [00:14:23] Right. If you want to get to one in a million failures per hour per year, whatever it is, to get enough statistical significance, you’d have to push very far in the amount of data that you collect.

Winn Hardin: [00:14:35] Both on the deterministic traditional safety model — I mean, that rule would also apply to the AI model. Is that correct? Am I correct?

Erik Reynolds: [00:14:41] Yeah, that’s on the AI model. It applies to the deterministic one as well. But the transparency allows for you to understand how it can fail a priori, before you do anything. So now you can limit down those more. So that’s a little bit of my bone to pick.

Winn Hardin: [00:14:59] As long as they’re only looking at one factor at any given moment.

Erik Reynolds: [00:15:01] Right. Exactly, exactly.

Winn Hardin: [00:15:03] I mean, if we’re looking at the impacts of — this may be a standard test ignorance on my part — but if we’re mainly just looking at, let’s say we have a slight power sag or something else that may impact multiple electronic systems, if you’re doing a deterministic system, you’re simply looking at one in isolation at a time, whereas the power stack spread across multiple units could actually have a very different response.

Erik Reynolds: [00:15:28] Yes, exactly. You’re 100% correct. And the difficulty is, with an AI system, we don’t know what dots the AI model is connecting to make its decision. It may be something totally crazy. I mean, there’s some examples where AI models were trained to classify dogs and cats as “This is a dog,” “This is a cat.” Pretty simple. Maybe you guys have seen this before. And the researchers found that it was making some crazy errors. Sometimes it would make a mistake and it wasn’t really explained. And they found out that different compression algorithms were used on some of the image sets. When it compressed the images that it was used to train them on, for some reason the model thought, oh, that compression algorithm, the way it’s compressed, ends up into the decision-making framework of whether it’s a dog or a cat. Of course, that doesn’t follow. And you only find that out after the fact, right? These are the kind of things that make it difficult to really validate an AI-based system because the possible things it’s taken in from the environment is anything and everything that it could possibly see, right? And so that’s what gives it some difficulty.

Winn Hardin: [00:16:38] Yeah, including metadata in that case. You know, you would need to scale that out. So that would seem to be answered best by the best cases, the best practices that you’re recommending during the first part, the part one of model validation. That’s not model validation. That’s just defining the setup. The training of the model and everything. In the second case — and you know most AI models always suffer from a lack of data, sufficient dataset, especially in negative scenarios where defects are present. We’ve got a lot more pictures of things going perfectly than we have things going south. Is that a big part of the IEC 22440 or your approach, where you say: All right, well, the way people overcome this in AI usually is to use different models to synthetically produce additional datasets of what could go wrong and then apply that to the model and look for the response. Is that similar to how you validate an AI model and safety?

Erik Reynolds: [00:17:41] Yeah, I think you’re on the right track there, but I’m going to go back one more step, to Francis Bacon, right?

Winn Hardin: [00:17:49] I’ve heard of him. It comes in hickory flavor. It’s great.

Erik Reynolds: [00:17:55] That’s right, that’s right. No. So the original scientific method is I have a hypothesis and then I test it to prove it wrong. I don’t test it to prove it right. So in this case, we’ve flipped the hypothesis around. Instead of saying, I’ve got a safety validation plan where I’m going to test to make sure that my system is safe, we flip it upside down and we say we have a safety validation plan to test and show that my system is unsafe. And what that does is it prescribes a border in that multidimensional trade space around where you reach a border where there’s a cliff, where all of a sudden the model loses its mind, right? And you define where that cliff is, and then you back off from that and say, if you operate within this space, the model is not shown to be unsafe here. And so I know that’s a little bit specific there, but the idea is we’re not trying to say models are safe. We’re trying to find where they’re unsafe and then back off to local regions of safety.

Aaron Hand: [00:19:04] But it also seems like AI has the potential to be even safer. Just going back to its ability to learn and its ability to increase in safety. It seems like AI as a general principle thinks of ways to do things that are different. I always think back about the early days of it learning to play Go, for example, and playing against these champion Go players who are like, “That’s a bad move. That move doesn’t make sense.” And yet AI would win. So it seems like it has the opportunity to see more things than we’re seeing necessarily as human beings and saying, well, if both of these systems go out, then you’re going to have a problem. Is that fair to say?

Erik Reynolds: [00:19:54] Yeah, I think so. And I’m really glad you brought up Go. I don’t play Go, but it’s the perfect example for this to describe the difference between a classical safety system and an AI-enabled one. A classical safety system is playing chess, right? So it can be very complicated and an infinite number of games that you can theoretically play because of the path-dependent nature of things. But every piece has a move that it can make, whereas in Go — I don’t play it, but as I understand it — there are only a very simple set of rules. But there are unbelievable amounts of options of where you can make moves, right? And so the possible space that you could operate on is much larger. So why is that important here? You’re exactly right. Safe or unsafe behavior in AI-enabled systems emerges. It isn’t prescribed. So in that case it can look down at the Go board and come up with what a human would say is a silly move. But because it’s looking at the whole board at once instead of looking at each individual piece, it’s able to produce a more optimal move. The flip side is what they probably didn’t show you in all those Go videos is there’s a bunch of times where it did silly things and it lost, right? They’re only going to show you the happy path, not the sad path. So that’s the trade-off between the two.

Winn Hardin: [00:21:28] But it learned that for every one of those failure modes, which is how humanity did it, only it’s doing it trillions of times in a minute to build up its level of knowledge.

Erik Reynolds: [00:21:41] Yeah, exactly. And I don’t claim to be an AI expert. There’s a lot of AI experts out there. What I try to be is knowledgeable about AI trustworthiness, right? So it’s hard to say AI as a category, because there’s so many different variants of it and they work in different ways and that sort of stuff. I’ll leave that to the experts. But if somebody’s developed something and I think this is a very performant model and we want to use it in a safety system, then that’s where I would come in and say, okay, we’re going to take a look at designing a way to test it. And I mentioned earlier the problem of the number of test cases that you would have to explore to fully prescribe. So number one, we’re flipping it on its head and saying we’re going to hypothesize that it’s unsafe and try to prove ourselves wrong. The second thing is, we’re going to use simulation to explore that space before we use physical, real-world testing to explore that space. And then it can be iterative. You can learn things in the simulated environment, go to the real world, and see if it’s an actual real-world behavior. And then based on what you learned there, go back to the simulated environment. And that’s an iterative test, analyze, test approach, which traditional safety is more of an analyze and then test. And so this one is iterative in nature, which I think is another significant difference between classical systems and AI-enabled systems for validation.

Aaron Hand: [00:23:14] Okay.

Winn Hardin: [00:23:15] So is vision or machine vision or computer imaging usually always considered part of your AI safety systems? And can you talk about the multimodal nature? You know, you’re looking at a whole bunch of different sensor types and data that you’re bringing in, right?

Erik Reynolds: [00:23:28] Yeah. So I think I heard this somewhere and it stuck with me. So the way I put AI safety systems is into three layers. One is the perception layer, which is kind of what we’ve been talking about so far. Can it classify? Is that a human, a dog, a cat, a robot, a grapefruit, whatever it is, right? That perception is about understanding the world. And it’s different from sensing because classical computer vision or other things like that, they do sensing. But this is about perception because the model has to have context to know what that thing is and what it is in relation to other things in the environment. So perception is different than sensing, right? So there’s the perception layer. Then there’s another layer that is the behavior layer. And now that it has that perception data, how does it decide what it needs to do? In the classical safety system, this would be the logic layer, right? How do I decide? Do I go to the safety stop or do I go to this or go to that? But in an AI system, especially like what I’ll call a free-range or a personal-use robot or something, it has to make a lot of combinations of decisions that are really more for philosophers and ethicists than for roboticists, right? For instance, I’m in the kitchen, the robot is moving a pot of boiling water from the stove to the counter to go pour out some pasta or whatever it is.

Erik Reynolds: [00:25:04] And the dog runs underneath its feet and knocks it over. And in a split second, it’s got to decide: Do I spill the hot water on the dog or on the toddler who’s sitting next to the dog? And it only has those decisions and it has to make it right then. And of course, when that happens to a human, we say, “Oh my gosh. That’s a terrible situation.” And then we have a method to adjudicate whether someone was negligent or criminal in that behavior. And the whole court system was built around it. We don’t have that for non-human things, right? You know, you can’t try it. So there’s a whole behavior layer and that’s kind of the ethical portion of that. Then there’s just what I’ll call the more locomotion and collaboration, just making decisions about what do I do next, right? We’ve all seen the robots folding laundry, the robots washing dishes, the robots doing all that sort of stuff. Those are discrete behaviors. But in a complex environment, it has to choose: Should I do this next? Should I do that next? And as it’s making those decisions, sometimes it has to say, “Wait, there’s a safety problem. I have to do this.” And so there’s that layer, which is I think the least well described in robotics, maybe more explored in automotive, where they have to make these decisions about pedestrians and other cars and that sort of stuff. And then the final one that I like to consider — so we have perception, we have behavior, and then we have control, which is what we would consider motor control or end effector control or the actual moving of the mechanical units, right? And so you may have an AI model or AI models working in each of these individual layers to perform safety functions. Or you may have them spanning those different layers. So you may have a model that is just a perception model. And all it’s doing is interpreting the world, feeding it to a behavior model that’s making decisions, feeding it to another control model that’s now making the actuation happen. But you may have what sometimes is called an end-to-end or a pixel-to-actuator model, a hierarchy of models that work across all of those. And it’s not discrete or even really knowable which exact one is making which decision, which makes it very difficult. What we’re focusing on, most of the work we’ve done is in perception at the moment. We do have methods to quantify the reliability of perception models for a safety application. Those are underway on active projects and are being validated and verified. And then we also have under development the perception and behavior pieces as well, at varying stages. And really what we’re trying to do is keep up with what clients need right now. So we have clients who come to us and say, I really want to deploy this product on such and such a time. Are you doing anything in this space? And can you accelerate it for us? And we try to accommodate that where we can.

Winn Hardin: [00:28:11] So it’s perception, behavioral, and control. Those are the three primary layers to consider, right? Okay. And you’re mainly spending your efforts on perception right now, and the behavioral and control are coming afterwards. I mean, you’ve got all the challenges of your standard deep learning AI models with agentic AI, because you’re giving it the ability to control direct outcomes without HIL — human in the loop — or anything else in safety systems. And then you’ve got hierarchy of events, which is often part of complex agentic AI challenges. Am I right in assuming that when you’re validating these different parts, that, back to your earlier comment about parameters, like really setting the contextual relationship for every particular activity and/or sub-activities feeding into a primary activity. Not in an infinite universe, but we’re testing it in this environment, right? Is that a key component of how one goes about AI safety?

Erik Reynolds: [00:29:17] Yeah. That’s absolutely right for our approach. We don’t want to boil the ocean. You’ll never get there. So instead we’re saying: What are you trying to do right now? And then if tomorrow or next week you try to do something else, there would need to be a certain testing profile for that operational environment as well. And that’s kind of congruent with classical safety systems. You know, every certificate says: Here’s the environment that it is certified for. If you take it to another environment, this isn’t valid anymore, right? So the trick is how broad to make that environment. Because you could narrow it down to a very small thing, but then it’s practically useless. You’re just doing it for a show to say you have an AI thing.

Winn Hardin: [00:30:06] Especially if you’re talking about totally unconstrained applications like humanoid robotics or service robotics, which seems to be a big focus. I guess, going out on a limb, are probably a lot of the models you’re trying to validate for safety systems.

Erik Reynolds: [00:30:20] Yeah, exactly. Humanoids are a special case. I know there’s a lot of controversy over whether that form factor is even viable or even appropriate. Aren’t there better ways to do this, especially in a controlled environment, like where industrial automation functions a lot? I wrestled with this a lot, trying to figure out, and I’ve decided to withhold judgment on form factors, to be solution neutral around that. I would like to say I would like to live in a world where humans can still do things. And if the robots look like us, then we can still do things. If they don’t, then it would be hard for us to still do the thing. There would be robot-only tasks and human-only tasks. And I don’t really like the idea of that world necessarily, but they don’t ask me. But a key thing for humanoids and other legged-type robots, which in the industrial world, I’m told by one of our engineering managers who is on this committee that they’re now called dynamically stable industrial mobile robots. They’re not called humanoids or quadrupeds. That’s what they’re called. And the big thing there is we don’t have safe-state definitions for those right now. What are they supposed to do? Like you can’t just call a safety stop or a safe torque off. They’ve got to develop all that stuff. And so what I’m hoping is that the work Reynolds and Moore is doing on validation of AI trustworthiness can keep pace with what the industry is developing. And that’s why we’ve got people on that standard and 22440 and a few others to try and have a voice in that and then also try to keep pace with it.

Aaron Hand: [00:32:05] So as you talked before, the standard is you think like nine months down the road? is that right?

Erik Reynolds: [00:32:12] Yeah. Don’t hold me to that. I’m not the convener. But I think the last scheduled publication date I saw for that is January 2027.

Aaron Hand: [00:32:21] Okay. Well, at least let’s think in theory, if a standard comes out in January 2027, what does that mean for the safety industry? Does that mean things start rolling out then? Or you can work better? I mean, when should we expect to see AI safety actually implemented?

Erik Reynolds: [00:32:44] Yeah. I think you’re going to see it in an uncertified way before then. It’s all already out there. People are doing pilot projects. People are doing things where you have a chaperone watching them, that sort of stuff. Right now you’ll see that all over. But as far as a certified one, we have clients right now who are saying as soon as that standard gets published, we want a certificate for it right now. That’s how fast the pace is on these things. So we’re trying to trying to keep the pace with that. But the flip side of it too is, once a standard is out there, it’s incumbent on you to determine: Should I be applying this standard to what I’m doing or not? Before there’s a standard, you can kind of say, well, there’s no standard, so . . .

Winn Hardin: [00:33:40] Wild West.

Erik Reynolds: [00:33:42] But once there’s a standard, you really got to take a look at it and say: Does this apply to what we’re doing? And of course that’s one of the services that Reynolds & Moore provides. They have people who can look at what someone’s doing and can say, yep, this might apply to you or it doesn’t or here’s how it may. That’s kind of their specialty.

Winn Hardin: [00:34:02] Yeah. And I know insurance companies tend to drive a lot of that discussion. I know that I’ve worked on a number of standards and . . .

Erik Reynolds: [00:34:09] We’re going to have to talk more, Winn. You’ve got quite an insight. A lot of people don’t see that. We talk to a lot of safety or compliance engineers, and they just think about getting a certificate, and they’re not really seeing why their company is pushing so hard for that, in a jurisdiction like the U.S., where it’s not like Europe, where it’s: Thou shalt do this or you can’t do business here. And the real driver is the insurance company won’t insure the building or the workers or any of the operations unless you’ve got third-party something, right? Most of the time a certificate is the strongest form. So, yeah, that’s the behind-the-scenes dynamic that is working. And of course we’re exploring that with some of our industry relations as well, because the big problem for insurance, as I understand it now, is they don’t know how to write an actuarial table for this, for an AI model, right? You don’t have data.

Winn Hardin: [00:35:17] Of course. We can’t tell you how it works. So you certainly can’t predict how it will work.

Erik Reynolds: [00:35:21] Yeah. And we don’t have 30 years of data of it running and all that sort of stuff. So a lot of insurance companies are writing actually exclusionary clauses that say if you use AI, this policy does not cover any accident that happens when AI is in use, right? They’re trying to kind of limit their downside exposure to that. So I would hope that as the new standard comes out, other standards that follow, as the approach that we’re trying to bring to market comes out, then insurance companies are able to say, okay, here’s something that we can understand and that we can apply and we can use. And it helps move us over this hump as an industry.

Winn Hardin: [00:36:02] Hundred percent. My experience is usually the European does that through regulation and through enforcement and fines. And here it’s the courts and the impacts of that without following the best practices of the industry, which usually get validated by the insurance companies once they can be sure that’s going to reduce their costs and make sure that they constantly make money. Buy insurance stocks, people, they never go out.

Erik Reynolds: [00:36:26] That’s right. That’s right.

Winn Hardin: [00:36:27] Unless it’s housing insurance.

Erik Reynolds: [00:36:30] Well, Winn, I’ve got to check my Ancestry.com and see if we’re related because you’re bringing up a lot of the arguments that I . . .

Erik Reynolds: [00:36:40] It’s really true. Part of the reason I moved from CEO to just founder at Reynolds & Moore is I have other companies that I’m founding and working with too. And one of them is a company called RMAI, which is a testing, inspection, and certification company that is focused exclusively on physical AI. The goal there is to close the gap that usually happens: Standard comes out; 10 years later somebody can get certified to it. We’re trying to close that gap down as much as possible. And the good news is several of the other testing, inspection, and certification companies are on board with it because it kind of accelerates their approach too, as part of the NVIDIA Halos Inspection Lab for their physical AI too. So we’re really trying to position that, as a service provider to fill that niche of: How are we going to get the first physical AI systems safety certified and trustworthy? And then after that we’ll see what happens.

Aaron Hand: [00:38:01] So not to waylay this conversation, but we were just talking this morning about what physical AI is and people’s varying definitions. So how would you define physical AI?

Erik Reynolds: [00:38:13] It’s AI that can touch me. From a safety perspective, that’s what it means to me. So right now, if I don’t want any safety impact from an LLM on my life, I just don’t engage with it for the most part, and then it has to convince me to do something because it can’t do it itself. But physical AI is something that is AI enabled that can touch me. That’s my definition of physical AI.

Winn Hardin: [00:38:42] I just had a nightmare scenario where LLMs get unlimited VoIP dialing and can then take over the entire robot spam text thing. Oh my God. That just freaked me out for a moment.

Erik Reynolds: [00:38:54] I think it’s already happened. I just saw a news article of a bartender, I think it was in Ireland, who was using AI-enabled robo dialing to call all of the bars in Ireland and ask them how much their Guinness was. Their Guinness beer. And then he would set his price lower or whatever it is. And it’s actually affecting the whole beer market in Ireland apparently. That’s at least what the headline said. But who knows. Maybe that article was AI written as well. So this is the difficulty.

Winn Hardin: [00:39:26] Speaking of applications. So, Erik, 27 possibly we have a green light. Things go crazy, although there’s already a lot of work going on behind the scenes. What applications or industries do you think are going to be the first safe AI champions? Robotics? Material handling? Vision? Where are you thinking? Multi-modal?

Erik Reynolds: [00:39:45] Oh, I didn’t answer your multi-modal question. I can come back to that if you want. But I think the first industry that we’re already seeing it in is automotive. And that’s because their entire regulatory structure is fundamentally different than most of the others.

Winn Hardin: [00:40:02] And this is autonomous operation not the manufacturing site?

Erik Reynolds: [00:40:07] Autonomous operation. I mean the self-driving cars, the Waymos, the Zoox, those kind of guys. Those are already on the road in many cities. Unfortunately, they’re already hitting people. They’re doing things. You know, I don’t mean to laugh at that, but that’s just going to happen. I think you’ll see it there first. And that’s mostly because automotive is what I’ll call by and large self-certification. The industry in the U.S. was kind of set up that way, so that it’s on the manufacturer to say: We’ve satisfied this and here’s how we manage it. And then the courts settle it when it doesn’t work, as you brought up earlier, Winn. Now the robotics world, whether that’s industrial robotics or personal robotics or whatever you want to say, those work kind of under the machinery approach to things, which is more about third-party certification. So I think that’s going to naturally lag, especially in Europe. And Reynolds Moore has an office in Europe, so we’re very close to that as well, about what is the European market going to do about physical AI. The difficulty in the U.S. is there’s a lot of folks that are going into quote unquote founder mode.

Erik Reynolds: [00:41:18] And I am a founder, so I guess I can say this. If you’ve heard about founder mode, it’s like, “I’m just going to do it and then we’re going to see what happens.” I don’t mean to cast disparity on anyone, but there’s a lot of folks saying, “We can’t slow down to figure out how that’s going to work. It’s got to catch up to us.” And so what we’re trying to do is catch up to what is already going to happen. So I would see it first coming out in automotive and then after that moving into the controlled environments for commerce. Industrial robotics, manufacturing, supply chain, that sort of stuff. I think the last one we’ll see actual safety certification is home use, hospital use, personal care. What my friends in Denmark call warm hands, which are the people who come and check on patients in a home health environment or whatever. There’s a labor shortage in that already that they’re hoping to fill with robotics. So that’s the rollout that I would see.

Aaron Hand: [00:42:20] I do want to go back to the fact that autonomous cars are hitting people. But statistically they’re not hitting as many people as the human drivers are hitting, right? So that just makes me think about if something goes wrong with AI, let’s say on a manufacturing floor, is it going to have the same sort of backlash as the autonomous cars hitting people, where they say, “See, it’s not safe”?

Erik Reynolds: [00:42:50] Yeah. Well, this is another soapbox I get on. So it’s on you for bringing it up. So in the automotive industry in general, about 40,000 people a year are killed in the U.S. on roads, every year, and that doesn’t include injuries and property damage and all that stuff. And we don’t hear anybody clamoring: “We need to shut down all the roads.”

Winn Hardin: [00:43:15] Right.

Erik Reynolds: [00:43:16] As a society, we have accepted that as the cost of doing business. For whatever reasons — I’m not making a moral judgment about whether that’s right or whether that’s wrong. It’s just accepted. So that argument of an autonomous car is killing less people than humans. From a strict utilitarian philosophy perspective, that’s better. Killing fewer people is better. I have six kids. All of them are teenagers or above. I’ve taught all of them to drive. I’m in favor of self-driving cars. Let me say this loudly. Not only for their driving capability but for the other people around them. You know, the biggest thing when I’m teaching them to drive, I tell them, “Look at the other person’s eyes.” When you’re at a stoplight or whatever it is. You can’t anymore. They’re all looking down at their phones, right? Or doing something else.

Winn Hardin: [00:44:12] But you can tell they’re looking down. As a motorcycle driver, for example, I can.

Erik Reynolds: [00:44:18] Oh yeah. That’s totally true. So I’m in favor of that. But number one, when a person hits someone else with a car, we have injury lawyers and courts and case law and all sorts of stuff and insurance companies who know about coverage and all that. And we have a way to work all that out. Do we have it for autonomous vehicles? If my autonomous vehicle hits —like there was a Waymo that hit a kid at a school in I think Washington or something. The kid’s okay. It hit him at like 9 miles an hour or whatever. I think it was a Waymo. I’m not trying to libel or slander anybody. I’m not sure of the definition of that. If that happens, who is at fault? Am I at fault because I own the car but I wasn’t driving it? Is the manufacturer at fault because the system didn’t work right? How does that get adjudicated? Eventually, 10 years from now, that’ll get sorted out with case law. But in the meantime, we don’t know. Now I think on the machinery side, anybody who’s worked in a manufacturing environment, every plant I have visited has a “This many days since a lost-time incident” chart on the wall. And you can argue whether you should have those or you shouldn’t, but for the whole industry, zero is the only acceptable number in that industry. Those are two totally different frameworks for evaluating whether AI is helping or hurting the safety situation. So Aaron, does that answer your question?

Aaron Hand: [00:45:53] It does. For the most part, I just think that my anticipation, I guess, is for people to say, “Hey, look, there was a safety issue with the AI system, and therefore that’s not the way to go.” Rather than look at, “But this AI system is still safer than the traditional system.” I guess that’s less likely to happen because in both cases you don’t have the scenario like you described with autonomous vehicles versus a human driver.

Erik Reynolds: [00:46:28] Yeah. It’s complicated by legislation as well. All 50 states have passed some sort of AI safety legislation, and most of them are somewhat modeled on the EU AI Act, or at least California’s was for a while. I can’t keep track of it because it’s always: This is past. It’s been shut down. It’s been whatever. But the difficulty is a lot of those are putting the liability back on the original model developer for the AI system. Even if that model was developed and put into this system and modified by that one and put in by that one. So we’re still trying to sort out what is the liability chain. And so in my mind, it goes all the way back, Winn, to your comment about the insurance companies are going to decide this, because they’re the ones who hold the bag, or reinsurers, who actually, as I understand it, insure the insurance companies, which kind of blows my mind a little bit. But that’ll be what sorts it out rather than the strict logic on the shop floor of, “Hey, this one is safer than that one, because we’ve actually reduced. . .” I agree with that logic, but there’s a lot of complicated factors at play.

Winn Hardin: [00:47:44] Yeah, there are a lot of factors. In the case of autonomous vehicles, not to go too far because you’re more expert than I am, Erik, but I mean you’ve got a mass application area. It’s going to massively impact and change society and everything. So there’s going to be a whole lot of data that’s going to be collected. And the case law will be worked out over long periods of time. You know, the install base, and there’s so many different players that make a decision in that, from the person who made the car to the model, the modification of the model, to the car owner, to the route that was chosen and all these different factors. In an industrial environment, it’s usually the company. There is one person at the top of this hierarchy discussion. And they know whether they’re making or losing more money by taking either a traditional safety approach or an AI safe approach. And that’s quickly going to shake out, you know. And back to an earlier point you made, Erik, I was kind of interested when you said the service model, the medical robotics or the service robotics would be like the third big uptake. And I agree with you just solely because there are so many fewer robotic or automation models in those environments right now. But if we were to look at where the money from the world is going to go over the next 20 years, it’s going to go massively skewed towards those type of applications. On the medical, on the health care side because of an aging global population and things of that nature. So fewer units now, but it’ll be interesting to see how fast they catch up.

Erik Reynolds: [00:49:14] Yeah, you’re absolutely right. I mean, well, any private equity firm right now would be buying up retirement homes, right? That’s the next thing with the aging population around the world.

Erik Reynolds: [00:49:28] Yeah, I think you’re right. I agree with you. So I think the switch to fully autonomous cars will happen very quickly, because it’ll be a latent feature in new cars that are coming out. And then eventually it’ll just get turned on, and then you’re going to have old fogies like me in my 2002 Chevy Silverado driving around with all these self-driving cars, and they’re all going to have to platoon around me with a wide — because I’m the unsafe driver now because of all that stuff. That’ll happen quickly. I think in industrial settings it will happen on a case-by-case basis. A company-by-company basis. Because you’re right. The automotive world — the world is your test lab. You can collect all this public data everywhere. You’ve got to pay attention to GDPR and other stuff like that. But there’s no limitation. But if Company A deploys AI safety systems in its operations, it’s not going to be too excited to share what it learned from that with Company B that’s it’s a competitor. And so there’s a different pace at which you can collect enough data to actually improve and accelerate. And then even beyond that, when you move into the medical world, I think there are a lot of folks who want to do that. But number one you’ve got to get over — that’s asking the least ready population to be the most proximate to physical AI systems, right? So people who are in elder care type situations, there’s probably nothing scarier to them than a robot coming up to pick them up off the bed. So that has to be crossed. That’s why I think it’ll take longer. I think eventually, when my kids have kids and they’ve all grown up with a robot nanny or whatever it is, it’ll just be natural to them. It won’t be a problem. But that’s going to take a generation to get there. That’s my opinion.

Winn Hardin: [00:51:35] I agree with you, unless there’s just simply not enough warm hands to go out there during those warm-hand applications, and then it’ll be by necessity not by desire. But TBD, and we’re certainly in an interesting point in society’s or civilization’s history right now with various population changes going on and everything else. We have taken way more time than I meant to today, Erik, and I do want to apologize for that, but I sure have enjoyed the time. Can I ask you one more quick question before we let you go: If someone was considering safe AI today or thinking of it as a future set, what’s the one or two things you would tell them to think about at this point?

Erik Reynolds: [00:52:17] Well I guess the first thing would be why? Can you do it with a classical safety system? And if the answer is yes, then you should probably do it with a classical safety system. If the answer is “I want to do something that you can’t do with a classical safety system,” then you should should consider AI. The second thing: You’ve got to answer the “How do you know that you know?” question. How do you know you can trust it? There will be a lot of folks and well-meaning engineers and designers and developers who will demonstrate that their AI system works to you, once, in front of you, or in a pilot project. But we’re managing the unexpected here. We’re managing the perfect storm: We had a storm last night and the roof developed a leak and a bat came in and landed on this. And that’s the weird situations that come up. Maybe that’s the third thing. Designing for average safety behavior does not work in AI systems because it’s the corner cases that dominate safety performance in AI systems. And you will encounter corner cases over time.

Winn Hardin: [00:53:38] Can’t wait to see how IEC 22440 — is that right?

Erik Reynolds: [00:53:42] That’s right.

Winn Hardin: [00:53:43] Can’t wait to see how that comes out, early next year. And maybe before that we can revisit this with you again, Erik, and get a little more download for our audience in terms of, well, this is how it’s coming in the public opinion, because there’s always those periods where we’re collecting a lot of different information. So I can’t wait to hear how that all goes.

Erik Reynolds: [00:54:01] Happy to do so. And I’ll try to be less long-winded next time.

Winn Hardin: [00:54:05] No, no, it’s not you. We asked too many questions, dude. We’re the ones who blow this thing up, so no worries. Erik, thank you so much for joining us today. If anyone in the audience has any questions for Reynolds & Moore or Erik specifically, please reach out to us directly. You can find us wherever you get your favorite podcast platforms. You can see this episode and past episodes and also contact us if you ever want to be a guest at manufacturing-matters.com. Until the next episode, thank you so much for joining us, Erik. Have a great day. Aaron, always a pleasure. See you back at the office, and we’ll get together and do this again soon. Thank you.

Erik Reynolds: [00:54:39] Thank you all.

Aaron Hand: [00:54:40] Thank you.