On this episode of Manufacturing Matters, Juan Aparicio, Founder & CEO of Reshape Automation and member of the A3 Artificial Intelligence Technology Strategy Board joined Jimmy Carroll at Automate 2023 to discuss the state of AI and deep learning today, including its practical and real-life applications, the importance of keeping a human in the loop, and its overall role in automation today. The discussion also covers vision-guided robotics advancements, ChatGPT hype, 3D printing, simulation, cloud computing, and more.

Jimmy Carroll: [00:00:00] Now. Good afternoon everybody. My name is Jimmy Carroll. I’m the vice president of operations at tech B2B marketing here at day three of Automate in Detroit. I have the pleasure of being joined by Juan Aparicio. Juan, thanks so much for taking the time.

Juan Aparicio: [00:00:13] Thank you. It is a pleasure to be with you and also with your audience.

Jimmy Carroll: [00:00:17] So, Juan, you are our technology board member for AI or A3 rather, is that correct?

Juan Aparicio: [00:00:22] That’s correct. I have been back in, I think even three years ago, uh, A3 was only comprised by a robotics vision and motion control, but I was an emerging technology that was becoming so important for our members that we thought that it was important. It should have the same level, the same, um, visibility than the other technologies. So that’s why we decided to kick this new board that was, uh, I think, official since since last year. Okay. And it’s very exciting. The companies like Nvidia, Microsoft, we are all working there to see how AI fits into the, uh, the world of automation and how does it change or revolutionize, um, this industry.

Jimmy Carroll: [00:01:08] So that’s a good point. I’d like to ask about that. Uh, it was maybe 7 or 8 years ago around the vision show in Stuttgart, where where a lot of people really started hammering deep learning. And from a marketing standpoint, as this technology that’s going to come out and change the world and overtake machine vision. And of course, it hasn’t transpired that way. But, um, there are real world ways that deep learning is adding value to people in manufacturing and beyond the factory floor, of course. But since we’re here at the automation show and we’re largely talking about factories and warehouses and things like that, what are some of the most common ways and maybe the most, the best fit ways that you’ve seen? Ai, deep learning, machine learning add value?

Juan Aparicio: [00:01:47] Yes. Very good. Intro. I, I think AI being such a, uh, overarching technology, that means a lot of different things for different people has has kind of bubble up into a hype. And now even more with technologies like ChatGPT where everybody can experience that up to the level where we are a little bit confusing performance and competency. Right? Yeah. Uh, the these models are very performant, but they are not super competent, right? They don’t have a model of the world when it comes to understanding the physics. They have a hard time, but they are very good at language, very good at completing. That doesn’t mean that we have to throw them away, right? A lot of people pass from, oh, this is so hype. This is perfect. It’s going to a human race is going to switch over to oh no, it’s hallucinating. I’m going to throw it to the garbage. Right. There are something in between. Now what happens, as you said, right? Already ten years ago, like, um, deep learning started to, to pop up. And also convolutional neural networks that are now extremely efficient, like you can have a $5 piece of, uh, hardware that runs a huge inference for neural networks and are very good at detecting and segmenting objects. Yes. So that technology is actually very applicable to a lot of our applications, as you said, right.

Juan Aparicio: [00:03:08] One of them, for example, is dexterity and manipulation and mostly the the picking side of things, right? Not the dexterous manipulation that the human can do. But for a lot of tasks, you just need to pick and place. Uh, been picking used to be the holy grail of, of robotics. Now companies like opera there is making more approachable. Photo Neil McMinn and others. They have solutions and pick it 3D. I can continue, right? Of course, because this is like many, many other players are offering solutions, some of them from the hardware point of view, very good hardware that can uh, uh, do structured light and create point clouds. And then with those very fine detailed point clouds, you can decide where to pick apart. Right. And things that are easy for a human are usually hard for a robot and vice versa. Right. And a robot can beat me at chess, right? If I play just on the logic side. But no robot can go up. Take a chessboard from a shelf and start playing and moving the parts yet. Right? And all of this to say that these technologies are getting more common and almost more commoditized, and vendors are now adding more value in terms of how you program, how you integrate with the rest of the ecosystem.

Jimmy Carroll: [00:04:27] Yeah, yeah, it’s really cool that you say that. It just reminds me of like walking the show floor when in between all the videos that we’ve been doing, I get the chance to walk the show floor and it’s it’s nice to see first, uh, vendors working together to solve unique problems. And there’s more of these application specific systems that do certain things, like bin picking, for example. But also, it’s nice to see the best way to say it is, is as as a number of different technologies advanced together, you’re able to solve new problems. So like with AI at first you know I some of these. The advancements were driven by improvements in processing power with GPUs and things like that. But as you’re saying now, there are small chips like the Sony IMX 500 that can do these things. And it’s it’s just a little chip. And it’s really cool to see that all these all these technologies working together to, to solve new problems.

Juan Aparicio: [00:05:16] So fully agree more integrated into cameras, uh, at different perspective. Structured light is one way, as we were saying, to get these point clouds to, to do uh to do inference on the, on the object or where do you pick also there are other companies doing now even with dual cameras and what they call for division the. But the interesting thing is that, uh, every successful application of AI has two ingredients out there. Either there is a human in the loop, so if it fails, a human comes and fix it. Or, uh, the cost of failure is very small. So that’s why you see a lot of beam picking applications at the beginning of implementing deep learning. Because it doesn’t matter if you fail to pick as soon or as far as you, uh, keep up with the cycle time, you’re okay. So you can fail a pick, but then you try again and then your cycle time is perfect. That’s why a lot of companies focus on the pick, not the pick and place, because the place is in a box in the logistics, the perfect use case. You just pick an object coming from a tote with a several objects. You just pick one and put it and fulfill the order. Perfect use case. Siemens and other company doing this now integrated even into a PLC. You don’t need even. We’re talking about computing now. Uh, I actually remember back in my when I started working on this technology in 2016, uh, or CPUs were not fast enough to do the inference.

Juan Aparicio: [00:06:45] So we worked with Intel to create a small module that we could attach to a PLC to do the, uh, this neural network computations. And by the time that we finish, CPUs were powerful enough so we could run it there. But back to my point of ingredients that make a AI application successful. It’s just that the the price of failure is is small, right? So been picking perfect. The moment that you do placing is a little bit more tricky because you get it wrong. The process can be really damaged. If you’re doing an assembly, you can really destroy the part you’re doing right. That’s why you don’t see that many yet. And what is important too, is design around those cases because it’s going to happen. Yeah. If you walk the shop floor, you will see many of these demos. They’re demos and you see them that if you watch closely for 15 minutes or half an hour, it’s going to it’s going to fail. Right? It’s how do you gracefully recover? What happens when you, uh, when you fail. Right. And you recognize that there is a failure and quickly call a human? Can you retry again? Can you push things around? Yeah. This is it’s more a systems problem than an algorithm problem in this moment.

Jimmy Carroll: [00:07:52] So that’s a really interesting point. Um, I think anecdotally I’ve heard and you can confirm whether or not this is true, but anecdotally I’ve heard of instances of business coming to systems integrators or vendors, OEMs and saying we think we need AI, but okay. For what? Right? How what’s what is the what approach have you taken or have you seen that work successfully when it comes to evaluating whether or not AI is actually a good fit for a given application?

Juan Aparicio: [00:08:18] Very good question. The AI happens a lot, right? With all these overhyped technologies, it looks like this is a silver bullet. Yeah, right. I don’t know how to make something intelligent or this thing work, but just added AI. That’s a solution. How do you make this work? Just add AI because it’s such a keyword that you can use as a key solution. A lot of companies, they brand it that way. But when you look under the hood, it’s just good old fashioned control. And now it’s branded as AI because it also sells like it has the marketing part that we shouldn’t also discount. Um, when a customer comes with a with, okay, I think you need a hammer and you’re looking for nails, but then you probably may be different, right? So it’s very important to reframe the situation and say, okay, what do you actually need and not spend any single don’t over engineer your solution. If you can do an automation task without a camera, you’re going to be much more robust, right? If you can fix your if you have the luxury to fix your the parts so the robot knows where to go, don’t go with the camera because it’s an extra hardware. There are many things that can go wrong. You need to calibrate it. You need to have the light. It’s another point of failure for your system. Now, if you cannot afford that, there are also other solutions like, uh, like a bowl that vibrates, right, that can also simulate the parts. Very easy for a camera. And then you need also the camera, but you don’t need to be in peak, which is very complex.

Juan Aparicio: [00:09:41] So traditional computer vision can solve that problem. Now if you go through the rabbit hole and I still don’t have the ROI, uh, then that’s where I make sense. Where more advanced methods to train a neural network to pick either cat based picking where you know the part and you care about the. Lacing or unstructured. Pick what you don’t know the card, but it has been a neural network has been trained to to pick the parts. And so it’s going into your solution and understanding what you need. What are your parameters in terms of ROI maintenance? What is your budget? What do you want to accomplish with that task? Is there a human around that’s very important, right? Yeah. This lights Out factory doesn’t really exist, right? It’s lights out for the first 15 minutes until something jumps. Yeah. And but then there are other less hype technologies like Teleoperation. They’re very cool company called Ola Alice that does teleoperation for industrial robots. Amazing demo where you can see I can control my robot in Seattle from my phone. Right? These are a lot of value and are very complementary. And back to my point or okay, is there a human in the loop? Maybe not physically there, but if I can teleoperate to the cell that already simplifies and starts to, uh, make it easier for these kind of algorithms that may fail, it’s not going to may it will fail. Yeah. Or it won’t succeed, or something unexpected will happen and then a human will. Humans are excellent to understand a situation that it hasn’t seen in its life, and find a solution for that.

Jimmy Carroll: [00:11:22] Yeah. And that human in the loop idea is, is obviously a very important one. But it’s it. Correct me if I’m wrong again one. But like it’s not just having that human in the loop who’s maybe a trained factory floor technician who can say, yes, this is this is what we need to do. This is the adjustment we need to make. But cliche turn right, but garbage in, garbage out. If you have these humans up front that can help identify defects and label something that’s within tolerance, um, so that you can reach an agreement and start with clean data to train this model. So the humans are important both up front and indefinitely for, for these AI models. Right.

Juan Aparicio: [00:11:58] Fully agree. Yes. And that is particularly important. Also we talk a lot about the grasping side of things, but also on the quality inspection. Many quality inspection companies now like Unitec’s Elementary, landing I and others offering, uh, machine learning models that you can train for quality inspection. Right. Very different than the traditional Cognex or Keyence models where it was feature based. You were looking into a feature like a scratch, and you know that either you have a scratch, you don’t have it, so you can train your algorithm very easy. But now with machine learning, you can actually just say, this is a good one, this is a bad one, this is a good one. And you train with a bunch of good models and everything that deviates is going to be a bad one. Now to your point. Agreeing like a human, agreeing what is good and what is bad is very important because otherwise they are going to be surprises. If your in your quality control, you cannot even agree. If this object is good or bad, then the algorithm is also not going to agree. As you said, garbage in, garbage out. And it’s very important also to train the customer on these new technologies. When you do machine learning, inspection is a stochastic process. So if something deviates, you’re going to have a lot of false positives, right? And you have dust suddenly and then you get, oh, everything is bad because there is dust. I didn’t know that dust was not supposed to be bad. Let me retrain the model and next time it’s going to learn. Right? Yeah. But it’s just this constant process of you are just. And I really like your analogy. You need to have a good, uh, set of good examples in order to get a good deal out of them. Algorithm out of the system.

Jimmy Carroll: [00:13:36] Yeah. Thank you for confirming that and for expanding on it and providing an eloquent response that I, that I couldn’t. But but that’s my understanding of it. And hearing it from someone like you is, is enlightening. But, um, I’m just thinking that, uh, kind of goes back to something. Earlier you said you brought up ChatGPT and people think of robots, and now they’re thinking of ChatGPT. So is ChatGPT going to take my job now? Are these robots going to take my job? Well, no. They’re really they’re really just going to change your job or create new jobs. Uh, you know, like with the human in the loop thing. Like somebody before that maybe have been labeling with their job was to just, you know, label data or manually inspect something. Those people are now involved with helping improve these models and then, you know, adding value in other places on the factory floor. But, um, it’s been nice to see over the past few years that the idea that these automation, these advanced automation technologies like robots and AI and machine vision are going to take people’s job, it seems like the general public has kind of cooled on that a little bit, because it’s really just not happening. There are jobs out there and people don’t want them fully agree.

Juan Aparicio: [00:14:35] And also the it’s very clear that technologies like GPT is not even going to replace the robot programmer, right. But it’s going to make their life easier. A lot of the code may be auto generated, but there is a person in the loop, uh, revisiting that right and reviewing it, making sure that it does what it should do. Right. And one example of actually application for the LMS this big. A large language. Models for our world will also be creating all these documents that are very tedious to process this back office work, right? It’s not the actual worker, but we thought at the beginning, a few years ago, we thought, oh, robots, really, as you said, taking our jobs because they’re going to be so dexterous. Actually, the dexterity hasn’t really changed. What has changed is the vision. Right now. They can see as we just talked before. And also you can generate language. So all these the way that we use it at rapid is actually creating a scope of work documents and not even creating them in the first place, because then a human will not take it. We’ll say to the customer and then maybe whatever, right. Because these models are hallucinate. But we put a bunch of text describing a task the model will ask us some questions about, oh, you may have forgotten about talking about what safety protocol are you using? Or if you are going to offer a training or if you’re going to use a camera or not.

Juan Aparicio: [00:15:57] And then you add completely unstructured text into a conference page or a word document and click generate. And it generates a scope of work that you can send to a customer. This kind of back of the office work and is now very automatable again, when people talk about taxing robots, which I think is a horrible idea. And how do you find a robot in this place? Because yeah, there may be a person spending four hours doing a scope of work and just writing documents that are very repetitive and not adding any ingenuity to that. But and their job is now a shorten and versus also a robot. That is a physical thing. Defining robot is also very hard to, uh, to just put some parameters around around that. And regarding automation. Right. But to your point, this technology is what we are really seeing is that they are not replacing jobs. Uh, they are automating tasks. Right? It’s about tasks and not jobs.

Jimmy Carroll: [00:16:58] Yeah, absolutely. One, I don’t want to take too much more of your time because I know you’ve peeled yourself away from the booth on the show floor, but I did want to ask you. You had a presentation. Uh, and I believe it’s it described the the convergence of multiple technologies to solve problems. Right. So that’s 3D printing and simulation and the cloud and AI. So can you talk to me about those, those things and how those might work together.

Juan Aparicio: [00:17:18] Yes, I think we extensively covered I still super excited about the future of this technology and what it can bring to the shop floor. And I think simulation is another very interesting technology. Less overhype is helping tremendously in all the stages of deploying a project, an automation project. At the beginning, you want to know which robot am I using, right? Which automation do I have the reachability that I need? Do I have also the cycle time that I need? Because these are commitments that you are giving to the customer, right? You need to. Usually a robot needs to match a process. And if the process requires you to palletize eight, uh, boxes per minute, then you need to hit it, right? Otherwise you’re going to have a customer that is not happy and you may have chosen the wrong hardware. Once you are deploying, you want to be as much as possible on the virtual world because it’s faster. It’s cheaper. Every time that you go to the real world, then you need to go there, do some modifications, right? The physical world. So and then the simulation tools are getting very, very good. Right. And they seem to real gap is getting shorter. There’s still a gap. And you still need to to play with the physical world. That’s the nice thing about robotics right is they work in the real world.

Juan Aparicio: [00:18:28] And but then even after we commission, if there is a ticket on the field, it’s always good to replay what is going on. Right. And see, okay, the actually this robot was in a singularity that we didn’t plan because it the part moved and suddenly you have a singularity simulation is is getting extremely useful for things like virtual commissioning and the other cloud. The other technology that you mentioned is cloud extremely useful to for particularly for the servicing part. Right. Companies that do robot as a service. That is the robot. But there is a service part, right, that we are on the hook. If the robot fails, we need to immediately fix it. The customer is not going to pay more in order for you to to fix the problem. The more you can do remotely, the more data you can get, the better, right? And that’s how we leverage cloud. Initially, a lot of manufacturers thought, okay, I don’t want to have my data on the cloud. It’s going to be less secure. I have it internally, but now they are realizing, oh, no, actually, it’s actually even more secure. Yeah, because there are a lot of cybersecurity that you can add that you can leverage from the IT world.

Juan Aparicio: [00:19:35] And then finally, uh, we talk about 3D printing, extremely exciting technology. That is also it passed from okay, we don’t need to manufacture anything else. Everybody is going to have a 3D printer at their homes to okay, now these 3D printers are just toys. Oh, no, it’s useful right? There are parts that do 3D printing metal that go into turbines. Right. These are this is real impact. And you can completely redesign the way that you do parts with less components and really fit in the need. And for us, what we have found that is tremendously useful. And there is another company that I’ve seen like Anubis, I think it’s called that they do. Similar approach is creating an effectors using, um, 3D printing because where the, uh, the kind of the tire meets the road is on the when the robot touches the part and every process is slightly different, right. Sometimes you need to pick apart and at the same time put in the machine and the finished. Good. Put it in a in a bag. Sometimes you need to grab this bottle, sometimes something different. Right. And it’s very difficult to have one standard gripper that works for all the objects in the world. Every manufacturer by definition, produces something slightly different.

Juan Aparicio: [00:20:52] So that’s where, uh, 3D printing is so useful because, uh, first of all, it’s now 3D printers are becoming very good. They are very durable. The fact that you produce and you can customize to your process. Right. And we have found out that that’s extremely valuable in order to deliver in such a broad amount of applications. But overall AI simulation, cloud, 3D printing, I think they are kind of the chocolate of the chocolate cake, right? They are. You want a chocolate cake? Yeah, the chocolate is nice, but it’s not the main ingredient. Right. You can, you know, have a cake with chocolate, the eggs, the flour. This is these are the real ingredients. And I think in, in this analogy, in this sweet analogy that I’m making, uh, the real ingredient is the good old fashioned control, the automation technologies that are really running the factories all over the world. Right. The PLC is the Hmmis, the EOS, the logic behind all of that, that is robust, right? And that thanks to that robustness, all these other technologies are adding flexibility, are reducing cost and making it easier to automate. But let’s not forget about the the pillar that is automation is when this, uh, traditional technologies and the more modern when they come together where magic happens at the end.

Jimmy Carroll: [00:22:09] I want I love that and I have a million other questions I’d like to ask. But again, I know I want to be respectful of your time. So, um, thank you very much for taking the time today. I really enjoyed speaking with you again. This is the, uh, Manufacturing Matters podcast. It’s manufacturing Matters.com if you have any questions or comments, please feel free to visit the website and reach out at one. Thanks again. Really appreciate it.

Juan Aparicio: [00:22:30] Thank you. It is a pleasure.