On this episode of Manufacturing Matters, Sina Afrooze, Co-Founder and CEO of Apera AI joined Jimmy Carroll at Automate 2023 to discuss trends and applications in vision-guided robotics, artificial intelligence software, 3D imaging, and what Apera calls “4D vision” for robotic applications. Afrooze also discusses difficult “non-conforming” applications where AI may be suitable, model training, edge AI processing requirements and advancements, the importance of automation amidst labor shortages, and the next wave of automation in factories.

Jimmy Carroll: [00:00:07] Hi, everybody. My name is Jimmy Carroll. I’m the vice president of operations at Tech B2B Marketing. And we’re here at day one of Automate 2023 in Detroit. And I have the pleasure of being joined by Sina Afrooze, who is the co-founder and CEO of Apera AI. Thanks so much for taking the time today. I really appreciate it.

Sina Afrooze: [00:00:22] Thank you for having me.

Jimmy Carroll: [00:00:24] Of course. Yeah. So Apera AI. One of the first things I noticed when I go to the website and I walk by the booth is 4D vision. 4D vision for robots. Let’s start there. What’s that mean and what does that mean for your customers?

Sina Afrooze: [00:00:36] So at a basic level, we call using simply 2D cameras, two of them with AI in the middle, that gives 2D plus 3D, 4D vision. So that’s maybe the marketing explanation for it, but the true explanation for it is really that we bring an extra dimensionality to 3D vision, and that’s where 3D vision has historically started by always using a 3D sensor to begin with to capture the scene in 3D, and then you start processing that 3D-sensed world in the form of a point cloud. And instead of doing that, which limits you to what the 3D sensor can sense, we started effectively just changing the equation and saying, start with simple 2D cameras, which is the basis of human vision, because we were inspired by that, and solve the big problem of sensing in 3D with AI, which now brings an additional dimension to the perception of the world, which is the luminance and the look of that item. So it’s not only just 3D sensing, it’s 4D sensing, because it’s always looking at not only the geometry of the part but also the look of the part.

Jimmy Carroll: [00:02:01] That’s interesting. So I have a question about that. The choice to go from 3D to 2D: was processing power ever considered there? I know that when it comes to deploying a deep learning or AI system, processing power is always going to be a factor. And that’s when more powerful computing and GPUs might come into play. But when you combine 3D point clouds and AI, that’s a lot of processing power. Was that a factor for you?

Sina Afrooze: [00:02:24] That was not why we stopped using 3D sensors. The primary motivation for that was today’s 3D sensing technology is pretty much boiled down to using structured lighting systems, where you have a projector and you have a camera, and the projector projects a known pattern onto the scene. And this projector, there’s nothing special about it. It’s just like your home projector. And just imagine you want to watch a movie at home anywhere but a dark room on a white screen. It just doesn’t work. Can you project onto a transparent glass? No you can’t. Can it project onto something that’s reflective like a mirror? No you can’t. So 3D sensors have a lot of limitations. So to limit yourself to what 3D sensors can sense immediately narrows down the set of applications that you can address. Consumer packaged goods: almost everything is either glass or shiny. Pharmaceutical: almost everything is transparent and translucent because they’re liquid filled. Automotive: a lot of material is stamped sheet metal parts that are very reflective. So we knew that if we want to truly solve the problem of vision guided robotics with a universal solution, we have to take a paradigm shift. And our paradigm shift was going back to basics of human vision, where you have simply two cameras arranged in a stereo way, and then the heavy lifting is done by AI. That was the true reason, not because of the processing power.

Jimmy Carroll: [00:04:03] Okay, fair enough. Yeah. So on to the AI. I don’t know, maybe it was six or seven years ago at the Vision Show in Stuttgart, and AI and deep learning start to come out as this really hyped technology that’s going to change everything. It hasn’t necessarily transpired that way, but in the past couple of years, you’ve seen the way that deep learning is actually adding real-world value for manufacturers and warehouses and other businesses. What are some ways that you’ve seen your customers adopt your technology and actually use deep learning to improve efficiency, boost productivity, drive revenue, things like that?

Sina Afrooze: [00:04:41] So typically our beachhead to all of our customers is what I call nonconforming applications. So these are the impossibles. They’re really shiny material; they’re translucent material. Anecdotal example: a really large Tier 1 automotive, they wanted to pick really shiny gears off a pallet. We’re the only company in the world that allows them to do that. So they started using Apera for that nonconforming application. But because the AI, and this is where the true value perhaps comes in, is trained to really deal with a lot of real-world challenges well, in terms of lighting change, in terms of vibration, in terms of difficult poses and arrangements of the part, the system behaved so reliably in their production that over one month it had zero micro stops, and when they compared it to existing installations on their factory floor that were vision guided robotics, they were blown away. So immediately they have now started on this cadence of effectively every couple of weeks ripping out one of those vision systems, replacing them with Apera, not because those vision systems are completely incapable of solving the problem. It’s just because they’re not as reliable as an Apera solution. And this ultimately comes down to what AI allows you to do. It’s just a lot more powerful in dealing with real-world challenges.

Jimmy Carroll: [00:06:21] Okay. Okay. Interesting. So I have a couple of follow-up questions there. From a high level, anyway, walk me through a sequence of events of how an Apera system might look at a problem differently. Like if you’re looking at some Tier 1 automotive part, like a traditional vision system does it this way. How does an Apera system do it differently?

Sina Afrooze: [00:06:43] Let’s look at how it would be done traditionally. Traditionally, again, you start with a 3D sensor. The 3D sensor gives you a point cloud, which is a 3D representation of the world from the view of the camera. And then that is fed to an algorithm, which is called CAD matching. And what it does is that it tries to match the humps and bumps that it sees in the point cloud with the shape that it gets from the CAD. Now, if there are slight variations or points of ambiguity about that part. For example, imagine if you have a bolt. You’ve got a fat head and you’ve got the long stretch. If that fat head is butting up against another bolt, now when the 3D camera looks at it, it’s not going to know whether this bolt is facing this way or facing that way. It’s not going to know that. Now, take it to Apera. What do we do? We train the AI 100% in simulation, using synthetic data, creating a superset of what can potentially happen in the real world. So all of these confusing arrangements. The AI has seen that before. The variety of lighting condition changes. AI has seen that before. What does the system look like under vibration, which is a reality of a factory. the AI has seen that before. That motion blur that happens because of vibration. The AI seen that before. So what that results in is a technology that is highly, highly reliable in dealing with real-world challenges that manufacturers deal with every day.

Jimmy Carroll: [00:08:36] Yeah, and I assume that the software is continuously learning of new corner cases and things like that that it encounters. And it kind of goes into the second question I was going to ask you, which is about data and how you train your data, but synthetic data. So that’s interesting. Processing power must come into play there, right? So do you use some high-powered GPU computing to do that? Is that what you guys do there?

Sina Afrooze: [00:09:00] So our software is highly optimized, but we still do use GPUs on the edge. It’s important to process on the edge because the manufacturers just can’t rely on the connectivity of the Internet because, you know. Say it’s an automotive, your line is down for an hour, that’s going to cost you $10,000. If you’re down for a couple of hours, you’ve already lost more than what an entire system would cost. So it’s incredibly important to always be online. And that delay that you may encounter by using cloud should also not exist. So the processing is done on the edge. It is using a GPU, but it’s not a massively complex PC. Think of it as a standard gaming PC perhaps. That sort of power. And yeah, that’s where the AI runs. But the training of AI, you mentioned synthetic data and that might be complex. And it is. It is complex, and it does require quite a bit of compute. But really the beauty of synthetic data AI training is how automated it is. It requires no manual labeling of data, which is something that people immediately think about when they think deep learning. In our scenario, there is absolutely no manual labeling of data required. And that’s why from the moment that somebody decides that I need to train for this application to the moment that it’s ready on their system and the robot can start using it, it’s a matter of 48 hours, sometimes down to 24 hours. And in fact, we’re pushing that to only a couple hours by the end of the year.

Jimmy Carroll: [00:10:52] I just think it’s a really cool example of edge computers with these powerful GPUs. As these technologies advance, it helps other technologies advance along with them. And same thing with some of the cameras I’m sure you’re using that are achieving faster frame rates and higher data rates and things like that. So it’s just really cool to see all these different technologies progressing together in a system like this one.

Sina Afrooze: [00:11:14] Absolutely.

Jimmy Carroll: [00:11:16] Let’s see. Another question I had was the topic of the labor shortage. The last few years have really highlighted the importance of automation, robotics and machine vision and AI and motion control, because there’s just a lack of people working and there’s the COVID 19-related disruptions and the great resignation and all that. What are some ways that you’ve seen, I mean, perhaps some of this is obvious, but what are some ways that you’ve seen your customers deploy your technologies to protect themselves against shutting lights off?

Sina Afrooze: [00:11:50] The primary ways that we’ve seen our customers use our technology is to do the dull and dirty tasks that humans just don’t want to do. In fact, one thing that’s become clear to us is humans historically tend to complain about their labor problems and how hard the job is, for a brief period. But because it’s their job, they kind of stop. And that’s one of the things that we’ve actually observed with one of our customers where, when we started working with them, they said, “Oh, this thing has to be automated. Everybody is complaining about it.” When we went back in a few months and we said, “Okay, let’s kickstart a project,” the general manager of that plant said, “You know what? I don’t really hear about it anymore. I wonder if they solved it.” And you go to the plant and nothing’s changed. The only thing that’s changed is that human thought, “Okay, I’m not being heard.” And they stopped complaining. Or the ones who didn’t like it, they quit. So effectively you’re forcing humans to continue doing this dull and dirty job, and the new generation of workers, they’re not putting up with it because there are a lot of new economy work that is available to them. Since the beginning of COVID, that’s a great example. There was a great resignation. Up until I think 2020, the number of job openings in factories in the U.S. was over 800,000. As of this January of 2023, that still remains at about 800,000 job openings in factories. People are not coming back to take these jobs because COVID opened their eyes to a new kind of job that is a lot more creative. And that’s primarily what we end up solving, the kind of jobs that humans no longer think that their human powers should be used to do day in and day out.

Jimmy Carroll: [00:14:05] You mentioned dull, dirty, and sometimes dangerous too. And it’s obviously those are prime application areas for robotics. Dull, dirty, and dangerous. That’s what we’ve always heard. What are you most excited about here, both for your company and for the space in general, the automation space in general?

Sina Afrooze: [00:14:24] You know, the automation space is truly going through a new revolution where all the things that happened during COVID was already important for manufacturers. But COVID really opened people’s eyes to how important reshoring and offshoring and nearshoring is, how important local manufacturing is and not relying on that supply chain, given all the things that are happening in terms of geopolitical events, wars, and the supply chain that happened during COVID. And the only way that we can get ourselves out of this inflationary regime that we’re in is truly to make products that people want very efficiently. And doing this process that products are manufactured in the likes of China and low-cost-labor factories in North America is not cost-efficient. That’s what causes inflation. The only way to solve this problem for generations to come is to continue to automate. And that’s what gets me really excited about automation. That’s why I think this is just the beginning of the world of industrial automation, just getting propelled to another level of impact to our lives. And particularly this year. What I’m seeing is a lot of companies coming up with innovative products, particularly with respect to collaborative robots, which allow themselves to work in small spaces right alongside humans, no cagings required around them. The programming of those robots, as well as some of the other companies that just provide software products for robots, make it accessible to people who don’t necessarily have a background for robot programming. Because as we need more automation, we need more people to be able to program these robots. And a lot of people don’t necessarily have that background. So bringing that sort of low-code, no-code environment, availability, I think that’s what gets me excited. And of course our company effectively enables a new kind of automation for a lot of things that are just not automatable without vision, without a really powerful vision. So a combination of these I think is what’s enabling the next wave of automation in factories.

Jimmy Carroll: [00:16:58] I absolutely agree. Earlier this year, at the A3 Business Forum, many people always talk about this presentation, but it’s the annual Alan Beaulieu Global Economic Outlook. And he said, throughout the next however many years, whenever there’s a recession, automation is what’s going to keep the economy afloat. And when we talk about the adoption of automation growing during COVID, yes, COVID helps I would say. But in a lot of ways, adoption of automation was probably going to grow during that time period no matter what. So it’s definitely a very exciting time to be a part of it and to be here at this show. So we touched on it a little bit. Robots by themselves, even so-called blind robots, that have no vision, can still be used for a lot of different things. But adding vision, particularly 3D vision, opens up a whole new world, makes them much more flexible. What are some other technologies. AI, for example, how does AI make robots more flexible, more advanced, and what new opportunities does it create?

Sina Afrooze: [00:18:01] So we’re just starting to apply AI to robots in factories by effectively allowing them to deal with a lot of real-world challenges in scenarios much better. But the next level of AI, everyone’s heard of probably ChatGPT and what large language models are enabling, and effectively it allows the machine to start having a better sense of decision making. And that is I think where the next impact of AI to robotics is going to be, how robots would be more autonomously able to handle corner case conditions that they’re not necessarily programmed for. This is something that today causes micro stops, which is I encountered a corner case and that’s not part of my program and I don’t know what to do. So AI can come in and help with that. But really I think a big theme here is, what the events of the past perhaps four years have shown us is reliance on a globalized economy can be a big national threat, basically when emergencies like COVID happen. and making sure that the American manufacturing really comes back to become, again, the top of the world with highest productivity, highest efficiency, most sophisticated technology, best level of automation is truly a national interest on par with everything else that is a national interest, like space technology, like defense, like health care. And that is I think why bringing AI and applying it to manufacturing today is also incredibly important for everyone.

Jimmy Carroll: [00:20:10] Yeah. Yeah, I love that. Thank you. Well, Sina, thank you so much for taking the time. You left the show floor to come talk to me, and I really appreciate it. Again, this is the Manufacturing Matters podcast. We here at day one at Automate. I’m Jimmy Carroll of Tech B2B Marketing. Thanks again, Sina.

Sina Afrooze: [00:20:26] Thank you, Jimmy. I really enjoyed this conversation.

Jimmy Carroll: [00:20:28] And if anybody has any questions or would like to be a part of the podcast in the future, please visit manufacturing-matters.com. Thanks.