Episode 12 – LIVE – Jared Glover from CapSen Robotics: Audio automatically transcribed by Sonix

Episode 12 – LIVE – Jared Glover from CapSen Robotics: this mp4 audio file was automatically transcribed by Sonix with the best speech-to-text algorithms. This transcript may contain errors.

Jimmy Carroll:
Hello, everybody. My name is Jimmy Carroll. I'm the vice president of operations at Tech B2B Marketing. Thanks for joining me. Manufacturing Podcast. Today I have the pleasure of being joined by Jared Glover, who is the CEO of CapSen Robotics. Jared, welcome and thank you for taking the time today. Really appreciate it.

Jared Glover:
Yeah. Thanks for having me, Jimmy.

Jimmy Carroll:
Of course. Yeah. So for those who are unfamiliar, can you please explain what your company does and how it started?

Jared Glover:
Yeah. So we make software that gives robots or robot arms more capabilities for handling clutter. So picking things out of bins, picking objects off of shelves, stacking d, stacking them, machine tending, you know, our software uses the latest techniques in AI, computer vision, motion planning control and enables those kind of new applications. And then in terms of how we started, I, I've been doing research in robotics for, for a couple of decades, started at Carnegie Mellon and then went to MIT for my PhD and did research in, you know, some of the technologies, the 3D vision in particular. And also on the motion planning side, I worked on a robot that learned to play ping pong, which was a really fun, fun, high speed project. Um, and so that just kind of naturally led into thinking about how it could be used in industry and then I pulled in my, my partner that started the company with Mark, I pulled him in. I had worked for him between my master's and and PhD at a finance company in New York. And, you know, he's been managing software teams for for decades. You know, already before we started the company, he'd been doing that and brought him in because I knew that, you know, a PhD like like me needed help to build really industrial strength technology. And so that's how we started.

Jimmy Carroll:
Interesting. Yeah. So when think of bin picking applications, think of those systems that typically involve the use of a 3D machine vision, a robotic manipulator and some sort of software. But but you know, bin picking is obviously much more than just vision and robots. So to you, Jared, what's the key to successful implementation of a bin picking system?

Jared Glover:
So I think there's really two things. So, you know, the software has to do more than just bin picking typically. Um, you know, typically it's the task for the robot isn't just to get an object out of a pile, get an object out of the bin, but it's a machine tending application, right? It has to insert the part into a machine in a particular way, or it has to assemble it with some other part, or it has to package it or what have you. Palletize And so the software has to be able to reason about all the steps of the task, not just the bin picking task typically, which is, you know, that's what we focused on, is giving the software all those additional capabilities. And then also, you know, you really from the user's perspective, from the customer's perspective, you really want to be working with a company, a technology company that can provide the whole solution, that can solve the whole application. And whether that's a prefab cell where they've done it a hundred times already, they've done exactly that application 100 times, or whether it's a new application, but where, you know, you have a partner, you have a vendor that is solving the whole problem, not just giving you a robot and a piece of software and you have to figure out the rest, including all the end effectors and fingers and all of that, which can be quite tricky. Sometimes you just want to want a partner that can solve the whole thing for you.

Jimmy Carroll:
So yeah, ultimately finding a partner that can task a robot to pick a part in a cluttered scene that presents a challenge. And many companies have struggled with that. So companies that have struggled with bin picking, how do they overcome this challenge?

Jared Glover:
Yeah, I mean, it's funny, That's a very typical customer for us. We often get customers that have tried to solve some bin picking related task, either on their own, just, you know, using tools that are available off the shelf, the software tools that are available off the shelf, or even with another partner, another robot company that has more traditional tools for for doing bin picking. Um, and so that's, that's a very typical story that they've, they've been trying for a while and haven't been successful and then they come to us and, and you know, we're able to solve things but you know some common problems are, you know maybe they have no touch areas on the objects that they're trying to pick or maybe they need to clear the bin. And they weren't able to do that before. Another really common one, like I mentioned a minute ago, is what are you doing with the part after you pick it up? Are you able to, you know, know how you're holding it well enough and re grasp it if you need to so that when you insert it into the machine, you're sticking the right end of the part into the machine 100% of the time. Right. It doesn't have to be like an inspection step afterwards to again, to like make sure that you're holding things in the right way or make sure you're placing things in the right way. What kind of what kind of motion constraints are there? Are you allowed to, like turn it upside down as you're moving it from point A to point B? Or do you have to keep it in a certain or fixed orientation? All those kind of things are are common issues. And again, that's why, you know, we've worked so hard to to make sure that we can solve the entire application for customers.

Jimmy Carroll:
Yeah. So one of the things that you mentioned I wanted to ask about partnerships. Partnerships are so important in this space. How do you, CapSen, think, you know, I'm talking to you in the past, Jared. I think that maybe in the past some people have considered you to be an integrator, but you're not an integrator necessarily. How do you partner with with both integrators and and other robotics companies?

Jared Glover:
So, so it's interesting. Um, you know, the first bean picking application we installed was picking hooks out of a bin for Wire and Spring Company here in Pittsburgh. Very complicated application. The hooks were entangled. They had to be disentangled and then re grasped so you could put the right end to end into a machine. And it's very precise application. And we ended up doing the full integration on that project to get the experience to know what had to be done, not because we wanted to be an integrator, but so, you know, we we'd eat our own dog food, so, so to say, and, you know, really understand what it was that integrators were going to need from us. But the plan all along and immediately after that, you know, we said, all right, now we're going to work with integrators. We don't want to do all of the electrical work, all of the fab work and all of that. We want to provide them with the tools they need and the support, you know, the the the services as well to make our software enable these applications without being the integrator. Um, and so what we found is that, you know, even with integrators have, you know, lots of capabilities, lots of technical staff that are excellent at programming robots, they are still very happy to have us program the bean picking part of, of the whole system.

Jared Glover:
Um, and you know sometimes get involved with end effector design and small, you know, hardware mechanical issues that come up with the whole system again because, you know, there's so much interplay between the different parts of the system. For a bean picking application, you really want the the technology provider like us to solve the whole application, which which involves mechanical and software components. Now the next stage of our evolution, though, is going to be taking what we've learned in doing, you know, solving these whole applications for every, every single install with our integrator partners and, and, and providing the tools, the same tools we use to do all the configuration and programming and end effector design, providing those as well to our integrator partners so that they can start to do, you know, the entire programming and configuration on their own. But but so far, again, just the whole practice of eating your own dog food, you know, we wanted to get really good at doing that ourselves and make sure that our tools are very robust and and easy to use. And and, you know, before we we try to get our partners to do all that heavy lifting. But that's the evolution that we're we're in the process of now.

Jimmy Carroll:
I know you also work with some robotics companies. How do you how do you partner with them typically?

Jared Glover:
With robotics companies, it's more of a sales and marketing partnership typically. So, you know, we we make joint marketing videos together and advertise on each other's websites. We're going to be in one of our robot partners booths at automate in May. Denso And actually probably in Calvary's booth as well, one of our integrator partners. But, but yeah, it's mostly a marketing partnership with them. Now that being said, we we support many different robot brands, many different camera brands, end effector brands. So our software itself is hardware agnostic, which is useful for the end customer that may have their own preferences or standards. But we definitely work closely with our robot partners on the marketing side.

Jimmy Carroll:
Okay, great. So let me circle back to the initial question before I got a bit sidetracked there. When it comes to your cats and their approach to bin picking in a cluttered scene, what what approach do you take?

Jared Glover:
It really depends on the application. As far as far as the overall approach. There's a lot of different tools we have in our software toolkit and hardware tool kit, but you know, there's some times where all the parts are rigid and they're exactly the same every time. And so, you know, you can use the full 3D model of the object and do 3D model matching what's called CAD matching oftentimes to understand exactly where the parts are in the bin or on the shelf and how to go and pick them up and manipulate them. There's other times where maybe the the the part is rigid and it's the same shape every time, but it's extremely, extremely shiny or transparent. And so you can't use a 3D camera to get a good point cloud to get a good a good 3D observation of of the bin. And so you have to use more machine learning, deep learning techniques, that sort of thing to to get the job done. So it really depends on the application. There's others where it's a sorting application, right? So you're picking something up without knowing what it is, and then you're holding it up in front of another camera. Maybe you're reading bar codes to help you differentiate. And and so the approach is different depending on the application. But but generally speaking, what we have worked the hardest on in our software is using geometric reasoning and planning. And so whether that means you've got the object geometry or not, you know, we have geometry models of everything in the environment. And even if you don't know the object geometry ahead of time, once you pick it up, you're looking at it so that you know exactly what it's shaped and how you're holding it. And then we're using all of that geometry information about the environment and the task so that our software can can plan the motions for the robot to accomplish the task.

Jimmy Carroll:
Yeah. So, so you mentioned software and that's where I was going to go next. So this is good. Um, I mean, obviously been picking involves more than just robots and 3D vision. You need software that can segment and identify an individual object and find that so that the robot can make a pick. What else can you talk about in terms of the software? Maybe for those who are less familiar, like the role and the importance of the software, which is kind of the heart of the system?

Jared Glover:
Yeah, so I've touched on a couple of them already, but there's four main software components in our system and in any, any 3D vision, you know, guided robotics software that's out there has to have these four components, right? So there's the vision software. So the robot can, you know, look at what's in the bin or what's around it and understand exactly where things are. Then there's motion planning software because, you know, the parts are jumbled and different configurations every time. So the robot has to move differently every time to pick them up and do things with them. And the motion planning software goes beyond just, you know, figuring out how to move the robot arm from point A to point B without colliding with things you're talking about. Okay, how do I where do I grasp on this part so that I'm not colliding with other objects in the bin? Which object do I pick in the first place? Um, you know, which which orientation do I pick the part in? Do I need to, like, put it down and re grasp it again to do what I need to do next? Do I need to disentangle it? Is it going to get tangled up with another part? Do I need to separate it somehow? Do do I need to like place it in a pattern or stack it or something? So all those things are part of the motion planning software. Then, you know, that's just planning what to do. Then it actually has to execute that plan on the robot. And so there's an execution and control component to the software which executes the motions and communicates with other devices, PLCs sensors and so on, and, you know, decides what to do in real time. Maybe there's some interrupts that say, hey, if you know, if you feel some force, you don't expect at this point in the in the motion, how do you recover from that? Maybe there's another way you can try to accomplish the same thing or maybe you just give up and get a user to come over.

Jared Glover:
So there's a lot of sophistication in the execution and control. And also like, can you do multiple things at the same time? Can you plan, you know, can you plan for the next motion? You take the image and plan for the next motion while you're still doing the previous pick and place task. So a lot of things that happen in parallel as well in the execution and control that give you the high throughput and and fast cycle times on these things. And then the final the fourth component is the user interface, which is all about, you know, not just how do you how do you control the system, how does the user control the system and see what it's doing. But it's how do you configure the robot in the first place? How do you program it and teach it? How do you, you know, teach it about the new parts, the new application, the environment, All of that has to be you have to have very sophisticated tools in the user interface to enable the configuration of of the system without writing new C plus plus code for every new application. That's something we we've worked very hard on is to make sure that we're not writing, you know, new C plus plus code to handle a new part or to take the robot to a new environment. You know, it's it's all done in configuration and the user interface and teaching that can be done by technicians that don't know how to program.

Jimmy Carroll:
So related to that question is something you touched on earlier and it's it's the topic of AI, right? So these days, many companies are touting so-called AI based bin picking, but maybe the term is misleading. Maybe it's more like a bin picking system that incorporates AI algorithms for certain functions. Right. So how do these AI algorithms help in picking applications, both in terms of, you know, kapsan and the greater marketplace?

Jared Glover:
So, you know, the way that AI is used is the term has evolved over the years. These days, I think it's mostly referring to like chat bots and other things like that or image generation. But really artificial intelligence encompasses pretty much anything sophisticated that computers and robots have done for the last 30 or 40 years. You know, starting with, you know, computers that play chess and search algorithms all the way up to machine learning, optimization, motion planning. You know, everything that we've talked about today on the software side is one type of AI or another. Now, you know, there's some types that people talk about more often these days when they're referring to deep learning, the generative stuff to generate text and images, all of that. Um, you know, in a lot of that, you know, so deep learning is used again as a tool. One of the tools that we use not not the only tool, but it's one of the tools that we have that enables certain applications, like those very shiny parts or transparent parts, that sort of thing. Um, but looking at some of the, the more recent tools like the chat bots, you know, that that type of AI is not going to solve the problems that we're working on anytime soon. It's not going to suddenly give robots human level capabilities and adaptability. You know, there's a big difference between having a conversation that sounds sort of reasonable and actually moving around and touching things in the real world and and and and doing assemblies and machine tending and all of that. There's a big difference. So that type of AI is not not solving anything for us yet, unfortunately.

Jimmy Carroll:
Thing, right? So in, in reality, in the space that you live in, what strengths do these AI, you know, machine learning, deep learning algorithms offer in been picking.

Jared Glover:
So so again, you know, if you look at the, you know, the umbrella use of the of the term, which is really what what it's meant for for decades. Um, you know, you can't do bin picking type tasks without you have to do motion planning, you have to do optimization and machine learning to solve bin picking. There's no other way. Now, if you if you're specifically talking about deep learning, for example, like I said before, there's some parts where deep learning is is definitely necessary and there's others where it's not, and there's some where you could solve it with deep learning or with a more classical geometric based algorithm. Um, and there's a lot where you want to combine them both. You know, we're, we're never just using deep learning. I think there's a couple of companies out there that claim maybe they're, they're trying to train end to end deep reinforcement learning systems. Uh, but you know, typically when you really talk to their engineers, you know, they'll say, oh, well, yeah, we do that for the first prototype, but to actually install it, we had to add all this other stuff, you know, the classical stuff to make it safe and to verify what it's doing is correct and all of that. So, you know, yes, eventually maybe these deep reinforcement learning systems that are trained end to end on a task, they could get sophisticated enough. But, um, you know, we're a long ways. There's a lot of basic research that has happened. It'll probably be, you know, after I retire that we're really talking about, about not having to program robots at all, that they just learn to accomplish a task on their own.

Jimmy Carroll:
Mm Yeah. So that's, there's a lot of interesting stuff there. Um, yeah, I'm just thinking along the same lines there are, there are companies in the marketplace today that that may go as far as to say that, you know, something like lighting doesn't matter, which probably isn't true. So, you know, in terms of the role of consistent lighting and robotic pick and place or bin picking, can you talk about that a little bit?

Jared Glover:
Um, yeah, it completely depends on the object material. So if you've got very matte objects, then lighting doesn't matter as much, especially if you're using, um, you know, structured light 3D cameras that project patterns. And then they're looking for those projected the projected light coming back. Um, you know, then the lighting really doesn't matter quite as much. You've got shiny objects though, you know, you do want to avoid having too much directed light focused on the objects. They can bounce back in unpredictable ways and cause, you know, blind spots for for cameras that can cause problems for 2D or 3D cameras as well as for humans sometimes. Right? So you want to minimize that sort of thing just by diffusing the light, not necessarily blocking out, blocking it out completely, but diffuse lighting. And if you've got a mix of parts, it's always safer to just use diffuse lighting where you can.

Jimmy Carroll:
Let's see. Jared. Um, we haven't really touched too, too much on, like, specific industries, so I know CapSen is involved in several, you know, general manufacturing, automotive, logistics and warehousing. But one that I wanted to ask about because I know you and I've seen some of your videos and everything is the medical space. Your technology's been deployed here and some really interesting ways, especially over the last few years with things like COVID 19, obviously, and some of the applications that have arisen as a result. Right. So maybe without naming some some of the customers, can you talk in broad terms about some of these recent deployments?

Jared Glover:
Yes. We've been getting a lot of interest in you from the medical industry on on using these type of bin picking robots. So we've done medical packaging for packaging, COVID test kits. We've done sorting, sorting medical supplies for hospitals, order fulfillment of, you know, cartons of medical supplies from larger boxes on flow racks. So really there's a wide range of applications that are that are useful in the medical industry and and gaining some traction. And it's really just the start. I mean, and this goes beyond medical, but there's so many applications where you've got objects that are randomly configured, right? You've got you've got stuff randomly piled in bins or on shelves or inside of a box. Even if it's somewhat structured, maybe it's, you know, it's aligned in grids, but you really don't know exactly where the rows of the grid are and where the separators are. Or maybe they're packed tightly together, you know, in all those situations. And any situation where you don't know exactly where something is, every time you need vision, you need you need some intelligence to the pick and place with the robot. So, you know, there's lots of applications in medical that we're gaining traction in for sure. But it's just, you know, it's a microcosm. The problems that that they have in medical with with bin picking are the same problems in every other industry. It's just think in medical, you know, they're having such acute supply chain issues and labor shortages that are really affecting the supply chain. And like in medical, if you have issues with with the supply chain, you know, somebody's surgery is getting delayed. So it's like it's really big consequences. Whereas if there's a you know, if there's a delay in the shipment of some nuts or bolts or something, you know, maybe, maybe you get your Tesla a couple days later, but like, that's that's not going to kill anyone, right? Yeah. So, yeah, medical, I think, has a very acute need for more automation. That's where we see more traction there right now.

Jimmy Carroll:
Supply chain. So you mentioned supply chain. Um, this is a topic that I wanted to bring up too. So, uh, Association for Advancing Automation A3 Their figures show that robot sales hit a record a record high last year. North American companies ordered, I think, more than 44,000 robots close to 2.4 billion. Uh, these numbers represent increases of I think 11 and 18%. So, so automation is obviously more important than ever before, and it will continue to be, um, you know, on the, on the topic of A3 at the, at the business forum this year during Alan Bailey's Global Economic Outlook keynote, which anyone who's been to the show knows is a great, great annual presentation of Catch. Um, he said automation will be key to keeping the economy going during uncertain times. So say all this because you know, this is all very exciting stuff, but what are your key takeaways from all of this? What are some ways that robot sales can continue to grow, including like new or expanding industries?

Jared Glover:
So it's interesting the growth story in robotics in North America, particularly the US. Recently, you know, there's been a lot of growth, but I think most of the growth has been playing catch up, right? Because company countries like South Korea and Japan and even China recently have really surged ahead. And robot robot adoption, they have way more robots installed per capita than we do, especially South Korea and Japan and some European countries. Um, and so we really lag behind here in the US for the last 10 or 20 years. So we're just playing catch up right now. It's long overdue and necessary. Um, you know, and I think, you know, a lot of the growth has not required a ton of intelligence, it's not required a ton of vision systems and AI. But you know, the next big wave of growth after that obviously is going to be with AI enabled robots. And that's an area where the US is number one, right? The US is number one in software. And so we can really capitalize on that here and, you know, get an advantage in the types of automation we have here compared to other countries. You know, I just read an article today that was talking about Japan having, you know, this huge lead in robotics, the hardware of robotics for so long and adoption. But they're lagging behind now in terms of AI and, and the the software aspect. So, um, you know, I think that's going to be the story in the coming 1 or 2 decades, is that the US is, is going to be the center of robotics again because of our advanced capabilities. But, you know, we need some help from, from from government to we need them to give companies the incentives to increase adoption faster because, you know, there's still some things holding holding us back in that in that respect.

Jimmy Carroll:
Hm Interesting. So yeah, so staying on the maybe on the topic of like the general space, the general industrial automation space, what are you most excited about? Both both in the general space and for your company, let's say, this year and beyond?

Jared Glover:
I mean, I get excited every time I see a robot do something new. So whether it's, you know, mobile manipulation, putting arms on mobile robots, I think that's going to be huge growth vector in the coming years, whether it's, you know, the fact that robots can handle more entangled parts or, you know, tend things that are moving, um, or like packing objects into really tight spaces and getting the motion control right for that and get really, really excited about things like that. We just solved a repackaging problem for an automotive company recently. That's that's something that, you know, has been a struggle for robots before shiny and transparent objects. So that's what that's what gets me excited. And I think all those things just adding new capabilities to robots and going mobile with manipulation, all of those things are going to be big drivers of growth.

Jimmy Carroll:
Jared you mentioned it earlier. Your technology will be in a couple of partner booths that automate. Um, how else can people see CapSen and this year or otherwise, how can they get in touch with you if they have questions?

Jared Glover:
Yeah, you can check out our website capsenrobotics.com and definitely come visit us at our booth and automate will be at the Denso booth and the Calvery booth and automate in Detroit this May.

Jimmy Carroll:
Well, Jared, thanks so much for joining us today and sharing all these insights. Really appreciate it. It's always a pleasure to talk to you.

Jared Glover:
Likewise. Thanks for having me.

Jimmy Carroll:
Of course. Yeah. And everyone tuning in, thanks for watching. We look forward to sharing more of these conversations with you in the near future. If anyone has any questions or ideas or would like to join the podcast, please visit us at manufacturing-matters.com. And thanks very much, everyone, and have a great day.

Sonix is the world’s most advanced automated transcription, translation, and subtitling platform. Fast, accurate, and affordable.

Automatically convert your mp4 files to text (txt file), Microsoft Word (docx file), and SubRip Subtitle (srt file) in minutes.

Sonix has many features that you’d love including collaboration tools, transcribe multiple languages, share transcripts, advanced search, and easily transcribe your Zoom meetings. Try Sonix for free today.