Kamel Saidi, an engineer within the Intelligent Systems Division at the National Institute of Standards and Technology (NIST) joins David Dechow, Consulting Director of Vision Technology and Education at TECH B2B Marketing in this episode of Manufacturing Matters. NIST works with industry and academia to create critical measurement solutions and helps develop standards with ASTM International to improve manufacturing processes through advanced technology. For example, the ASTM Committee E57 on 3D Imaging Systems (including the subcommittee on Industrial Machine Vision Systems) is creating standards to evaluate 3D imaging’s performance in manufacturing applications, including depth error, depth resolution, and bin picking. Saidi also discusses how teams of researchers working hands-on with robots and vision systems benefits the resulting standards – and the manufacturers who use them.

David Dechow: [00:00:00] Hi. We’re here with another episode of Manufacturing Matters at the Vision Show in Boston, where we’re talking to Kam Saidi. And he’s with NIST and ASTM. And we’re going to talk about what that is and what they do. So, Kam, could you tell us a little bit more about — people have heard these acronyms, I’m sure, and they may have an idea of really what it means and how those words and letters tie in. Could you give us a little background on what NIST and ASTM are and how those organizations work.

Kamel Saidi: [00:00:39] Sure. Okay. Thanks for having me. So NIST is the National Institute of Standards and Technology. We are an agency under the US Department of Commerce, and our mission is to promote US competitiveness and improve efficiency in order to do a lot of things, including improving our quality of life. And so I work in the engineering lab at NIST, and under the engineering lab, there’s obviously many divisions. I’m in the Intelligence Systems Division, and the Intelligence Systems Division is focused on making things, improving things for manufacturing specifically. And so we have programs on, for example, additive manufacturing, looking at specifically for metal additive manufacturing. So looking at how to improve the process of the metal additive manufacturing, how to understand what happens at the laser interface with the powder, and so on. And so we do a lot of work in that. It’s a program on its own. We have a program — the program that I’m under is measurement science for manufacturing robotics, and that’s focused specifically on everything to do with robots and sensors and so on to improve the manufacturing process. So that’s NIST. ASTM — it used to stand for American Society for Testing Materials, but now it’s just ASTM. They rebranded. It’s ASTM International, so I don’t work at ASTM International, but I am involved in several standards committees under ASTM. Specifically for this, I’m involved in ASTM Committee E-57 on 3D imaging systems, and I’m the vice chair of that committee. And then I’m also the chair of the Subcommittee on Industrial Machine Vision Systems. So that’s E-57.23, and ASTM is a standards development organization. They develop standards for all sorts of things. NIST also helps develop standards for all sorts of things. So that’s how these two are tied together.

David Dechow: [00:02:57] Excellent. Can you tell us a little bit more about E-57 and how that’s going to be helping manufacturing and particularly manufacturers involved with imaging and 3D imaging?

Kamel Saidi: [00:03:10] Sure. So E-57 is a committee that was started back in 2006. And at the time we were working with the U.S. construction industry actually on trying to improve their processes. And so I was involved in research on robotics for construction and so on. And we quickly, quickly realized — actually we realized back in 2001 — that lidar was going to play a significant role in a lot of these technologies, especially when you’re dealing with robotics. And so we started working with a lot of companies, you know, the biggest names in lidar that you can think of and construction companies and so on, and also quickly we realized that there were no standards for these things. So people were trying to use them and did not understand their performance. So in 2006 we established E-57, ASTM 57 on 3D imaging, and started developing standards for those kinds of terrestrial-based lidars. Our focus has shifted because of the importance of manufacturing to the U.S. and the nation, and so we switched a lot of our work to focus on 3D imaging for manufacturing applications. And so, specifically in the subcommittee that I just mentioned, we are working on several standards to address the performance of some of these systems that are used in manufacturing.

Kamel Saidi: [00:04:39] So, for example, 3D imaging systems that are used for bin-picking applications. How well do they work? Or 3D imaging systems that are used on AMRs to do obstacle detection, people detection. How well do those work? And I know there are lots of standards, 3D imaging systems in terms of interfaces and communication protocols and so on, or safety for those that are rated for safety applications. We’re not dealing with any of those standards. We’re looking at specifically the performance. So when you use a depth sensor, for example, and you’re getting the depth to a certain object, how reliable is that number? Can you put a bounds on it, on certainty, bounds on that number. Similarly for resolution. So these are two standards that we’re working on, depth error and depth resolution. And then of course we’re working on bin picking. And how do you even — what kind of metrics do you need to understand the performance of bin picking? And then how do you measure those metrics? And the fourth one that we’re working on is, of course, how do you go about selecting a 3D machine vision system for a certain application? And that’s something we’re working with you on.

David Dechow: [00:05:53] Yes. And I find that very, very rewarding and also very valuable. And tell us a little more about those work items, and people get involved externally, right?

Kamel Saidi: [00:06:04] That’s right. So NIST. So just for those who are not familiar with NIST, NIST is a non-regulatory agency. So we don’t set standards, we don’t set codes, we don’t enforce them. We work with industry to help develop the measurement science. We have a lot of capabilities obviously at NIST for doing ground truth measurements, for example. We have all sorts of fancy instruments that we don’t expect anybody else to have necessarily, but we use those to help develop the tests and so on. And so for all of the tests, the standards that we work on, we like to work with industry on them. And most of the time they’re led by industry. So, for example, the depth error standard that we’re working on is led by Intel RealSense. The bin-picking standard is led by Apera, and then you lead the one on the best practices for selecting a 3D imaging system. And so we are there as support. We’re also very familiar with the standards development process. So we kind of help guide that as well. But we’re there to help gather data, produce experiments, analyze data, and provide that information to the committee. So we have participation, obviously from not just industry but academia and other government organizations who are interested in these kinds of activities. For example, some of the things we do is develop testbeds as well. So we have developed a bin-picking testbed at NIST, where we have six different bin-picking systems interfacing with two different robots. That way we can then support that standards effort in comparing and figuring out how best to test these systems, what’s fair across different kinds of systems, and not just using one system to develop that test.

David Dechow: [00:08:03] I’m glad you mentioned that because that’s a really interesting part that I found of NIST. You actually have a team that is hands-on with components and with manufacturing systems, and they work with those and learn from that and learn how to apply the standards or how to work with the standards. Isn’t that right? Can you elaborate on that a little more?

Kamel Saidi: [00:08:24] Yes. So we’ve got, under the Measurement Science for Manufacturing Robotics Program, we’ve got six — sorry, seven — different projects going on that deal with different aspects of robotics in manufacturing. And so we’ve got teams working on the safety aspect, working on the human–robot collaboration aspect. What are the interfaces that are needed for those, how well do they work, and so on. Looking at the manipulation and the dexterity of the manipulators and the grippers and so on and others. Agility, for example. In our project, which is looking at the perception aspect of it, we’ve got obviously a team of researchers who are, like you said, hands-on with the robots, with the vision systems, and trying to understand how they work. And then we also look at what other standards obviously are out there and what’s missing. And so a big part of what we do is to try and understand how those standards work. We obtain the standards, we run the standards on the systems that we have. But unfortunately, there aren’t many standards for these applications, and that’s why we’re working on this.

David Dechow: [00:09:37] In closing, could you give our audience and the manufacturing people who are listening your take on how standards in general and the things that you work on every day or work with every day, how those really ultimately benefit them in the manufacturing environment, just as a general thought?

Kamel Saidi: [00:09:57] Yeah, that’s a very good question that we get asked all the time. And sometimes it’s hard to explain the benefits of standards in general, but just think of common things. And this is what I’ll talk about in my talk later today. But imagine if you were going to board a plane and there were no standards for certifying pilots. I wouldn’t board a plane if I knew that pilots were just haphazardly chosen. Or choosing a car with fuel economy. Based on fuel economy, you want something that’s fuel efficient. But imagine if there were no fuel economy standards. And so each manufacturer would tell you, my car works this way or my car is this fuel efficient. You would have a hard time. Those are the kinds of benefits for people in having standards. It’s easier to understand the benefits of standards from the user perspective, just like I explained. From the manufacturer perspective — and when I say manufacturer, I mean the manufacturers of the vision systems, for example in this case — it’s a bit harder, especially when you have manufacturers that are very popular, let’s say, and they have a corner on the market. They say, well, my market share is pretty good, so why do I need standards? I’m doing really well. Eventually they come around because they see how standards are helping the smaller manufacturers of these systems. And when people start asking, when people start understanding how these systems perform and asking the people who are trying to sell them machine vision systems: What do you mean by accuracy? What do you mean by point precision? Is that tested to something that I can understand? Show me the test. And so when they start, when the small manufacturers get on board and start specifying their systems based on standards, the big manufacturers usually come around.

David Dechow: [00:11:55] Wonderful. Wonderful. Well, thank you very much for your time today. And we’ll join you again on another episode of Manufacturing Matters.