Companies are increasingly producing goods at different locations, working with multiple supply companies. There is no guarantee that all the components they receive are labeled with barcodes or type plates, which means that objects often need to be recategorized in the receiving area. Employees then have to manually search the catalog for similar parts to unambiguously identify them for further logistics processing – a painstaking and time-consuming task. Automated, digitalized recognition would be helpful to speed up the process. Researchers at Fraunhofer IPK in Berlin are now working on this task – applying machine learning methods. They use what are known as convolutional neural networks (CNNs) to recognize manufactured components, such as screws, clamps, nozzles, pipes, tubes and cables, as well as microcontrollers and other electronics.
Generating training data for industrial processes
“CNNs have become the standard in image processing. Reliably recognizing, say, 1000 everyday objects requires one million images for these networks to use as training data. Our job was to generate an algorithm for industrial applications even with little data so that, in our case, components with no code can be recognized automatically, easing the burden on receiving staff. Our goal is to enable the algorithm to easily distinguish even highly similar objects from each other, such as screws of the same standard but different sizes, or turbochargers from different production series,” says Jan Lehr, a
researcher at Fraunhofer IPK. “We use specially developed algorithms to limit the search radius to five or ten objects, so employees no longer have to search through the entire range typically found in a large warehouse.”
To realize this, Lehr and his colleagues, working on a project funded by the German Federal Ministry of Economics and Energy (BMWi), developed a detection system called Logic.Cube. The objects to be recognized, which have a maximum edge length of 40 centimeters, are placed in the cube-shaped device with an integrated scale and are photographed by up to nine cameras. An image processing algorithm measures the objects’ height, width and length in order to calculate what size box or shelf space is needed. At the same time, the resulting image set is stored in a database, together with the material number. This image data is used to train the AI algorithm to enable it to recognize a wide array of different components.
High recognition rates with few images
Since not every company will consider purchasing the Logic.Cube, the Fraunhofer IPK research team ported the detection system’s functionality to a browser-based, operating system-independent app that works on smartphones, tablets, laptops and desktop computers. To do this, they had to expand the training dataset to include smartphone data and retrain the algorithm. “We tested the algorithms with a hundred components that had been photographed in a wide variety of scenes, with 50 images taken of each part. Within seconds, the app shows the user five or fewer potential matches, independent of lighting, background and scenery. The recognition is so robust that it can replace manual searching,” says the engineer. “This saves workers an incredible amount of time. We managed to achieve high recognition rates with a minimum of images.” The researchers reached recognition rates of 98 percent in Logic.Cube and reduced the search radius from 4500 images to five. They aim to get the same success rate with the app.
The images will be uploaded over the internet or the company’s intranet and stored in a local edge cloud, which is also where the actual image processing and recognition will take place. “The AI algorithms run on the server, and the smartphone or tablet app is the client,” explains Lehr. The overall system is designed in such a way that, when used, it continuously collects additional data. After a certain amount of time, this new data can be used to retrain the algorithms, resulting in a continuously self-improving system.
Currently the researchers are working to expand the set of image data and to digitalize the catalog and integrate it into the app. It should soon be possible to initiate all subsequent processes, such as further processing and potential reorders, through the edge cloud. Lehr and his colleagues are also optimizing the algorithms to enable recognition of even highly similar-looking objects. In the test runs they conducted, the system correctly recognized even screws of the same standard but different sizes.
Application scenario: surface inspection
The app can also be used for surface inspection. AI-based image processing is used to examine the surfaces of incoming components as soon as they are received. It shows whether parts have any scratches or corrosion, or whether they were damaged during transport or are just a bit dirty. AI methods are used to mark the relevant spots on the image so workers can immediately check the damaged areas. “Manufacturing companies are often still cautious when it comes to AI. We would be very pleased if our research work could help counter this skepticism and boost acceptance among employees,” says Lehr.
Fraunhofer-Gesellschaft
