41 www.drivesncontrols.com April 2025 MACHINE VISION n drawer to be opened. The vision software uses this image to create a coordinate system which it shares with the robot, allowing the robot to open the drawer. In the next step, the 3D camera takes a picture of the contents of the drawer. This is used to determine how many vials are stored and where they are located. The system also checks for errors, such as whether individual ampoules are upright or have tipped over and cannot be picked up. The robot’s gripper can handle 46 vials at a time. Once a drawer has been unloaded, the camera determines the position of the drawer handle, allowing the robot to close it. This is then repeated with the remaining eight drawers until the trolley has been unloaded completely. The application, in which the robot operates in a largely autonomous fashion, presented several challenges. “The development of the robot cell took place under tremendous time pressure,” recalls Stephan Trunk from Goldfuß Engineering. “The demand for the vaccine had to be met as quickly as possible, which also served as an intrinsic motivation for our employees.” The main challenge was to achieve suitably high precision. The position of the vials has to be detected with an accuracy of 0.1mm. With a working area of 800 x 600mm and a depth of 600mm, this is no easy task. The valuable contents of the vials make this requirement even more critical. The vials could not be damaged under any circumstances. However, loading and unloading them had to be done quickly, in part to facilitate the rapid vaccine delivery. “The vision system also faced two special challenges,” Daniel Simon explains. “On the one hand, it was necessary to work with dierent materials. The glass vials and the metal trolleys have transparent or reective surfaces, and are therefore dicult to detect. “The system will only work if the robots can operate autonomously in 3D space,” he continues. “To do this, the machine vision software must include powerful 3D vision technologies. There are not many machine vision software products that achieve the necessary performance and robustness in this regard. Based on our many years of experience with Halcon, we knew that the software has an extensive library with many extremely powerful methods.” Various machine vision technologies are used in the robot cell. One requirement is for “hand-eye calibration”. This is essential for any application in which cameras work together with robots. During this process, the robot’s and camera’s coordinate systems are synchronised. This makes it possible to match the movements of the robot precisely with the camera images. The hand-eye calibration function delivers high accuracy when determining the relative positions of the camera and the robot, thus making it possible to determine the exact positions and orientations of the vials in relation to the robot. This calibration is the basis for all further machine vision operations in the application. The system also uses the stereo vision 3D vision technology in the Halcon software for 3D reconstruction. It also facilitates quality control or position detection of 3D objects, and calculates 3D coordinates on object surfaces using one or more cameras. Stereo vision is particularly well suited for the precise measurement of elevations. One function in the technology is multigrid stereo – an advanced method of interpolating the 3D data in homogeneous parts of the image. This method yields greater accuracy for small objects. “We managed to develop a completely new robot cell and get it up and running stably in just six months,” recalls Stephan Trunk of Goldfuß Engineering. “We are proud to have contributed to accelerating the production of a vaccine against a dangerous virus.” The system has achieved the strict requirements in terms of speed and precision. “This project illustrates all of the potential oered by machine vision,” Daniel Simon concludes. “Thanks to the successful implementation, we are very motivated to automate other challenging tasks. There are a number of promising projects already underway with Goldfuß.” n The vision system is installed in a way that eliminates the need to mount a camera on the robot’s gripper, allowing it to handle up to 46 vials at a time.
RkJQdWJsaXNoZXIy MjQ0NzM=