Drives & Controls April 2022

34 n MACHINE VISION April 2022 www.drivesncontrols.com 3D camera guides AGV to move reels of yarn A Bavarian company, Ontec Automation, has developed a mobile robotic system for picking up and placing heavy loads. The company’s autonomous Smart Robot Assistant combines an AGV (automatic guided vehicle) with a flexible robot arm and a 3D stereo camera system. It can lift Euro pallets in container or industrial formats, as well as mesh pallets in various sizes, and has a maximum load capacity of 1.2 tonnes. A customer in the textile industry is using the system to load coil creels automatically. The AGV picks up pallets with yarn spools, transports them to a designated creel and loads it for further processing. Using a specially developed gripper, it can pick up as many as 1,000 yarn packages during an eight-hour shift, and push them onto creel mandrels. A 3D camera on the gripper arm identifies the sizes and positions of the coils. The pallets loaded with industrial yarn spools are picked up from the floor and transported to the creel location. There, the gripper positions itself above the pallet. An image trigger is sent to the camera using software developed by Ontec that communicates with the vehicle’s PLC and can read out and pass on data. The software controls communications between the vehicle, gripper and camera. The camera knows when the vehicle and the grabber are in the right position to take a picture. A point cloud of the image reports the coordinates of the coils on the pallet to the robot. The robot can then pick up the coils precisely and process them. When a layer of the yarn spools has been cleared, the camera takes a picture of the packaging material between the yarn spools and provides a point cloud of this as well. The data is processed to provide the robot with information to guide a needle gripper to remove the intermediate layers. “This approach means that the number of layers and finishing patterns of the pallets do not have to be defined in advance and even incomplete pallets can be processed without any problems,”explains Ontec software developer, Tim Böckel.“The gripper does not have to be converted to use the needle gripper. There is a normal gripping component for the coils, and a needle gripping component for the intermediate layers.” Ontec chose an IDS Ensenso N45 3D camera with a 1.3MP sensor and Gigabit Ethernet interface for the application. The compact camera’s stereo electronics are decoupled from the housing, allowing the use of a lightweight plastic composite. “Challenges with this application can be found primarily in the different lighting conditions in different rooms of the hall and at different times of the day,”Böckel explains. In difficult conditions, a high-contrast texture is projected onto the object being imaged using a pattern mask with a random dot pattern. This supplements structures on featureless surfaces. Parameters such as exposure time, binning, and depth measuring range can be adjusted live, allowing the camera to recognise a large number of pixels, including any position changes, using the auxiliary structures projected onto the surface, and thus to create complete depth information of the scene. This ensures the necessary precision. “The camera only takes one image pair of the entire pallet to achieve a faster throughput time, but it has to provide the coordinates from a relatively large distance with an accuracy in the millimetre range to enable the robot arm to grip precisely,”explains Matthias Hofmann, an Ontec IT specialist for application development.“We therefore need the high resolution of the camera to be able to safely record the edges of the coils with the 3D camera.”The localisation of the edges is important to be able to gauge the position from the centre of the spool to the gripper as accurately as possible. Further development of the system is planned, including among other things, using cameras to help the AGV to navigate. Ontec is evaluating using the cameras to allow the vehicle to react more flexibly to obstacles – for example, to classify them and drive around them. n A German company has developed a mobile robotic transport system for a textile application that requires reels of yarn to be collected and placed on creels. The process is guided by a 3D camera that identifies the sizes and positions of the coils. The Ontec AGV uses a vision-guided robot arm to pick up pallets with yarn spools, transport them to a creel and load them for further processing