A visual servoing system for interactive human robot object transfer

This work focuses on the main problems of interactive object transfer between a human worker and an industrial robot: the recognition of the object with partial occlusion by barriers including the hand to the human worker, the evaluation of object grasping affordance, and coping with inaccessible grasping points. The proposed visual servoing system integrates different vision modules where each module encapsulates a number of visual algorithms responsible for visual servoing control in humanrobot collaboration.
Nội dung trích xuất từ tài liệu:
A visual servoing system for interactive human robot object transferJournal of Automation and Control Engineering Vol. 3, No. 4, August 2015A Visual Servoing System for InteractiveHuman-Robot Object TransferYing Wang, Daniel Ewert, Rene Vossen, and Sabina JeschkeInstitute Cluster IMA/ZLW & IfU, RWTH Aachen University, Aachen, GermanyEmail: {ying.wang, daniel.ewert, rene.vossen, sabina.jeschke}@ima-zlw-ifu.rwth-aachen.debatches efficiently, it is desirable to combine theadvantages of human adaptability with robotic exactnessand efficiency. Such close cooperation has not yet beenpossible because of the high risk of endangerment causedby conventional industrial robots. In consequence, robotand human work areas had strictly been separated andfenced off. To enable a closer cooperation, robotmanufacturers now develop lightweight robots for safeinteraction. The light-weight design permits mobility atlow power consumption, introduces additionalmechanical compliance to the joints and applies sensorredundancy, in order to ensure the safety of humans incase of robot failure. These robots allow for seamlessintegration of the work areas of human workers androbots and therefore enable new ways of human-robotcooperation and interaction. Here, the vision is to havehuman and robot workers work side by side andcollaborate as intuitively as human workers would amongthemselves [1]-[4].Among all forms of human-robot cooperation,interactive object transfer is one of the most common andfundamental tasks and it is also a very complex and thusdifficult one. One major problem for robotic visionsystems is visual occlusion, as it dramatically lowers thechance to recognize the target out of a group of objectsand then perform successive manipulations on the target.Even without any occlusion, objects in a special positionand orientation or close to a human, make it difficult forthe robot to find accessible grasping points. Besides, inthe case of multiple available grasping points, the robot isconfronted with the challenge of deciding on a feasiblegrasping strategy. When passing the object to the humancoworker, the robot has to deal with the tough case ofoffering good grasping options for the human partner.A visual servoing system is proposed to address theabove-mentioned concerns in human-robot cooperation.Our work considers an interaction task where the robotand human hand over objects between themselves.Situational awareness will be greatly increased by thevision system, which allows for the prediction of thework area occupation and the subsequent limitation ofrobotic movements in order to protect the human bodyand the robotic structure from collisions. Meanwhile, thevisual servoing control enhances the abilities of roboticsystems to deal with the unknown changing surroundingsand unpredictable human activities.Abstract—As the demand for close cooperation betweenhuman and robots grows, robot manufacturers develop newlightweight robots, which allow for direct human-robotinteraction without endangering the human worker.However, enabling direct and intuitive interaction betweenrobots and human workers is still challenging in manyaspects, due to the nondeterministic nature of humanbehavior. This work focuses on the main problems ofinteractive object transfer between a human worker and anindustrial robot: the recognition of the object with partialocclusion by barriers including the hand to the humanworker, the evaluation of object grasping affordance, andcoping with inaccessible grasping points. The proposedvisual servoing system integrates different vision moduleswhere each module encapsulates a number of visualalgorithms responsible for visual servoing control in humanrobot collaboration. The goal is to extract high-levelinformation of a visual event from a dynamic scene forrecognition and manipulation. The system consists ofseveral modules as sensor fusion, calibration, visualization,pose estimation, object tracking, classification, graspingplanning and feedback processing. The general architectureand main approaches are presented as well as the futuredevelopments planned.Index Terms—visual servoing, human-robot interaction,object grasping, visual occlusionI.INTRODUCTIONRobots are a crucial part of nowadays industrialproduction with applications including e.g. sorting,manufacturing as well as quality control. The afflictedprocesses gain efficiency owing to the working speed anddurability of robotic systems, whereas product quality isincreased by the exactness and repeatability of roboticactions. However, current industrial robots lack thecapability to quickly adapt to new tasks or improvisewhen facing unforeseen situations, but must beprogrammed and equipped for each new task withconsiderable expenditure. Human workers, on the otherhand, quickly adapt to new tasks and can deal withuncertainties due to their advanced situational awarenessand dexterity.Current production faces a trend towards shorterproduct life cycles and a rising demand for individualizedand variant-rich products. To be able to produce smallManuscript received July 1, 2014; revised September 15, 2014.©2015 Engineering and Technology Publishingdoi: 10.12720/joace.3.4.277-283277Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015[5]. A few decades ago, technological limitations (theabsenceofpowerfulprocessorsandtheunderdevelopment of digital electronics) failed someearly works in meeting the strict definition of visualservoing. Traditionally, visual sensing and manipulationare combined in an open-loop fashion: first acquireinformation of the target, and then act accordingly. Theaccuracy of operation depends directly on the accuracy ofthe visual sensor, the manipulator and its controller. Theintroduction of a visual-feedback control loop serves asan alternative to increas ...