Comfortable Robot to Human Object Hand-Over

1. UNIVERSITY OF PARMA, ITALY Comfortable Robot to Human Object Hand-Over Jacopo Aleotti aleotti@ce.unipr.it Vincenzo Micelli micelli@ce.unipr.it Stefano Caselli caselli@ce.unipr.it

2. Outline • Introduction and motivation • Related work • Proposed approach: • 3D object modeling and segmentation from range data • Human detection and robot hand-over planning • Detection of contact between human hand and object • Experiments • Conclusions

3. Introduction and motivation • object hand-over: humans usually offer objects taking into account • etiquette factors, such as object usage or habits • socially acceptable robots should behave in a comfortable way for humans • novel approach for direct robot to human hand-over: • object is delivered so that the most appropriate part • is oriented towards the person

4. Relatedwork • user comfort in object delivery tasks • [Sisbot et al., 2007, 2010, 2011] • Human Aware hand-over, optimization visibility, • safety and comfort parameters [Young SangChoiet al., 2009] ‘’Hand It Over or Set It Down: A User Study of Object Delivery with an Assistive Mobile Manipulator’’, direct object delivery must consider posture and body size of the receiver. [Kimet al., 2004] ‘’Three Handover Methods in Esteem Etiquettes Using Dual Arms and Hands of Home-Service Robot’’, pure simulated environment on simple convex objects

5. System setup • ComauSMART SiX(6 dofs) • 2-finger gripper (Schunk PG-70) • eye-in-hand planar laser • (Sick LMS 400) • estimated accuracy 1.5cm • fixed range sensor (Kinect) • Assumptions: • one object in the scene • complete scan available

6. Proposedmethod 3D range data acquisition, objectmodeling and segmentation Human detection • Robot hand-over planning Task execution and detection of contact

7. 3D modeling and segmentation • performed offline (when the user has not yet triggered the hand-over task) • time consuming (robot operates slowly to achieve accurate reconstruction) • range data are acquired by moving the robot arm along a pre-computed path • range data are stored in a point cloud in the robot reference frame

8. 3D modeling and segmentation point cloud is filtered by removing noisy data (statistical outliers) points of the dominant plane are removed through sample consensus remaining points constitute the object cluster point cloud cluster is triangulated (Power Crust+Poisson) the reconstructed mesh is segmented into parts (Reeb Graph) hammer jug

9. 3D modeling and segmentation • Reeb graphs • A Reeb graph is a data structure describing the evolution of a scalar function over a mesh. • Topologicalchanges of connected components are encoded in the nodesof the graph. • given a surface S and a real, continuous function f: S → R • the Reebgraph is the quotient space of f in S×R by • the equivalence relation • (X1, f(X1))~(X2, f(X2)) which holds if and only if • f(X1) = f(X2) and if the two points X1 and X2 are in • the same connected component of f -1(f(X1)) • the Reeb graph depends on • the scalar function f • the number of quantization levels of f • f is the integralgeodesicdistance • (computationallyexpensive, invariantto rotation)

10. Human detection • hand-over task startswhen the person approaches the robot • fixed Kinect sensor performs body tracking (20-joint body) • user can assume an arbitrary position in front of the robot • after standing still for 4s, body position, • orientation and height are computed • average errors: • body position 7cm • body orientation  10◦ • acceptable for a hand-over task

11. Human detection Asimulationenvironmentisgenerated for robot hand-over planning (object, robot, human mode, environment)

12. Robot hand-over planning • robot to human hand-over task is planned in the simulated environment • based on the OpenRAVEengine • Two steps: • 1) Grasp planning • Randomizedofflinealgorithm for robot graspsynthesis • samplingforce-closuregrasps on all the segmented parts of the object but the target part • samplingaround the principalaxis of inertia of the part • leaves the user room for grasping the object by the target part • pre-grasps configurations are saved • in a grasp map

13. Robot hand-over planning 2) Robot motion planning • object is delivered so that the target part (e.g. handle) is oriented towards the user • goal configuration depends on both the position and the orientation of the user • object centroid is placed at the same height of the chest vp

14. Robot hand-over planning 2) Robot motion planning • object is delivered so that the target part (e.g. handle) is oriented towards the user • goal configuration depends on both the position and the orientation of the user • object centroid is placed at the same height of the torso vp

15. Robot hand-over planning 2) Robot motion planning • object is delivered so that the target part (e.g. handle) is oriented towards the user • goal configuration depends on both the position and the orientation of the user • object centroid is placed at the same height of the torso

16. Robot hand-over planning 2) Robot motion planning • object is delivered so that the target part (e.g. handle) is oriented towards the user • goal configuration depends on both the position and the orientation of the user • object centroid is placed at the same height of the torso

17. Experiments • hammer is grasped by the robot from the head • object is delivered so that the handle is oriented towards the user

18. Detection of contact • after touching the object for 2s the robot gripper is opened • users are free to use whatever hand they are more comfortable with

19. Experiments • USER STUDY 16 subjects (12 males and 4 females, 25 ± 3.3 years, 80% right handed) • each subject performed two trials with the same object (proposed|perturbed) • no practice session, no information about how the object will be delivered • object (hammer, jug, blow torch) was randomly chosen by the moderator • the perturbed case forced the users to grasp the object in an unnatural way • proposed approach was judged as: • more comfortable, with significant difference (Wilcoxon signed-rank test, p < 0.005) • more safe, with significant difference (Wilcoxon signed-rank test, p < 0.01)

20. Experiments • average time: the online phase of the task requires about 44s • in the perturbed approach the user takes twice the time to take the object

21. Conclusions • A complete system for robot to human hand-over • The approach facilitates user’s comfort when receiving the object • The appropriate part of the object is oriented towards the user • Topological shape segmentation based on Reeb graphs • User study confirmed that the user feels that • objects are delivered in a comfortable way • that complies with objects affordances. • Ongoing work: environments with multiple objects