Robotic Grasping: Planning and Manipulation Systems

1. Robotic Grasp Planning Peter Allen Department of Computer Science Columbia University

2. Robotic grasping is a complex field • hand design: high level (number of fingers, kinematic structure, etc.) and low-level (mechanism design, motors, materials, etc.); • hand control algorithms: high level (find an appropriate posture for a given task) and low-level (execute the desired posture); • information from sensors (tactile, vision, range sensing, etc.); • any pre-existing knowledge of objects shape, semantics and tasks (e.g. a cup is likely to be found on a table, should not be held upside-down, etc.); • all of these add up to a Grasp Planning System...and more!

3. Human Grasping vs. Robotic Grasping • Human performance provides both a benchmark to compare against, and a working example that we can attempt to learn from. However, it has proven very elusive to replicate: • the human hand is a very complex piece of equipment, with amazing capabilities; • humans benefit from an unmatched combination of visual and tactile sensing; • human continuously practice grasping and manipulation, the amount of data they are exposed to dwarfs anything tried so far in robotics; • are we setting the bar too high\ • Speed stacking!

4. Robotic Manipulation • Process allowing a robot to make physical changes to the world around it. • Includes moving objects, joining objects, reshaping objects, etc. • Moving objects can be done by grasping, pushing, carrying, dropping, throwing, etc. • Task accomplished by a manipulator with some sort of end-effecter.

5. Grippers vs. Hands • Structured environments • Reliable • Simple • Low cost • Unstructured environments • Adaptable • Complex • Expensive Suction Magnet Parallel Jaw Barrett Utah/MIT Robonaut

7. Simulation for Grasp Planning • Integrated grasp analysis • Grasp quality, weak point, force optimization • Perform many grasps quickly • Faster than using a real arm and hand • Build a library of saved grasps • Recall grasp when object is encountered again

8. GraspIt!: A Tool for Grasping Research • Library of hands and objects • Intuitive user interface • Visualize grasp wrench space • Quality measures evaluate grasp • Dynamic simulation • Grasp Planning

9. GraspIt: A Tool for Grasping Research • Library of hands and objects • Intuitive user interface • Visualize grasp wrench space • Quality measures evaluate grasp • Dynamic simulation • Grasp Planning

10. GraspIt Components World Construction User Interface Contact Determination • read object models • read link models • read kinematics • assemble robots • view 3D scene • change hand pose • auto-grip • manually move joints • Matlab interface • detect collisions • adjust contact to object surface • find contact area • add friction cones Grasp Analysis Wrench Space Visualization Rigid Body Dynamics • compute grasp wrench space • use metrics on space • grasp force optimization • create projections of GWS • worst case disturbance • compute object motions Grasp Planner Control Algorithms • generate and test grasps • PD controllers

11. Hand Construction • The hand kinematics (in D-H notation) specify the transforms between links, and can handle coupled joints. • The 3D link geometries are accurately described in CAD model files. • The format is flexible and can easily model many of the available complex articulated hands.

12. Hand Kinematics

13. Hand/Arm Library Barrett Rutgers Robonaut Parallel Jaw DLR Puma Paloma

14. Grasp Analysis • Occurs when a contact is formed or broken • Computes space of forces and torques that can be applied by the grasp • Quality measures numerically evaluate grasp • Provides a means to evaluate grasps • Compare grasps of one hand, one object • Compare grasps of many hands, one object • Compare grasps of many hands, across a task specified object set.

15. Wrench Spaces • In 3-space, a wrench is a 6D vector composed of a force and a torque: • The space of wrenches that may need to be applied during a task is the task wrench space. • The space of wrenches that can be applied by a grasp is the grasp wrench space. • A possible quality measure:

16. Special Types of Grasps • A force-closure grasp completely restrains the object. • Origin is contained within grasp wrench space. • A manipulable grasp can impart arbitrary velocities on the object without breaking contact.

17. Friction Cones • Friction at a contact point allows forces in directions other than the contact normal • COF, m, is determined by the contacting materials • Estimate friction cone as convex sum of a force vectors on the boundary assuming a unit normal force, .

18. 2D Example Object: square lamina Grasp: 3 point contacts with a COF of 0.3 In 2D, friction cones do not need to be approximated. y x

19. f1,1 d1 dmax Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x t i: contact number j: (1-8) around cone boundary fy fx

20. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

21. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

22. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

23. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

24. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

25. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

26. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

27. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

28. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

29. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

30. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

31. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

32. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

33. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

34. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

35. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

36. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

37. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

38. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

39. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

40. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

41. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

42. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

43. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, y x i: contact number j: (1-8) around cone boundary

44. Grasp Wrench Space • Objective: find total space of wrenches that can be applied by a grasp of unit magnitude. • Grasp vector: • Define with norm: • Sum magnitude of contact normal forces is 1. • Compute grasp wrench space using qhull:

45. Ball radius e: 0.201 Hull volume v: 0.902 Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space.

46. Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space. Ball radius e: 0.201 Hull volume v: 0.902

47. Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space. Ball radius e: 0.201 Hull volume v: 0.902

48. Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space. Ball radius e: 0.201 Hull volume v: 0.902

49. Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space. Ball radius e: 0.201 Hull volume v: 0.902

50. Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space. Ball radius e: 0.201 Hull volume v: 0.902