GraspIt!: A Versatile Simulator for Robotic Grasping

1. GraspIt!: A Versatile Simulator for Robotic Grasping Andrew T. Miller Columbia University

2. Outline • Motivation • Simulator Overview • Simulator Front-end • Grasp Analysis • Dynamic Simulation • Performing a Grasping Task • Future Work

3. Simulation for Hand Design • Mechanical prototypes are costly • Spend time and money • Evaluate design in simulation • Examine range of motion, finger placement, kinematics, link geometry • Ensure it can grasp desired objects • Change design easily

4. Simulation for Grasp Planning • Integrated grasp analysis • Grasp quality, weak point, force optimization • Visualize grasp from all angles • Perform many grasps quickly • Faster than using a real arm and hand • Build a library of saved grasps • Recall grasp when object is encountered again

5. GraspIt: A Tool for Grasping Research • Library of hands and objects • Intuitive interface allows many grasps to be tested quickly • Visualize grasp wrench space • Quality measures evaluate grasp

6. GraspIt Components Hand/Object Construction User Interface Contact Determination • read object model • read link models • read kinematics • assemble hand • view 3D scene • change hand pose • auto-grip • manually move joints • detect collisions • adjust contact to object surface • find contact area • add friction cones Front end Grasp Analysis Wrench Space Visualization Object Dynamics Back end • compute grasp wrench space • use metrics on space • grasp force optimization • create 3D projections of GWS • compute object motion

7. Hand Construction • The hand kinematics (in D-H notation) specify the transforms between links, and can handle coupled joints. • The 3D link geometries are accurately described in CAD model files. • The format is flexible and can easily model many of the available complex articulated hands.

8. … Header info and palm description ... #-----------------------------------f0--------------------------------------- #number of joints 3 #number of links 3 #Transforms from palm origin to base of the finger #EXAMPLE: #r 90 x (rotation by 90 degrees about x axis) #rr 3.0368729 y (rotation by 3.0368729 radians about y axis) #t 1.92188 134.5 39.0717 (translation in mm) t 25.0 0 -1.0 r 180 y #Joint Descriptions (1 joint per line) #DOFNum theta d a alpha DOFminval DOFmaxval #(joints are ordered from closest to palm outward) #(linear equations are of the form: q#*k+c [no spaces!]) #(coupled joints must come after joints they are coupled to) d0+90 0 50 -90 0 180 d1+5 0 70 0 0 144 d1*0.333333333+40 0 55 0 0 144 #Link Descriptions (1 link per line) #filename material lastJoint #(links are ordered from closest to palm outward) #(lastJoint is the last joint in the chain which can affect this link) link1 plastic 0 link2 plastic 1 link3 plastic 2 #-----------------------------------f1--------------------------------------- etc... Hand Configuration File

9. Hand Library Barrett Rutgers Robonaut Parallel Jaw DLR

10. User Interaction • Inventor 3D viewer • Rotate hand • Translate hand • Auto-grasp • Manipulate individual DOF’s • Simple and intuitive • Direct interaction gives sense of hand’s capabilities

11. Collision Detection • PQP system (Larsen et al.) • Each body is a polygonal soup • Creates bounding volume hierarchies • Fast recursive algorithm tests for overlap, or provides minimum distance

12. Finding Contacts • Binary search until link is < 0.1mm away • Record all triangle pairs within this distance • Compute points of overlap between each pair • Only keep contact points on region boundary • Mark points with red friction cones Search for instant of contact Determine contact region Collision detected

13. Grasp Analysis • Occurs when a contact is formed or broken • Computes space of forces and torques that can be applied by the grasp • Quality measures numerically evaluate grasp • Provides a means to evaluate grasps • Compare grasps of one hand, one object • Compare grasps of many hands, one object • Compare grasps of many hands, across a task specified object set.

14. Wrench Spaces • In 3-space, a wrench is a 6D vector composed of a force and a torque: • The space of wrenches that may need to be applied during a task is the task wrench space. • The space of wrenches that can be applied by a grasp is the grasp wrench space. • A possible quality measure:

15. Special Types of Grasps • A force-closure grasp completely restrains the object. • Origin is contained within grasp wrench space. • A manipulable grasp can impart arbitrary velocities on the object without breaking contact.

16. Friction Cones • Friction at a contact point allows forces in directions other than the contact normal • COF, m, is determined by the contacting materials • Estimate friction cone as convex sum of a force vectors on the boundary assuming a unit normal force, .

17. Contact Wrenches • Each force acts at a position on object. • Compute the corresponding wrench with respect to object’s center of gravity: • l- torque scale factor, i: contact number j: (1-8) around cone boundary

18. Grasp Wrench Space • Objective: find total space of wrenches that can be applied by a grasp of unit magnitude. • Grasp vector: • Define with norm: • Sum magnitude of contact normal forces is 1. • Compute grasp wrench space using qhull:

19. Two Measures of Quality • Assume task wrench space is unknown. • Estimate with wrench space ball - good grasps will resist all wrenches equally well. • Previously proposed measures of quality: • Radius, e, of the largest wrench space ball that can fit within the unit grasp wrench space. • Volume, v, of unit grasp wrench space.

20. Grasp Wrench Space Projections • To visualize the 6D grasp wrench space project it to 3-space by fixing three coordinates. • A useful choice: • See the forces that can be applied without applying a net torque (or vice versa).

21. Visualizing the Results Numeric quality & Worst case indicator Wrench Space Projection

22. Comparing Grasps

23. Comparing Grasps

24. Comparing Grasps

25. Comparing Grasps

26. Grasp Force Optimization • Given a specific wrench to resist, find the optimal force to apply at contact points • Forces should obey torque limit constraints and avoid friction cone boundaries • Previously done by using linearized friction cones and linear programming [Kerr & Roth ’86] • overly conservative

27. Grasp Force Optimization • GraspIt implements LMI technique by Han et al. ’00, • Does not require friction cone approximation • Feasible Set: • Minimize: • Determinant Maximization with LMI constraints • Using hand Jacobian, it can also compute optimal joint torque values Weighting factor Contact forces Inverse friction cone LMI

28. Grasp Force Optimization

29. Simulating Object Dynamics • Added realism • Can examine grasp formation process • Formulate non-penetration and friction cone constraints as a Linear Complementarity Problem [Stewart & Trinkle ’00] • Lemke’s algorithm solves LCP to find contact forces that satisfy constraints. force OR acceleration must be 0

30. Simulating Object Dynamics

31. Simulating Object Dynamics

32. Connecting GraspIt to theReal World Object Pose Estimation Frame Grabber Robot/Hand Control Grasp Execution Task Monitoring Presented at ICRA ‘01

33. Environment Modeling • Easy to import environmental objects like workbench. • Puma arm with inverse kinematics module also supplied. • Prevents user from planning infeasible grasps.

34. Real-Time Vision System • Supply GraspIt with object pose. • Needs to track object that is partially occluded by the hand. • Since we have an object model, why not use model-based pose estimation and tracking? • Single camera located 2m from workspace, calibrated wrt robot coordinate system.

35. Camera Parameters 3D Projection Matrix Render Model Fitting 3D Model Locate Visible Edges Manual Initialization POSE Normal Flow Affine Motion Real-Time Pose Estimationand Tracking • Position estimation based on Dementhon and Davis. • Tracking based on Araujo and Brown’s extension of Lowe.

36. Tracking System Example Wireframe model Selected points are used to determine pose Pose estimated, and model overlaid Normal flow estimation

37. Grasp Execution Experiments Pose estimated User plans grasp Grasp performed

38. Grasp Execution Experiments Pose estimated User plans grasp Grasp performed

39. Grasp Execution Experiments Pose estimated User plans grasp Grasp performed

40. Grasping and Visual Servoing Once pose has been estimated, grasp may be planned, then executed.

41. Grasping and Visual Servoing Destination planned in GraspIt. Goal position is projected into image. Image based servoing drives robot to goal.

42. Planning a Grasping Task in GraspIt

43. Planning a Grasping Task in GraspIt

44. Planning a Grasping Task in GraspIt

45. Contributions • Grasping simulator • Flexible method to specify hand construction • Interactive interface • Contact determination system • Inclusion of material properties • Integrated grasp analysis theory into simulation system • GWS, quality measures, GFO • Visualization techniques • Object Dynamics: First implementation of S-T algorithm with large scale and interactive bodies • Integration of simulation system into grasping task • A useful tool for research in grasping and hand design: • Currently being used by a number of research groups for design and synthesis: NASA, Sandia, Rutgers, U of MD

46. Future Work • Evaluate manipulable grasps • Provide method of specifying task wrench space • Human hand model • Full dynamic simulation • Automatic grasp selection • Combine high-level and low-level synthesis techniques • Learning techniques