Research Summary, Robert Platt Jr.

Grasp Control

Background

A fundamental problem in robot manipulation, known as grasp synthesis, is deciding where to place manipulator contacts in order to be able to grasp and lift an object. A significant body of research addresses this as a planning problem, generally relying on precise geometric and environmental models. My work is based on Coelho and Grupen's force residual and moment residual grasp controllers. The force residual controller calculates a net residual force that would be applied by the contacts if they were to apply equal unit forces and calculates how the force residual would change with small contact displacements. The force residual controller displaces the contacts in directions that minimize the force residual. Similarly, the moment residual controller displaces grasp contacts in directions that minimize the moment residual.

Null Space Composition of Force and Moment Residual Control Laws

I make several contributions to this approach. First, the gradients of the force and moment residual control laws are expressed in object surface coordinates. They are combined by projecting the gradient of the moment residual controller into the null space of the gradient of the force residual controller. This makes the resulting controller more robust to local minima in wrench residual. While there many local minima exist in the moment residual function, the force residual function is unimodal for convex objects. As a result, the composite grasp controller can be shown to converge to quality grasps for arbitrary convex objects. The video below illustrates this approach. The manipulator interactively makes contact with the object, senses local contact information, and displaces the contacts toward configurations with low wrench residual.

Relevant Publications:

Platt, R., Fagg, A. H., Grupen, R. Nullspace Composition of Control Laws for Grasping, In the Proceedings of the 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2002 (pdf: 316KB; ps: 596KB)

Platt, R., Learning and Generalizing Control-Based Grasping and Manipulation Skills, Chapter 4 , Ph.D. Dissertation, Computer Science Department, University of Massachusetts at Amherst, September, 2006.

Sliding Grasp Controller

I also showed that the composite grasp controller can be combined with a hybrid force/position controller inner loop. In this approach, the hybrid force/position controller displaces contacts tangent to the sensed surface normal while continuing to apply a small inward force on the object. The grasp controller executes at a slower loop rate, providing a constant stream of new position references in object coordinates. This approach allows the grasp controller to collect orders of magnitude more contact information than is possible if the robot only probes the object. The resulting controller finds grasps faster and more robustly as illustrated in the video below.

VIDEO: Grasp controller, sliding. (AVI, 6M)

Statically-Stable Dexterous Manipulation

Grasp control has implications beyond grasping. In statically-stable manipulation, the robot transitions through a sequence of stable grasps. The key is devising a method of safely transitioning between grasps without violating a grasp constraint (i.e. without dropping the object). I proposed that these transitions can be realized by combining multiple grasp controllers in the task null space. Given that one grasp is active, my approach executes the grasp controller corresponding to the next grasp in sequence in the task null space of the existing grasp constraint. This approach realizes the new grasp without the need to explicitly characterize the space of manipulator-object configurations. This approach was tested in the context of bi-manual dexterous manipulation of a large beach ball by Dexter, the UMass bimanual humanoid robot, illustrated in the sequence of frames below.

The video below shows Dexter manipulating the beach ball.

VIDEO: Dexterous manipulation of a beach ball. (AVI, 4M)

Relevant Publications:

Platt, R., Fagg, A. H., Grupen, R. Manipulation Gaits: Sequences of Grasp Control Tasks, Proceedings of the 2004 IEEE Conference on Robotics and Automation (ICRA), New Orleans, Louisiana, April 2004 © 2004 IEEE. (pdf: 175KB)

Schema-based Generalization

Feedback controllers such as the grasp controller described above eliminate the need for the robot to plan its motion and force in precise detail. However, it may still be necessary for the robot to change its strategy as a function of problem context. In addition, it may be difficult for a human programmer to specify ahead of time what strategy to use. Learning based on unconstrained policy search is an inadequate solution to this problem because of the large number of possible policies that the robot would need to explore. Rather, I proposed an approach to generalizing control knowledge based intuitively on Piaget's notion of the schema. The key to this approach is defining a criteria for deciding which sub-task solutions are similar to a generalized solution and which are different. Using a framework known as the control basis, I described the set of available controllers as a language generated by a context free grammar. This language gives similar representations to controllers with related functionality and allows a generalized solution (the schema) to be instantiated in a different but related ways. When presented with a new problem, the robot can experiment with different instantiations of the schema to find a solution. I proposed a variant of policy iteration that searches instantiations of the schema for solutions expected to achieve schema objectives. Taken together, grasp control and schema-based learning constitute a comprehensive approach to grasp synthesis and other force domain problems. An important feature of this approach is the way that the task is divided into two parts that use sensor data of different resolutions. In the first part, coarse information is used to select a general strategy. In the second part, precise (but local) force feedback is used to precisely position manipulator contacts.

Bagging Groceries

Figure A. Left: the object to be grasped. Right: the first and second moments of the segmented object that are used to establish grasp context.

The combined approach was demonstrated in the context of bagging groceries. Problem context for the grasp was established based on the first and second moments of the covariance matrix for a ``blob'' segmented from a camera image (Figure A). The set of reach-grasp schema instantiations described a space of relative hand-object poses to which the robot could reach. The system was trained to grasp a small set of grocery items (Figure B) using the coarse-to-fine approach outlined above. For each of the training objects, the robot learned the appropriate relative hand-object pose. Then, performance was tested for a much larger set of everyday grocery items (shown in Figure C) that had not been experienced before and shown to have improved. The result was a dexterous robot capable of grasping and lifting a large set of common household items.

Video of the process of learning to grasp an object is shown below.

VIDEO: The process of learning to grasp a horizontally placed box. (AVI, 20M)

Video of the resulting grocery item grasps is shown below.

VIDEO: Grasping a series of grocery items. (MPG, 15M)

Figure B. The set of grasp training objects.

Figure C. The set of grasp test objects.

Relevant Publications:

Platt, R., Fagg, A. H., Grupen, R., Learning Grasp Context Distinctions that Generalize, IEEE-RAS International Conference on Humanoid Robots, Genoa, Italy, December, 2006

Platt, R., Fagg, A. H., Grupen, R., Improving Grasp Skills Using Schema Structured Learning, International Conference on Development and Learning, Bloomington, Indiana, May, 2006