Robots brain teaches machines to retrieve objects they have not seen before

CSAIL says that kPAM or Keypoint Affordance Manipulation is more accurate than other similar technologies. After discovering all the coordinates of an object, it determines what it can do with it. For example, if it sees a mug with a handle, it can hang it on a hook by the handle. If it determines that it looks on a pair of shoes, it can place the shoes on a tripod. "Understanding a little more about the object ̵[ads1]1; the location of a few key points – is enough to activate a wide range of useful manipulation tasks. And this particular representation works magically well with today's state-of-the-art machine learning perception and planning algorithms," said the study's senior author, MIT Professor Russ Tedrake

The researchers hope to develop the system further to machines powered by kPAM can do larger tasks, such as emptying the dishwasher and wiping down the kitchen. They also hope that the system can manufacture factory robots in a larger manipulation machine in the future.