AAAI Publications, Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence

Font Size: 
Learning to Parse and Ground Natural Language Commands to Robots
Jayant Krishnamurthy, Thomas Kollar

Last modified: 2012-07-15


This paper describes a weakly supervised approach for understanding natural language commands to robotic systems. Our approach, called the combinatory grounding graph (CGG), takes as input natural language commands paired with groundings and infers the space of parses that best describe how to ground the natural language command. The command is understood in a compositional way, generating a latent hierarchical parse tree that involves relations (such as "to" or "by") and categories (such as "the elevators" or "the doors"). We show an example parse-grounding tree and show that our system can successfully cluster the meanings of objects and locations.

Full Text: PDF