AAAI Publications, Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence

Font Size: 
Incremental Referent Grounding with NLP-Biased Visual Search
Rehj Cantrell, Evan Krause, Matthias Scheutz, Michael Zillich, Ekaterina Potapova

Last modified: 2012-07-15


Human-robot interaction poses tight timing requirements on visual as well as natural language processing in order to allow for natural human-robot interaction. In particular, humans expect robots to incrementally resolve spoken references to visually perceivable objects as the referents are verbally described. In this paper, we present an integrated robotic architecture with novel incremental vision and natural language processing and demonstrate that incrementally refining attentional focus using linguistic constraints achieves significantly better performance of the vision system compared to non-incremental visual processing.


incremental visual filtering; incremental natural language understanding; referent grounding; visual attention; visual scene understanding; integrated language and perception

Full Text: PDF