Understanding Natural Language Commands

Natural language is an intuitive and flexible modality for human-robot interaction. A robot designed to interact naturally with humans must be able to understand instructions without requiring the person to speak in any special way. We are building systems that robustly understand natural language commands produced by untrained users. We have applied our work to understanding spatial language commands for a robotic wheelchair, a robotic forklift, as well as a micro-air vehicle. More information is at http://spatial.csail.mit.edu.

Publications

Thomas Kollar, Stefanie Tellex, Deb Roy and Nicholas Roy. “Grounding Verbs of Motion in Natural Language Commands to Robots”, International Symposium on Experimental Robotics (ISER), New Delhi, India, Dec. 2010. [PDF]
Stefanie Tellex, Thomas Kollar, George Shaw, Nicholas Roy, and Deb Roy. “Grounding Spatial Language for Video Search,” Proceedings of the Twelfth International Conference on Multimodal Interfaces (ICMI), 2010. (Winner, Best Student Paper award.) [PDF]
Thomas Kollar, Stefanie Tellex, Deb Roy and Nick Roy, “Toward understanding natural language directions,” Human-Robot Interaction 2010. [PDF]

Understanding Natural Language Commands

Publications

People

Stefanie Tellex

Tom Kollar