Computer Science Thesis Oral

Thursday, June 14, 2018 - 9:00am


8102 Gates Hillman Centers



Language-Based Bidirectional Human And Robot Interaction Learning For Mobile Service Robots

We believe that it is essential for robots that coexist with humans to be able to interact with their users in a seamless way. This thesis advocates the use of language as a rich and natural interface for the interaction between robots and humans. We assume that a mobile service robot, such as the CoBot robot, is equipped with domain information of its environment and is able to perform tasks that involve autonomously navigate to desired goal positions. The thesis provides the ability for the robot and the human to interact in natural language, while introducing a novel bidirectional approach for the exchange of commands  and information between a robot and its users.

In the human-to-robot direction of interaction, we assume users provide a high-level specification of what the robot should do. This thesis enables a mobile service robot to understand (1) requests to perform tasks, and (2) questions about the robot experience as stored in its log files. Our approach introduces a dialogue-based learning of groundings of natural language expressions to robot actions and operations. These groundings are learned into knowledge bases accessible by the robot.

In the robot-to-human interaction direction, this thesis enables a robot to match the detail of the explanations it provides to the user request. Moreover, we introduce an approach for a robot to pro-actively report in language on the outcome of a task after finishing its execution. The robot contextualizes information about the task execution by comparing it with its past experience.

In a nutshell, this thesis contributes a novel language-based bidirectional interaction approach for mobile service robots, where robots learn to understand and execute commands and queries by users, and also robots take the initiative to offer information in language to users about their experience. So the language exchange is initiated by the humans and also by the robots.

We evaluate the work both on the actual CoBot robots and on constructed simulated and crowd-sourced data.

Thesis Committee:
Manuela Veloso (Chair)
Jaime Carbonell
Stephanie Rosenthal
Xiaoping Chen (University of Science and Technology of China)

For More Information, Contact:


Thesis Oral