Challenges to decoding the intention behind natural instruction

Raquel Torres Peralta, Tasneem Kaochar, Ian R. Fasel, Clayton T. Morrison, Thomas J. Walsh, Paul R. Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Currently, most systems for human-robot teaching allow only one mode of teacher-student interaction (e.g., teaching by demonstration or feedback), and teaching episodes have to be carefully set-up by an expert. To understand how we might integrate multiple, interleaved forms of human instruction into a robot learner, we performed a behavioral study in which 44 untrained humans were allowed to freely mix interaction modes to teach a simulated robot (secretly controlled by a human) a complex task. Analysis of transcripts showed that human teachers often give instructions that are nontrivial to interpret and not easily translated into a form useable by machine learning algorithms. In particular, humans often use implicit instructions, fail to clearly indicate the boundaries of procedures, and tightly interleave testing, feedback, and new instruction. In this paper, we detail these teaching patterns and discuss the challenges they pose to automatic teaching interpretation as well as the machine-learning algorithms that must ultimately process these instructions. We highlight the challenges by demonstrating the difficulties of an initial automatic teacher interpretation system.

Original languageEnglish (US)
Title of host publication2011 RO-MAN - 20th IEEE International Symposium on Robot and Human Interactive Communication, Symposium Digest
Pages113-118
Number of pages6
DOIs
StatePublished - Sep 26 2011
Event20th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2011 - Atlanta, GA, United States
Duration: Jul 31 2011Aug 3 2011

Publication series

NameProceedings - IEEE International Workshop on Robot and Human Interactive Communication

Other

Other20th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2011
CountryUnited States
CityAtlanta, GA
Period7/31/118/3/11

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Challenges to decoding the intention behind natural instruction'. Together they form a unique fingerprint.

  • Cite this

    Torres Peralta, R., Kaochar, T., Fasel, I. R., Morrison, C. T., Walsh, T. J., & Cohen, P. R. (2011). Challenges to decoding the intention behind natural instruction. In 2011 RO-MAN - 20th IEEE International Symposium on Robot and Human Interactive Communication, Symposium Digest (pp. 113-118). [6005273] (Proceedings - IEEE International Workshop on Robot and Human Interactive Communication). https://doi.org/10.1109/ROMAN.2011.6005273