Challenges to decoding the intention behind natural instruction

Raquel Torres Peralta, Tasneem Kaochar, Ian R. Fasel, Clayton T Morrison, Thomas J. Walsh, Paul R Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Currently, most systems for human-robot teaching allow only one mode of teacher-student interaction (e.g., teaching by demonstration or feedback), and teaching episodes have to be carefully set-up by an expert. To understand how we might integrate multiple, interleaved forms of human instruction into a robot learner, we performed a behavioral study in which 44 untrained humans were allowed to freely mix interaction modes to teach a simulated robot (secretly controlled by a human) a complex task. Analysis of transcripts showed that human teachers often give instructions that are nontrivial to interpret and not easily translated into a form useable by machine learning algorithms. In particular, humans often use implicit instructions, fail to clearly indicate the boundaries of procedures, and tightly interleave testing, feedback, and new instruction. In this paper, we detail these teaching patterns and discuss the challenges they pose to automatic teaching interpretation as well as the machine-learning algorithms that must ultimately process these instructions. We highlight the challenges by demonstrating the difficulties of an initial automatic teacher interpretation system.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE International Workshop on Robot and Human Interactive Communication
Pages113-118
Number of pages6
DOIs
StatePublished - 2011
Event20th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2011 - Atlanta, GA, United States
Duration: Jul 31 2011Aug 3 2011

Other

Other20th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2011
CountryUnited States
CityAtlanta, GA
Period7/31/118/3/11

Fingerprint

Decoding
Teaching
Robots
Learning algorithms
Learning systems
Feedback
Demonstrations
Students
Testing

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Human-Computer Interaction

Cite this

Torres Peralta, R., Kaochar, T., Fasel, I. R., Morrison, C. T., Walsh, T. J., & Cohen, P. R. (2011). Challenges to decoding the intention behind natural instruction. In Proceedings - IEEE International Workshop on Robot and Human Interactive Communication (pp. 113-118). [6005273] https://doi.org/10.1109/ROMAN.2011.6005273

Challenges to decoding the intention behind natural instruction. / Torres Peralta, Raquel; Kaochar, Tasneem; Fasel, Ian R.; Morrison, Clayton T; Walsh, Thomas J.; Cohen, Paul R.

Proceedings - IEEE International Workshop on Robot and Human Interactive Communication. 2011. p. 113-118 6005273.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Torres Peralta, R, Kaochar, T, Fasel, IR, Morrison, CT, Walsh, TJ & Cohen, PR 2011, Challenges to decoding the intention behind natural instruction. in Proceedings - IEEE International Workshop on Robot and Human Interactive Communication., 6005273, pp. 113-118, 20th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2011, Atlanta, GA, United States, 7/31/11. https://doi.org/10.1109/ROMAN.2011.6005273
Torres Peralta R, Kaochar T, Fasel IR, Morrison CT, Walsh TJ, Cohen PR. Challenges to decoding the intention behind natural instruction. In Proceedings - IEEE International Workshop on Robot and Human Interactive Communication. 2011. p. 113-118. 6005273 https://doi.org/10.1109/ROMAN.2011.6005273
Torres Peralta, Raquel ; Kaochar, Tasneem ; Fasel, Ian R. ; Morrison, Clayton T ; Walsh, Thomas J. ; Cohen, Paul R. / Challenges to decoding the intention behind natural instruction. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication. 2011. pp. 113-118
@inproceedings{b170827aada343dcab8f70d9c3683c28,
title = "Challenges to decoding the intention behind natural instruction",
abstract = "Currently, most systems for human-robot teaching allow only one mode of teacher-student interaction (e.g., teaching by demonstration or feedback), and teaching episodes have to be carefully set-up by an expert. To understand how we might integrate multiple, interleaved forms of human instruction into a robot learner, we performed a behavioral study in which 44 untrained humans were allowed to freely mix interaction modes to teach a simulated robot (secretly controlled by a human) a complex task. Analysis of transcripts showed that human teachers often give instructions that are nontrivial to interpret and not easily translated into a form useable by machine learning algorithms. In particular, humans often use implicit instructions, fail to clearly indicate the boundaries of procedures, and tightly interleave testing, feedback, and new instruction. In this paper, we detail these teaching patterns and discuss the challenges they pose to automatic teaching interpretation as well as the machine-learning algorithms that must ultimately process these instructions. We highlight the challenges by demonstrating the difficulties of an initial automatic teacher interpretation system.",
author = "{Torres Peralta}, Raquel and Tasneem Kaochar and Fasel, {Ian R.} and Morrison, {Clayton T} and Walsh, {Thomas J.} and Cohen, {Paul R}",
year = "2011",
doi = "10.1109/ROMAN.2011.6005273",
language = "English (US)",
isbn = "9781457715716",
pages = "113--118",
booktitle = "Proceedings - IEEE International Workshop on Robot and Human Interactive Communication",

}

TY - GEN

T1 - Challenges to decoding the intention behind natural instruction

AU - Torres Peralta, Raquel

AU - Kaochar, Tasneem

AU - Fasel, Ian R.

AU - Morrison, Clayton T

AU - Walsh, Thomas J.

AU - Cohen, Paul R

PY - 2011

Y1 - 2011

N2 - Currently, most systems for human-robot teaching allow only one mode of teacher-student interaction (e.g., teaching by demonstration or feedback), and teaching episodes have to be carefully set-up by an expert. To understand how we might integrate multiple, interleaved forms of human instruction into a robot learner, we performed a behavioral study in which 44 untrained humans were allowed to freely mix interaction modes to teach a simulated robot (secretly controlled by a human) a complex task. Analysis of transcripts showed that human teachers often give instructions that are nontrivial to interpret and not easily translated into a form useable by machine learning algorithms. In particular, humans often use implicit instructions, fail to clearly indicate the boundaries of procedures, and tightly interleave testing, feedback, and new instruction. In this paper, we detail these teaching patterns and discuss the challenges they pose to automatic teaching interpretation as well as the machine-learning algorithms that must ultimately process these instructions. We highlight the challenges by demonstrating the difficulties of an initial automatic teacher interpretation system.

AB - Currently, most systems for human-robot teaching allow only one mode of teacher-student interaction (e.g., teaching by demonstration or feedback), and teaching episodes have to be carefully set-up by an expert. To understand how we might integrate multiple, interleaved forms of human instruction into a robot learner, we performed a behavioral study in which 44 untrained humans were allowed to freely mix interaction modes to teach a simulated robot (secretly controlled by a human) a complex task. Analysis of transcripts showed that human teachers often give instructions that are nontrivial to interpret and not easily translated into a form useable by machine learning algorithms. In particular, humans often use implicit instructions, fail to clearly indicate the boundaries of procedures, and tightly interleave testing, feedback, and new instruction. In this paper, we detail these teaching patterns and discuss the challenges they pose to automatic teaching interpretation as well as the machine-learning algorithms that must ultimately process these instructions. We highlight the challenges by demonstrating the difficulties of an initial automatic teacher interpretation system.

UR - http://www.scopus.com/inward/record.url?scp=80052996110&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052996110&partnerID=8YFLogxK

U2 - 10.1109/ROMAN.2011.6005273

DO - 10.1109/ROMAN.2011.6005273

M3 - Conference contribution

AN - SCOPUS:80052996110

SN - 9781457715716

SP - 113

EP - 118

BT - Proceedings - IEEE International Workshop on Robot and Human Interactive Communication

ER -