Interaction analysis of the ALICE chatterbot

A two-study investigation of dialog and domain questioning

Robert P. Schumaker, Hsinchun Chen

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

This paper analyzes and compares the data gathered from two previously conducted Artificial Linguistic Internet Chat Entity (ALICE) chatterbot studies that were focused on response accuracy and user satisfaction measures for six chatterbots. These chatterbots were further loaded with varying degrees of conversational, telecommunications, and terrorism knowledge. From our prior experiments using 347 participants, we obtained 33 446 human/chatterbot interactions. It was found that asking the ALICE chatterbots "are" and "where" questions resulted in higher response satisfaction levels, as compared to other interrogative-style inputs because of their acceptability to vague,binary, or clichéd chatterbot responses. We also found a relationship between the length of a query and the users perceived satisfaction of the chatterbot response, where shorter queries led to more satisfying responses.

Original languageEnglish (US)
Article number5272316
Pages (from-to)40-51
Number of pages12
JournalIEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans
Volume40
Issue number1
DOIs
StatePublished - Jan 2010

Fingerprint

Linguistics
Internet
Terrorism
Telecommunication
Experiments

Keywords

  • Artificial linguistic internet chat entity (ALICE)
  • Chatterbot
  • Dialog platform
  • Domain-specific knowledge
  • Evaluation
  • Knowledge delivery

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Human-Computer Interaction
  • Information Systems
  • Software

Cite this

@article{ffbb4e0b54b141abaa20857277a109f6,
title = "Interaction analysis of the ALICE chatterbot: A two-study investigation of dialog and domain questioning",
abstract = "This paper analyzes and compares the data gathered from two previously conducted Artificial Linguistic Internet Chat Entity (ALICE) chatterbot studies that were focused on response accuracy and user satisfaction measures for six chatterbots. These chatterbots were further loaded with varying degrees of conversational, telecommunications, and terrorism knowledge. From our prior experiments using 347 participants, we obtained 33 446 human/chatterbot interactions. It was found that asking the ALICE chatterbots {"}are{"} and {"}where{"} questions resulted in higher response satisfaction levels, as compared to other interrogative-style inputs because of their acceptability to vague,binary, or clich{\'e}d chatterbot responses. We also found a relationship between the length of a query and the users perceived satisfaction of the chatterbot response, where shorter queries led to more satisfying responses.",
keywords = "Artificial linguistic internet chat entity (ALICE), Chatterbot, Dialog platform, Domain-specific knowledge, Evaluation, Knowledge delivery",
author = "Schumaker, {Robert P.} and Hsinchun Chen",
year = "2010",
month = "1",
doi = "10.1109/TSMCA.2009.2029603",
language = "English (US)",
volume = "40",
pages = "40--51",
journal = "IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.",
issn = "1083-4427",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "1",

}

TY - JOUR

T1 - Interaction analysis of the ALICE chatterbot

T2 - A two-study investigation of dialog and domain questioning

AU - Schumaker, Robert P.

AU - Chen, Hsinchun

PY - 2010/1

Y1 - 2010/1

N2 - This paper analyzes and compares the data gathered from two previously conducted Artificial Linguistic Internet Chat Entity (ALICE) chatterbot studies that were focused on response accuracy and user satisfaction measures for six chatterbots. These chatterbots were further loaded with varying degrees of conversational, telecommunications, and terrorism knowledge. From our prior experiments using 347 participants, we obtained 33 446 human/chatterbot interactions. It was found that asking the ALICE chatterbots "are" and "where" questions resulted in higher response satisfaction levels, as compared to other interrogative-style inputs because of their acceptability to vague,binary, or clichéd chatterbot responses. We also found a relationship between the length of a query and the users perceived satisfaction of the chatterbot response, where shorter queries led to more satisfying responses.

AB - This paper analyzes and compares the data gathered from two previously conducted Artificial Linguistic Internet Chat Entity (ALICE) chatterbot studies that were focused on response accuracy and user satisfaction measures for six chatterbots. These chatterbots were further loaded with varying degrees of conversational, telecommunications, and terrorism knowledge. From our prior experiments using 347 participants, we obtained 33 446 human/chatterbot interactions. It was found that asking the ALICE chatterbots "are" and "where" questions resulted in higher response satisfaction levels, as compared to other interrogative-style inputs because of their acceptability to vague,binary, or clichéd chatterbot responses. We also found a relationship between the length of a query and the users perceived satisfaction of the chatterbot response, where shorter queries led to more satisfying responses.

KW - Artificial linguistic internet chat entity (ALICE)

KW - Chatterbot

KW - Dialog platform

KW - Domain-specific knowledge

KW - Evaluation

KW - Knowledge delivery

UR - http://www.scopus.com/inward/record.url?scp=72949088640&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=72949088640&partnerID=8YFLogxK

U2 - 10.1109/TSMCA.2009.2029603

DO - 10.1109/TSMCA.2009.2029603

M3 - Article

VL - 40

SP - 40

EP - 51

JO - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.

JF - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.

SN - 1083-4427

IS - 1

M1 - 5272316

ER -