Squeezing out Gaming Behavior in a Dialog-Based ITS

Peter Hastings, Elizabeth Arnott, and David Allbritton. Squeezing out Gaming Behavior in a Dialog-Based ITS. In Proceedings of Intelligent Tutoring Systems 2010, Springer, Berlin, 2010.

Download

[106.2kB pdf] 

Abstract

Research Methods Tutor (RMT) is a dialog-based intelligent tutoring system which has been used by students in Research Methods in Psychology classes since 2003. Students interact with RMT to reinforce what they learn in class in five different topics. In this paper, we evaluate a different population of students and replicate our prior research: despite the relatively small amount of exposure during the term to RMT compared to other course-related activities, students learn significantly more on topics covered with RMT (Arnott, 2008). However, we did not find the same advantage for the dialog-based tutoring mode of RMT over the CAI mode. When transcript analyses indicated that a small but significant number of students were gaming the system by entering empty or nonsense responses, we modified the tutor to require reasonable attempts. This did lead some students to reform their gaming ways. In other cases, however, it resulted in disengagement from tutoring at least temporarily because reasonable answers were not recognized.

BibTeX

@InProceedings{Hastings:its2010,
  author = 		 name:ped,
  title = 		 {Squeezing out Gaming Behavior in a Dialog-Based {ITS}},
  booktitle = {Proceedings of Intelligent Tutoring Systems 2010},
  year = 	 2010,
  editor = 	 {V. Aleven and J. Kay and J. Mostow},
  address = 	 {Berlin},
  publisher = {Springer},
  cvnote = {Acceptance rate: 38\%. CORE Conference Rank: A.},
  abstract = {Research Methods Tutor (RMT) is a dialog-based intelligent tutoring
  system which has been used by students in Research Methods in
  Psychology classes since 2003.  Students interact with RMT to
  reinforce what they learn in class in five different topics.  In
  this paper, we evaluate a different population of students and
  replicate our prior research: despite the
  relatively small amount of exposure during the term to RMT compared
  to other course-related activities, students learn significantly
  more on topics covered with RMT (Arnott, 2008).  However, we did not
  find the same advantage for the dialog-based tutoring mode of RMT
  over the CAI mode.  When transcript analyses indicated that a small
  but significant number of students were gaming the system by
  entering empty or nonsense responses, we modified the tutor to
  require reasonable attempts.  This did lead some students to reform
  their gaming ways.  In other cases, however, it resulted in
  disengagement from tutoring at least temporarily because reasonable
  answers were not recognized.}
}