Research Methods Tutor: Evaluation of a dialogue-based tutoring system in the classroom

Elizabeth Arnott, Peter Hastings, and David Allbritton. Research Methods Tutor: Evaluation of a dialogue-based tutoring system in the classroom. Behavior Research Methods, 40(3):694–698, Psychonomics Society Publications, 2008.

Download

[128.2kB pdf] 

Abstract

Research Methods Tutor (RMT) is a dialogue-based intelligent tutoring system for use in conjunction with undergraduate psychology research methods courses. RMT includes five topics that correspond to the curriculum of introductory research methods courses: ethics, variables, reliability, validity, and experimental design. We evaluated the effectiveness of the RMT system in the classroom using a non-equivalent control group design. Students in three classes (n = 73) used RMT, and students in two classes (n = 52) did not use RMT. Results indicated that the use of RMT yielded strong learning gains of .71 SD above classroom instruction alone. Further, the dialogue-based tutoring condition of the system resulted in higher gains than the textbook-style condition (CAI version) of the system. Future directions for RMT include the addition of new topics and tutoring elements.

BibTeX

@ARTICLE{Arnott:brm2008,
  author = 	 name:epd,
  title = 	 {Research Methods Tutor: Evaluation of a dialogue-based tutoring system in the classroom},
  journal = 	 {Behavior Research Methods},
  year = 	 {2008},
  publisher =    {Psychonomics Society Publications},
  volume =       {40},
  number =       {3},
  pages =        {694--698},
  cvnote = {Impact factor = 2.458, g-index = 175},
  abstract =     {Research Methods Tutor (RMT) is a dialogue-based intelligent tutoring system for use in conjunction with undergraduate psychology research methods courses.  RMT includes five topics that correspond to the curriculum of introductory research methods courses: ethics, variables, reliability, validity, and experimental design.  We evaluated the effectiveness of the RMT system in the classroom using a non-equivalent control group design.  Students in three classes (n = 73) used RMT, and students in two classes (n = 52) did not use RMT.  Results indicated that the use of RMT yielded strong learning gains of .71 SD above classroom instruction alone.  Further, the dialogue-based tutoring condition of the system resulted in higher gains than the textbook-style condition (CAI version) of the system.  Future directions for RMT include the addition of new topics and tutoring elements.}
}