Diane F. Halpern, Claremont McKenna College, Art Graesser, University of Memphis

1. Diane F. Halpern, Claremont McKenna College, Art Graesser, University of Memphis Keith Millis, Northern Illinois University, and the Talented Humans and Avatars from Team Aries

4. Scientifically Tested

5. IES Grant that is incorporating what we know about the science of Learning A program to teach critical thinking/ scientific reasoning skills using what we know about current students (play games on line, use teaching agents, and more). Grant is with Keith Millis at Northern Illinois University and Art Graesser at University of Memphis—(they are the brains behind this project) ARIES Acquiring Research Investigative and Evaluative Skills (a brief look early in the project) Special thanks to other ARIES folks—Patricia Wallace, Zhiqiang Cai, Heather Butler, Carol Forsyth, Anne Britt, Joseph Magliano, Katja Wiemer

6. ARIESAcquiring Research Investigative and Evaluative Skills Animated Agents—a teacher and a student guide the student through the tutor lessons and will be an expert on scientific inquiry—the heart of critical thinking For example: “My roommate and I got into an argument yesterday on who was more influential on hip hop: James Brown or Stevie Wonder” to which the Teacher-Agent might respond, “You know, you could have resolved the argument by using what scientists call an operational definition.”

7. Some Key Concepts in Scientific Inquiry Developing Research Ideas Theories, hypotheses, pseudoscience, falsifiability The Independent and Dependent Variables Operational definitions, reliability, accuracy, precision, validity, objectivity of scoring Experimental Control Comparison groups, random assignment, subject bias, attrition/mortality The Sample & Experimenter Representative, sample size, experimenter bias, conflict of interest Drawing conclusion Alternative interpretations, limits of correlation research, quasi-experimental designs, replication of results

8. ARIESAcquiring Research Investigative and Evaluative Skills We use “science of learning” in the program Self-explanation Generate reasons why a study is faulty or not faulty Reciprocal teaching Students teach the fellow student Spacing, testing effects Students must recognize concepts across many examples Variable encoding Psychology, biology and chemistry problems Authentic learning Case studies are magazine, news articles, advertisements Motivation, engagement Consequences for their performance Auto-tutor platform that allows students to hold a dynamic conversation with the learner (dialog interactivity). Principles of serious gaming.

9. Operation ARIES! ARIES: Acquiring Research Investigative and Evaluative Skills Educational game Story line Aliens from the Aries constellation are attempting to colonize Earth Alien spies are teaching poor science and selling products based on faulty research Goal: recruiting & training new FBS (Federal Bureau of Science) agents to help identify the alien spies, and thus prevent being colonized.

10. Operation ARIES!

11. Aliens taking over the earth with bad science—now that is scary!!

12. Overview: Operation ARIES! • Level 1: Learn about science by reading a Big Book of Science written by the aliens • Learn basic concepts • Level 2: Help identify potential aliens by evaluating case studies (i.e., faulty studies) • Recognize concepts in authentic contexts • Level 3: Interrogate potential aliens • Discriminate good from flawed research in an active way

13. How can you become an FBS agent? Step 1: Take a Science Training Course You will learn key concepts in the scientific method like control groups, validity, independent and dependent variables (20 in all). You will read (and be tested on) a science book used by the Fuath spies. You will be guided by an FBS handler and be joined by a fellow (animated) student

15. One important method that can be used to understand things is called the experimental method. It is a complicated method, so we will teach you about it in different stages as you progress through this manual. Suppose the Beans wanted to know if they can learn while they sleep. To test this, they would need to start with an explanation of how sleep learning might occur. Here is an example of one possible explanation: Sleep learning occurs because people can hear while they sleep, and the information they hear gets remembered even if they don’t recall how or when the learning happened. So, if one of the Beans wanted to know if it were possible to learn while one sleeps, he would need to start with one possible explanation of sleep learning—how it might work. The starting point or explanation is called a theory.

17. Assigning Trialog Type (individualizing instruction) • High knowledge • Teachable Agent • Reinforce and use existing knowledge • Intermediate Knowledge • Standard Tutorial Dialogue • Partial knowledge, zone of proximal development • Low Knowledge Vicarious Learning • Low knowledge good for vicarious learning (Craig et al., 2004)

18. Do trialogs increase learning? • Misconceptions • Examples and transfer • Discriminate relevant from irrelevant information • Testing effects • Motivation

19. Trialog (.42 correct) vs. no trialog (.35 correct) (ANCOVA with prior knowledge partialed out, p < .05, effect size = .41)

20. Brief Sample of Training Program

21. How can you become an FBS agent? Step 2: Analyze Case Studies You will analyze examples of research written by the Fuaths. They are from magazines, the Internet, and newspapers, covering topics in Psychology, Biology and Chemistry. With help from your FBS handler and a Fuath defector, you will learn to identify flaws in the research. Pay attention because the Fuaths use the flaws to communicate with one another. You will likely uncover important clues to their plans.

22. Example of Research to EvaluateStudents Have to Ask Questions

23. How can you become an FBS agent? Step 3: Interrogate Suspected Alien ‘Scientist’ Spies You are close to finding and stopping the mastermind behind the invasion. FBS has captured a number of scientists, some of whom are Fuaths and some whom are Human. It is your job to tell the difference between them by asking questions about their research. If a study is faulty, then you have found a Fuath and you are one step closer to saving the Earth. Be alert! http://thumbs.dreamstime.com/thumb_109/11675061860q4102.jpg

24. Question-Answer Cycles • Player poses a question • Selected or typed in • The question is asked by the interrogator • The question is answered by suspect • A flaw is revealed or not on one or more category • Player evaluates the answer • Flaw, no flaw, or a flaw that was recognized by the suspect

25. Interrogation Module • Player learns how to ask questions and evaluate answers about research • Learning by questioning (Rosenshine, Meister, & Chapman, 1996) • Questioning requires reading carefully, be active, and self-regulation (Wisher & Graesser, 2007) • Questioning can be taught (Beck et al., 1997; King, 1994; Palincsar & Brown, 1984) • Becoming more important in learning environments (Linn, et al., 2004)

26. Here is an example of a session in which the student evaluates information to decide if the person doing the research is a human (good research) or an alien (flawed research)

27. Summary • We are half-way through toward completing Operation ARIES! • We are beginning to evaluate • Engagement, usability, and learning • ARIES incorporates many learning principles • Self-explanation, reciprocal teaching, spacing, testing effects, variable encoding, authentic learning, motivation, engagement • Making engaging games that lead to deep learning is still a new frontier in education

28. Contact Information Dr. Diane F. Halpern Claremont McKenna College 850 Columbia Ave. Claremont, CA 91711 (909) 607-9647 For more information, send an e-mail to diane.halpern@cmc.edu