The Retention Effects of Dyad Test-Taking in Graduate School Students: An Empirical Analysis

1. The Retention Effects of Dyad Test-Taking in Graduate School Students: An Empirical Analysis The 31st Annual International Lilly Conference on College Teaching November 18, 2011 Presented by Dr. Terry J. Schindler Dr. Karl Knapp Dr. Sheela N. Yadav

2. Session Agenda • Review of Relevant Literature • Relationship to The Scholarship of Teaching and Learning (SoTL) • Research Methodology • Results of the Research • Discussion • Q & A

3. Literature Review

4. Alternatives to Individual Test-Taking • Cooperative groups/pairs • Dyad test-taking • Paired test-taking • Collaborative test-taking

5. Cooperative Test-Taking • An alternative to the traditional method of students taking tests - without outside references or assistance • Associated with the increased quantity of recall (Dallmer 2004, Lambiotte et al. 1987, Rossignol 2004).

6. Dyad Test-Taking • Defined as two students taking and examination together (Rossignol 2004) • Various methods have been used to investigate differences between dyad test-taking and taking tests individually

7. Prior Research - Hanshaw 1982 • Tests were administered in two consecutive parts • Part I: • Same exam administered to two groups of students assigned to different rooms. • One group completed the exam in pairs, while the other group worked alone. • Part II: • New test distributed – students who worked in pairs in Part I took the subsequent test alone and vice versa.

8. Prior Research - Kagan and Kagan 2009 • Underlying premise: • Students learn more if they are held individually accountable for their actions. • When taking a test as a team: • Not all students needed to answer each problem • Allow students to hide behind teammates who know the answers. • Students first took tests alone to account for this need of individual accountability.

9. Prior Research –Cortrightet al. 2003 • Study of collaborative-group testing • Students answered questions in the traditional format as individuals • Immediately after completing the exam individually, in groups of two, students answered a subset of randomly selected questions from the initial exam (design was intended to explore the effectiveness of collaborative-group testing on test performance and the level of retention). • Following a second exam (four weeks later), pairs of students again answered a subset of questions from the initial exam, which had been already completed in the traditional format as individuals. • Analysis of the subset of questions was used to determine the level of retention of the original test material. • This same process was also repeated four weeks later .

10. Prior Research (cont.) • In dyad testing research, two basic procedures are used in administering examinations to dyads: • Paired students encouraged to discuss/debate each test question until a mutual conclusion is reached; one set of answers is produced and one test is submitted (Billington 1994, Dallmer 2004, Rossignol 2004) • Paired students collaborate but each student completes and submits their own copy of the exam (Hanshaw 1982, Woody et al. 2008).

11. Results are Mixed • Significant differences exist between paired and unpaired testing (Hanshaw1982) • Given the finding that students who work in cooperative groups achieve higher mean scores than students who work alone, Hanshaw (1982) recommends students be given more opportunities to work in cooperative pairs (Hanshaw 1982).

12. Results are Mixed • Senior-year baccalaureate nursing students • a significant difference between dyad testing and individual testing results is noted (Rossignol 2004). • Student retention of course content is reduced when students complete the original examinations individually. Student retention is improved when students complete the original examinations in groups (Cortright et al. 2003).

13. Results are Mixed • Retention of previously learned material is short-lived. Collaborative testing improves student retention of course content (Cortright et al. 2003, Bloom 2004). • A summary of two studies conducted to find the effects of collaborative test taking on mathematics retention by third-grade students provided contradictory results (Billington 1994). • Collaborative testing appears to influence strongly the ability for students to retain information for one week (Billington 1994).

14. The Scholarship of Teaching and Learning (SoTL)

15. Challenges and Opportunities in the Field of SoTL • Increasing the research emphasis of the scholarship of teaching and learning, especially on the learning part. • Increasing the area of SoTL that focuses on the teaching and learning of graduate students. • Doing more SoTL beyond the level of the classroom and take it to the course , program, and department levels. Source: McKinney, K. (2007). Enhancing Learning Through The Scholarship of Teaching and Learning. Anker Publishing

16. Emphasis on Learning • Faculty who are interested in sharpening their teaching focus continually ask, “How can I help students learn better?” Angelo and Cross (1993) • Classroom research (or classroom assessment) describes the attempt by teachers who regularly study their classrooms in order to evaluate what and how students are learning (Brookfield 2006) • Instructors across disciplines continue to seek new methods to evaluate learning (Woody et al. 2008).

17. Teaching and Learning of Graduate Students • There is limited research about the benefits of cooperative test taking for college students in which students are allowed to work on their test together (Dallmer 2004). This is especially true at the graduate level. • Examples of college-level studies include: • Undergrads – Fundamentals of Science classes (Hanshaw 1982) • Introductory psychology students (Lambiotte et al. 1987) • Freshman and sophomores – Introduction to Theater (Bloom 2009) • Senior-year baccalaureate nursing students (Rossignol 2004) • Senior-level Psychology of Prejudice classes (Woody et al. 2008) • Graduate students in an Education Research class (Dallmer 2004).

18. SoTL Beyond the Level of the Classroom to the Course Level • Previous studies have compared the results of dyad or group test-taking to individual test-taking: • Within the same class session (Rossignol 2004) • After a period of hours (Bloom 2009, Lambiotte et al. 1987) • After three weeks (Woody et al. 2008).

19. SoTL Beyond the Level of the Classroom to the Course Level This research tests the hypothesis that dyad test-taking improves retention of course contentby graduate students over the period of a traditional fifteen-week semester and addresses all three of the challenges and opportunities suggested by McKinney (2007).

20. Research Methodology

21. Research Methodology • Quasi-Solomon four group design • Two control groups • Two experimental groups • Each instructor facilitated one control and one experimental group to control for bias introduced by the researchers • Pre-test sensitization not an issue - semester Researcher 1 Researcher 2 Instrument Design Control Group 1 Experimental Group 1 Experimental Group 2 Control Group 2 Comparison of Results Martella, Nelson, & Marchand-Martella, 1999

22. Instrument • Pre- and post-test consistent with course objectives for MBA capstone class • Multiple choice questions • Sequence of questions same as sequence of material coverage

23. Population & Sample • 155,637 masters degrees in business granted in the U.S. in 2008 • Sample of convenience • Students enrolled in the capstone MBA classes • Mostly non-residential adult students with significant business experience • Control & experimental classes the same night (enabled observation and held environmental conditions stable between groups) Number of Students in Sample Experimental group 1 22 Experimental group 2 20 Missed pre/post -1 41 Control group 1 14 Control group 2 17 Missed pre/post -2 29 (National Center for Education Statistics 2009)

24. Experimental Methodology • Pre-test administered • Control group takes three course examinations individually • Experimental group takes same three course examinations in randomly selected dyad pairs • Answers discussed, but may differ by student • Students given the option to take exams individually • Post-test administered • Pre- and post-test results matched to measure learning over the course of the semester

25. Results

26. Combined Results • No statistical difference between experimental and control groups (p=0.129, t-statistic=-1.54) • Analysis of Variance (ANOVA) found one statistical difference between the four groups • Second control group increased learning by 26.4%, outperforming the second experimental group by 13.1% (p=0.025, t-statistic=-2.35) • Opposite of prediction in literature and theory

27. Second Control Group • Second control group performed significantly worse of the pre-test (27.3%) as compared with the other three groups (36.8%) • By the end of the semester the control group performed statistically the same as the other groups on the post-test • Final post-test results between groups similar (p=0.422, F=.950).

28. Discussion

29. Unexpected Results • Literature and research predicted positive retention and learning from dyad test-taking • Experimental groups did not statistically benefit from dyad test-taking

30. Potential Explanations • Prior research done on primary and undergraduate students • This research was conducted with predominantly adult graduate students • Type and/or age of the learners could cause variation in effectiveness of dyad test-taking

31. Qualitative Observations • Student reaction to dyad test-taking was almost universally positive • Energy and atmosphere of the test-taking environment was extremely positive • Challenging approach if the potential for social loafing is high on examinations

32. Limitations & Assumptions • Small sample size • Sample of convenience • Need for further replication • Lack of the ability for generalization to larger population • Pre- and post-test instrument too rigorous • Adjusted for amount of student business experience • Adjusted for this being the capstone class in MBA

33. References • Angelo, T. A. and Cross, K. P. (1991). Classroom Assessment Techniques: A Handbook for College Teachers, Second Edition. Jossey-Bass Publishers • Billington, R. (1994). Effects of Collaborative Test Taking on Retention in Eight Third-Grade Mathematic Classes. The Elementary School Journal, Volume 95, Number 1 • Bloom, D. (2009). Collaborative Test Taking. College Teaching, Volume 57, Issue 4 • Brookfield, S. D. (2006). The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom, Second Edition. Jossey-Bass: A Wiley Imprint • Cortright, R N., Collins, H. L., Rodenbaugh, D. W. and DiCarlo, S. E. (2003). Student Retention of Course Content is Improved by Collaborative-Group Testing. Advances in Physiology Education, Volume 27: Number 3 • Dallmer, D. (2004). Collaborative Test Taking With Adult Learners. Adult Learning, volume 15, Issue ¾ • Dimitrov, D. M. and Rumrill, Jr., P. D. (2003). Speaking of Research: Pretest-posttest designs and measurement of change, Work 20, IOS Press pp. 159-165 • Hanshaw, L. G. (1982). Test Anxiety, Self-Concept, and the Test Performance of Students Paired for Testing and the Same Students Working Alone. Science Education, Volume 68, Issue 1 • Kagan, S. and Kagan, M. (2009). Kagan Cooperative Learning.Kagan Publishing • Lambiotte, J. G., Dansereau, D. F., Rockllin, T. R., Fletcher, B., Hythecker, V. I., Larson, C. O., and O’Donnell, A. M. (1987). Cooperative Learning and Test Taking: Transfer of Skills. Contemporary. Educational Psychology 12 pp. 52-61 • Martella, R. C., Nelson, R., & Marchand-Martella, N. E. (1999). Research Methods: Learning to Become a Critical Researach Consumer. Needham Heights: Allyn & Bacon. • McKinney, K. (2007). Enhancing Learning Through The Scholarship of Teaching and Learning. Anker Publishing • National Center for Education Statistics. (2009). Digest of Education Statistics. Retrieved from http://nces.ed.gov/programs/digest/d09/tables/dt09_272.asp • Rossignol, M. A. (2004). Dyad Testing: Promoting Skills Used in the Workplace. Nurse Educator, Volume 29, Issue 2 • Woody, W. D., Woody, L. K., and Bromley, S. (2008) Anticipated Group Versus Individual Examinations: A Classroom Comparison. Teaching of Psychology 35 pp 13-17