Recent Research on Effective Ways to Improve Students’ Learning

1. William Langston Recent Research on Effective Ways to Improve Students’ Learning

2. Additional Information • Primary source: Dunlosky, Rawson, Marsh, Nathan, & Willingham (2013) • http://psi.sagepub.com/cgi/reprint/14/1/4?ijkey=Z10jaVH/60XQM&keytype=ref&siteid=sppsi • Roediger’s (2013) editorial: • http://psi.sagepub.com/cgi/reprint/14/1/1?ijkey=Aq5/rcztL2GbI&keytype=ref&siteid=sppsi

3. Thematic Considerations • Theme 1: • Encoding is a cooperative exercise between the student and the instructor: • What can students do during encoding to improve performance? • What is under the instructor’s control?

4. Thematic Considerations • Theme 2: • We need a lot more research.

5. Thematic Considerations • “…we cannot point to a well-developed translational educational science in which research about learning and memory, thinking and reasoning, and related topics is moved from the lab into controlled field trials (like clinical trials in medicine) and the tested techniques, if they succeed, are introduced into broad educational practice” (Roediger, 2013, p. 1).

6. Thematic Considerations • Theme 3: • It’s virtually impossible to know whether or not studying has been effective.

7. Thematic Considerations • Calibration of comprehension (Glenberg, Sanocki, Epstein, & Morris, 1987): • Read some material. • Rate how you’ll do on a test (verbatim information, an inference test, an immediate test, or a delayed test). • Regardless of the kind of test, people’s confidence in how they would do was unrelated to how they actually did.

8. Research Goal • Evaluate 10 techniques suggested by the literature or popular with students. • Four criteria: Generalize to… (p. 6) • Learning conditions (e.g., amount of practice, incidental or intentional) • Student characteristics (e.g., age, prior knowledge) • Materials (e.g., vocabulary, lecture content, maps) • Criterion tasks (e.g., recall, problem solving)

9. Dunlosky et al. (2013, p. 6)

10. High Moderate Low Dunlosky et al. (2013, p. 6)

11. Research Goal • Let’s explore the two that scored as most effective… • Detailed description of a sample study. • Assessment. • Surprising generalization study.

12. Practice Testing • Karpicke & Roediger (2008) • “The standard assumption in nearly all research is that learning occurs while people study and encode material. Therefore, additional study should increase learning.” (p. 966) • “Retrieving information on a test, however, is sometimes considered a relatively neutral event that measures the learning that occurred during study but does not by itself produce learning.” (p. 966)

13. Practice Testing • Karpicke & Roediger (2008) • What is the effect on long term memory of additional study trials once the information has been successfully retrieved? • What is the effect on long term memory of additional retrieval attempts once the information has been successfully retrieved?

14. Practice Testing • “Standard” study conditions: • Standard method: Learn list, test, learn list, test… (ST) • Dropout approach: Learn list, test, drop items that were successfully recalled, learn list, test, drop, learn, test… (SNT) • Assumption is that additional study of learned items takes away time from unlearned items.

15. Practice Testing • To separate testing effects from study effects, two more conditions were added: • Drop items from testing after successful retrieval, keep studying them. (STN) • Drop items from study and test after successful retrieval. (SNTN)

16. Practice Testing • Design:

17. Practice Testing • Method: • Students learned 40 Swahili-English word pairs (e.g., mashua-boat). • First trial study all 40, test all 40. • After that, follow the dropout procedure for each list. • Four total study-test trials.

18. Practice Testing Karpicke & Roediger (2008, p. 966)

19. Practice Testing • Learning phase results: Karpicke & Roediger (2008, p. 967)

20. Practice Testing • Learning phase: • All groups achieved nearly 100% performance during the learning phase. • No difference in the learning curves. • What happens one week later?

21. Practice Testing • Predictions: Once an item has been correctly recalled once… • Study matters, retrieval does not: • (ST = STN) > (SNT = SNTN) • Retrieval matters, study does not: • (ST = SNT) > (STN = SNTN) • Retrieval and study both matter: • ST > SNT ? STN > SNTN • Results…

22. Practice Testing Karpicke & Roediger (2008, p. 967)

23. Practice Testing • Additional notes: • STN had around 88 more trials than SNTN. No gain in retention. • ST about 83 more than SNT. Likewise, no gain in retention. • ST had 77 more than STN, huge gain in retention. • SNT had about 82 more than SNTN, likewise huge gain. • Where you put your 80 trials really matters.

24. Practice Testing • Additional notes: • What about student perceptions? • Predict all will be about the same (50%). • Makes sense given learning data. • Obviously they are all wrong. But, it highlights how easily tricked we are.

25. Practice Testing • Conclusions: • “…shows a striking absence of any benefit of repeated studying once an item could be recalled from memory.” (p. 968) • “The benefits of repetition for learning and long-term retention clearly depend on the processes learners engage in during repetition.” (p. 968)

26. Practice Testing • Why? • Direct effects (Roediger & Karpicke, 2006): “Changes in learning that arise from the act of taking a test itself” (Dunlosky et al., 2013, p. 30). • Carpenter (2009): Search activates related information in semantic memory that provides additional retrieval cues on subsequent tests.

27. Collins & Loftus (1975, p. 412)

28. Practice Testing • Why? • Mediated effects (Roediger & Karpicke, 2006): “changes in learning that arise from…the amount or kind of encoding that takes place after the test” (Dunlosky et al., 2013, p. 30). • Zaromb and Roediger (2010): Testing improves organization in memory.

29. Practice Testing • Generalization—Learning conditions (p. 30-31): • Free recall, short-answer, fill-in-the-blank, multiple-choice, etc. practice tests. • Some hints that recall-based or generation practice tests are more effective; evidence supports cross-format generalization. • Definite dosing effect, more is better; does interact with spacing (next section).

30. Practice Testing • Generalization—Student characteristics (p. 31-32): • Effects seen in ages from preschoolers to adult learners. • Hints at individual differences: No effect of knowledge base on magnitude of testing effect, possible effect of ability with higher-ability participants getting more benefit.

31. Practice Testing • Generalization—Materials (p. 32-33): • Paired associates (e.g., foreign language vocabulary), fact knowledge, a variety of text genres, videos, etc.

32. Practice Testing • Generalization—Criterion tasks (p. 33-34): • Free recall, short-answer, fill-in-the-blank, multiple-choice, etc. show benefits. • Also inference making, application, and far transfer (apply concepts from one domain to a novel domain). • Retention intervals are typically 1 week, but have gone as long as 11 months with benefits at all intervals.

33. Practice Testing • Issues (p. 34-35): • Student training in most effective use. • Availability. • Feedback to reduce perseveration of errors.

34. Practice Testing • “What is the last name of the person who panicked America with his book Plague of Fear?” (Kornell, Hays, & Bjork, 2009, p. 990).

35. Practice Testing • You couldn’t answer the question because it was fictional. • People were asked fictional questions and given feedback OR they read the answers. Memory for the answers was compared.

36. Practice Testing • What is the effect of NOT being able to answer a question on memory? • In other words, if you try to answer a question on a test and you can’t, then you get feedback, what does that do to memory? • Overall, people did better when tested before hearing the answer.

37. Kornell, Hays, & Bjork (2009, p. 992)

38. Practice Testing • What goes in the blank? • Hot - _____ • Pond - _____

39. Practice Testing • Pond – frog is a low probability one, so you’re less likely to generate it. • When you get feedback after trying to answer, memory is better than if you just read it.

40. Kornell, Hays, & Bjork (2009, p. 994)

41. Practice Testing • The effort to retrieve improved memory, even when you didn’t have the answer to retrieve. Why? Kornell et al. (2009) • Retrieval helps with deeper processing (it connects to more things in memory because you’re rooting around in there looking for the answer). • Helps weed out bad paths to retrieval (“I know it isn’t over here”). • Generates more cues.

42. Practice Testing • “A practical implication of the current research is that educators and learners should introduce challenges into learning situations, including using tests as learning events, even if doing so increases initial error rates” (Kornell et al., 2009, p. 997).

43. Distributed Practice Bahrick (1979; taken from Dunlosky et al., 2013, p. 36)

44. Distributed Practice Bahrick (1979; taken from Dunlosky et al., 2013, p. 36)

45. Distributed Practice • Spacing: How much time you take between study episodes influences what you will get (Cepeda, Vul, Rohrer, Wixted, & Pashler, 2008). Cepeda et al. (2008, p. 1096)

46. Distributed Practice • Cepeda et al. (2008): Vary the study gap between study sessions and the retention interval to find the optimal combination for various retention intervals.

47. Distributed Practice Cepeda et al. (2008, p. 1097)

48. Distributed Practice • Cepeda et al. (2008): The result is that there is an optimal gap where spacing out study sessions leads to improvements in memory. • It’s different for different retention intervals. • Waiting too long between study sessions can also hurt you, but not as much as too short.

49. Distributed Practice Cepeda et al. (2008, p. 1098)

50. Distributed Practice • Cepeda et al. (2008): The overall function is also pretty cool…