1 / 66

Think-Aloud Protocols

Think-Aloud Protocols. February 5, 2010. Today’s Class. Probing Question Think-Aloud Protocols Assignments. Probing Question for Friday, February 5. Let’s say that the NSF has just given you a $100,000 grant to study student note-taking during classroom lecture

onella
Download Presentation

Think-Aloud Protocols

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Think-Aloud Protocols February 5, 2010

  2. Today’s Class • Probing Question • Think-Aloud Protocols • Assignments

  3. Probing Question for Friday, February 5 • Let’s say that the NSF has just given you a $100,000 grant to study student note-taking during classroom lecture • i.e. you can assign one minion to the task full-time for a year • Which qualitative method would you prefer to use, and why? • What factors would you be especially attentive to, to ensure validity of your results?

  4. Today’s Class • Probing Question • Think-Aloud Protocols • Assignments

  5. I need a volunteer • A fast typist • To try to record things people say while thinking aloud • Only record the person thinking aloud

  6. I need two more volunteers

  7. I need two more volunteers • Volunteer #1: Will think-aloud • Volunteer #2: Will run the think-aloud

  8. What should Volunteer #2 rememberwhile running the think-aloud?

  9. What should Volunteer #2 rememberwhile running the think-aloud? • Use appropriate instructions • Prompt the subject • Write down interesting events • Focus on what the subject does and says, not “why” • Don’t help the subject

  10. Long division • Volunteer #1 will think-aloud • Volunteer #2 will run the think-aloud • So, Volunteer #2, give Volunteer #1 some instructions…

  11. Long Division 7140 16, 429, 782, 600

  12. How did it go? • Volunteer #1: • How hard was the task? • Did thinking aloud make it harder?

  13. How did it go? • Volunteer #2: • How hard was it to get him/her to think-aloud? • Did you worry about disrupting him or her? • Audience: • Any (friendly and constructive!) criticism for Volunteer #2?

  14. Thoughts/comments • From anyone • On why this task might have been difficult/easy for the person thinking aloud, or the experimenter • (or anything else)

  15. Two more volunteers!

  16. Drawing a scatterplot • Volunteer #1 will think-aloud • Volunteer #2 will run the think-aloud

  17. Draw a scatterplot of this fake data

  18. How did it go? • Volunteer #1: • How hard was the task? • Did thinking aloud make it harder?

  19. How did it go? • Volunteer #2: • How hard was it to get him/her to think-aloud? • Did you worry about disrupting him or her? • Audience: • Any (friendly and constructive!) criticism for Volunteer #2?

  20. Thoughts/comments • From anyone • On why this task might have been difficult/easy for the person thinking aloud, or the experimenter • (or anything else)

  21. Two more volunteers!

  22. The 9-Dot Problem(Kershaw & Ohlsson, 2004)

  23. The 9-Dot Problem(Kershaw & Ohlsson, 2004) Volunteer #2 will connect all 9 points with just 4 straight lines(you don’t already knowthe answer, right?)

  24. The 9-dot problem

  25. How did it go? • Volunteer #2: • How hard was the task? • Did thinking aloud make it harder?

  26. How did it go? • Volunteer #1: • How hard was it to get him/her to think-aloud? • Did you worry about disrupting him or her? • Audience: • Any (friendly and constructive!) criticism for Volunteer #1?

  27. Thoughts/comments • From anyone • On why this task might have been difficult/easy for the person thinking aloud, or the experimenter • (or anything else)

  28. Insight Problems • The biggest problem with thinking-aloud as a technique. • When people are trying to figure something out, and it requires “insight”, thinking-aloud can be very disruptive. • Therefore, if your participant is really baffled, • Now might not be exactly the time to push them to “KEEP TALKING” • In general, it’s best to avoid using think-aloud in these situations • What else might you use to study process in these situations?

  29. Two more volunteers! • One more time

  30. Trogdor! • http://www.homestarrunner.com/trogdor.html

  31. How did it go? • Volunteer #1: • How hard was the task? • Did thinking aloud make it harder?

  32. How did it go? • Volunteer #2: • How hard was it to get him/her to think-aloud? • Did you worry about disrupting him or her? • Audience: • Any (friendly and constructive!) criticism for Volunteer #2?

  33. Thoughts/comments • From anyone • On why this task might have been difficult/easy for the person thinking aloud, or the experimenter • (or anything else)

  34. OK enough for now • What have we learned about think-alouds from all of this? • Bonus points for expressing your thought as a Beyer & Holtzblatt-style aphorism

  35. What could you do? • If you want to do a think-aloud on some uninterruptable process?

  36. Example • Take, for example, a helicopter pilot who needs to drop a ladder to someone while still flying

  37. Crashing computer OK,Crashing helicopter NOT OK

  38. What could you do?

  39. Task-splitting • You might want to do a think-aloud of the pilot’s processes as (s)he does each of these tasks separately: ie, a think-aloud as (s)he lowers a ladder, and a think-aloud as (s)he flies the plane • This would show you the pilot’s major sub-goals, decision-points, and evaluation criteria for each task

  40. Retrospective Think-Aloud • Videotape the participant as they are doing the task • Take a 10 minute break • Then show the video to the participant and have them do colour-commentary. • Take notes on their colour-commentary • (Or even tape-record it)

  41. Retrospective Think-Aloud is Also Useful… • When you are studying someone who is so expert at an action, that their actions are completely automatized. • In this case, thinking-aloud can disrupt or change someone’s behavior.

  42. Two more volunteers!

  43. Volunteer #1 • Volunteer #1: Please tell me the cash value of the change you have in your pocket. Don’t think aloud.

  44. Volunteer #2 • Volunteer #2: Please THINK ALOUD while you determine the cash value of the change you have in your pocket.

  45. Volunteer #2 • Volunteer #2: Please THINK ALOUD while you determine the cash value of the change you have in your pocket. • Volunteer #1, is this the procedure you used?

  46. Thoughts? Questions?

  47. Analysis

  48. Analysis • Let’s take the longest of the protocols that our noble live transcriber has produced • Can you email it to me? • I’ll put it up on the screen

  49. Segmentation • It’s better to do segmentation from an audio recording than a transcript in general, but for this time…

  50. Segmentation • Your thoughts on where we should segment?

More Related