1 / 35

Psychophysics 3

Psychophysics 3. Research Methods Fall 2010 Tamás Bőhm. Signal detection theory. Aka. sensory decision theory (SDT) A model & a data analysis method for decision problems with uncertainty (noise) Originates from World War II: aircraft detection on radar signals

baird
Download Presentation

Psychophysics 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm

  2. Signal detection theory • Aka. sensory decision theory (SDT) • A model & a data analysis method for decision problems with uncertainty (noise) • Originates from World War II: aircraft detection on radar signals • Today: widely used in psychophysics, medicine, radiology and machine learning

  3. Signal detection theory • Experiment setup: • In some trials a stimulus (signal) is presented, in others there is no stimulus; • Observer reports if she/he saw a signal or not • Calculate how many times the observer detected a signal when she/he was presented one (hit rate) • Is the hit rate all we want to know? Two observers achieved the same hit rate. Are they certainly behaving the same way? • NO, we also need to know how many times the observer said “I see” when there was no signal (false alarm rate)

  4. Signal detection theory • Confusion matrix: contains all the information about the observer’s performance

  5. Signal detection theory • Confusion matrix: contains all the information about the observer’s performance • As columns add up to 100%, it is enough to know one item from each column 40 trials 20 = 100% 20 = 100% 18 = 90% 6 = 30% 2 = 10% 14 = 70%

  6. Signal detection theory • Perfect detection: 100% 0% 0% 100%

  7. Signal detection theory • No detection at all (1st example): always reporting “Seen” 100% 100% 0% 0%

  8. Signal detection theory • No detection (2nd example): always reporting “Not seen” 0% 0% 100% 100%

  9. Signal detection theory • No detection (3rd example): flipping a coin 50% 50% 50% 50%

  10. Signal detection theory • No detection (4th example): reporting “Seen” in 30% of the trials (no matter what is presented) Rows equal  no detection 30% = 30% 70% = 70%

  11. Signal detection theory • Receiver operating characteristic (ROC): 100% hit rate 100% false alarm rate

  12. Signal detection theory • Receiver operating characteristic (ROC): 100% hit rate 90% 30% 10% 70% 100% false alarm rate

  13. Signal detection theory • Receiver operating characteristic (ROC): 100% Perfect detection hit rate 100% 0% 0% 100% 100% false alarm rate

  14. Signal detection theory • Receiver operating characteristic (ROC): 100% No detection: always “yes” hit rate 100% 100% 0% 0% 100% false alarm rate

  15. Signal detection theory • Receiver operating characteristic (ROC): 100% No detection: always “no” hit rate 0% 0% 100% 100% 100% false alarm rate

  16. Signal detection theory • Receiver operating characteristic (ROC): No detection: reporting “yes” in 50% of the trials (flipping a coin) 100% hit rate 50% 50% 50% 50% 100% false alarm rate

  17. Signal detection theory • Receiver operating characteristic (ROC): No detection: reporting “yes” in 40% of the trials 100% hit rate 40% 40% 60% 60% 100% false alarm rate

  18. Signal detection theory • Receiver operating characteristic (ROC): No detection: reporting “yes” in 30% of the trials 100% hit rate 30% 30% 70% 70% 100% false alarm rate

  19. Signal detection theory • Receiver operating characteristic (ROC): No detection: reporting “yes” in 60% of the trials 100% hit rate 60% 60% 40% 40% 100% false alarm rate

  20. Signal detection theory • Receiver operating characteristic (ROC): 100% Diagonal: no detection hit rate 100% false alarm rate

  21. Signal detection theory • SDT model: • No way to remove the noise • But sensation can be separated from decision by using ROCs SL ≥β YES Sensation level (SL) Signal present/absent Sensation Decision NO SL < β Noise Criterion (β)

  22. Signal detection theory SL ≥β YES Sensation level (SL) Signal present/absent Sensation Decision NO SL < β (Noise) Criterion (β) Without noise: perfect detection is possible signal absent criterion signal present probability sensation level

  23. Signal detection theory SL ≥β YES Sensation level (SL) Signal present/absent Sensation Decision NO SL < β (Noise) Criterion (β) signal absent criterion signal present 100% 0% probability 0% 100% sensation level

  24. Signal detection theory SL ≥β YES Sensation level (SL) Signal present/absent Sensation Decision NO SL < β Noise Criterion (β) signal absent(noise only) signal present(signal+noise) criterion Noise: smears the distributions perfect detection is impossible (if the two distributions overlap) probability sensation level online demo

  25. Signal detection theory Sensation level Sensation level http://www-psych.stanford.edu/~lera/psych115s/notes/signal/

  26. Signal detection theory hit rate Sensation level false alarm rate Sensation level

  27. Signal detection theory β = 6 β = 6 ROC curve β = 8 β = 8 hit rate β = 10 β = 10 false alarm rate

  28. Signal detection theory • Criterion (β): specifies where we are on the ROC curve • The ROC curve is specified by sensory capacities only(discriminability) β hit rate false alarm rate probability sensation level

  29. Signal detection theory • Discriminability: how well the observer can separate the presence of signal from its absence~ overlap between the two distributions~ bowing out of the ROC curve • Measured by d’ (discriminability index,also called sensitivity) http://www-psych.stanford.edu/~lera/psych115s/notes/signal/

  30. Signal detection theory d’: selects the ROC curve β: specifies a point on the selected ROC curve • same information as hit rate & false alarm rate, but: hit rate, false alarm rate:both reflect sensation & decision characteristics;cannot separate the two d’: depends only on sensation β: depends only on decision β The two processes are separated http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.ROC.html

  31. Fechner’s methods: Is a stimulus detectable? Yes or no? Clear-cut threshold value (with some variability) that can be measured Stimulus intensity > threshold  detectable Stimulus intensity < threshold  not detectable Dichotic outcome, categorical model Signal detection theory: How well is it detectable? How sensitive the observer is to the stimulus? Measured by d’ The higher d’ is, the more the stimulus is detectable d’ = 0  not detectable at all Scalar outcome, dimensional model Signal detection theory

  32. SL ≥β YES Decision NO SL < β Criterion (β) Different task Forced-choice: eliminates the criterion SDT: separates the criterion Correct Incorrect Signal detection theory • Problem with Fechner’s methods: criterion Sensation level (SL) Stimulus Sensation (Noise)

  33. Signal detection theory Psychophysical measurements with SDT: • Create a stimulus set with a range of intensities (like in the method of constant stimuli) • Test each stimulus many times with each observer • On each trial, either present a randomly selected stimulus or do not present anything • Ask the observer if he/she detected the stimulus • Calculate the hit rate and false alarm rate for each observer, for each stimulus intensity • Use the formula/table to calculate d’ for each case • Examine how d’ changes with intensity: the higher d’ is for a stimulus intensity, the greater the observer’s ability to detect this intensity http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.SignalDetection.html

  34. Signal detection theory • Main results: changes in d’ values Caudek–Rubin Vision Res. 2001

  35. Signal detection theory • There is also a β value for each d’ value • It can be informative about the decision behavior: • Balanced: false alarm and miss rates are equal • Liberal: the observer says “yes” whenever there may be a signal • Conservative: decision is yes only when it is almost certain that there is a signal balanced conservative liberal probability sensation level

More Related