560 likes | 650 Views
Perceiving, Remembering & Knowing in Scene Cognition: Where are the Divisions? Helene Intraub University of Delaware. Framework for Scene Cognition. Visual construct Spatial construct “Dividing lines” between perception and memory. Visual Starting point: representation of the
E N D
Perceiving, Remembering & Knowing in Scene Cognition: Where are the Divisions? Helene Intraub University of Delaware
Framework for Scene Cognition • Visual construct • Spatial construct “Dividing lines” between perception and memory
Visual Starting point: representation of the picture Very short-term stores Icon Transsaccadic memory CSTM VSTM What is transferred? Lost? Added? Locus of effect? Very short term stores STM Visual Working Mem LTM
Intraub & Richardson (1989) JEP:LMC • LTM [minutes – 2 days] (Koriat et al., 2000; Annual Review) nonlinguistic version • RSVP trios, 1 sec masked interval • Intraub, Gottesman, Willey & Zuk (1996), • Bertamini, Jones, Spooner & Hecht (2005)
Boundary Extension (BE) Roediger (1996) “… the phenomenon falls at the boundary between perceiving and remembering”
Spatial Starting point: spatial representation of the scene the pic only partially reveals Sky Outdoors Street surface Neighborhood
Spatial Cognition Starting point: spatial representation above behind right left In front below Graded representation Imagine, response to object imagine in front, faster than in back “Resolution” better than front than back (Franklin, N., Henkel, L., & Zangas, T., 1995; Franklin & Tversky, 1990)
Perception vs. Memory? Visual: BE is the result of this active process Spatial: scene understanding is a graded 3-D representation The region that will constitute the BE error is already there. BE is the result of a decision – which part of the entire scene representation had a sensory source – was seen before?
When does BE occur? • Visual construct: • No a priori prediction • Early no BE, over some pd of time it emerges • Which early visual memory system? • Spatial approach • Predicts – when sensory info is gone • Conscious memory includes surrounding swath of space
Visual construct Diff possibilities Either way Either way May or may not Spatial Construct When sensory input is gone Weaker record, more BE Weaker weaker, more BE Same spatial scene rep Timing? Retention interval Stimulus duration Divided attention? Haptics?
Early time-course experiments are in collaboration with: Christopher A. Dickinson
Exp 1 36 Ss 36 Trials (4 types) Scene (250 ms) Interrupt 42 ms Remains on Screen ______ Boundary Rating
Much more Wide-angle Slightly more Wide-angle Slightly more Close-up Much more Close-up Trial Type: Stimulus and Test Identical +2 +1 0 Identical view: -1 -2 CC WW
Much more Wide-angle Slightly more Wide-angle Slightly more Close-up Much more Close-up Trial Type: Stimulus and Test Identical +2 Close-ups Wide-angle +1 0 Identical view: -1 -2 CC WW
Identical View during Presentation and TEST 42 ms Interruption CC vs. WW: t(35) = 3.29, p = .002
Much more Wide-angle Slightly more Wide-angle Slightly more Close-up Much more Close-up Trial Type: Stimulus and Test Differ +2 +1 0 Identical view: -1 -2 CW WC
Much more Wide-angle Slightly more Wide-angle Slightly more Close-up Much more Close-up Trial Type: Stimulus and Test Differ +2 Close-Wide Wide-Close +1 Identical view: 0 -1 -2 CW WC Park, Intraub, Yi, Widders & Chun; fMRI/PPA/RSC
Slightly Different View during Presentation and TEST 42 ms Interruption CW vs. WC (deviation from same – asymmetry): t(35) = 6.64, p < .001
Summary BE occurred when memory ought to have been excellent! • Memory Load: 1 picture • Task is known & repeated across 36 trials • Target location was fully predictable • Extremely brief interruption of sensory input: • 42 ms (commensurate w/saccade) • Eye tracking, also found across actual saccade (103 ms)
Graded Representation Effect of Presentation time? 100 ms 250 ms 500 ms 1.0 .50 0 -.50 -1.0 • 100 250 500 • Duration (ms)
Intraub, Daniels, Horowitz & Wolfe (submitted) Divided Attention = Greater BE 750 ms
BE occurs following haptic exploration of space. Blindfolded-signted, and an Early deaf and blind woman (Lebers Syndrome) – Intraub (2004) Cognition
Mono Large (3 cm) Mono Small (.6 cm)
Quinn & Intraub (2007) Child Development 3-4 month old infants (and 6-7 month old infants)
Scene Understanding: Spatial Construct • Visual – BE develops (pic rep – then fill out) • Perception/memory issue • Spatial – BE area is already there • Which part of the full spatial representation was originally sensory information? • Reality monitoring (Johnson & Raye, 1981) • Usually LTM • But may apply to any scene memory decision • Perception v. memory dividing line? Not an issue • Graded representation –prior sensory, amodal, knowledge
World is continuous… But sensory input is not! Brain’s goal: Represent the scene • Don’t retain spurious boundaries Sensory input is gone – mental rep remains • Regions are not tagged as “high acuity”, “low acuity”, amodally “filled in” or “filled out” • On-line, layout projection facilitate view integration Supporting a coherent representation that is always based on discrete successive views.
Acknowledgments Kristin O. Michod (RA) Undergrad RAs • Dan Bensonoff • Dave Drowos • Jason Rand • Kari Schweifel • Mattie Wilson • Erika Furlong We thank NIMH • David Grieco Kristin’s latest research, “Conceptual Masking: Is it really all about the concept or does layout matter?” Submitted to VSS ’07
Special thanks to.. • James E. Hoffman • Anna Papafragou • Neil Intraub For generously spending time listening, commenting, questioning… New RM-related research: w Anna and Ozge (Turkish evidentials and ReMon) w Jessica Hughes (BE and ReMon)
Visual Cognition BE developed over the interrupt Spatial Cognition The content that we ultimately define as BE was part of the representation. Once sensory input is gone… Cannot distinguish peripheral from amodal… Can distinguish from less constrained knowledge
Spatial Layout Boundary Extension: • Infants as young a 3-4 months Quinn & Intraub (in press).Child Dev. • Children 6–80 y.o. Adults Seamons et al., (2002) • Adult haptic exploration • Deaf & blind adult (Lebers Syndrome) • Blindfolded adults Intraub (2004). Cognition.
Labeling Cognitive Processes Heuristic – Perceiving, remembering Issues “New Look” psychologist 1940’s/1950’s Beyond the stimulus/or Perception as Categorization Jerome Bruner ( ) Hochberg (1968, 1978)– or one of the chapters Quote about schema “Levels of processing” 1970’s Memory a byproduct of perceptual processes Craik & Lockhart ( 1972) Situated Perception (or Embodied) 90’s Barsalou and others
Exp 2: BE across a Saccade • EyeLink II (sampling at 500 hz; chinrest) • N = 24 • 36 trials • Stimulus and test on opposite sides of screen • Stimulus locations held constant for each S • Stimulus right; Test on left for ½ Ss • Stimulus left ; Test on right for ½ Ss
250ms Boundary Rating Exp 2:Across a Saccade? Launch Saccade To this location
250 ms Boundary Rating Exp 2:Across a Saccade? Launch Saccade Mean total time = 298 ms Initiate saccade = 195 Saccade duration = 103 (approx. 11° amplitude) Full screen is 28° x 21°
Target onset is GAZE Contingent (EyeLink II, chinrest) When eyes land: Identical ViewDifferent View
Target onset is GAZE Contingent (EyeLink II, chinrest) When eyes land: Identical ViewDifferent View
Target onset is GAZE Contingent (EyeLink II, chinrest) When eyes land: Identical ViewDifferent View 59% Single Fixation; 41% two fixations CC vs. WW: t(23) = 2.11, p = .046 CW vs. WC (absolute values): t(23) = 3.23, p = .004
Single Saccade ONLY trials When eyes land: Identical ViewDifferent View Filtered DATA – Single saccade only
Single Saccade ONLY trials When eyes land: Identical ViewDifferent View Filtered DATA – Single saccade only
Spatial Cognition Visual Cognition