150 likes | 166 Views
Multimodal Virtual Environments: Response Times, Attention, and Presence. B93902033 陳柏叡. Introduction. Efficiently combine sensory information from two or three channels Three channels: vision, audio, haptic
E N D
Multimodal Virtual Environments:Response Times, Attention, and Presence B93902033陳柏叡
Introduction • Efficiently combine sensory information from two or three channels • Three channels: vision, audio, haptic • Differentiates from older technologies - communicating only via a single channel • The underlying cognitive mechanisms are still elusive and unknown
Environments Richness Results in a Complete and Coherent Experience • Single channel conveys limited and insufficient information to the senses - Engenders a lower sense of presence • Multimodal VE provide a greater extent of sensory information to the observer - Sense of being present is felt stronger
Multimodal VE are Mimicking Reality Better • Many of daily experiences in the real world are fundamentally multimodal • Grasping an object – visual, haptic • Gastronomic pleasure – taste, smell, vision • Multimodal VE have a clear advantage, in mimicking a multimodal phenomenon
Better Integration and Filling in of Missing Information • Synthetic VEs provide fewer sensory cues than physical environments • User needs to interpolate sensory stimuli to create a functional mental modal • Result in an enhanced sense of presence
Faster Mental Processing Enables Deeper and Richer Experience • Using Reaction Time to compare the brain processing speed of unimodal, bimodal, and trimodal signals • Faster processing at the perceptual stage allow users more time in consequent cognitive stages • Contributing to the great sense of presence
Experimental Design • Materials – touch-enabled computer interface, haptic device(force-feedback) • Participants – 16 students(unaware of the purpose of the experiments) • Under the guidelines of the ethical committee and with its approval
Stimuli • Holding the stylus in their hand as if they were writing • On each trail the computer generated a sensory stimulation – uni, bi, trimodal • Visual – changing color • Auditory – compound sound • haptic – resisting force
Procedure • Pressing a button on the stylus as soon as they detected the stimuli • Computer generated the stimulation randomly • Consisted of six blocks of trails (three with dominant hand, three with the other hand) • Each of blocks consisted of 105 single trail, in which each of the seven conditions
Result-Unimodal • V – dominant hand 430ms (SD=94) not dominant 436ms (SD=92) • A – dominant hand 330ms (SD=103) not dominant 326ms (SD=76) • H – dominant hand 318ms (SD=99) not dominant 334ms (SD=91)
Result-Bimodal • VA – dominant hand 302ms (SD=78) not dominant 304ms (SD=70) • HV – dominant hand 294ms (SD=75) not dominant 306ms (SD=77) • HA – dominant hand 272ms (SD=81) not dominant 280ms (SD=69)
Result-Trimodal • dominant hand 430ms (SD=94) not dominant 436ms (SD=92) • Between-Hand comparisons - Revealed insignificant differences in the VA and HA conditions, and only marginal difference in HV conition
Multimodal Stimulation and Attention • Multisensory enhancement is modulated by attention • Hypothesizing that the brain allocates a greater amount of attention to multimodal and employing a larger neural network
Attention and Presence • The link between attention and presence is already documented in literature
Conclusion • At the initial perceptual stage – • Multimodal information is processed faster • Activating larger neural network, increasing user’s attention • At the consequent cognitive stages, more time to • Acquire a wider range of details and subtle for the display • Fill in missing information • Integrate all these informative cues • Contributing to a greater sense of presence