1 / 29

Enhancing Communication with Crossmodal Tactile and Visual Signaling

Explore the effects of integrating tactile and visual signals in communication for effective action in various contexts. This study delves into how presenting information through multiple senses influences sense perception, processing, and understanding. Discover the implications of crossmodal interactions in real-world scenarios.

mkirksey
Download Presentation

Enhancing Communication with Crossmodal Tactile and Visual Signaling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CROSSMODAL EFFECTS IN TACTILE AND VISUAL SIGNALING Can we get the message through?

  2. Effective action depends on… Communicating Processing Sensing Presenting Understanding the situation

  3. Picture of thermometer in HMMWV taken 31 May, ‘04 in central Baghdad.

  4. Sensing

  5. Sensing… with the senses covered…

  6. Effective action depends on Communicating

  7. Communicating… with the senses in extreme context

  8. GPS L-Band Full FBCB2 on Lower Tactical Internet TOC - ABCS CO/PLT Leaders/Sgts CO/PLT Voice Nets SQD Leader SQD Leader SQD Leader Squad Voice Net Squad Voice Net Squad Voice Net TM Leaders TM Leaders Battalion SA Net GPS DBCS (T) BFT (FBCB2) DBCS (P) Who else needs to know?

  9. Literature Review • Gilson & Fenton (1974)A tactile display presenting aircraft control information was comparable to using a visual display and in some cases actually produced a reduction in tracking error, altitude variations decreased as well as speed variations. • Zlontik (1988) The use of tactile signaling helped pilots and air traffic controllers overcome some of their challenges. • Gilland & Schlegel (1994) reported that tactile communication applied to the head lowered performance on a concurrent task. • Sklar & Sarter (1999) Showed tactile signals are able to be perceived concurrently with auditory or visual information. • Cholewiak, Brill, & Schwab (2004) found that for the torso, a ring of eight loci of vibration is the most one can resolve with accuracy exceeding 90%. • Hoop, Smith, Clegg & Heggestad (2005) Tactile signals aided in interruption management. • Pettitt, Redden, & Carstens (2006)found that tactile presented arm and hand signals could be interpreted faster than standard visually presented signals when undergoing dynamic movement tasks.

  10. Tactile Patterns emulating Standard Army Hand Signals* (FM 21-60) Tactons*…

  11. Merlo, J.L., Terrence, P.I. Stafford S.C., Gilson, R. & Hancock, P.A., Redden E.S., Krausman A., Pettitt, R., White, T.L., & Carstens, C.B. (2006). The effects of dynamic environments and physiological stress on tactile communication and signaling. Proceedings of the Society for Human Performance in Extreme Environments 4th Annual Meeting, San Francisco, CA. Merlo, J.L., Stafford S.C., Gilson, R. & Hancock, P.A. (2006). The effects of physiological stress on tactile communication. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, San Francisco, CA. Merlo, J.L., Terrence, P.I. Stafford S.C., Gilson, R. & Hancock, P.A., Redden, E.S., Krausman, A., Pettitt, R., White, T.L., & Carstens, C.B. (2006). Communicating through the use of vibrotactile displays for dismounted and mounted soldiers. Proceedings of the Twenty-fifth Army Science Conference, Orlando, FL.

  12. Spence and Driver (2004) Attention and then perception is largely a multi-sensory construction Stein and Meredith (1993) bi-modal and tri-modal neurons have stronger cellular response with two sensory modalities than with uni-modal stimulation Wallace, Meredith, Stein (1998) found that combinations of two different sensory stimuli have been shown to significantly enhance the response of Superior Colliculus neurons above those evoked by either uni-modal stimulus Allman and Meredith (2007) found uni-modal visual neurons had their visual response modulated by concurrently presented auditory stimuli Current research has explored these conflicts of congruent and incongruent cross-modal paradigms in many of the possible permutations and combinations of very basic visual, auditory, kinesthetic and tactile stimuli (Soto-Faraco, Lyons, Gazzaniga, Spence, & Kingstone, 2002; Soto-Faraco, Morein-Zamir, & Kingstone, 2005; Soto-Faraco, Spence, & Kingstone, 2004a; Spence & Driver, 1997; Spence & Walton, 2005). Multi-Modal Benefits

  13. visual only (video presentation of the signals) tactile only (tactile presentation of the signals) both visual and tactile simultaneous and congruent (i.e. the same signals were presented both visually on the video and through the tactile system) both visual and tactile simultaneous and incongruent (i.e. the visually presented signal did not match the presented tactile signal). Experimental Method

  14. Equipment • an acoustic transducers • displace 200-300 Hz sinusoidal vibrations onto the skin • C2’s contactor is 7 mm, with a 1 mm gap separating it from the tactor aluminum housing • 17 grams • 8 tactors mounted on an elastic belt with acoustic padding • 8 X 45 degrees = 360 degrees • Tactor Control Box (TCB) Stores tactile messages and contains bluetooth receiver • Total system under 2 lbs

  15. N NW NE 2 1 W E 8 FRONT 7 3 LEFT RIGHT SE SW 6 4 BACK 5 S Spatial layout of Tactile Display

  16. Method

  17. Attention

  18. RALLY RALLY

  19. Equipment Hardware Software • Windows XP, tablet • LabVIEW 8.2 National Instruments • Microsoft Office 2003 • SPSS 11.5 • Windows Media Player • Samsung Q1 Ultra Mobile • Intel Celeron M ULV (900 MHz) • 7” WVGA (800 x 480) LCD • Sound dampening earmuffs • reduction rating of 11.3 decibels at 250 Hz

  20. Experimental Procedure • Training session • Learned both visual and tactile signals simultaneously • Advanced only after mastery • Lasted approximately 10 minutes • 20 Participants • 10 Males • 10 Females • Mode age 19 years • Signals • Attention • Halt • Move Out • NBC • Rally • Experimental block • 10 trials visual only • 10 trials tactile only • 20 congruent trials • 20 incongruent trials • 60 total trials per block • 2 x Blocks = 120 trials

  21. Screen Capture from Visual interface Participants chose the appropriate signal as fast a possible after signal presentation. The tactile only presentation showed a video of the soldier standing at the rigid position attention.

  22. Initial Results Response Time x Condition t(19)=-3.98, p≤.01 t(19)=-2.16, p≤.04 t(19)=-2.25, p≤.04 Oneway ANOVA ( F(1, 19)=473.445, p<.000, partial eta squared = .961, observed power, β= 1.000 )

  23. Accuracy x Condition t(19)=4.03, p≤.01 t(19)=1.27, p≤.22 t(19)=2.00, p≤.06

  24. Initial Results

  25. Initial Results Block 1 Block 2

  26. Data should provide valuable information on: Congruency effects Selected attention Interference effects Tactile signal efficiency Modality interactions Future research for tactile signaling includes: Spatial references Temporal importance Alternate inputs Discussion

  27. Questions?

More Related