Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang University, Seoul, Korea

1. Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang University, Seoul, Korea

2. Introduction

3. Introduction • The social virtual environments (VEs) VEs Virtual avatar Social VEs

4. Alcoholism Social phobia Schizophrenia Neuroscience Introduction • The research and therapy using the social VEs Social VEs

5. Introduction • The functional MRI environments Conventional device High magnetic and electronic fields MR room Display device Tracking device Tactile device …  For social VEs, the other method is needed!!!

6. Introduction • In this study, we purpose that • development the social VEs for social cognitive neuroscience study using MR compatible devices. • a pilot study performed on the developed social VEs.

7. Methods

8. development the social VEs • fMRI environment for social interaction Inside of magnet room Outside of magnet room Filter panel MR compatible camera Computer Real-time video image MR compatible HMD marker Marker detection on the hand Calculate maker position Apply hand position to virtual environments

9. (a) Rejection (b) Acceptance development the social VEs • Contents for the social VEs

10. development the social VEs

11. A pilot study • Participants

12. A pilot study • fMRI experiment design

13. A pilot study • Questionnaire (5 point response scale ) • Questionnaires of social cognition about the other avatar • Did you feel positive feeling about the other avatar’s response? • Did you feel negative feeling about the other avatar’s response? • Did you think that other avatar’s response was suitable in the situation? • Did you feel that the other avatar rejected your offer? • Did you feel that the other avatar hated you or had negative emotion about you? • Did you feel that the other avatar took threatening attitude to you? • Did you feel that the virtual other ignored or excluded you? • Did you feel that the virtual other was aware of you? • Questionnaires of social realism about the other avatar • How real the other avatar’s action was? • Did you feel that you really shook hands with (or was rejected by) the virtual other ?

14. A pilot study • fMRI data acquisition - The functional MRI conducted with 1.5T machine (GE Medical System). • BOLD (Blood Oxygenate Level Dependent) • EPI sequence, Gradient Echo • 64x64x30 matrix with 3.75x3.75x5-mm spatial resolution, FOV: 240mm, • Slice thickness: 5mm, TR: 2s, TE: 14.3, FA=90, # of slices: 30 • High resolution anatomical images • Fast Spoiled Gradient Echo sequence • 256x256x116 matrix with 0.94x0.94x1.50-mm spatial resolution, FOV: 240mm, • Slice Thickness: 1.5mm, TR: 8.5s, TE: 1.8s, FA: 12, # of slices: 116).

15. A pilot study • fMRI analysis • - Analysis of Functional NeuroImages (AFNI, Ver. 2006_06_30_1332) • - subjects whose haed movements are more than 1mmare excluded Statistical Analysis Preprocessing 3D construction General Linear Model Analysis in each subject Coregistration Temporal Normalization Group Analysis Temporal Smoothing (3-point Low pass filter ) Paired t-test within group (uncorrected P < 0.001) Spatial Normalization (MNI N27 templete, 2x2x2mm3) Spatial Smoothing (Gaussian filter, 8 mm FWHM)

16. Results & Discussions

17. Results & Discussions • Questionnaire (5 point response scale) Results & Discussions • The questionnaire results. - Social realism about other avatars • How real the other avatar’s action • really shake hands with (or was rejected by) the virtual other ? - 1 point : not at all 5 point : very much

18. Results & Discussions • Questionnaire (5 point response scale) Results & Discussions • The questionnaire results. - social cognition about other avatars • positive feeling • negative feeling • other avatar’s response was suitable • the other avatar rejected participant offer • hated participant or had negative emotion • threatening attitude • ignored or excluded participant • awareness of participant - 1 point : not at all 5 point : very much

19. Rejection > Acceptance L prefrontal gyrus (-17, 53, 20) R rolandic operculum (37, -3, 20) L cerebelum (VI) (-25, -45, -22) L superior temporal sulcus (-37, -31, 6) Results & Discussions • Acceptance > Rejection R insula lobe (37, -3, 0) L thalamus (-11, -17, 8) uncorrected P <0.001, cluster size > 80 microliter

20. Results & Discussions • Acceptance > Rejection • mirroring other’s emotion • related to empathy R insula lobe • Social cognition related • friendship and teamwork L thalamus

21. L prefrontal gyrus L superior temporal sulcus Results & Discussions • Rejection > Acceptance • understanding other’s • mental states • Social emotion • (ex. embarrassment) • Social emotion • (ex. embarrassment)

22. Summary • In this study, we develop the VEs for social cognitive neuroscience study that it is possible to interact with the virtual avatars. • This social VEs would provide social environment to patients with deficit in social cognition, such as patients with schizophrenia, social phobia and etc., even though just only handshake with other. • The social interaction methods using maker would be practicable in various social cognitive neuroscience studies.

23. Thank you Prof. Ph.D candidate M.S candidate Hyeongrae Lee Neuro-mechanism On Virtual Environment Lab. Dept of Biomedical Engineering In Hanyang University E-mail: hensgi@bme.hanyang.ac.krHomepage: http://bme.hanyang.ac.krFAX: +82-2-2296-5943Tel: +82-2-2290-0691