Interactive and Immersive Technologies: Experience and Research at Saint-Petersburg State University

2. Interactive and immersive technologies: research and experience of application Saint-Petersburg State University of Aerospace Instrumentation Institute of Computing Systems and Programming Department of computing systems and networks Laboratory of computer graphics, virtual and augmented reality

3. Directions of activities Scientific and experimental research and development of innovative methods and tools for human-computer interface. Development and maintenance of various high-tech products for education, culture, industry, transport, energy, health, security, sport and others. Educational programs in the direction “Informatics and computer techniques“ • Bachelor of technique and technology. • Master of technique and technology (Multimedia systems and computer graphics).

4. Areas of research and experiments • Interactive and immersive technologies • Virtual reality • Mixed reality (Augmented reality, Augmented virtuality) • Augmented reality • Addition and combination of different sensations - visual, auditory, tactile, etc • Use markers, images, geo-location, in- and outdoor, single- and multi-user • Augmented virtuality • Collaborative VR/MR • Visual, voice, text and nonverbal communications in collaborative work • Autonomous characters with intelligent behavior • Multimodal interfaces • Combination of different modalities • Use of human physiological parameters - pulse, blood pressure and others. • Adaptive interface • Changes in the properties of the VW/ MR (presentation, navigation), depending on the user's behavior • Personalized interface

5. Applications Scope • Science • Industry • Health and Medicine • Culture • Museum • Archaeology • Art • Tourism • Education • Fitness • Games • Advertising • and other Examples of technological solutions • Interactive and immersive 3/4 D representations of objects and events with contextual information • Serious games with elements of geopositioning • Guides (navigators) presenting information about surrounding objects • 3/4D passport of the objects and events • Electronic educational resources • Interactive map • and other • App • Desktop • Projection Surface • Mobile • Wearablе • Simulators • Devices • 3D Vision • VR HMD • AR Glasses • trackers • gloves • multiscreen • Simulators • 3/4/5 D • etc.

6. Devices and simulators

7. Network of Motion Simulators Purpose • Training • Education • Entertainment • Ergonomics • Science and Technology Center • Culture Solutions • i3D models of terrain and buildings • i3D models car, plane/drone, trolley • i3D virtual tours (Project Yam Fortress) • MultiUser

8. Cybernetic Bicycle Purpose • Entertainment  • Cultural tourism • Health and Medicine • Sport, Fitness and Wellness Solutions • i3D model SUAI • i3D model of bike with character • Cyberbike • Single- or multiuser modes

9. VR and AR devices Purpose • Training • Education • Entertainment • Ergonomics • Industry • Science and Technology Center • Health and Medicine Solutions • i3D models of terrain and buildings • i3D models car, plane/drone, trolley • i3D model of human

10. Industrial i3D Applications Purpose • Virtual prototyping and simulation • After-sales service - cars, ships, planes, buildings, etc • Maintenance • Repair • Assembly Work Instructions • Training • Operator Training Simulator • Normal Conditions • Abnormal Conditions • Transient Conditions • Emergencies Solutions • i3D models of terrain, buildings, plant • i3/4D models of complex objects, equipment, failures and emergency situations • Augmented reality • IETM for ship and other systems

11. Russian Ilizarov Scientific Center for Restorative Traumatology and Orthopaedics Purpose • Advertisement of Center • Innovation in medicine • Conservation of Ilizarovov‘s heritage • Entertainment for hospital patients • Guide for the Center • Education Solutions • i3D model of buildings (exterior and interior) and surrounding area • i3D Ilizarov apparatus, human organs, etc. • Augmented reality (single/multi-marker) • Virtual audio-visual excursions and free walk

12. Managing psychological States of Human Training of attention • Sequence of tasks for attention training consisting of displaying basic and distracting objects. • Registration and authorization of users, different modes of difficulty, processing of current statuses and statistics of users on the server. • Treatment of phobias in public speaking • The virtual class of students reacting neutrally, positively or negatively to the speaker's speech. • Changing a state of mind with • breathing techniques • Audiovisual virtual environment providing assistance in synchronization of breathing in a certain technique. • Selection of a desired state - focus, calming, happiness, energizing, core. • Selection of different levels of difficult

13. Participants Museum Agency of the Leningrad Region Historical museum of Kingisepp Institute of History of Material Culture RAS IBRD + FIBP SUAI VR Lab Historical reconstructions of Yam Fortress Purposes Preservation and promotion of historical information about the lost object of cultural heritage The creation of the resource for economic and social development of the region (cultural tourism) The program “Preservation and Use of Cultural Heritage in Russia”

14. Historical reconstructions of Yam Fortress • i3D models of fortresses and warriors (XIV + XVI-XVII + XVIII centuries) • interactive excursions and free walks

15. Historical reconstructions of Yam Fortress Historical excursions • Virtual audio-visual excursions • Gestural Interaction models Historical games • Defense and explosion of the fortress • Users move freely in the game space • 1 or 2 players Augmented reality • Museum (20) • Territory (5) Mobile, Android, iOS

16. Historical reconstructions of Yam Fortress • Personalities (37) • Fortresses (14) of the North-West of Russia • Monuments (19)

17. Historical reconstructions of Yam Fortress

18. Historical reconstructions of Yam Fortress

19. Alexandrinsky Theater Purpose • Previsualization of stage and spectacle • Historical research • Teaching and the creative process • Publishing with augmented reality Solutions • i3D model of stage and auditorium • i3/4D model of "Masquerade“ spectacle • Gestural Interaction models • Augmented reality (single/multi-image)

20. State Museum of the History of Religion Purpose • Extension of museum exhibitions • Religion of Ancient Greece (Delphi) • Christian relics on the Holy Land • Cultural tourism • Historical research • Education Solutions • i3D model of the Temple of Apollo • i3D Church of the Holy Sepulcher • Multimedia information system • Virtual audio-visual excursions • Free walk • Quiz • Versions - electronic kiosk and CD

21. Russian as foreign language Purpose • Education • Edutainment • Cultural tourism Solutions • i3D model of the Building of the Twelve Collegia (SPbSU) • 3D Character models • Model educational situations • Single- or multiuser modes • Information and test system

22. Games

23. Thank you for your attention http://guap.ru/labvr guap.cgvr@gmail.com