1 / 30

Sosuke Hoshino , Yutaka Ishibashi,

QoE Assessment in Olfactory and Haptic Media Transmission: Influence of Inter-Stream Synchronization Error. Sosuke Hoshino , Yutaka Ishibashi, Norishige Fukushima, and Shinji Sugawara Department of Scientific and Engineering Simulation Nagoya Institute of Technology Nagoya 466-8555, Japan.

manchu
Download Presentation

Sosuke Hoshino , Yutaka Ishibashi,

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QoE Assessment in Olfactory and Haptic MediaTransmission: Influence of Inter-StreamSynchronizationError Sosuke Hoshino, Yutaka Ishibashi, Norishige Fukushima, and Shinji SugawaraDepartment of Scientific and Engineering Simulation Nagoya Institute of Technology Nagoya 466-8555, Japan

  2. Outline • Background • Purpose • Olfactory and Haptic Media Display System • Method of Assessments • QoE Assessment Method • QoE Assessment Results • Conclusions and Future Work

  3. Background Vision, auditory, olfaction, tactile, gustation Multi-sensory communications have been emerging. This study focuses on these media. Olfactory, haptic and visual media Transmissionover the Internet • Network delay • Delay jitter • Packet loss Degradation of qualityof experience (QoE) We need to carry out mediasynchronization control. For efficient media synchronization control It is necessary to clarify the influence of inter-stream synchronization error on QoE.

  4. Purpose Previous Work Influence of inter-stream synchronization error between olfactory media and video is investigated on QoE. (inter-stream synchronization errors between about 30 ms and around +20 ms are hardly perceived). The haptic media are assumed to be synchronized with the visual media for simplicity. No one has investigated influence of inter-stream synchronization error between olfactory and haptic media on QoE. This Work We deal with harvesting fruit in a 3-D virtual space for an olfactory and haptic media display system. By QoE assessment, we investigate the influence of inter-stream synchronization error between olfactory and haptic media.

  5. Olfactory and Haptic Media Display System • A user can touch and catch a grapefruit by manipulating PHANToM. • The smell of grapefruit is diffused by SyP@D2 when a grapefruit is • picked since the picked grapefruit approaches the user’s nose.

  6. Olfactory Display (SyP@D2) SyP@D2 displays a smell to a user by blowing air intoa smell cartridge attached to the main body. • Distance between a user and SyP@D2: About 0.3 m • Intensity of diffusion: Maximum It takes about 2 seconds for a user to perceive the smell.

  7. Functions of Olfactory and Haptic Media Display System Every millisecond Every 10 milliseconds

  8. Method of Assessments (1 of 4) We assume that olfactory and haptic media are synchronized when a user starts to perceive a smell on picking a grapefruit. When we advance output timing of olfactory media When we delay output timing of olfactory media Smell of grapefruit Olfactory media Smell threshold value Buffering Reaction force when pulling a grapefruit Reaction force by weight of grapefruit Haptic media A user catches and starts to pull a grapefruit. The user’s force exceeds 2.45 N. The user releases the grapefruit. : Starting to output the smell : Outputting the reaction force when picking a grapefruit from a tree

  9. Method of Assessments (2 of 4) We generate temporal difference by outputting olfactory media earlier or later than haptic media. We start to output the olfactory media when the force given by a user exceeds smell threshold value. We generate a delay by buffering of the olfactory media. However, If a user picks grapefruit quickly, it is impossible for smell to reach a user before picking grapefruit even if we use the smell threshold value. We employ picking time in order to enable the smell to reach the user before picking grapefruit.

  10. Method of Assessments (3 of 4) Olfactory and haptic media are synchronized when a user starts to perceive a smell on picking a grapefruit. Smell of grapefruit Olfactory media Smell threshold value Picking time Haptic media Reaction force when pulling grapefruit Reaction force by weight of grapefruit A user catch and start to pull a grapefruit. The user’s force exceed 2.45 N. The user release the grapefruit. If the user’s force continues to exceed 2.45N for a specified time, the grapefruit is picked. picking time : Starting to output the smell : Outputting the reaction force when picking a grapefruit from a tree

  11. Method of Assessments (4 of 4) As we increase the picking time, it takes a longer time for a user to pick a grapefruit. It is impossible for smell to reach a user before picking a grapefruit. The feel of picking a grapefruit in our system is different from that of reality. short long picking time We make two assessments (Assessments 1 and 2).

  12. QoE Assessment Method (1 of 2) • Assessments 1 and 2 • Inter-stream synchronization quality • Task: Each subject operates PHANToM and picks grapefruits in the virtual space. • Assessment item: • Total assessment time in each of • Assessments1 and 2: • About half an hour • We make Assessment 1 first. • Then, we make Assessment 2 • on a different day. We obtain mean opinion score (MOS).

  13. QoE Assessment Method (2 of 2) • Subjects: • Number : 20 • Age: Between 21 and 30 • Subjects pick grapefruits while listening to classical music • (The Four Seasons  by Vivaldi). • Reason: To prevent the subject from being influenced by sound of blowing from SyP@D2. • Subjects take a 30-second rest every judgment. • Reason: To prevent him/her from being influenced by lingering smell in the air and human olfactory adaptation.

  14. QoE Assessment Results (1 of 5) • Inter-stream synchronization quality in Assessment 1 (picking time: 0 ms) It takes approximately 2 seconds to perceive smell. About 1380 ms The subjects hardly perceive the inter-stream synchronization error when it takes from about 620 ms to around 1750 ms to perceive the smell after picking a grapefruit About 250 ms Average of inter-stream synchronization error Constant delay of olfactory media Outputting olfactory media later Outputting olfactory media earlier

  15. QoE Assessment Results (2 of 5) • Inter-stream synchronization quality in Assessment 1 (picking time: 0 ms) It takes approximately 2 seconds to perceive smell. About 1380 ms The inter-stream synchronization error is allowable when it takes from about 620 ms to around 2400 ms to perceive the smell after picking a grapefruit. About +400 ms We cannot assess the inter-stream synchronization quality when the inter-stream synchronization error is smaller than about 1380 ms. Outputting olfactory media earlier Outputting olfactory media later

  16. QoE Assessment Results (3 of 5) • Inter-stream synchronization quality in Assessment 2 (picking time: 1500 ms) It takes approximately 2 seconds to perceive smell. The subjects hardly perceive the inter-stream synchronization error when it takes from about 500 ms to around 1000 ms to perceive the smell after picking a grapefruit. About 1500 ms Around1000 ms Constant delay of olfactory media Average of inter-stream synchronization error Outputting olfactory media earlier Outputting olfactory media later

  17. QoE Assessment Results (4 of 5) • Inter-stream synchronization quality in Assessment 2 (picking time: 1500 ms) It takes approximately 2 seconds to perceive smell. The inter-stream synchronization error is allowable when it takes from about 110 ms to around 1400 ms to perceive the smell after picking a grapefruit. Around 600ms About 1890ms Outputting olfactory media earlier Outputting olfactory media later

  18. QoE Assessment Results (5 of 5) • Comparison between Assessments 1 (picking time: 0 ms)and 2 (picking time: 1500 ms) The MOS value of the inter-stream synchronization quality in Assessment 2 is smaller than that in Assessment 1. We adopt results of Assessment 2 for safety. To clarify the reason of this result is for further study. Outputting olfactory media later Outputting olfactory media earlier

  19. Conclusions • When it takes from about 500 ms to around 1000 ms to perceive the smell after picking a grapefruit, the inter-stream synchronization error is hardly perceived. • When it takes from about 110 ms to around 1400 ms to perceive the smell after picking a grapefruit, the Inter-stream synchronization error is allowable. • The difference in the feel of picking between two picking times is not large. We investigated the influence of inter-stream synchronization error between olfactory and haptic media on QoE. We also examine which of picking times (0 ms and 1500 ms) is felt more real.

  20. Future Work • We plan to clarify the reason why the MOS value of the inter-stream synchronization quality in Assessment 2 (picking time: 1500 ms) is smaller than that in Assessment 1 (picking time: 0 ms). • We will investigate the influence of inter-stream synchronization error between olfactory and haptic media when the diffusion of the smell stops after harvesting a grapefruit. • We will examine the influences of packet loss and network delay jitter on QoE. • We will study media synchronization control which takes advantage of the results obtained in this study.

  21. Olfactory display (SyP@D2) • We can attach six cartridges to SyP@D2. • SyP@D2 can diffuse each smell at any time. • There are some parameters of SyP@D2 which we can set up. • Name of smell, Duration of diffusion, Intensity of diffusion • Maximum wind speed of SyP@D2: approximately 0.15 m/s We set the distance between a user and SyP@D2 to about 0.3 m. We measured the average time that it takes for the user to perceive a smell.

  22. QoE Assessment Method We assume that olfactory and haptic media are synchronized when a user ceases to perceive a smell on releasing a grapefruit (if the user release the grapefruit, the grapefruit disappears). • Inter-stream synchronization quality • Task: Each subject operates PHANToM and picks grapefruits in the virtual space. • Assessment item: • Total assessment time: • About half an hour • Subjects: • Number : 20 • Age: Between 21 and 30 We obtain MOS.

  23. QoE Assessment Results (when the diffusion of the smell stops after harvesting a grapefruit) • Inter-stream synchronization quality About 960 ms About 520 ms I 95% confidence interval About +400 ms About 1310 ms Inter-stream synchronization error [ms] Outputting olfactory media earlier Outputting olfactory media later

  24. QoE Assessment Results • Comparison between Assessments 1 (picking time: 0 ms)and 2 (picking time: 1500 ms) When the inter-stream synchronization error is between about 30 seconds and around +20 seconds, the inter-stream synchronization error between olfactory media and video is allowable. The allowable range in this study is narrower than that in previous study. This is because we need higher interactivity in the case where the olfactory and haptic media are perceived together than in the case where the olfactory media and video are perceived together.

  25. QoE Assessment Results (haptic and visual media) • Inter-stream synchronization quality when we change the haptic media I 95% confidence interval Inter-stream synchronization error [ms]

  26. QoE Assessment Results (haptic and visual media) • Inter-stream synchronization quality when we change visual media I 95% confidence interval Inter-stream synchronization error [ms]

  27. Method of Assessments (5 of 5) We change the smell threshold value from 0 N to 2.10 N at intervals of 0.35 N in order to alter the average of the inter-stream synchronization error.

  28. QoE Assessment Method (2 of 3) • We also investigate difference in feel of picking • a grapefruit. • Task: Subjects pick some grapefruits for each value of picking times (0 ms and 1500 ms) and are asked to choose one from among three options. • Total assessment time of this assessment: • Around three minutes • We make this assessment immediately after carrying out Assessment 2. 1: Feel in the case where picking time is set to 0 ms is better. 2: Feel in the case where picking time is set to 1500 ms is better. 3: There is almost no difference between the two cases.

  29. Difference in Feel of Picking between Two Picking Times The difference in the feel of picking between two picking times is not large.

More Related