1 / 27

A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition

A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition. † Fujiko Yara (fujiko@mdbl.sfc.keio.ac.jp) ‡ Naofumi Yoshida (naofumi@mdbl.sfc.kei.ac.jp) ‡ Shiori Sasaki (sashiori@mdbl.sfc.keio.ac.jp) † Yasushi Kiyoki (kiyoki@mdbl.sfc.keio.ac.jp)

yanni
Download Presentation

A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition †Fujiko Yara (fujiko@mdbl.sfc.keio.ac.jp) ‡Naofumi Yoshida (naofumi@mdbl.sfc.kei.ac.jp) ‡Shiori Sasaki (sashiori@mdbl.sfc.keio.ac.jp) †Yasushi Kiyoki (kiyoki@mdbl.sfc.keio.ac.jp) † Faculty of Environmental Information, Keio University ‡ Graduate School of Media and Governance, Keio University

  2. I feel lonely, tired…. Fufufu♪ Lighting ♪ ♪ Picture ♪ ♪ ～ ～ Music ～ Smell Application Example and Demonstration • We will show the demonstration of our system. • Our system makes a user’s feeling change by rendering media data continuously.

  3. Overview In this presentation, we show • Implementation method of continuous media data rendering system to visualizing a psychological impression-transition. • Experimental results for the feasibility of our system.

  4. Background Issues Data Engineering Research Field • Several media-data-search methods based on a vector space models have been proposed. ex. Music retrieval [IK_2005][IKNS_2003] Image retrieval [KK_1994] Psychology Knowledge • Impression-Transition Models have been defied. ex. Hevner-model [H_1937] CIS (Color Image Scale) -model [IK_2005] Ijichi, A. and Kiyoki, Y. : A Kansei Metadata Generation Method for Interpretation, Information Modeling and Knowledge Bases, 16, 170-182, 2005. [H_1937] Hevner, K : The affective value of pitch and tempo in music. American Journal of Psychology, 49, 621-630, 1937. [KK_1994]Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41. [IKNS_2003] Ishibashi, N. Kiyoki, Y. Nakagami, Y. and Sato, A. : An Impressionistic Metadata Extraction Method for Music Data with Multiple Note Streams, DBSJ Letters, Vol.2, No.2, October 2003, pp.61-64 . [KK_1993] Kitagawa, T. and Kiyoki, Y. :The mathematical model of meaning and its application to multidatabase systems, Proceedings of 3rd IEEE International Workshop on Research Issues on Data Engineering Interoperability in Multidatabase Systems, April 1993, 130-135

  5. result 1 start-query goal-query Transition metric Lw1 Distance metric ■ ▲ Lw2 start-query ■ ▲ goal-query Iw2 ・・・ ▲ ▲ ■ ■ ▲ Lw3 CIS-model ■ start-media data ▲ ▲ Ｌw1 ■ ▲ ■ Iw3 ▲ ■ terminal-media data ■ ■ ■ ▲ ▲ ■ ■ Hevner-model Impression-Transition DB Vector Space Start-point-query Goal-point-query Start-media data Goal-media data Multi Media DB Sensor ＤＢ Color ＤＢ Music ＤＢ System Overview

  6. Allocated by impression-words CIS-model Allocated by color gradation Hevner-model Impression-Transition Model and Derivation of Kansei • The necessity of continuous impression-transition. • In psychological impression-transition models, media data are allocated continuously by something to define. • We think the continuous relationships of these models show derivation of Kansei stay constant.

  7. ■ ▲ ■ ▲ ▲ ▲ ■ ■ ▲ ■ ▲ ▲ ■ ▲ ■ ▲ ■ ■ ■ ■ ▲ ▲ ■ ■ ■Impression-Words in Longman Dictionary▲　Impression-Words expressing Media data • [KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41. • Longman Dictionary of Contemporary English, Longman, 1987 Implementation Method (1/8) • Step 1 : Creating an impression-words vector space of media data and mapping the impression-words of media contents into it. • We have implemented this system using the Mathematical Model of Meaning (MMM) [KK_1994]. • The MMM search space is created by using the Longman Dictionary of Contemporary English. • The impression-words of media data are mapped into this impression-word vector space.

  8. Implementation Method (2/8) • Step 2 : Creating database representing the route of the impression-transition. • By using psychological models as impression-transition models, we create databases which express the route of the impression-transition. • Using the Hevner-model and the CIS-model (Color Image Scale model) as impression-transition.

  9. ■ terminal-query start-query ▲ ■ ・ ・ ▲ ▲ ▲ ▲ ▲ ■ ▲ start-impression ■ ■ ■ terminal-impression ▲ ■ ■ ■ ■ ▲ ▲ ■ ■ Implementation Method (3/8) • Step 3 : Submitting two query words (starting-query and terminal-query) into the impression space. • A user submits two query words into MMM search space created in step1. • Two query are not always words used in Longman Dictionary, so in next step, two query words are converted into the words used in Longman Dictionary. ■Impression-Words in Longman Dictionary ▲　Impression-Words expressing Media data ● Query words

  10. ■ terminal-query starting-query ▲ ■ ・ ・ ▲ ▲ ▲ ▲ ▲ ■ ▲ start-impression ■ ■ ■ terminal-impression ▲ ■ ■ ■ ■ ▲ ▲ ■ ■ Implementation Method (4/8) • Step 4 : Converting two query words as two impression words (starting-impression and terminal-impression). • The starting-query is converted into the semantically closest word (starting-impression) within the impression words included in the route representing the impression transition (Step2). • The terminal-query is also converted into the semantically closest word (terminal-impression). ■Impression-Words in Longman Dictionary ▲　Impression-Words expressing Media data ● Query words

  11. start-impression terminal-impression ■ ▲ ▲ ▲ ■ ■ ▲ ■ start-media data ▲ ■ ▲ ▲ terminal-media data ■ ■ ■ ▲ ▲ ■ ■ Implementation Method (5/8) • Step 5 : Converting impression-words to media data (starting-media data and terminal-media data) ■Impression-Words in Longman Dictionary▲　Impression-Words expressing Media data

  12. CIS-model Hevner-model Implementation Method (6/8) • Step 6 :Extracting two media data from vector space and mapping into the impression-transition models respectively. • Two media data (Starting-media data and terminal-media data) are mapped into the database based on the impression-transition models. • The appropriate route is chosen for connecting starting-media data and terminal-media data continuously.

  13. Allocated by impression-words CIS-model Allocated by color gradation Hevner-model Implementation Method (7/8) • Step 7 : Choosing the route on the impression-transition model for rendering media data for visualization by color data. • To realize rendering the media data continuously, the route from starting-media data to terminal-media data is chosen. • According to taking time, the way how to walk on the impression-transition is decided. • The way how to walk must be uniformed like chess.

  14. Implementation Method (8/8) • Step 8 ：Rendering the sequence of output media data generated in Step7. • We render the set of color data generated in Step7 along the selected route using the impression-transition model. • A personal computer is used to display the rendering of the set of color data.

  15. (start-point-query=“confuse”,goal-point-query=”comfortable”(start-point-query=“confuse”,goal-point-query=”comfortable” R=shortest (右回り)) Results Example (start-point-query=“confuse”,goal-point-query=”comfortable”, R=longest (左回り)) (start-point-query=“confuse”,goal-point-query=”comfortable” R=第４節5(B)) Experimental Results (1/3)[confuse → comfortable] • These results of the set of color data show the feasibility of our continuous media data rendering system for visualizing the change of psychological impression-transition. CIS-model Hevner-model

  16. CIS-model Hevner-model (start-point-query=”merry”, goal-point-query=”calm” R=shortest (右回り)) (start-point-query=”merry”, goal-point-query=”calm” R=longest (左回り)) Results Example (start-point-query=“merry”, goal-point-query=“calm” R=longest (Right and Down)) (start-point-query=“merry”, goal-point-query=“calm” R=longest (Down and Right)) (start-point-query=“merry”, goal-point-query=“calm” R=shortest (Left and Down)) (start-point-query=“merry”, goal-point-query=“calm” R=shortest (Down and Left)) Experimental Results (2/3)[merry → calm] • These experiments have shown the applicability of our method for user’s various requirement of impression-transition.

  17. CIS-model Hevner-model Results Example (start-point-query=“simple” goal-point-query=“graceful”, R=longest (Right and Down)) (start-point-query=“simple”goal-point-query=“graceful”, R=longest (Right and Down)) (start-point-query=“simple”goal-point-query=“graceful”, R=longest (Right and Down)) (start-point-query=“simple”, goal-point-query=“graceful”) (start-point-query=“simple”goal-point-query=“graceful”, R=longest (Right and Down)) Experimental Results (3/3)[simple → graceful] • Our method has applicability for various strength of relationship between two query words, even if the two impression-words have a weak relationship in the vector space.

  18. Summary • Our method makes it possible to implement visualizations of the continuous change of impression-transition, according to impression-words expressed for starting point and terminal point. • By the implementation and experiments using color data as output media data, we have clarified the feasibility of our method for visualizing the change of impression-transition from starting point to terminal point by using the research results of musical psychology and color psychology.

  19. Future Work • We will design aggregate functions for color data expression in the experiments using psychological word groups by the Hevner-model. • We will approach to the computation mechanisms of continuous transition of impression.

  20. References • [KK_1993] Kitagawa, T. and Kiyoki, Y. :The mathematical model of meaning and its application to multidatabase systems, Proceedings of 3rd IEEE International Workshop on Research Issues on Data Engineering Interoperability in Multidatabase Systems, April 1993, 130-135 • [KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41. • [KKH_1995] Kiyoki, Y. Kitagawa, T. and Hitomi, Y. : A fundamental framework for realizing semantic interoperability in a multidatabase environment, Journal of Integrated Computer-Aided Engineering, Vol.2, No.1, Jan.1995, 3-20. • [AS_1994] Aiello, R. and Slobada, J.A.: Musical perceptions, Oxford University Press, 1994. • [IK_2005] Ijichi, A. and Kiyoki, Y.:A Kansei Metadata Generation Method for Interpretation, Information Modeling and Knowledge Bases, 16, 170-182, 2005. • Longman Dictionary of Contemporary English, Longman, 1987 • [H_1937] Hevner, K. : The affective value of pitch and tempo in music, American Journal of psychology, 49, 621-630. • [IKNS_2003] shibashi, N. Kiyoki, Y. Nakagami, Y. and Sato, A. : An Impressionistic Metadata Extraction Method for Music Data with Multiple Note Streams, DBSJ Letters, Vol.2, No.2, October 2003, pp.61-64 .

  21. Thank you for your attention.

  22. Derivation of Kansei • 印象遷移モデルは一定を約束している • 別のストーリを持つモデルは多く存在し、 • 微分係数が一定でないものが多い。

  23. Allocated by impression-words １ ２ 20 ３ 19 ４ CIS-model ・ bright ・ Allocated by color gradation conspicuous 15 ・ ・ １０ Hevner-model The Impression-Transition Model and Liner relationship • There is no relationship between color and impression-words. • There is no relationship between color and impression-words.

  24. 言葉を連続にした、照明色ＤＢの構築 • 仮定１：心理情況遷移の仮定は連続し、 　　　　　　かつサイクルをなす。 • 仮定２：色相サイクルと、心理情況遷移 　　　　　　サイクルは対応する。 • 用語の設定 　　言葉：心理情況を示す単語

  25. 仮説の検証マンセルの色円を用いた場合 • 色円の代表色に番号をつける。 • 言葉と色との間に線形性があるかどうかを、千々岩先生の色別イメージ・プロフィール(色の印象評価アンケート結果)を用い検討する。 １ ２ 20 ３ 19 ４ ・・ ・・ 15 ・・ ・・ １０ ・・ 上記の体表色は、マンセルにより20等分されたものである。 １ １

  26. マンセルの色円と言葉の間に線形性はあるか？マンセルの色円と言葉の間に線形性はあるか？ • 結果　　・・・・ないっぽい

More Related