1 / 28

A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition

A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition. † Fujiko Yara (fujiko@mdbl.sfc.keio.ac.jp) ‡ Naofumi Yoshida (naofumi@mdbl.sfc.kei.ac.jp) ‡ Shiori Sasaki (sashiori@mdbl.sfc.keio.ac.jp) † Yasushi Kiyoki (kiyoki@mdbl.sfc.keio.ac.jp)

shiloh
Download Presentation

A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Continuous Media Data Rendering System For Visualizing Psychological Impression-Transition †Fujiko Yara (fujiko@mdbl.sfc.keio.ac.jp) ‡Naofumi Yoshida (naofumi@mdbl.sfc.kei.ac.jp) ‡Shiori Sasaki (sashiori@mdbl.sfc.keio.ac.jp) †Yasushi Kiyoki (kiyoki@mdbl.sfc.keio.ac.jp) † Faculty of Environmental Information, Keio University ‡ Graduate School of Media and Governance, Keio University

  2. (ﾟﾟ;)(｡｡;))((;ﾟﾟ)(;｡｡) (*´ｪ`*) The statement of user is “confuse”. The statement of user is “comfortable”. Lighting ♪ ♪ Picture ♪ ♪ ～ ～ Music ～ Smell confuse comfortable Application Example and Demonstration • Our system enables us to render the continuous impression-transition for making user’s feeling change. • Our system calculates the relationship between media data and impression. • And then renders set of media data continuously. • The demonstration shows an example of rendering media data. Demonstration Input : from “confuse” to “comfortable” Output : Sequence of colors by emotionally continuous transition Demonstration STRT FINISH t

  3. Three Main Features in our system • Our system enables us to render media data with continuous impression-transition to guide user’s feelings by psychological models. • There are a lot of variety of routes. Because of many models and routes, our system renders the appropriate set of media data in the several scenes for users. • Our system use the psychological impression-transition models that can be applied to render various media continuously.

  4. Allocated by impression-words CIS-model Allocated by color gradation Hevner-model Our Understanding of the Impression-Transition Models We believe that it is necessary to visualize psychological impression-transition for rendering continuous media data emotionally. • In psychological impression-transition models, many elements (for example, color data, impression-words) are continuouslyallocated by something to define. • CIS (Color Image Scale) model allocates color data into 2 dimensional manner, and Hevner-model allocates impression-words into 1 dimensional manner. • We think the continuous relationships between each elements in these models show emotionally continuous transition.

  5. Overview • In this presentation, we show an implementation method and effectiveness of our method . • We demonstrate several experiments for the feasibility. • We explain about our understanding of Impression-Transition Model and how to apply to our system.

  6. Our target DB Design Retrieval technology Rendering technology Rendering media data with emotionally sequence • [KK_1993] Kitagawa, T. and Kiyoki, Y. :The mathematical model of meaning and its application to multidatabase systems, Proceedings of 3rd IEEE International Workshop on Research Issues on Data Engineering Interoperability in Multidatabase Systems, April 1993, 130-135 • [KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41. • [IKNS_2003] shibashi, N. Kiyoki, Y. Nakagami, Y. and Sato, A. : An Impressionistic Metadata Extraction Method for Music Data with Multiple Note Streams, DBSJ Letters, Vol.2, No.2, October 2003, pp.61-64 . • [IK_2005] Ijichi, A. and Kiyoki, Y.:A Kansei Metadata Generation Method for Interpretation, Information Modeling and Knowledge Bases, 16, 170-182, 2005. Background Issues • Metadata of Multimedia management system • Media data searching method and system [IK_2005] [KK_1994] • Metadata extraction method and system [IKNS_2003] • Multi Media Data Base • Meta Data Base System [KK_1993] • Data Base Management system Retrieval Results • Psychological Field • Psychological Field • Image transaction technology

  7. start-query goal-query result 1 ▲ ▲ Vector Space CIS-model ・・・ terminal-point query starting-point query starting-media data ■ ▲ ◆ Hevner-model ◆ ■ ▲ terminal-media data ▲ ▲ Impression-Transition DB ▲ ▲ ■ terminal-impression word starting-impression word ■ ■ ■ ▲ ■ Start-media data Goal-media data Start-point-query Goal-point-query ■ ▲ ▲ ■ ■ Multi Media DB System Overview Metric for rendering emotionally continuous transition Metric for distances on vector spaces Sensor ＤＢ Color ＤＢ Music ＤＢ

  8. terminal-point query starting-point query starting-media data ■ ▲ ◆ ◆ ■ ▲ terminal-media data ▲ ▲ ▲ ▲ ■ terminal-impression word starting-impression word ■ ■ ■ ▲ ■ ■ ▲ ▲ ■ ■ Basic Method (1/2) • Calculation in Metric for distances on vector space. • Creating an impression-words vector space to calculate the impression retrieval. • Impression-words and media data are mapped into this vector space. • Converting 2-queries into 2-media-data. ▲ ▲ ■Impression-Words in Longman Dictionary ▲　Impression-Words expressing Media data ◆ Query words

  9. ● ・・・ ● ● ● ● ● ・・・ ● ● ● ● ● ● ● ● ● ● ・・・・・・・ Based Point Based Point W (n) W (1) W (n-1) ・・・ W (1) W (2) W (3) W (n) W (2) ● ● ・・・ ● ● ● ・・・ ● ● ● ● ● W (n-2) ・・・ W (n-1) ● ● ・・・ ● ● ● ● ● Basic Method (2/2) • Calculating in Metric for rendering emotionally continuous transition. • Using transition model representing something to meant. • Creating database of transition’s routes. • Defining many parameters to decide the route. • Mapping 2-media data (start-media data and terminal-media data) into transition model. • Converting data on transition model into output media. CIS-model Hevner-model Sets of rendering output media data

  10. ■ ▲ ■ ▲ ▲ ▲ ■ ■ ▲ ■ ▲ ▲ ■ ▲ ■ ▲ ■ ■ ■ ■ ▲ ▲ ■ ■ ■Impression-Words in Longman Dictionary▲　Impression-Words expressing Media data • [KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41. • Longman Dictionary of Contemporary English, Longman, 1987 途中！ Implementation Method (1/8) • Step 1 : Creating an impression-words vector space of media data and mapping the impression-words of media contents into it. • We have implemented this system using the Mathematical Model of Meaning (MMM) [KK_1994]. • The MMM search space is created by using the Longman Dictionary of Contemporary English. • The impression-words of media data are mapped into this impression-word vector space.

  11. 途中！ Implementation Method (2/8) • Step 2 : Creating database representing the route of the impression-transition. • By using psychological models as impression-transition models, we create databases which express the route of the impression-transition. • Using the Hevner-model and the CIS-model (Color Image Scale model) as impression-transition.

  12. ■ terminal-query start-query ▲ ■ ・ ・ ▲ ▲ ▲ ▲ ▲ ■ ▲ start-impression ■ ■ ■ terminal-impression ▲ ■ ■ ■ ■ ▲ ▲ ■ ■ 途中！ Implementation Method (3/8) • Step 3 : Submitting two query words (starting-query and terminal-query) into the impression space. • A user submits two query words into MMM search space created in step1. • Two query are not always words used in Longman Dictionary, so in next step, two query words are converted into the words used in Longman Dictionary. ■Impression-Words in Longman Dictionary ▲　Impression-Words expressing Media data ● Query words

  13. ■ terminal-query starting-query ▲ ■ ・ ・ ▲ ▲ ▲ ▲ ▲ ■ ▲ start-impression ■ ■ ■ terminal-impression ▲ ■ ■ ■ ■ ▲ ▲ ■ ■ 途中！ Implementation Method (4/8) • Step 4 : Converting two query words as two impression words (starting-impression and terminal-impression). • The starting-query is converted into the semantically closest word (starting-impression) within the impression words included in the route representing the impression transition (Step2). • The terminal-query is also converted into the semantically closest word (terminal-impression). ■Impression-Words in Longman Dictionary ▲　Impression-Words expressing Media data ● Query words

  14. start-impression terminal-impression ■ ▲ ▲ ▲ ■ ■ ▲ ■ start-media data ▲ ■ ▲ ▲ terminal-media data ■ ■ ■ ▲ ▲ ■ ■ 途中！ Implementation Method (5/8) • Step 5 : Converting impression-words to media data (starting-media data and terminal-media data) ■Impression-Words in Longman Dictionary▲　Impression-Words expressing Media data

  15. CIS-model Hevner-model 途中！ Implementation Method (6/8) • Step 6 :Extracting two media data from vector space and mapping into the impression-transition models respectively. • Two media data (Starting-media data and terminal-media data) are mapped into the database based on the impression-transition models. • The appropriate route is chosen for connecting starting-media data and terminal-media data continuously.

  16. Allocated by impression-words CIS-model Allocated by color gradation Hevner-model 途中！ Implementation Method (7/8) • Step 7 : Choosing the route on the impression-transition model for rendering media data for visualization by color data. • To realize rendering the media data continuously, the route from starting-media data to terminal-media data is chosen. • According to taking time, the way how to walk on the impression-transition is decided. • The way how to walk must be uniformed like chess.

  17. 途中！ Implementation Method (8/8) • Step 8 ：Rendering the sequence of output media data generated in Step7. • We render the set of color data generated in Step7 along the selected route using the impression-transition model. • A personal computer is used to display the rendering of the set of color data.

  18. (start-point-query=“confuse”,goal-point-query=”comfortable”(start-point-query=“confuse”,goal-point-query=”comfortable” R=shortest (右回り)) Results Example (start-point-query=“confuse”,goal-point-query=”comfortable”, R=longest (左回り)) (start-point-query=“confuse”,goal-point-query=”comfortable” R=第４節5(B)) Experimental Results (1/3)[confuse → comfortable] • These results of the set of color data show the feasibility of our continuous media data rendering system for visualizing the change of psychological impression-transition. CIS-model Hevner-model

  19. CIS-model Hevner-model (start-point-query=”merry”, goal-point-query=”calm” R=shortest (右回り)) (start-point-query=”merry”, goal-point-query=”calm” R=longest (左回り)) Results Example (start-point-query=“merry”, goal-point-query=“calm” R=longest (Right and Down)) (start-point-query=“merry”, goal-point-query=“calm” R=longest (Down and Right)) (start-point-query=“merry”, goal-point-query=“calm” R=shortest (Left and Down)) (start-point-query=“merry”, goal-point-query=“calm” R=shortest (Down and Left)) Experimental Results (2/3)[merry → calm] • These experiments have shown the applicability of our method for user’s various requirement of impression-transition.

  20. CIS-model Hevner-model Results Example (start-point-query=“simple” goal-point-query=“graceful”, R=longest (Right and Down)) (start-point-query=“simple”goal-point-query=“graceful”, R=longest (Right and Down)) (start-point-query=“simple”goal-point-query=“graceful”, R=longest (Right and Down)) (start-point-query=“simple”, goal-point-query=“graceful”) (start-point-query=“simple”goal-point-query=“graceful”, R=longest (Right and Down)) Experimental Results (3/3)[simple → graceful] • Our method has applicability for various strength of relationship between two query words, even if the two impression-words have a weak relationship in the vector space.

  21. Summary and Future Work • Our method makes it possible to implement visualizations of the continuous change of impression-transition, according to impression-words expressed for starting point and terminal point. • By the implementation and experiments using color data as output media data, we have clarified the feasibility of our method for visualizing the change of impression-transition from starting point to terminal point by using the research results of musical psychology and color psychology. • We will design aggregate functions for color data expression in the experiments using psychological word groups by the Hevner-model. • We will approach to the computation mechanisms of continuous transition of impression.

  22. References • [KK_1993] Kitagawa, T. and Kiyoki, Y. :The mathematical model of meaning and its application to multidatabase systems, Proceedings of 3rd IEEE International Workshop on Research Issues on Data Engineering Interoperability in Multidatabase Systems, April 1993, 130-135 • [KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41. • [KKH_1995] Kiyoki, Y. Kitagawa, T. and Hitomi, Y. : A fundamental framework for realizing semantic interoperability in a multidatabase environment, Journal of Integrated Computer-Aided Engineering, Vol.2, No.1, Jan.1995, 3-20. • [AS_1994] Aiello, R. and Slobada, J.A.: Musical perceptions, Oxford University Press, 1994. • [IK_2005] Ijichi, A. and Kiyoki, Y.:A Kansei Metadata Generation Method for Interpretation, Information Modeling and Knowledge Bases, 16, 170-182, 2005. • Longman Dictionary of Contemporary English, Longman, 1987 • [H_1937] Hevner, K. : The affective value of pitch and tempo in music, American Journal of psychology, 49, 621-630. • [IKNS_2003] shibashi, N. Kiyoki, Y. Nakagami, Y. and Sato, A. : An Impressionistic Metadata Extraction Method for Music Data with Multiple Note Streams, DBSJ Letters, Vol.2, No.2, October 2003, pp.61-64 .

  23. Thank you for your attention.

  24. Derivation of Kansei • 印象遷移モデルは一定を約束している • 別のストーリを持つモデルは多く存在し、 • 微分係数が一定でないものが多い。

  25. ▲ ▲ Vector Space terminal-point query starting-point query starting-media data ■ ▲ ◆ ● ・・・ ◆ ■ ● ● ▲ ● terminal-media data ▲ ▲ ● ● ・・・ ● ● ● ● ● ▲ ● ● ● ● ▲ ■ ● terminal-impression word ・・・・・・・ starting-impression word ■ ■ Based Point Based Point ■ ▲ W (n) W (1) W (n-1) ■ ・・・ W (1) ■ ▲ W (2) W (3) ▲ ■ W (n) ■ W (2) ・・・ W (n-2) W (n-1)

  26. c1 lofty c2 c8 spiritual dignified heavy inspiring serious gloomy depressing vigorous sacred awe pathetic sad exalting emphatic sober solemn dark tragic martial majestic doleful mournful robust ponderous frustrated melancholy c3 c7 triumphant yielding agitated exhilarated longing dreamy exciting dramatic plaintive sentimental impetuous soaring tender yearning restless passionate sensational c6 c4 calm leisurely gay cheerful c5 lyrical satisfying happy bright quiet serene quaint joyous merry soothing tranquil graceful fanciful delicate humorous light whimsical sprightly playful

  27. ○ ● ：ＭＤ ：印象語 始点の印象語 終点の印象語 S:Start-query t:goal-query 心理学的印象モデルCo群 ○ ○ ○ ○ ○ ○ ○ Hevner’s model[4,5] ○ ○ ○ ○ ○ ○ ○ ○ ・・・・・・ ○ CIS model[6] ○ Neuton’s model Mansel’s model メディアデータＭＤで出力 メディアデータＭＤで出力 ● ● ● ● ● ● ● ● ・・・・・・・・ R Hev Hevner’ modelでのルート群 ・・・・・・・・ n ・・・・・・・・ ● ● ● ● ● ● ● ● ● ● R CIS ・・・・・・・・ ● ● ● ● ● ● ● ● ● ● CIS model でのルート群 ● ● ● ● ● ● ● ● ● ● ・・・・・・・・ ● ● ● ● ● ● ● ● ● ● n ● ● ● ● ● ● ● ● ● ● ・・・・・・・・ ● ● ● ● ● ● ● ● ● ●

  28. c1 lofty c2 c8 spiritual dignified heavy inspiring serious gloomy depressing vigorous sacred awe pathetic sad exalting emphatic sober solemn dark tragic martial majestic doleful mournful robust ponderous frustrated melancholy c3 c7 triumphant yielding agitated exhilarated longing dreamy exciting dramatic plaintive sentimental impetuous soaring tender yearning restless passionate sensational c6 c4 calm leisurely gay cheerful c5 lyrical satisfying happy bright quiet serene quaint joyous merry soothing tranquil graceful fanciful delicate humorous light whimsical sprightly playful

More Related