Sharing multimedia and context information between mobile terminals

1. 2006 지식기반 시스템 수업 발표 자료: 정보생성 Sharing multimedia and context information between mobile terminals T. Horozov, N. Narasimhan, and V. Vasudevan Soft computing Laboratory Yonsei University 박문희 Dec 5, 2006 International Symposium on Applications and the Internet (SAINT'06), pp.124-129, 23-27 January 2006.

2. Outline • Introduction • Techniques for creating and sharing multimedia • Crossing application boundaries • Maps • Context information • Prototype • Summary and discussion

3. Introduction • Sharing of experiences using mobile is becoming more common • Due to physical limitations of mobile terminals • Requiring particular UI solution to present and process multimedia • Context awareness of mobile terminals • Sharing and presenting contexts by showing context of their members • e.g context based call operation and messaging • an approach for a user interface for mobile terminals • Sharing multimedia and context information • Map-based interface and domain object model-based user interface

4. Crossing application boundaries • Distinct applications • Map-based positioning , taking photos or video shots, playing the media • sharing the media files created and showing the current context of each user • Should not be any separate applications, but objects and operations that can manipulate those objects • The Naked Objects framework • maintaining correspondence of the domain or business object model and the UI • Our object model for enabling multimedia communication • People in the community • The multimedia files they create and share • A shared map acting as a container object for people and multimedia files

5. Maps • Characters of geographical maps • Direct representations of the real world already familiar to users • Useful for presenting and finding electronic services having an unambiguous geographical location • The ability to visualizeheterogeneous objects being either physical like people or immaterial like video clips, if including location information • Utilize this feature in this user interface • Sharing context information and multimedia objects on a map • Minimal user efforts to communicate their position and the context • The oldest media objects can be removed from the map

6. Context information • Represent the current state of the object or its environment as pictures • Classification of UI pictures for small interface • Iconic • Index • Symbolic picture • In this system, • Iconic picture like Table 1. • First class • user activity • Second class • environment • device activity Table 1. Context information with classes used in user interface

7. Prototype • Context-aware map-based interface for accessing situated services with mobile terminals • On the Compaq iPAQ3660 PDA • Positioning: via WLAN • Context based control: via an external sensor • Exploring the Naked Objecs framework • for extending the Naked Objects platform by implementing an OVM for PocketPC style devices • Fig 1. Screenshots of a UI

8. Summary and discussion • UI solution for mobile terminals presenting and sharing multimedia with context information • UI solution • Sharing interesting from the environment • Multimedia data are presented on the map as icons • Online sharing of context information (activity, device and environment) with simple , not descriptive symbols • Future works • Continuing the integration of the map interface to the Naked Objects platform • Usability test in real usage situations

9. 2006 지식기반 시스템 수업 발표 자료: 정보생성 ComicDiary: Representing Individual Experience in a Comics Style • Yasuyuki Sumi, Ryuuki Sakamoto, Keiko Nakao, Kenji Mase Soft computing Laboratory Yonsei University 박문희 Dec 5, 2006 UbiComp 2002: Ubiquitous Computing : 4th International Conference, Göteborg, Sweden, September 29 - October 1, 2002.

10. Outline • Associative work • Introduction • Using comics to exchange individual experiences • Related works • Implementation of ComicDiary • System architecture of ComicDiary • Story generating • Rendering comic frames • Comic examples and user evaluations • Examples of generated comic diaries • User evaluations and discussion • Summary and conclusion

11. Associative work • Life-like character-based systems developed in the context of our personal guidance system • PalmGuide: a handheld tour guidance system • AgentSalon: display showing conversationsbetween personal agents according to their users’ profiles and interests • ComicDiary: a system representing individual experiences in the style of a comic

12. Introduction • C-MAP: Context-aware Mobile Assistant Project • To build a tour guidance system personalized according to its user’s individual contexts • To facilitate knowledge communications among community members • ComicDiary • A tool of to record their touring summaries • The process of generating a comic diary • determining the story stream of a comic diary from fragmentary data of the user’s touring records • rendering comic frames for assembling the necessary characters and scene backgrounds

13. ComicDiary Fig. 1. Examples of hand-drawn diary in the style of a comic

14. Using comics to exchange individual experiences • Representing Individual Experiences in a Comic Style • Characteristics of comic report • Structured according to the subjective viewpoint • Comic character has its own identity and personality as alter ego • Exaggerates her personal impressions projecting them onto the alter-ego character • Highlighting impressive events • describing the surroundings and other visitors • Jokes and small talk – entertainment value • not appropriate for precisely conveying detailed events • excellent for sharing personal impressions and edpisodes

15. Related Works • Story generation in the context of AI and cognitive science • [Rummelhart’75] tried to build a grammatical schema of stories by analyzing traditional fables • [Schank and Abelson’77] regarded AI issues e.g. problem solving and planning, as story generation • Several systems of story generation [Okada’92, Ogata’96] • Comic style as a user interface representation • Comic Chat for representing an online chat history as a comic [Kurlander’96] • Video Manga for visualizing a video summary in comic style [Uchihashi’99] • Chat Scape for using captured camera images to represent an online chat history as a comic [Ayatsuka’01]

16. Implementation of ComicDiary • The first prototype of ComicDiary • Participants to an academic conference Fig2 . Scenario of system usage Fig3 . Accumulation of personal information

17. System architecture of ComicDiary • System Architecture of CmicDiary • Macromedia Flash running on web browsers • Diary generation was processed at a web server • Processing for diary generation • Clients certify users • user log in / automatically sent from the PalmGuide • Requests to generate comic diaries with the user IDs are sent to the server via CGI • Server extracts the user’s personal data from the community DB • Generate a comic diary • A Flash file (SWF) and parameter data are sent to the client

18. Story Generation • Story Generation Engine • Use personal profile, community profile • Potential streams of a comic story are prepared as a constraint networkrepresenting mutual dependenciesamong scenes

19. Rendering Comic Frames • To increase the variety of comic frames from limited resources • Superimpose several layers • Reuse background layers • Fig.6 Frame composition by multiple layers

20. ComicDiary Illustrations • 44 illustrations for each character • 8 kinds of characters : 352 appearances of main character layers • Templates of word layers • Presentation title • User name • Fig. 7. Comic-part data (of main character layer)

21. Examples: For presenters Generated Comic Diary (1) • 3 types of story outlines • For presenters, for active attendees, for non-active attendees • Fig. 8. Example of generated comic diary (for a presenter) 4 frames were used for the user's presentation scene, followed by scenes of meeting with other participants

22. Examples: For active attendance Generated Comic Diary(2) • Starts with a cheerful atmosphere • Embedded with many scenes of attending presentations • Fig. 9. Example of generated comic diary (for active attendance)

23. Examples: For non-active attendance Generated Comic Diary(3) • Presented with a negative (but amusing) mood • Fig. 10. Example of generated comic diary (for non-active attendance)

24. A diary in hypertext style • Ordinary method

25. User Evaluations and Discussion • 52 users participated in the digital assistant project in 2001 • After 5 weeks, 16 replied of survey. How many people do you show your comic diary to others? Do you think the comic diary encourage conversations? Does the comic exactly represent your memory? which is better to show to other people, comic style or hyper-text style?

26. Summary and Conclusions • Represent individual experience and interests in a comics style • ComicDiary: a character agent • To summarize data ubiquitously obtained • To facilitate the exchange of personal experiences among users • As a digital assistant service for conference participants • Future works • Knowledge processing techniques • To build a framework of flexible user-modeling from the user data • To increase the variety of stories generated according to the user-model • Computer graphics techniques • To employ a morphable model of characters to decrease the cost of preparing comic parts for a charater