1 / 62

MIDAS Multi-device Integrated Dynamic Activity Spaces

MIDAS Multi-device Integrated Dynamic Activity Spaces. Unmil P. Karadkar. Ph.D. Candidate Dept. of Computer Science Texas A&M University. Research Questions. Theory Which aspects of information content do humans find valuable? Techniques

ophrah
Download Presentation

MIDAS Multi-device Integrated Dynamic Activity Spaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University

  2. Research Questions • Theory • Which aspects of information content do humans find valuable? • Techniques • How to adapt content while optimizing for human perception? • How to enable heterogeneous device co-use? • Frameworks • What are the attributes of multi-device systems? • How options are available to system designers for each attribute? • System • Design a flexible architecture to embody multi-device system behaviors • Proof-of-concept implementation and analysis • Practice • Harness device multiplicity to accomplish significant tasks

  3. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  4. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  5. 6” 7” 4.5” Custom 10” 14” 17” 75” 21-24” 40”- 60” 3.5” 13”- 17” • Display characteristics • Network bandwidth • Processing power • Interaction modalities

  6. Societal Changes • User demographics • Nature of activity • Access location • Affordability • Free phones with contract • $150 e-readers • Multiplicity • iPhone, iPad, MacBookPro Heterogeneous contexts of information access

  7. MIDAS Multi-device Integrated Dynamic Activity Spaces M I D A S

  8. Design Goals • Deliver information elements to multiple devices simultaneously • Content • Interaction • Rely on widely available infrastructure • Wired and wireless computer networks • Mobile phones, e-readers, tablets, desktop computers • Sensors embedded in the environment • Support content transformation • Within form (image scaling, text summarization) • Across forms (text to speech, audio extraction from video)

  9. Design Goals • Strive to retain content integrity from a human perspective • Select transformations that a user perceives to be closest to the original form • Direct content elements to devices that can present them optimally • Reconfigure presentation dynamically in response to changes in device availability • New devices become available • Current devices become unavailable

  10. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  11. Background Individual devices Opportunistic device co-use Ubiquitous Computing Infrastructure-enabled device co-use device-based content adaptation • Service separation • Yahoo!, CNN, Weather Channel, UT Mobile • Image distillation • Digestor (Fox 1996) • Text and Web form summarization • Pythia (Bickmore 1997) • Power Browser (Buyukkokten 2002) • Proteus (Anderson 2001) • Web page content extraction • Baseline SVPs + device-based PVPs (Chua 2007) • Location-aware services • Computer for the 21st century (Weiser1991) • Java Ring (Olsen 2001) • IntelliBadgeTM(Cox2003) • Mobile devices in interactive spaces • Personal Server (Want 2002) • InfoStick(Kohtake1999) • Elope (Pering2005)

  12. Background—Device co-use • Pebbles(Myers 1997) • Desktop and PDA co-use • Extend Windows applications to PDAs • PDAs as input and control devices • WebSplitter(Han 2000) • Split Web content across a user’s devices • Support for multiple views • ANMoLE (Haneef 2004) • Dynamic content reformulation (splitting video) • Integration of phone and computer networks

  13. Background—Device co-use • Ubiquitous Display System (Aizawa 2002) • Large public displays and mobile phones • Control from mobile phone • Multibrowsing (Johanson 2001) • Desktops, notebooks, PDAs, and large displays • Specially encoded links for content routing • Interface Distribution Daemon(Luyten 2005) • Web interface as well as stand-alone applications • Multiple content allocation modes • Usability analysis - Completeness and Continuity

  14. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  15. 12C Framework • Concurrency • Simultaneous multi-device content presentation • Control • Interaction support • Comity • Permissible diversity in device characteristics • Completeness • Presentation of relevant information elements • Coverage • Distribution of content to optimize device usage • Conversion • Content transformation to suit target device characteristics

  16. 12C Framework • Composition • Relationship of components to the interface • Coherence • Consistency in presentation of content elements • Coordination • Reallocation of elements when devices change • Continuity • Support for users to interpret and evaluate system state • Constancy • Reuse of content instances that have been shown before • Confidence • Security and trustworthiness in communication

  17. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  18. Development Platform Web or something else?

  19. WorldWide Web • WWW • Dominant infrastructure • World-wide audience • Stateless server • Client-initiated communication • Content embedded within structure • <p>, <h1> • Changing devices means restarting the activity • Standardized browsers • Little support for custom behavior

  20. context-aware Trellis (caT) • Petri net-based hypertext system (Furuta, Stotts - 1989) • Formal, graph-based model • Browsing semantics (user privileges, time of day, location) • Decouple specification from information content • Content not subsumed within structure • <img>, <a> • Separation allows devices to select suitable content • Decouple content from presentation • Browsers present content autonomously • Stateful server • Support for multiple clients • Easy propagation of user actions and content to multiple devices

  21. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  22. Specification Information Service MIDAS Browser 2 Browser 2 Browser 2 Browser 3 Browser 3 Browser 3 Browser 4 Browser 4 Browser 4 Device 1 Device 2 Device 3 Users Architecture caT Browser

  23. Specification Information Service MIDAS Browser 2 Browser 2 Browser 2 Browser 3 Browser 3 Browser 3 Browser 4 Browser 4 Browser 4 Device 1 Device 2 Device 3 Users Architecture Device Manager Browser Coordinator Resource Realizer Application Coordinator Application Coordinator Browser

  24. Specification content handles Information Service user actions content handles instance handle status, user actions instance properties instance handle, actions MIDAS content user actions content handle content instance properties Browser 2 Browser 2 Browser 2 Browser 3 Browser 3 Browser 3 Browser 4 Browser 4 Browser 4 Resource Repository Device 1 Device 2 Device 3 Users Architecture Device Manager (instances, devices) Browser Coordinator Resource Realizer Application Coordinator Application Coordinator Browser

  25. Device Manager Information Service Fully Replicated Interaction Replicated Instance Optimal MIDAS Optimal Content Handles, properties Device Selection Scheme Content-Device Map Instance properties Instance Scorer Device properties, interaction mode Presentation only Input only Interactive Browser Coordinator Resource Realizer

  26. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  27. Content Transformation • Images • Scaling (automatic) • Color reduction (automatic) • Text • Text extraction from formatted documents (automatic) • HTML, Word, PDF • Summarization (semi-automatic) • Formatted data such as tables (semi-automatic) • Visual or audio rendering (automatic) • Video • Audio extraction, down-sampling (automatic) • Video summarization (semi-automatic)

  28. Resource Manager Authoring support Device dependence for content instances Automatic transformation of images (ImageMagick) Author validation

  29. Perception-based Image Transformation • Images • Changes to attributes of photographs • Fastest growing information content on the Web • Goal of user study • Understand human perception of nearness • Scaling of images • Color reduction (and gray scaling) • Obtain actionable metrics to design rules for providing content in its most suitable form

  30. Study Design • Two stages • Stage 1: Image classification • Stage 2: Similarity of image pairs • Subject demographics • Five subjects per stage • Graduate students and staff • Different disciplines

  31. Stage 1 • Subjects classified 100 photographs • Taken by four photographers • Everyday and travel pictures • Prints of digital photos to make piles • No guidelines regarding classification • Subjects created between 12 and 40 categories • I coalesced categories based on image content • Four final categories

  32. People Nature Structures Text Image Types

  33. Stage 2 • Subjects viewed image pairs on identical displays • No overlap or switching windows • Each pair differed in size or color • Variables manipulated independently • 20 image pairs for each variable • Answered three questions • Similarity of images (9-point scale) • Suitability of automatic substitution (yes/no) • Acceptance of informed substitution (9-point scale)

  34. Scaling

  35. Color Reduction

  36. 1 3 160x120 2 (1-bit) 320x240 4 (2-bit) 640x480 16 (4-bit) 800x600 256 (8-bit) 1024x768 16 Million (24-bit) 2 4 Data Analysis Colors Size

  37. Effect of Information Content Loss Color depth change Scaling Scores on a 9 pt. Scale 1024 X 768 X 24 vs. 160 X 120 X 24 320 X 240 X 24 vs. 320 X 240 X 1 (no loss) (98 %) (no loss) (96 %)

  38. Similarity by image type • Images of nature and structures scale well • Human faces • Legibility of textual elements • Algorithms for face and text detection in images exist Scaling

  39. Scaling– Image Substitution • No clear trends in terms of distance • Subjects allowed automatic substitution • Corresponded with higher replacement suitability

  40. Color Reduction – Image Substitution • Acceptance of automatic substitution is lower than that for scaled images • The suitability ratings are correspondingly higher

  41. (9 - Scoredist, size) Wsize x x Wtype 9 (9 - Scoredist, color ) (9 - Scoredist, color ) + Wtogray + Wcolor x x 9 9 Perception-informed Rules • Principles • Prefer scaling over color reduction • Avoid scaling of images that contain people and text • Gray scaling is the best option when used by itself • Normalized score: Original image = 0 (best score) Image Rating = 0.4 Wtype 0.6 Wtype (people, text) = 0.53 Wtype (nature, structures) = 0.47 Wtogray= 1.0 (grayscale) Wtogray= 0.0 (otherwise)

  42. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  43. Outline Motivation Related Work 12C Framework Approach MIDAS Architecture Content Transformation Demonstration Analysis Contributions Future Work

  44. Analysis - Design Goals ✔ • Deliver information elements to multiple devices simultaneously • Content • Interaction • Rely on widely available infrastructure • Wired and wireless computer networks • Mobile phones, e-readers, tablets, desktop computers • Sensors embedded in the environment • Support content transformation • Within form (image scaling, text summarization) • Across forms (text to speech, audio extraction from video) ✔ ✔

  45. Analysis - Design Goals • Strive to retain content integrity from a human perspective • Select transformations that a user perceives to be closest to the original form • Direct content elements to devices that can present them optimally • Reconfigure presentation dynamically in response to changes in device availability • New devices become available • Current devices become unavailable ✔ ✔ ✔

  46. Analysis - 12C Framework Concurrency Control Comity Completeness Coverage Conversion Composition Coherence Coordination Constancy Continuity Confidence Present information elements on multiple devices • Inclusion of devices in MIDAS is optional • WebSplitter, ANMoLE • A user can choose the interaction mode for each device • Inclusion of public devices is not supported • Ubiquitous Display System • Content is directed to devices automatically • Multibrowsing, Ubiquitous Display System

  47. 12C Framework Concurrency Control Comity Completeness Coverage Conversion Composition Coherence Coordination Constancy Continuity Confidence Interaction support • Flexible – dependent on device choices • Most architectures support centralized or distributed • It is possible to have no input devices at all • May change during a session

  48. 12C Framework Concurrency Control Comity Completeness Coverage Conversion Composition Coherence Coordination Constancy Continuity Confidence Permissible diversity in device characteristics • MIDAS places few restrictions on devices • Pebbles, Ubiquitous display system • Device must run the Browser Coordinator • No requirement of proximity to a user • Remote devices may be included - printers

  49. 12C Framework Concurrency Control Comity Completeness Coverage Conversion Composition Coherence Coordination Constancy Continuity Confidence Presentation of relevant information elements • Ensures completeness by presenting all resource ids • Resource Manager aids content authors • Support includes textual description as fallback • IDD assertion is based on fixed content instances • ANMoLE and MIDAS convert content

  50. 12C Framework Concurrency Control Comity Completeness Coverage Conversion Composition Coherence Coordination Constancy Continuity Confidence Distribution of content to optimize device usage • Flexible – dependent on Device Manager mode • Fully replicated - redundancy • Interaction replicated • Instance optimal - coverage for individual instances • MIDAS optimal – system-wide coverage • Coverage may not apply to some systems • Ubiquitous Display System, Multibrowsing

More Related