1 / 48

Display Management: Sensing User Intention and Context

This research paper explores the feasibility of using low-power sensors to match user behavior and reduce energy consumption in display management. It presents the FaceOff architecture and prototype, along with evaluation studies on responsiveness and energy savings.

waring
Download Presentation

Display Management: Sensing User Intention and Context

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Display Management:Sensing User Intention and Context (HOTOS 2003)

  2. Outline • Motivation and Research Objective • FaceOff Architecture and Prototype • Evaluation • Best Case Feasibility Study • Responsiveness Study • Future Work • Related Work • Dark Windows

  3. Motivation • Current energy management techniques tied to process execution • Can we use low power sensors to match I/O behavior more directly to user behavior and reduce system energy consumption? Sensing User Intention and Context for Energy Management

  4. Case Study: FaceOff • Displays: • Typically responsible for large power drain • Power State can be controlled by software • State transition strategies naïve A display is only necessary if someone is looking at it.

  5. Image Capture Face Detector Main Control Loop No Face=off Face=on

  6. Prototype • IBM ThinkPad T21 running RedHat Linux • Base Power Consumption = 9.6 Watts • Max CPU = 8.5 Watts over Base • Display = 7.6 Watts • Logitech QuickCam Web Cam • Power Consumption = 1.5 Watts • Software components: • Image capture, face detection, display power state control

  7. Face Detection • Skin detection used for prototype • Real time proprietary methods exist

  8. Outline • Motivation and Research Objective • FaceOff Architecture and Prototype • Evaluation • Best Case Feasibility Study • Responsiveness Study • Future Work • Related Work • Dark Windows

  9. Best Case Feasibility Study • What is the potential for energy savings? • Assume perfect accuracy • Best case user behavior – start it and leave. • Tradeoff of energy costs: • CPU/Camera vs. Display • Effect on System Performance • Network file transfer (113 MB) • CPU intensive process (Linux kernel compile) • MP3 Song (no display necessary)

  10. File Transfer Traces

  11. Kernel Compile Traces

  12. Energy and Time Comparisons

  13. MP3 Application • Playing an MP3 • Display not necessary • Song completes before default timeout turns off display • Energy comparison • 3,403 J with FaceOff vs. 4,714 J with Default • 28% energy savings • No noticeable effect on playback

  14. Responsiveness Study • Use full prototype including skin detection • Establish baseline timing • Examine Responsiveness • varying system load • varying polling rate

  15. Responsiveness Timing polling latency detection latency Face arrives (or departs) Image acquired detection complete display signaled Total responsiveness latency

  16. Baseline Detection Latency • Measured over a period of one hour with no programs other than background processes running • Latency increased over time • Started at ~110ms • Increased to ~160ms • Why? • Appears to be an effect of Linux scheduler reducing priority of long running jobs

  17. Detection Latency over Time

  18. Detection Latency Under Load

  19. Outline • Motivation and Research Objective • FaceOff Architecture and Prototype • Evaluation • Best Case Feasibility Study • Responsiveness Study • Future Work • Related Work • Dark Windows

  20. Varying Polling Rate • Reduce overhead by reducing polling rate • Increases responsiveness latency • Adaptive polling rate • Eliminate polling in presence of UI events • Begin polling as duration without UI events increases and face is detected • Reduce polling when no face present • Similar problem with latency increase upon return

  21. Optimization with Motion Sensor • Combine adaptive polling & motion sensing • Meet responsiveness requirements with minimal FaceOff system overhead • Eliminate image polling when no motion • Switch display state on immediately when motion detected and restart image polling

  22. Implementation • Prototype using X10 ActiveHome Wireless Motion Sensor and Receiver • Receiver connects to serial port • Reading port blocks until sensor triggers • Takes up to 10 seconds to recharge • Promising addition to FaceOff system

  23. More Roles for Sensors • Touch Sensor • Detect picking up of a PDA • Light, Sound sensors • Adjust display brightness (Compaq iPAQ) • Adjust speaker volume • 802.11 Signal Strength sensor • Determine possibility of offloading computation

  24. Enhanced Sensors • “Active Camera” • Perform some or all of the face detection • Color filtering • Preprocessing skin color segmentation • Low Power processor for external sensor control, computation

  25. Discussion: Other ideas for using sensors to save energy?

  26. Future Work • Continue work on optimizing responsiveness • Comprehensive user study • Survey of usability • Characterization of usage patterns • End-to-end experiment • Implementation with available very low power camera/motion sensor and prototype for small device (handheld)

  27. Conclusions • Context information offers promising method of energy management • FaceOff illustrates feasibility of approach • Available very low power sensors as well as optimization techniques would improve upon the FaceOff energy savings

  28. Outline • Motivation and Research Objective • FaceOff Architecture and Prototype • Evaluation • Best Case Feasibility Study • Responsiveness Study • Future Work • Related Work • Dark Windows

  29. Related Work • Display Power Management • Industry Specifications • APM, ACPI, DPMS • Zoned Backlighting • Energy-Adaptive Display System Design • Attentive/Perceptual UIs • Smart Kiosk System: Gesture analysis • CAMSHIFT: Game control • IBM PupilCam: Head gesture recognition

  30. ACPIAdvanced Configuration and Power Initiative Brought to you by Intel, Microsoft, and Toshiba and designed to enable OS Directed Power Management (OSPM). • Goal is to be able to move power management into software for more sophisticated policies • Abstract OS-HW interface • Replaces APM interface

  31. What ACPI Offers • Standardization industry-wide (Vendors to support ACPI in products instead of building their own power mgt) • System and device power states • Thermal model • Thermal zones, indicators, cooling methods • BIOS interfaces • Motherboard configuration tables • Interpreted control methods • Plug-and-play • Complexity moved into OS

  32. What ACPI Offers • System • Mechanisms for putting computer as a whole in sleep/wake states • Devices • ACPI tables describe motherboard devices • Power states • Controls for managing states • Processor • Detecting idle state and swapping to low power • Batteries • Querying and controlling battery behavior

  33. S4 S3 S2 S1 G1Sleep Power States G: global states apply to entire system and are visible to user D: states of individual devices S: sleeping states within the G1 state C: CPU states G3mech off wakeup G0-S0working Legacy G2-S5Soft off Dxmodem x DxHDD xDxcdrom x Cxcpu

  34. ACPI Internal Structure

  35. Transmeta Crusoe ACPI Power States

  36. Applications OnNow WIN32 ext OS ACPI Spec HW OSDM: OnNow SetSystemPowerState • initiate sleep state, query apps(?) SetThreadExecutionState • specifies level of support needed (e.g. display required) WM_POWERBROADCAST • a message notifying of power state changes to which applications can respond SetWaitableTimer • ensure PC is awake at scheduled time RequestDeviceWakeup RequestWakeupLatency - to specify latency requirements GetSystemPowerStatus and GetDevicePowerState

  37. Outline • Motivation and Research Objective • FaceOff Architecture and Prototype • Evaluation • Best Case Feasibility Study • Responsiveness Study • Future Work • Related Work • Dark Windows

  38. Energy-Adaptive Display System Designs for Future Mobile Environments S. Iyer, L. Luo, R. Mayo, P. Ranganathan, MOBISYS 2003

  39. Opportunity • New technology: OLEDs – Organic Light Emitting Diodes • Energy consumption on a per-pixel basis by determining each pixel’s brightness and color • Energy consumption of different regions of screen to be changed independently • No separate backlight • In development by Kodak, Sanyo, Sony,…

  40. Dark Windows • Software support – modifications to windowing system to ensure energy is spent mostly on window-of-focus (capturing user’s area of visual interest) • Non-active screen area is changed (dimmed or re-colored for energy optimization)

  41. Justification • Usage study – what are the user’s needs and how well do they match display characteristics? • Characterization of display usage in Microsoft Windows by 17 “typical” users • Application logger – recorded, for up to 14 days, when user was active. • Window of focus (the one accepting keyboard input) – its size, location, title • Size of total screen area used (all non-minimized windows

  42. Screen Usage Results • On averageonly 59% of area used by window-of-focus,additional 17% bybackground windows • High variability among users • Large fraction of smaller windows have very low content (notifications, alerts) – don’t need full color characteristics of display to convey it.

  43. Dark Windows Design • Prototyped on X Windows under Linux • Used VNC – Virtual Network Computing Server – provides a virtual representation of display – virtual framebuffer where pixels can be manipulated between application and display • Track window-of-focus, apply modifications to pixels outside of it

  44. Modifications • Half Dimmed – areas outside window-of-focus are dimmed to 50% brightness • Fully Dimmed – areas outside are turned off • Gray Scale – areas outside are changed to gray by setting rgb to average value. • Green Scale – areas outside are changed to green which is lowest power color for OLEDs. Dims to 67%

  45. Evaluation • Energy benefits measured by generating synthetic trace from usage study and playing trace on prototype. • 15 inch OLED displays were not available so they used a software power model to calculate power consumption • Controller power set to 0.5W • Driver power – 1W • 1024x768 pixels individually consume -- red power 4.3mW, green power 2.2mW, blue power 4.3mW. • User experience – users want to see background but willing to use dark windows

  46. Results based on default Teal Background* * Benefits depend on original Choiceof backgroundand windowcolors.

  47. Conclusions • While benefits depend on usage scenario, significant energy savings can be achieved with these optimizations • Further opportunities for application specific adaptivity • More meaningful notions of area of focus can be defined (e.g. most recent email message, most recently changed region of screen) • Better match to (low) content – e.g., notifications could be audible signal instead of popup window

More Related