Design Methodologies & Infrastructure

1. Design Methodologies & Infrastructure Daniel P. Siewiorek Carnegie Mellon University June 2012

2. Outline • User • Resource Management • Design Methodology

3. Outline • User • Resource Management • Design Methodology

4. The Aura ProjectDistraction-free ComputingSM School of Computer Science Carnegie Mellon University

5. 7 PENT . PRO 10 PENT . 80486 80860 68040 6 Number of devices 10 80386 SLOPE = 10X INCREASE IN 7 YEARS 68030 80286 68020 5 10 8086 6801 4 4004 8080 10 6802 6800 (Source: Walt Davis, Motorola) 3 10 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 Year Samples Introduced Moore’s Law Reigns Supreme Transistors per Processor a

6. Glaring Exception Human Attention Adam & Eve 2000 AD

7. Aura Thesis The most precious resource in computing is human attention • Aura Goals • reduce user distraction • trade-off plentiful resources of Moore’s law for human attention • achieve thisscalably for mobile users in a failure-prone, variable-resource environment

8. Wireless Campus as of June 2000 300+ basestations Academic andAdministrative Buildings Residence Halls, Parking, etc

9. Research Components HCI/Agents/Speech Tasks Task-driven computing Rapid service configuration Services Energy-aware adaptation Energy locality Multi-fidelity computation Failure-resilient data access OS/Network Service-oriented QOS Power/bw-aware devices Wearable computers Physical Devices Aura Research Framework Users

10. This Talk Task-Driven Computing Energy-Aware Adaptation Energy-Locality Multi-Fidelity Computation Service-Oriented Network QOS Omitted for Brevity Speech Recognition Language Translation Multimodal User Interfaces User Interface Adaptability Data Adaptability Network Adaptability Software Composition Proxies/Agents Collaboration Wireless communication Robustness, Reliability, Rapid Failover Security & Privacy Technologies Being Explored

11. Problem • Humans interact at too low a level with computer • URLs, filenames, program names, etc. • very explicit, step-by-step interactions • like programming in machine language! • Result • brittle behavior • many details change with failures, platform changes, etc. • consume mobile user’s attention

12. Users Tasks Services OS/Network Physical Devices Solution: Task-Driven Computing • Support user intentions • capture high-level intent as tasks • raise level of abstraction of user interactions • Support mobility • suspend/resume on different platforms and locations • dynamically reconfigure to match available resources • Support proactivity • active guidance from system • corrections, alternatives, persistence

13. Environment C Instantiation Feedback Mobility/ Resource change Feedback Instantiation Feedback Instantiation Environment B Environment A Mobility/ Resource change Adapting to Environment User Task access Feedback Task Space

14. Diverse Users, Tasks and Preferences Aura Task API Diverse Services, Contexts Environments and Resources

15. Energy-Aware Adaptation

16. Problem • Battery power is a critical resource when mobile • Current approaches only offer limited extensions of mission life • no dramatic battery improvements foreseen • low-power hardware is important, but not enough • “Wireless transmission will dominate energy demand in future dismounted soldier systems.” • NRC report on Energy-Efficient Technologies for the Dismounted Soldier, 1997 • Explicit user management of energy consumes attention

17. System Balance Changes Fraction of System Power 199319952001 Compute 63% 29% 10% Display 19% 29% 10% Radio 18% 43% 80% Total 33W 9W 2W

18. Users Tasks Services OS/Network Physical Devices Solution: Energy-Aware Adaptation • Applications change behavior as battery drains • Collaborative relationship between OS & applications • OS monitors energy supply & demand • notifies applications when to change fidelity • Extension: goal-directed adaptation • user provides time estimate • OS ensures goal is met • mid-course corrections accommodated

19. Idle Xanim X Server Odyssey WaveLAN Kernel Hardware Power-Mgmt. Baseline Premiere-B Premiere-C Reduced Window Combined Energy Impact: Video 2500 2000 1500 Joules 1000 500 0 Video 1 Video 2 Video 3 Video 4

20. Idle Janus Odyssey WaveLAN Kernel Baseline HW-Only Power Mgmt. Remote - Reduced Model Reduced Model Hybrid - Reduced Model Remote Hybrid Utterance 4 Utterance 3 Utterance 2 Energy Impact: Speech 160 140 120 Energy (Joules) 100 80 60 40 20 0 Utterance 1

21. Energy Locality

22. Users Tasks Services OS/Network Physical Devices Energy Locality • Goal: Maximize computation per battery charge • Processor speed setting to adapt to computer architectural bottlenecks (e.g. cache, memory bus) and non-ideal battery behavior • Impact: • Over four orders of magnitude difference in energy to perform the same task as a function of User Interface, Data Representation, Compute versus transmit trade-off • Approach: Embedded Network Proxies • Reduce bits transmitted • Adaptive scheduling to smooth demand. Delivered battery capacity determined by peak rather than average demand

23. Example of Energy Locality grams of battery weight User Information InterfaceRepresentComputerRadioTotal Speech Text 1.25*10-2 3.4*10-5 0.0128Recognition Audio Digitized 8.33*10-8 2.4*10-3 0.0024 Digitization Sound Video Picture 1.1*10-4 13.3 13.3 Compression

24. Energy Locality • Online Web services allow interaction with dynamically changing data • Traditionally, clients are treated as either workstations or terminals • Use of the communication link is mainly to move data • Alternative: Control-oriented design • Applications are groups of services that manipulate each other • Communication link is used to move instructions (I.e. control)

25. Example: Weather Browser • Composite data: maps, with periodically updating weather information • Underlying data is integrated from a multitude of Web services • Dynamically updating data normally prevents caching at the client • Control-oriented implementation resulted in • A factor of 20 times bandwidth reduction compared to terminal or workstation • Large savings in application latency from caching

26. Weather Browser Partitionings

27. Cumulative Communication Costs

28. Cumulative Communication - Warm Cache

29. Multi-Fidelity Computation

30. Problem • Good performance on interactive mobile apps very difficult • resource-poor hardware • (size, weight, energy constraints) • demanding applications (e.g. augmented reality) • wireless energy drain if computing shipped off-site • serious user discomfort & distraction with poor performance • Catch-22!

31. Solution: Multi-Fidelity Computation • Fundamentally rethink our model of computing • Classic notion of algorithm • fixed correctness criteria • variable amount of resources consumed to meet this • Adaptation for mobility suggests a different viewpoint • “Do the best you can using no more than X units of resource” • correctness criteria no longer fixed • multiple notions of “correct”; each is a level of fidelity

32. Users Tasks Services OS/Network Physical Devices Why is this Concept Useful? • Resource consumption can now be independent variable • Special case: let system find “sweet spot” • “Give the best result you can cheaply” • knee in fidelity-resource usage curve • Interactive mobile applications fit this model well • users are tolerant of imperfect results, if so labeled • if needed, they can insist on full-fidelity result • often, they realize original query was off the mark • if query P expensive, may prefer answer to “close” query Q • “sharp cliffs” in performance imply big wins • cache miss over low-bandwidth wireless link painful • huge gap in capabilities of local and remote computing engines • low-power considerations

33. Example: Architect’s Aid • Augmented reality for rapid preview of design changes • wearable computer with heads-up display • used on-sitefor remodeling projects • Architect and customer can explore alternatives together • initial “quick & dirty” rendering using local computation • many iterations to shrink design space • when a design looks promising, request high fidelity • Rendering algorithms span wide range of cost & fidelities

34. Motivation • Continuous monitoring of context consumes a significant amount of power • Power for data acquisition: sensor sampling • Power for data processing: feature creation, sample classification • Continuous sampling gives a lot of redundant information for each contiguous behavior being classified

35. Do we really need every sample? • There is a significant amount of inertia to human behaviors • Periodic sampling can remove at least a portion of the redundant samples • Human behaviors have different duration profiles • Can modify sampling strategy given current user context to improve balance of samples • Hopefully without adversely affecting monitoring performance by too much

36. Data Set • Daily activities of 4 members of our research group were recorded with self-annotation using the eWatch sensor platform • ~ 3 days per user, an average of 7.5 hours/day • annotated using an application developed for the eWatch • activities annotated: sitting, sitting in a meeting, standing, walking, eating, cooking, running, driving, sitting on a bus, standing on a bus, other • total data-set is 98 hours of annotated data

37. Varying Sampling Frequency Classification Accuracy vs Sampling Rate Classification Accuracy vs Estimated Battery Lifetime

38. Selective Sampling • Results are for different sampling schemes, using annotated data (i.e. assuming perfect classification) Classification Accuracy vs Percent of Samples Used Classification Accuracy vs Estimated Battery Lifetime

39. Performance Metrics • Accuracy • Percentage of time that the system predicted activity matches the true, current activity • Missed Activities • Percentage of activities during which no samples are taken • Average Latency • Average number of seconds between the true transition between two activities and detection of the transition

40. Results (Accuracy)‏ • Using only 10% of the available sample windows • Markov samesubject achieves 98% accuracy • Markov nonsubject achieves 96.5% accuracy • Uniform achieves 96% accuracy

41. Results (Missed Activities)‏ • Using 10% of available sample windows • Markov samesubject missed 5% of activities • Markov nonsubject missed 10% of activities • Uniform missed ~ 8%

42. Results (Average Latency)‏ • Using 10% of available sample windows • Markov samesubject latency is 15 seconds • Markov nonsubject latency is around 30 seconds • Uniform latency is around 25 seconds

43. Service-Oriented Network QOS

44. Problem • Mismatch between network QOS and user-perceived QOS • network support for QOS of limited value by itself • Today’s QOS interfaces too restrictive for proactivity

45. Users Tasks Services OS/Network Physical Devices Solution: Service-Oriented Network QOS • Bridge gap between high-level requirements and low-level QoS mechanisms • Impact • improve perceived quality of network service • simplify use of network QOS mechanisms • more predictable behavior • Rich two-way communication between services & network • services specify requirements in high-level terms • network provides “traffic report” on multiple time scales • offer “network weather map” for proactive feedback

46. Users Tasks Services OS/Network Physical Devices Service-Oriented QoS • Goal: bridge the gap between delay-oriented user requirements and bandwidth-oriented network control. • Users care when they get their data, not how it was transferred • Impact: reduce latency of message transfer. • Approach: consider a longer time horizon when controlling network traffic.

47. Many Application Domains Mobile Worker - wearable computer communicating with infrastructure computers to assemble and access information Smart Office- real time speech recognition, authorization, display sensitive representation Smart Conference Room - real time video, language translation, face identification, speech to text meeting summary, eye tracking Time Travel- replay events from any viewing angle, interact with 3 D object models generated from video Smart Project Room- display and manipulation of information and their relationships Flight Recorder- capture and indexing of audio, video, and information events representing a group shared experience base Smart Vehicle- position sensitive information access, navigation, entertainment

48. Outline • User • Resource Management • Design Methodology

49. Predicting User Interaction Time and System Energy

50. Keystroke-Level Model (KLM) [Luo] • Card, Moran, and Newell (1980) • Quantitative prediction • Validity and predictive value widely examined in HCI • Scope: • Serial operations • Expert user • Routine tasks • Error-free operation • Describe a task by placing operators in a sequence • Task execution time = Σ all operators involved The Human Information Processor (from CMN80)