Ideas for Cooperative Disk Management with ECOSystem

1. Ideas for Cooperative Disk Management with ECOSystem Emily Tennant Mentor: Carla Ellis Duke University

2. Outline • Introduction • Building blocks for cooperative file operations in ECOSystem • Read Requests • Write Requests • Unresolved Issues • Experimental Plan

3. Vision and Goals • The next step for ECOSystem • Energy efficient policy space • Energy aware applications • Vision • Create a set of API extensions permitting applications to pass application-specific information to the OS. • Use this information to manage disk accesses more efficiently. • Specific Goals • Simple interface • Minimal changes – backward compatibility • Energy savings in terms of currentcy

4. Related Work • Recent submission to OSDI conference: • Andreas Weissel, Björn Beutel, Frank Bellosa. Cooperative I/O—A Novel I/O Semantics for Energy-Aware Applications. • Goal: “demonstrate benefits of application involvement in operating system power management.” • Coop-I/O – an approach to reduce power consumption of devices (disk) • Encompasses all levels of the computer system • Hardware • Operating system • I/O interface for energy-efficient applications

5. Main Elements – Coop-I/O • New cooperative file operations • read_coop(), write_coop(), open_coop() • New parameters: time-out and cancel flag • Delay activating disk for length of time-out parameter • Possible to abort accesses after time-out period • Energy-efficient update mechanism • Update of cached disk blocks is batched to maximize the time hard disk can spend in standby mode. • OS controls hard disk modes • Disk drive is switched into low-power mode according to an adaptive algorithm. • “device-dependent time-out with early shutdown (DDT/ES)”

6. Cooperative I/O in ECOSystem • New cooperative file operations • Extend system calls (read, write) with extra parameters. • Deferrable disk accesses • Options for abortable file operations? • Energy-efficient update mechanism • Updates deferred to create bursty disk access • Energy-efficient update strategies integrated into write process • OS control of disk drive • Motivation of adaptive algorithm for powering down disk drive?

7. Outline • Introduction • Building blocks for cooperative file operations in ECOSystem • Read Requests • Write Requests • Unresolved Issues • Experimental Plan

8. Bidding • How do we couch the cooperative, deferrable file operations of Coop-I/O in terms of ECOSystem’s currentcy? • “Bidding” process • Inflated entry price delays disk spinup to ensure that processes have enough currentcy to execute and generate more disk requests. • Each process “bids” the amount of currentcy it is willing to contribute towards the entry price. • Is bidded currentcy considered available for use? Entry Price P1 P2 Total Bid P3 Disk spins up Time

9. Priorities • Motivating Question: How can we implement the bidding process in a useful and intuitive interface (similar to Coop-I/O)? • Currentcy is dynamic – difficult for application to assign directly. • Create static priorities. • Map priorities to a currentcy amount for bid. • What type of priorities could be created? • Integer sets (1-10, 1-100) • Real-numbered intervals • Dynamic priorities Time-out Priority Currentcy Time spent waiting Priority (and currentcy bid)

10. Dynamic Priorities • Allow priority to change over time. • As a process waits for disk access, its priority can increase, causing an increase in the amount of currentcy allocated to its bid. • A Simple Case: • No feedback required from application • Integer priority levels 1-10 • Include time-out parameter in system call • If disk access not scheduled within time-out, jump up one priority level. • Other possibilities • Priority function (priority vs. time) given by application • Upcalls to application allow on-the-fly changes in priority • Issues • Default priorities, time-out, etc.

11. Mapping Priority to Bid Resource Container • Takes place within OS • Involves resource container • May require resource container alteration • Bid, priority, time-out, etc. Available_currentcy Ticket Percentage of available currentcy Priority? priority Available_currentcy Bid Ticket Percentage of entry price Time-out?

12. A Simple Mapping Model • Assume numeric priority. • Priority corresponds directly to percentage of available currentcy that is allocated to bid. • Example: priority = 5, available currentcy = 1000mJ • BID = 500mJ • Remainder of available currentcy can be used (CPU, NIC) while process waits to access disk. • Overhead issues BID = available_currentcy * (priority/10)

13. The ability to save energy/currentcy by aborting a disk access is a desirable concept. Can abortability be considered in terms of currentcy? ECOSystem Possibilities: Time-out must be a specified, non-infinite value for all possibilities. Boolean system call parameter Abortable accesses can have decreasing priority over time. Unique priority level (i.e. 0) Does not require extra parameter Can automatically allocate zero currentcy (no bid) Tackling the Exception of Abortability • Coop-I/O: • Cancel flag as system call parameter • Abort disk access after waiting through time-out period.

14. Outline • Introduction • Building blocks for cooperative file operations in ECOSystem • Read Requests • Write Requests • Unresolved Issues • Experimental Plan

15. System call is generated. read(……...., [priority],[time-out]); Verify that data is uncached and disk is inactive. Transform priority bid. “Issue bid” – store bid amount in a data structure. For dynamic priorities, enter timed waiting function/loop. Update daemon waits until bids  entry price (reads & writes). If waiting function returns at time-out: If abortable, cancel access. Else, increase priority and “re-issue” bid. When bids  entry price, disk spins up. Disk access is scheduled. Data is read into buffer cache. Resource container is debited for cost of access. Sample Read Disk Accesses

16. Write Disk Access Issues • More complex! • What does “deferrable write” mean? Do we defer writing to the buffer cache or flushing the buffers to the disk? • How do we guarantee consistency of file system when write requests are aborted? • Coop-I/O: early commit/abort strategy • Delays writing data to buffer cache until it can be committed. • Drive is active at time of write request • Drive will be activated by another committed write (dirty buffers exist). • Time-out has been reached and write request is non-abortable. • Forces processes to wait before writing to buffers. • Defers both writing to buffer cache and flushing buffers to disk.

17. Write Disk Access, cont. • ECOSystem: Unified write/update policy defers disk access • Simplify non-abortable writes • Data can be written to buffer immediately – no need to defer! • Process bids to flush buffer to disk drive (only requires resource container). • What happens to the bidding process when buffers are overwritten? • Implement abortable writes with early commit/abort strategy. • Must immediately identify abortable writes. • Do not have to issue a bid. • Bidding process must take place before writing to buffer. • All dirty buffers are always flushed to disk.

18. Update Mechanism Coop-I/O • Separate update policy. • Four Drive specific-cooperative update strategies: • Write back all buffers. • Update cooperatively. • Update each drive separately. • Update on shutdown. • ECOSystem • Update strategy • Bidding process creates bursty disk access. • If disk is active, updates are scheduled immediately. • Disk spinup/spindown controlled by existence of scheduled disk accesses. • Decreasing entry price guarantees updates. • FlushStart vs. entry price

19. Sample Write Disk Accesses Generate system call Abortable? No Yes Is buffer already cached? Is buffer already cached? No Yes No Yes Is disk active? Is disk active? Is disk active? Write to buffer. No No No Bid/Wait. Is disk active? Bid/Wait (for read). Bid (for read). No Abort. Read buffer. Bid. Write to buffer. Read buffer. Abort. Flush buffer. Write to buffer. Flush buffer. Write to buffer. Flush buffer. Flush buffer.

20. Outline • Introduction • Building blocks for cooperative file operations in ECOSystem • Read Requests • Write Requests • Unresolved Issues • Experimental Plan

21. Unresolved Issues • 1. How do we accumulate bids? • 2. How do we charge for access? Do we weight charge by number of bytes read?

22. Experimental Plan • Design – 6/14/02 • Implementation of deferrable reads – 6/26/02 • Implementation of deferrable writes – 7/5/02 • Synthetic benchmarks/testing – 7/12/02 • Rewrite cooperative application – 7/19/02 • “Real” application testing – 7/24/02 • Implementation of abortable reads/writes • Testing of abortable reads/writes – as time allows (7/29/02)

23. Synthetic Benchmarks Simple read/write programs. Preliminary results. Create scenarios where energy-savings are most obvious. Simulate different workloads: Multiple simultaneous cooperative tasks Mix cooperative and non-cooperative tasks. Test same buffer access issues. “Real” Application Audio/video player, image viewer. Read file from hard disk. Test performance in non-optimized, “real-life” situations. Compare results on: ECOSystem unthrottled (Coop-I/O) Current ECOSystem implementation ECOSystem with cooperative implementation Testing What do we want to test?

24. Summary • Extended system calls for file operations. • Priority can be specified by application. • Priority determines amount of currentcy in bid. • Total bids for disk access must exceed entry price. • Abortable reads/writes may be cancelled after specific period of time. • Interaction between decreasing entry price and bids for disk access works toward efficiently batching disk accesses while guaranteeing that non-abortable accesses occur.