1 / 17

Accounting Manager

Accounting Manager. Taking resource usage into your own hands Scott Jackson Scott.Jackson@pnl.gov http://www.emsl.pnl.gov/docs/mscf/gold. Typical problems faced by HPC managers. Introduction.

eagan
Download Presentation

Accounting Manager

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Accounting Manager Taking resource usage into your own hands Scott Jackson Scott.Jackson@pnl.gov http://www.emsl.pnl.gov/docs/mscf/gold

  2. Typical problems faced by HPC managers 2

  3. Introduction Gold is an open source accounting system that allocates and tracks resource usage on High Performance Computers. • A resource bank (allocation management system) • Much like a bank, it associates a cost to computing resources and allows resource credits to be allocated to users and projects. As jobs complete or as resources are utilized, projects are dynamically charged and resource usage recorded. • An accounting system • Can be dynamically customized to record any type of accounting data – pacct, sar, node availability, performance data, etc. • An information service • Also functions as a powerful generalized information service useful in a variety of situations, such as providing meta-scheduling mappings of machines to resources, applications, accounts, users, etc. 3

  4. Background • SciDAC • Scientific Discovery through Advanced Computing – A DOE initiative to improve the impact of scientific computing • Scalable Systems Software Project (SSS) • Research, develop and support an integrated suite of systems software and tools for the effective management and utilization of the highest scale computational resources. 5 yr project involving 12 sites • QBank • Gold is based on a successful program, QBank, that has been used for years on government and university computing systems. 4

  5. Allocation Architecture Universe of Projects Universe of Users Account: 1234 Name: Chemistry Account: 1234 Name: Chemistry Universe of Machines 5

  6. 0 6 3 2 4 1 5 Charge Dynamic Charging Accounting Manager (Gold) 0 Deposits are made in Account 1 A Job is Submitted 2 A Quote is Requested 3 A Reservation is Made 4 The Job is allowed to Start Scheduler (Maui) 5 The Job Completes 6 The Reservation is Removed and a Charge is Issued Resource Manager (PBS, LL, LSF) Reservation Quote Deposit Reservation 6

  7. 4 5 2 0 Accounting Manager (Gold) Resource Manager (PBS, LL, LSF) 1 3 6 Dynamic Charging • Make Deposits, etc. • Submit Job • Obtain Quote • Make Reservation • Start Job • Job Completes • Issue Charge 7

  8. Dynamic Charging 0 Allocation Manager (Gold) Resource Manager (PBS, LL,LSF) 4 • Make Deposits, etc. • Submit Job • Locate Feasible Systems & Obtain Quote • Stage Job • Balance Check • Make Reservation • Start Job • Job Completes • Issue Charge 3 8 7 6 2 5 3 Meta-Scheduler (Silver) Scheduler (Maui) 1 8

  9. Expiring Allocations 9

  10. Nested Accounts University 25 25 25 Biology Chemistry Physics 40 50 Chem 101 Chem 201 40 40 Actinides Workshop 10

  11. Web-based GUI 11

  12. Other Gold Features Transparency Features Historical Journaling Customizable Accounting Dynamically Customizable Strong Security 12

  13. Other Gold Features Guaranteed Quotes Credit and Debit Models CPU Memory Disk QOS Powerful Querying Flexible Charging 13

  14. Enables Grid Computing 14

  15. Distributed Accounting Gold facilitates resource sharing across administrative domains (Computational Grids) by supporting distributed accounting and allocation while preserving local autonomy • Problems: • Security concerns (privacy, protection from fraud) • Want to maintain autonomous control of own resources • Need to know how much it is going to cost you (contract) • You want your own local accounting record (a receipt) • Solutions: • Traceback policy screening • Distributed accounting • Charge Quotes • Strong authentication and encryption 15

  16. Custom Accounting Sites can create or modify record types on the fly enabling them to meet their custom accounting needs. This capability turns Gold into a generalized information service. [root@altix1 gold-1.0.a2.0]# gold -v gold> Object Create Name=Motorcycle gold> ObjectAttribute Create Object=Motorcycle Name=Name PrimaryKey=True gold> ObjectAttribute Create Object=Motorcycle Name=Color gold> ObjectAttribute Create Object=Motorcycle Name=HorsePower ... gold> Motorcycle Create Name=Yamaha Color=Blue HorsePower=1200 Color Name HorsePower ----- ------ ---------- Blue Yamaha 1200 Successfully created 1 Motorcycle ... gold> Motorcycle Modify Name==Yamaha HorsePower=1500 Color Name HorsePower ----- ------ ---------- Blue Yamaha 1500 Successfully modified 1 Motorcycles 16

  17. Status • Status: • Currently in beta release (4Q04) • General release targeting 2Q05 • Download: • http://www.emsl.pnl.gov/docs/mscf/gold • For more information: • Scott.Jackson@pnl.gov • http://www.emsl.pnl.gov/docs/mscf/gold • http://sss.scl.ameslab.gov/home.shtml • http://www.scidac.org/ScalableSystems 17

More Related