1 / 7

Batch Queue Prediction

Batch Queue Prediction. Warren Smith. Recommendation. Scheduling Working Group recommends that TeraGrid deploys Karnak Current QBETS deployment not sustainable Service deployed outside TeraGrid Cannot be moved onto TeraGrid Source code unavailable Licensing not known

rooney-lucas
Download Presentation

Batch Queue Prediction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Batch Queue Prediction Warren Smith

  2. Recommendation • Scheduling Working Group recommends that TeraGrid deploys Karnak • Current QBETS deployment not sustainable • Service deployed outside TeraGrid • Cannot be moved onto TeraGrid • Source code unavailable • Licensing not known • Developer focus is elsewhere (Eucalyptus) • Working group evaluated Karnak and Moab • Also discussed Catalina as an alternative to Moab • Full report (draft) linked from sched-wg page • http://www.teragridforum.org/mediawiki/index.php?title=Scheduling_WG TeraGrid Quarterly - Batch Queue Prediction

  3. Karnak Advantages • Cost • Kanak is free, Moab has a cost • At least cost of the TeraGrid Moab contract (not currently being used) • Additional cost, if more sockets needed than in contract • License • Karnak is open source - Apache 2 • Moab is commercial - access to source code is possible, but only while an agreement continues • Capabilities • Karnak provides information on expected accuracy of it’s predictions, Moab does not • Karnak provides historical statistics about jobs • Prediction accuracy • Karnak is currently slightly better, and is expected to improve • RP effort • For Karnak, RPs just install GLUE 2 information provider • For Moab, RPs without Moab install/configure/maintain Moab in monitor mode • For Moab, RPs would install a new Moab prediction service • Time to deploy • Karnak should be faster to deploy • Prototype already available - http://karnak.teragrid.org:9998/index.html • Requires less effort from RPs than deploying Moab in monitor mode • Doesn’t require any additional contract negotiations TeraGrid Quarterly - Batch Queue Prediction

  4. Moab Advantages • Stability • Moab is production, commercial code • Karnak is research software on the way to production • Support • Moab has a development & support team • Karnak has a single developer (Warren) • Documentation • Moab has lots • Karnak documentation in progress • Familiarity • Moab is familiar to a number of TeraGrid RPs TeraGrid Quarterly - Batch Queue Prediction

  5. Preliminary Tasks • Finalize report (12/11/2009) • Provide new GLUE 2 version (RP testing 12/7/2009) • Publishes job information to secure MDS • Port to additional schedulers • Moab (complete) • Condor (complete) • Cobalt (in progress) • PBS at PSC (jobs ordered by scheduling priority) TeraGrid Quarterly - Batch Queue Prediction

  6. Karnak Tasks • Publish REST interface description (1/8/2010) • Initial optimization of predictions for TeraGrid (1/15/2010) • Provide client programs (1/22/2010) • Publish user documentation (2/5/2010) TeraGrid Quarterly - Batch Queue Prediction

  7. Usage-Related Tasks • Integrate into TeraGrid user portal (2/2010?) • Integrate into metaschedulers (e.g. Condor-G matchmaking) (2/13/2010) • Condor-G class ads created from GLUE 2 • Karnak predictions created from GLUE 2 TeraGrid Quarterly - Batch Queue Prediction

More Related