1 / 12

Batch Computing at Altera

Batch Computing at Altera. Condor, Quill and The Enterprise. About Altera. “The Programmable Solutions Company” Pioneer of SOPC technology Founded in 1983 $1.02 billion in 2004 sales 2,300 employees 14,000+ worldwide customers. About Programmable Solutions. Programmable Logic

elia
Download Presentation

Batch Computing at Altera

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Batch Computing at Altera Condor, Quill and The Enterprise

  2. About Altera • “The Programmable Solutions Company” • Pioneer of SOPC technology • Founded in 1983 • $1.02 billion in 2004 sales • 2,300 employees • 14,000+ worldwide customers

  3. About Programmable Solutions Programmable Logic Devices (PLDs) Intellectual Property(IP) Development Software

  4. About Me • Senior Software Engineer at the Toronto Technology Center • B.A.Sc. in Engineering Science from the University of Toronto • Joined Altera in 2001 • Focus on distributed computing

  5. Where It All Began • Developed in Toronto • Centralized scheduling system • Multiple queues • Priority/FIFO execution • No limit on resource claims • Engineer-designed, custom API

  6. Change Is Good, Right? • Multi-OS support • Redundancy and fault tolerance • Easy expansion beyond Toronto • Easy-to-use API • New features • Improve matchmaking • Capacity planning  Really Important!

  7. META SCHEDULER SOAP PriorityEngine CONDOR TTC DB Pain Free Migration CONDOR POOL USERTOOLS TTCPOOL

  8. Time Stands Still • Nice-style priorities [1:N] • Use priority factor to ensure PN negotiates before PN+1, PN+2, etc. • RUP(PN) = 0.5 • EUP(PN)/EUP(PN+1) = ½ • Freeze RUP values in time • PRIORITY_HALFLIFE = 100000000000000000000 • Let jobs at PN get all VMs in the system • NEGOTIATOR_IGNORE_USER_PRIORITIES = True

  9. Translation Services <cluster> <id>1</id> <priority>2</priority> <os>windows</os> <group>fitter</group> <job> <id>1</id> ... </job> <job> <id>2</id> ... </job> ... </cluster> +AlteraClusterID = 1 +AlteraGroup = fitter requirements = OpSys = ... +AccountingGroup = P1 AlteraTargetOs = windows ... +AlteraJobID =1 ... queue +AlteraJobID = 2 ... METASCHEDULER

  10. SQL! SQL! Everywhere! METASCHEDULER USAGE HISTORY POSTGRESQL DBMS CONDOR QUILL STATUSINFO CONDORCOLLECTOR SYSTEM AUDITS

  11. Better Planning Through Pictures

  12. From Here, Where? • Roll out across the enterprise • Scaling with multiple schedds • Quill++ • DBMS for configuration management (with R. Nordlund & J. Stowe from The Hartford)

More Related