1 / 24

Introduction to the BOINC software

David P. Anderson Space Sciences Laboratory University of California, Berkeley. Introduction to the BOINC software. Outline. Abstractions The BOINC server software The BOINC client software and runtime system. Design goals. A few applications, lots of jobs High performance

cathal
Download Presentation

Introduction to the BOINC software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. David P. Anderson Space Sciences Laboratory University of California, Berkeley Introduction to the BOINC software

  2. Outline • Abstractions • The BOINC server software • The BOINC client software and runtime system

  3. Design goals • A few applications, lots of jobs • High performance • millions of jobs per day • Scalability • Fault tolerance

  4. Abstractions • Platform • App version • a collection of files, one of which is an executable main program • associated with a platform • App • a set of app versions that all perform roughly the same computation • may have versions for different platforms • may have different versions for one platform (GPU, non-GPU)

  5. Abstractions • Workunit (job) • a collection of input files • associated with an app (not an app version!) • attributes • resource estimates and bounds • latency bound • Result (job instance) • a collection of output files • associated with a workunit

  6. Anatomy of a BOINC project MySQL database daemons and periodic tasks servers project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/ clients

  7. Work generator • Creates input files • Creates workunits • One per app • Flow control • disk space • DB size MySQL database work generator project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/

  8. Specifying a job • Workunit template • XML document describing • input files (logical, physical names) • job attributes • Result template • XML document describing output files • create_work() • specifies templates, app, input files

  9. Validator • Check result validity • Compare replicas • May be app-specific MySQL database validator project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/

  10. Validation • Clients may • return bad results • exaggerated claimed credit • Strategies • app-specific consistency checking • replication • fuzzy comparison • homogeneous redundancy • adaptive replication

  11. Assimilator • Processes completed results • App-specific MySQL database assimilator project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/

  12. Summary • Create app, app versions for different platforms • Develop work generator • Develop validator • Develop assimilator Isn’t there a simpler way?

  13. Single-job submission • Assemble your input files and executable, then boinc_submit --input foo --output blah program • How this works: • uses “wrapper” app • executable is part of workunit • templates are created automatically • What it doesn’t do: • multi-platform • validation

  14. Job dispatch MySQL database feeder transitioner project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/ share-memory job cache scheduler (CGI or FastCGI) clients

  15. File transfer Apache project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/ clients file upload handler

  16. Janitorial daemons MySQL database DB purger project root/ bin/ cgi-bin/ download/ 00/ .. 3ff/ html/ log_*/ templates/ upload/ 00/ .. 3ff/ file deleter

  17. Ways to deploy a BOINC server • Linux server • Server VM for VMWare • Server VM for Amazon EC2

  18. The BOINC runtime system application BOINC runtime Directory structure: BOINC/ projects/ lhcathome/ physical_name0 physical_name1 setiathome/ slots/ 0/ logical_name0 (link file) logical_name1 1/ fraction done CPU time share-memory message-passing suspend resume quit BOINC client

  19. Basic API • boinc_init() • creates a thread that handles messages • boinc_finish() • creates a “finish file” • boinc_resolve_filename() • maps logical to physical file names

  20. Checkpointing • boinc_time_to_checkpoint() • call at points where you can checkpoint • boinc_checkpoint_done() • call when you’re finished checkpointing

  21. Compound applications • Examples: • coordinator program runs several worker programs in sequence • “switcher” program probes CPU architecture, selects which executable to run • Variants of boinc_init() let you specify which app is main program, and how messages are handled • Each message type must be handled by 1 process

  22. Long-running applications • Trickle-up messages • Trickle-down messages • Intermediate file transfers

  23. Legacy applications • The BOINC wrapper • takes XML “job file” • handles all messages

  24. GPU and multithread apps • Server • you supply a function that takes an app version and a host, and returns resource usage and estimated FLOPS • the BOINC scheduler chooses the best version • Client • senses and reports coprocessors (e.g. NVIDIA GPUs) • coprocessor-aware scheduling and work fetch

More Related