1 / 17

Scaling Many-core Applications with COTS Clustering

or Call for Hammers on My (crappy) Idea. Scaling Many-core Applications with COTS Clustering. Haibo Chen Parallel Processing Institute Fudan University. Outline. Motivation VMM-based COS clustering for many-core applications Emulator clustering for many-core OS emulation.

mahon
Download Presentation

Scaling Many-core Applications with COTS Clustering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. orCall for Hammers on My (crappy) Idea Scaling Many-core Applications with COTS Clustering Haibo Chen Parallel Processing Institute Fudan University

  2. Outline • Motivation • VMM-based COS clustering for many-core applications • Emulator clustering for many-core OS emulation

  3. Multi-core is emerging • 4 cores are prevalent • 10s to 100s cores in five years • System software lags behind architecture

  4. Software not scalable in commodity OS

  5. The problem • Many OS components are designed and evolved from UP • Page fault, buffer cache • Not easy to fix • Years of improvements in Linux community not only fix problems • But also discover more problem as hardware evolves

  6. Solution • Long term: design new OS • Usually non-trivial • Backward compatibility issues • Long time to mature • Short/middle term solution • Can we provide scalable system software building from cots?

  7. Research Efforts • The Corey Philosophy • No share by default • Share efficiently if needed • Two preliminary efforts applying Corey philosophy • A VMM-based, SSI commodity OS cluster to host many-core application • SSI emulator cluster to study OS scalability problem

  8. Many-core apps(e.g. MapReduce, webserver) Uniform Interface Linux Linux Linux Linux Resource sharing Virtual Machine Monitor C C C C C C C C C C C C C C C C

  9. Research Points (1/3) • Study resource management for diverse applications among COS • Which should be shared? • Buffer cache? Address space? • Main memory? I/O buffer rings • Which should be exclusive? • TCP/IP stack?

  10. Research Points (2/3) • Adjust COSes to suite different sharing patterns • Modifying VMM to support inter-VM sharing of resources

  11. Research Points (3/3) • How an high-performance software emulation layer can be designed to seamlessly support existing apps • System call interposition • System call translation • System call forwarding

  12. Other Concerns • Heterogeneity in future multi-core? • Managing and hiding heterogeneity • Efficient uses of cache • Everyone talks about this

  13. SIMULATING FUTURE MANY CORE USING MULTI-CORE

  14. Many-core emulator • The need for many-core emulator • Study OS behaviors for future many cores • Bottlenecks • Debugging • Difficulty in many-core emulation • Scalability

  15. Proposed Research • “Divide and conquer” • Categorize the sharing and exclusive use of resource in an emulator • A thin emulation layer connecting the emulation clusters

  16. Many-core Operating Systems Uniform Interface C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU QEMU Resource Sharing and Communications Linux Kernel C C C C C C C C C C C C C C C C

  17. Thanks • Q&A

More Related