200 likes | 217 Views
This article discusses the implementation of a component assembly compiler for high-performance computing (HPC), focusing on the programming model requirements, software component models, component implementation choices, and algorithmic skeletons. The goal is to optimize performance and support for parallelism in HPC applications.
E N D
On MDE to implement a Component Assembly Compiler for HPC Journées IDM, June the 7th 2011 Julien Bigot*, Christian Pérez* Laboratoire de l’Informatique du Parallelisme (LIP) ENS Lyon
Scientific applications Domain specialist code Long life cycle (~30 years) Computation intensive Hardware Heterogeneous Supercomputers Computing clusters Computing grids Clouds Parallel Short life cycle (~3 years) Programming model Low level, fine-tuning of code Hand made … and remade Context: High Performance Computing ? programming models Structural Mechanics Optics Dynamics Thermal Satellite modeling SAN SAN Cluster Cluster LAN WAN Supercomputer Supercomputer Grid
Outline • Context: High Performance Computing (HPC) • High Level Component Model (HLCM) Modeling and Implementation • Feedback on the Model Driven Approach • Conclusion
Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components
Components black boxes Interactions through well defined interaction points Application Set of interconnected instances High level view of architecture Separation of concern Domain code: components Support: component model Eased reuse Well defined interface & behavior Software components d a b CmpA CmpB c c1 c2
Software Component Models • Industrial Component Models (CORBA Component Model, Enterprise JavaBeans, …) • Transparent distribution support (CORBA, JavaRMI, …) • Overhead for same host components • Research Component Models (Fractal, …) • Interesting features (Hierarchy, Connectors, …) • Features handled at runtime, additional costs • High Performance Component Models (CCA, …) • Acceptable overheads • Low level of abstraction => Hardware adaptation complex
Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Component implementation choice
Distinction type/implementation Type: name + ports Implementation: behavior Choice Depending on available hardware Multiple alternative implementations CmpA I2 I1
Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Component implementation choice 1st class Software Connectors with implementation choice
1st class entity A Name Set of roles Multiple alternative implementations Hardware optimization possible Interaction code well identified Multiple versions possible More application specific interactions 1st class software connectors CmpA CmpB ports CmpB CmpA UP user provider roles
Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Algorithmic skeletons, generic programming & hierarchy Component implementation choice 1st class Software Connectors with implementation choice
Generic programming & hierarchy:Algorithmic skeletons Farm<W, I, O, N> Farm W I O W Disp<I> Coll<O> . . . N times W
Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Compilation based approach Algorithmic skeletons, generic programming & hierarchy Component implementation choice 1st class Software Connectors with implementation choice No need for new components, only a new assembly model
HLCM Compilation • Reuse an existing component model (L²C) L²C: • Low Level • Primitive components only • Various interaction • C++ • MPI • CORBA • … PIM PSM HLCM: • Genericity • Hierarchy • Connectors • Multi-Implem MDA Hardware description PDM
HCLM (Meta)Modeling • Using the Eclipse Modeling Framework (EMF) • PIM • Main classes: ComponentType, PortType, Connector, … • Primitive implementations: abstract • Some patterns identified: genericity, implementation choice • 130 Ecore metaclasses (490 lines of EMFatic) • 2000 utility Java lines • PSM • Main classes: ComponentInstance, Connexion, … • References to the PIM in each instance • 41 Ecore metaclasses (160 lines of EMFatic) • 800 utility Java lines
HCLM Transformation • Four step transformation • Parsing HLCM: 400 lines Xtext annotated grammar • Compilation: 4000 lines plain Java • PSM to Backend • Backend Model Dump • Compilation • Choosing implementations (delegated to an external “oracle”) • Concretizing generic elements • Flattening composites • Until convergence: only primitive elements • Final dump • Often single hand made step Backend specific
Feedback (1/2) • Modeling tools: huge gain of time • Modeling HLCM • Initial attempt with plain Java: ~1 month work • Second attempt with Ecore: ~2 days work • Direct concrete syntax to model parsing (Xtext) • Ease of comprehension • Most concepts intuitive with UML background • Various level of modeling confusing (model vs. metamodel) • E.g. connectors • Fixed set of interactions: various connectors in the metamodel • 1st class connectors: various connectors in the model • Combination predefined connectors + user defined : ??? • E.g. genericity: manipulation of metamodel elements in the model About same state
Feedback (2/2) • Model transformation • Transformation tools not ready 2 years ago (Kermeta , QVT, ATL) • Plain java transformation • API not human friendly • Collection access => whole collection traversal • Omnipresent type casts (constraint not expressible in Ecore) • Error complex to associate with user error • Constraint language evaluation required • Real use • Applications: Parallel method calls, shared memory, Domain decomposition • Backends: L²C , Pure Java, CCM, Charm++, … • Transformation speed: 1500+ components, ≤ 10secs
Conclusion • HLCM: a component assembly model for HPC • Hierarchy, Generic Programming, Connectors, Alternative Implementations • Efficiency possible on a wide range of hardware • A model based implementation • Lot of time gained • Some tools not used due to immaturity • Usable in real world • Perspectives / future work • Re-evaluate modeling tools (ATL, OCL, …) • Introduce the time dimension in HLCM • Models@runtime ?