ParoC++: A Requirement-driven Parallel Object-oriented Programming Language

1. ParoC++: A Requirement-driven Parallel Object-oriented Programming Language Tuan-Anh Nguyen, Pierre Kuonen University of Applied Sciences of Fribourg

2. Outline • Changes in HPC methodology • Parallel Object • ParoC++: language and its programming paradigm • Experiment results • Conclusion ParoC++: requirement-driven parallel objects

3. Changes in HPC methodology • Emerging of Grid • Large scale computation requirements • Fast internet connections • The move of HPC programming paradigm from: program-centric approach to service-centric approach ParoC++: requirement-driven parallel objects

4. HPC Programming • Program-centric: Execute my program on my specified resources • Traditional approach • Lack flexibility to deal with computation on demand • MPI • Service-centric: Provide me some services, no matter where they are located • New approach • Service requirements: computing power, network bandwidth, storage space,… • More flexible in heterogeneous, distributed environment ParoC++: requirement-driven parallel objects

5. Our approach • Service-centric • Object-oriented parallel programming paradigm (parallel objects) • Object-oriented Programming System (ParoC++) ParoC++: requirement-driven parallel objects

6. Why object-oriented? • Well-investigated software engineering method • Higher level of abstraction • Nature of objects: relatively independent entities • Parallelism: inter-object and intra-object • Inter-object parallelism: coarse grain: concurrent interactions among objects • Intra-object parallelism: medium to fine grain: concurrent invocations of methods (operations) ParoC++: requirement-driven parallel objects

7. Parallel Objects • Coherence with the OO programming paradigm: interaction between objects via method invocations and through object interfaces • Extending sequential objects: an object can be located on a remote resource, in a separate memory address space • Parallel object: • Shareable • Various method invocation semantics • Transparent object allocation • Be able to describe the requirements during object’s lifetime • No explicit send/receive ParoC++: requirement-driven parallel objects

8. Shareable objects • A parallel object can be accessed by many other parallel objects simutaneously I am shareable ParoC++: requirement-driven parallel objects

9. Invocation semantics • Interface semantics • Asynchronous • Synchronous • Object-side semantics • Sequential: first come first serve. During the execution, no other request will be served. • Concurrent: serve requests concurrently • Mutex: unique execution of method ParoC++: requirement-driven parallel objects

10. Object creation/destruction • Dynamic object creation/destruction • Transparent to the user • Creation process: • Find a “suitable” resource • Locate and transmit the object code • Start object code on the resource • Set up the object interface • Object destruction: • Interface side: free memory if necessary • Object-side: destroy object only if there is no reference to that object ParoC++: requirement-driven parallel objects

11. Requirement-driven objects • Each parallel object has a user-specified object description (OD) • OD describes the requirements of parallel objects • OD is used as a guideline for allocating resource and object migration • OD can be expressed in terms of: • Maximum computing power (e.g. Mflops) • Communication bandwidth with its interface • Memory needed • OD can be parameterized on each parallel object (based on the actual input) • Two types: strict description and non-strict OD ParoC++: requirement-driven parallel objects

12. Object Description • Strict OD • Describing the requirements that the resource must be fully satisfied • Example: power=100MFlops; Network=100Mbit; • Non-strict OD • The fully satisfaction is preferable but partial satisfaction is acceptable • Example: power=100MFlops : 50MFlops; Memory=256MB : 128MB; ParoC++: requirement-driven parallel objects

13. Object1 Object4 Object2 Shared Object Object5 Object3 Programming Paradigm main ParoC++: requirement-driven parallel objects

14. ParoC++: a prototype of parallel object • Programming language: a super set of C++ to support parallel objects • ParoC++ compiler: translating ParoC++ code to ANSI C++ • Runtime services: running ParoC++ applications in heterogeneous environments ParoC++: requirement-driven parallel objects

15. parclass Integer { public: Integer(int wanted, int minp) @{ power=wanted : minp;}; Integer(char *machine) @{ host=machine; }; asyncvoid Set(int val); concint Get(); mutex syncvoid Add(Integer &other); protected: int data; }; Integer::Integer(int wanted, int minp) { } Integer::Integer (char *machine) { } void Integer::Set(int val) { data=val; } int Integer::Get() { return data; } void Integer::Add(Integer &other) { data+=other.Get(); }; ParoC++ Example ParoC++: requirement-driven parallel objects

16. ParoC++ Example int main(int argc, char **argv) { try { Integer o1(100,80),o2(50,40); o1.Set(1); o2.Set(2); o1.Add(o2); cout<<”value=”<<o1.Get(); } catch (int e) { cout<<”creation fail”; } } O1 main Set Add Set O2 ParoC++: requirement-driven parallel objects

17. ParoC++ architecture • System-wide services: resource discovery, remote execution • Application-scope services: code manager, monitor • Communication: TCP/IP with Sun XDR as data representation model High performance applications ParoC++ programming language System wide services App-scope services Comm Heterogeneous environment ParoC++: requirement-driven parallel objects

18. Object construction and method invocation Object implementation Interface XDR converter XDR converter Object Broker Resource discovery Communication Exec service NET Code manager ParoC++: requirement-driven parallel objects

19. Cocurrency control • Inter-object: method invocations • Intra-object: shared data vs. event sub-systems • Event sub-system: • Object-scope: inside each parallel object • A method can raise or can wait for an event • Deal with synchronization and signaling • Mutual exclusive execution: new keyword “mutex” • Object-scope • mutex { statement1; statement2;…} ParoC++: requirement-driven parallel objects

20. Results: network bandwidth • Ping-pong program between 2 objects (method invocations) and 2 MPI process (MPICH) • Linux P4 1.6 GHz, Fast Ethernet ParoC++: requirement-driven parallel objects

21. Example application: Pattern and Defect Detection System (PDDS) • A part of ForAll project, an European project financed by Swiss Government • CTI project No 5130.1 in the EUREKA European program • Done in the collaboration with EPFL • Find pattern positions and detect defects on tissue images, all in real-time • Type of tissues: non-uniform tissues with rectangular patterns ParoC++: requirement-driven parallel objects

22. Local maximal Smaller value PDDS algorithms • Pattern template: • Tissue image: • Inputs: a pattern template and a tissue image • Shift the template over the tissue image • For each position, compute the similarity between the template and the sub-image • Pattern position: local maximal of similarity • Criterion for the similarity: cross correlation ? ParoC++: requirement-driven parallel objects

23. ParoC++ implementation ParoC++: requirement-driven parallel objects

24. PDDS: Results (1) • Cluster P4, 1.6GHz, Fast Ethernet • Sparc workstations, 360MHz, Fast Ethernet ParoC++: requirement-driven parallel objects

25. Environment changes PDDS: results (2) • Adapt to the changes of the environment • “Main program” monitor the analysis speed at ImageBuf • Dynamic change of the requirement • If the monitored speed is not sufficient, more Analyzer objects will be allocated • Environment: Linux/Pentium 4 and Solaris/Sparc ParoC++: requirement-driven parallel objects

26. Conclusion • We have addressed the question: How to tailor HPC applications to the highly heterogeneous environment • A new concept of parallel objects has been proposed • Object description: requirement-driven objects • Dynamic creation, transparent access • Various invocation semantics • ParoC++: An implementation of parallel object • Programming language: extension of C++ • Runtime system • Experiments, both at service-levels and application-levels, show • Performance and scalability have been achieved • Ability to adapt to the computation on demand and to the changes in dynamic environments ParoC++: requirement-driven parallel objects