1 / 40

INRIA Background & ActiveEon’s Mission 2. ProActive : Programming Scheduling Resourcing

Accelerate on Windows Desktops , HPCS, Clouds w ith ProActive Parallel Suite. D. Caromel , C. Delbé. Speed: Application + Management: Productivity. INRIA Background & ActiveEon’s Mission 2. ProActive : Programming Scheduling Resourcing 3 . Use Cases: Genomics

devlin
Download Presentation

INRIA Background & ActiveEon’s Mission 2. ProActive : Programming Scheduling Resourcing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Accelerate on Windows Desktops, HPCS, Clouds with ProActiveParallel Suite D. Caromel, C. Delbé Speed: Application + Management: Productivity INRIA Background & ActiveEon’s Mission 2. ProActive: Programming Scheduling Resourcing 3. Use Cases: Genomics Finance

  2. 1. Background & Mission

  3. INRIA OASIS Team • A Research team, about 35 persons • 2004: First ProActive User Group • 2009, April: ProActive 4.1, Distributed & Parallel: From Multi-cores to Enterprise GRIDs & Clouds

  4. ActiveEon and INRIA synergy The best of research and industry! Customer Driven Developments • Customers Partners Research Contracts High Added Value Services Located in Sophia Antipolis, between Nice and Cannes, Visitors Welcome!

  5. ActiveEon Company Overview (1) • Co-developing, Support for ProActiveParallel Suite • Worldwide Customers: Fr, UK, USA Selected Partners:

  6. ActiveEon Company Overview (2) • Open Source Software Editor Born of INRIA in 2007 • Core mission: Scale Beyond Limits • Co-develops ProActive Parallel Suite® • Provides high level services for ProActive • Ten years experience in development of parallel computing

  7. 2. ProActiveParallel Suite

  8. ProActiveParallelSuite Java Parallel Toolkit Multi-Platform Job Scheduler Resource Manager Used in Production everyday: 50 Cores  300 Cores early 2010 • Strong Features: • Java Parallel Programming + Legacy + Integration + • Portability: Windows, Linux, Mac + • Versatility: Desktops, Cluster, Grid, Clouds = Perfect Flexibility

  9. ProActive Parallel Suite Three fully compatible modules Clutch Power: Solid Building Blocks for Flexible Solutions Programming Scheduling Scheduling Resourcing Resource Management

  10. ProActive Programming

  11. Groups: Broadcast and Scatter (aka MPI) ag JVM c3 c3 c3 c3 c3 c3 c3 c1 c1 c1 c1 c1 c1 c1 c2 c2 c2 c2 c2 c2 c2 JVM JVM s s JVM • Broadcast is the default behavior • Use a group as parameter, Scattered depends on rankings cg ag.bar(cg); // broadcast cg ProActive.setScatterGroup(cg); ag.bar(cg); // scatter cg 11

  12. Dynamic Dispatch Group c4 c4 c4 c6 c6 c6 c5 c5 c5 c7 c7 c7 c8 c8 c8 c0 c0 c0 c9 c9 c9 c3 c3 c3 c1 c1 c1 c2 c2 c2 JVM Slowest ag cg JVM Fastest JVM ag.bar(cg); JVM 12

  13. Optimizing GUI: IC2D

  14. Optimizing GUI: IC2D

  15. Optimizing GUI: IC2D

  16. ProActiveScheduling 16

  17. ProActive Scheduling Big Picture • Multi-platform Graphical Client (RCP) • File-based or LDAP authentication ProActive Scheduler • Static Workflow Job Scheduling, Native and Java tasks, Retry on Error, Priority Policy, Configuration Scripts,… ProActive Resource Manager • Dynamic and Static node sources, Resource Selection by script, Monitoring and Control GUI,… RESOURCES • ProActive Deployment capabilities: Desktops, Clusters, Clouds,…

  18. TaskFlow Example : Picture Denoising Split Denoise Denoise Denoise Denoise Merge • with selection on native executable availability (ImageMagik, GREYstoration) • Multi-platform selection and command generation • with file transfer Job

  19. ProActive Resourcing 19

  20. Desktops + Clusters + Clouds 20

  21. Versatility: Desktops, Grids, Clouds (EC2),Multi-Core, VMs (Hyper-V, Xen, VMware, Kvm) ProActive Scheduler ProActive Resource Manager Timing Policy 12/24 Dynamic Workload Policy Static Policy LSF SSH EC2 Dedicated resources Amazon EC2 Desktops

  22. RESOURCING User Interface

  23. 2 mn Video:Scheduler, Resource Manager

  24. 3. Use Cases

  25. Bio-Tech:GenomicSequencing

  26. Resources set up SOLID machine from Cluster 16 nodes Desktops Nodes can be dynamically added! Amazon EC2 Clouds

  27. Scalability:Internal SOLID Cluster to INRIA Cluster Reference point with 16 cores (same as in SOLiD machine) 4 Time faster from 20 to 100 Speed Up of 80 / Th. Sequential 50 Hours  35 Minutes • The distributed version with ProActiveof Mapreads has been tested on the INRIA cluster with two settings: the Reads file is split in either 30 or 10 slices • Use case: matching 31 millions sequences with the human genome (M=2, L=25)

  28. Benchmark: Internal vs. EC2 cloud only For only $3,2/hour, the EC2 setup has nearly the same performances as the local SOLiD cluster

  29. ProActivein Finance with Price-It Excel

  30. Use case: Price-It and ProActive • Price-It® • C++ library developed by Pricing Partners • Pricing solution dedicated to highly complexfinancial derivatives • Accelerated Price-It® • A collaboration between Pricing Partners and ActiveEon • An implementation based on ProActiveScheduling • Specification and constraints • Accelerate Price-It® Excel product • Built on Price-It® library, this product integrates an interface with Excel for input data management and results display • Focus on highly parallelizable Greek computation • Operating system: Windows

  31. Price-it Sequential Computing C/C++ Price-it® library Price-it® Computation DLL Before distribution

  32. Price-it Accelerated by ProActive C/C++ Price-it® library Price-it® Computation DLL Smooth integration of legacy code! Using distribution Scheduler Price-it® Computation DLL Price-it® Computation DLL Worker Worker

  33. Accelerated Price-It performances More than 3 times faster with only 4 nodes! Even 6 times faster with 9 nodes! 4 nodes 5 nodes 6 nodes 7 nodes 8 nodes 9 nodes Sequential Distributed • Use case: Bermuda Vanilla, Model American MC • Test conditions • One computation is split in 130 tasks that are distributed • Each task uses 300ko

  34. Windows HPCS Benchmarks

  35. Overall User Domain and Use Cases Use Cases: • Finance • Life Science and BioTech • Engineering: Accelerating Matlab® and Scilab • IT • Numerical SPMD

  36. Wrap Up

  37. ProActive Parallel Suite • Flexibility • Clutch Power • Portability: • Windows, Linux, Mac • Versatility: • Desktops, Grids, Clouds Java Parallel Toolkit Multi-Platform Job Scheduler Resource Manager Free Professional Open Source Software Community and Enterprise Editions • Choose the Edition you need CommunityEnterprise Users support Professional support www.activeeon.com/downloads

  38. Extra Material

  39. ActiveEon: Mission, Services, Product Mission: • Accelerate your applications: Development, Deployment, Execution • Optimize your Hardware Infrastructure utilization • Easy and Accessible Parallel Computing & Clouds High-End Services & Support: • Training, Consulting, Integration, Development • Technical Support and Subscription • Partnerships (ISVs, VARs for OEM) Open Source Products:

  40. ProActive : Active objects JVM A A WBN! A ag =newActive (“A”, […], VirtualNode) V v1 = ag.foo (param); V v2 = ag.bar (param); ... v1.bar(); //Wait-By-Necessity JVM ag v2 v1 V Wait-By-Necessity is a Dataflow Synchronization Java Object Active Object Req. Queue Future Object Proxy Thread Request 40

More Related