1 / 49

Aneka A Platform for Enterprise Grid/Cloud Computing

Aneka A Platform for Enterprise Grid/Cloud Computing. Christian Vecchiola Ph.D. GRIDSLab, Computer Science Software Engineering The University of Melbourne csve@unimelb.edu.au. Outline. Introduction Cloud Computing Aneka Features Architecture Programming Models Hands on Aneka

avalon
Download Presentation

Aneka A Platform for Enterprise Grid/Cloud Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Aneka A Platform for Enterprise Grid/Cloud Computing Christian Vecchiola Ph.D. GRIDSLab, Computer Science Software Engineering The University of Melbourne csve@unimelb.edu.au

  2. Outline • Introduction • Cloud Computing • Aneka • Features • Architecture • Programming Models • Hands on Aneka • Projects Aneka a Platform for Enterprise Grid/Cloud Computing

  3. Introduction Aneka a Platform for Enterprise Grid/Cloud Computing

  4. Uhm, I am not quite clear…Yet another buzzword..? Cloud Computing Amazon EC2 QoS SaaS IaaS Virtualization Amazon S3 PaaS Storage Pricing Billing Web Services VMWare Resource Metering Service Level Agreement Hypervisors Utility Computing Provisioning on Demand Security Manjrasoft Aneka Public Cloud Web 2.0 Privacy Private Cloud Enterprise Cloud Google AppEngine SalesForce.com Mosso Aneka a Platform for Enterprise Grid/Cloud Computing

  5. The “Cloud” is the default symbol of the internet in diagrams. • The broader term of “Computing” encompasses: • Computation • Coordination logic • Storage Cloud Computing is about moving computing from the single desktop pc/data centers to the internet. Cloud Computing • About the Cloud Cloud Computing Aneka a Platform for Enterprise Grid/Cloud Computing

  6. Cloud Computing • More formally…. • Fox et al. (Berkeley) Cloud Computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the datacenters that provide those services. The services themselves have long been referred to as Software as a Service (SaaS), so we use that term. The datacenter hardware and software is what we will call a Cloud. • Buyya et al. A Cloud is a type of parallel and distributed system consisting of a collection of interconnected and virtualized computers that are dynamically provisioned and presented as one or more unified computing resources based on service-level agreements established through negotiation between the service provider and consumers. Aneka a Platform for Enterprise Grid/Cloud Computing

  7. These are the two main differences from Grid Computing. Cloud Computing • So what ...? • Internet-based (the Internet is central..) • The entire stack (hw & sw) is leveraged to the internet • HaaS  Amazon S3 • IaaS  VMWare, Amazon EC2 • PaaS  Google AppEngine, Mosso, Aneka • SaaS  SalesForce • Computing becomes a real utility used by all the masses • researchers • end users • companies • service providers (web hosting) • The economic aspect become more important (billing, pricing, QoS, SLA) Aneka a Platform for Enterprise Grid/Cloud Computing

  8. Google Apps (Gmail, Docs,…) Accounting Salesforce.com Pricing and Billing Web 2.0 Interface Programming API Scripting & Programming Languages Admission Control Google AppEngine Microsoft Azure QoS Metering Manjrasoft Aneka Virtualization Private Cloud VM Management & Deployment SLA Negotiation Amazon S3, EC2 Public Cloud OpenNebula, Eucalyptus Data Centers Other Grids/Clouds Clusters Mosso Storage Anatomy of a Cloud Software as a Service Financial Applications Scientific Computing Social Networks Amusement Platform as a Service CDNs Infrastructure as a Service Aneka a Platform for Enterprise Grid/Cloud Computing

  9. Embracing the Cloud • Benefits • Infinite compute resource available on demand • Hardware virtualization • Accessibility anytime and anywhere • Internet (web based) access • Elimination of the upfront commitment of users • Reduced costs due to dynamic hardware provisioning • Pay per use basis (and also other models) • No need to plan for peak load in advance • Easy management • Software versioning and upgrading Aneka a Platform for Enterprise Grid/Cloud Computing

  10. Will it be fitting for me? Embracing the Cloud • Migration Risks • Privacy • Who access your data? • Security • How much you trust your provider? • What about recovery, tracing, and data integrity? • Political and legal issues • Who owns the data? • Who uses your personal data? • Government • Where is your data? • Amazon Availability Zones Aneka a Platform for Enterprise Grid/Cloud Computing

  11. References • Web pointers • cloud-computing@googlegroups.com • http://twitter.com/cloudcomp_group • http://sites.google.com/site/cloudcomputingwiki/ • http://wiki.cloudcommunity.org/ • http://en.wikipedia.com/cloud_computing • http://cloudslam09.com/ Aneka a Platform for Enterprise Grid/Cloud Computing

  12. Aneka Aneka a Platform for Enterprise Grid/Cloud Computing

  13. Features • Characterization • Middleware for Enterprise Grids/Clouds • Service oriented architecture • .NET/Mono based environment • Languages: • C#, C++, VB, Delphi, Java/IKVM… • … and 20 more languages • Platforms: • Windows XP/2000/2003 • Linux & Mac OS X Aneka a Platform for Enterprise Grid/Cloud Computing

  14. Designed to be a configurable middleware with the aim of supporting an open ended set of abstractions for distributed computing and deployment scenarios Features • Why Aneka? • Aneka (sanskrit): many, in many ways, many in one… • This means: • Multiple programming/deployment models • Multiple scheduling strategies • Multiple authentication models • Multiple persistence backends • Multiple platform and OSs Aneka a Platform for Enterprise Grid/Cloud Computing

  15. Private Cloud Public Cloud Aneka Aneka fits into the cloud architecture at the platform layer. This means that it provides a programming based interface for developing distributed application and a virtual execution environment in which the applications developed according to the published APIs can run. Features • Aneka & Clouds Software as a Service Platform as a Service Infrastructure as a Service Aneka a Platform for Enterprise Grid/Cloud Computing

  16. Features • Current Applications • Scientific • Distributed evolutionary computation • Proteine structure prediction • Commercial • Engineering: Go Front (China): Train models rendering • Media and games: platform for on-line gaming • Financial: risk analysis • Office automation: Excel integration • Educational • Image filtering • Image rendering • Distributed systems teaching Aneka a Platform for Enterprise Grid/Cloud Computing

  17. Aneka enterprise Cloud Aneka Container work units Executor Executor Executor Executor internet work units Scheduler Workers Manager(s) internet Manager Manager Client Applications Architecture • System Overview Aneka a Platform for Enterprise Grid/Cloud Computing

  18. Characterization Scheduling Execution Storage Persistence Monitoring Directory Service Negotiation Resource Allocation Core Security Communication Layer Architecture • Anatomy of a Container Aneka a Platform for Enterprise Grid/Cloud Computing

  19. Architecture • Services • Scheduling Services: • Basic scheduling services • Support for… • … independent bag of task scheduling • … different and pluggable scheduling algorithms • … advanced reservation aware schedulers • … automatic/custom task resubmission on failure • … node quarantine/black listing • Execution Services: • Basic sandboxing • Support for legacy code execution (PSM) • Storage: • Multi-protocol based storage service (FTP) Aneka a Platform for Enterprise Grid/Cloud Computing

  20. Programming Models • Development • Aneka is Platform as a Service cloud middleware • This means: • It exposes an API for development • It provides access to the cloud at programming level • More precisely… • It provides different programming models Aneka a Platform for Enterprise Grid/Cloud Computing

  21. Executor Executor Executor Executor Scheduler Manager Programming / Deployment Model Aneka • Scenario Aneka enterprise Cloud public DumbTask: ITask { … public void Execute() { …… } } for(int i=0; i<n; i++) { … DumbTask task = newDumbTask(); app.SubmitExecution(task); } internet work units internet Aneka a Platform for Enterprise Grid/Cloud Computing

  22. MapReduce Model Task Model scheduling TaskScheduler MapReduceScheduler execution TaskExecutor MapReduceExecutor coordination TaskManager MapReduceManager client component Task abstractions Mapper units of execution Reducer Programming Models • Overview infrastructure end users Aneka a Platform for Enterprise Grid/Cloud Computing

  23. Programming Models • Currently supported: • Task Programming Model • Thread Programming Model • MapReduce Programming Model • Parameter Sweeping Model • .. Implement your own.. Aneka a Platform for Enterprise Grid/Cloud Computing

  24. Programming Models • Task Programming Model • Used to model Independent Bag of Tasks (BoT) applications • The application is a collection of execution unit • Each execution unit is not related to the others • There is no order in the execution of the units • If there is any ordering or sequencing it is performed by the client application and NOT by the middleware Aneka a Platform for Enterprise Grid/Cloud Computing

  25. Programming Models • Task Programming Model • Aneka Support • User API: • Interface ITask • Only one operation: Execute • GridTask class • ITask wrapping • Support for remote execution • Middleware: • Task Scheduling Service • Coordinates and sequences the execution of GridTask instances • Task Execution Service • Provides the execution environment for the GridTask instance • Task Application Manager • Task specific submission component Aneka a Platform for Enterprise Grid/Cloud Computing

  26. [Serializable] public class MyTask: ITask { public void Execute() { // implement your logic here ……… } } public class TaskSample { … public void Setup() { Configuration conf = newConfiguration(); conf.SchedulerUri = “tcp://localhost:9090/”; this.app = newGridApplication<GridTask,TaskManager>(conf); this.app.WorkUnitFailed += newEventHandler<WorkUnitEventArgs>(this.OnWorkUnitFailed); this.app.WorkUnitFinished += newEventHandler<WorkUnitEventArgs>(this.OnWorkUnitFinished); ……… } } public class TaskSample { … public void Execute() { for(int i=0; i<100; i++) { MyTask task = newMyTask(); GridTask gt = newGridTask(task); this.app.ExecuteWorkUnit(gt); } ……… } } public class TaskSample { … public void OnWorkUnitFailed(object sender, WorkUnitEventArgs args) { Console.WriteLine(“Task {0} failed!”, args.WorkUnit.Id); ……… } public void OnWorkUnitFinished(object sender, WorkUnitEventArgs args) { Console.WriteLine(“Task {0} finished!”, args.WorkUnit.Id); ……… } } Programming Models • Task Programming Model • Developing Task based Applications • Implement the ITask interface • Create a GridApplication instance • Create and submit tasks • Collect results. Aneka a Platform for Enterprise Grid/Cloud Computing

  27. Programming Models • Task Programming Model • Task Model Samples • Convolution • PovRay • Gauss (TaskDemo) • ExcelGrid DEMO Aneka a Platform for Enterprise Grid/Cloud Computing

  28. Programming Models • Thread Programming Model • Based on the concept of distributed thread • Like a local thread but executed remotely • Implements a subset of the common operations on thread • Start • Stop • State Query • Join • Provides a quick way for porting on a distributed middleware, multi-threaded applications Aneka a Platform for Enterprise Grid/Cloud Computing

  29. Programming Models • Thread Programming Model • Aneka Support • User API: • GridThread class • Distributed version of System.Threading.Thread • Support for remote execution • Middleware: • Thread Scheduling Service • Coordinates and sequences the execution of GridThread instances • Thread Execution Service • Provides the execution environment for the GridThread instance • Thread Application Manager • Thread specific submission component (client component) Aneka a Platform for Enterprise Grid/Cloud Computing

  30. [Serializable] public class MyThreadWorker { public void DoWork() { // implement your logic here ……… } } public class ThreadSample { … public void Setup() { Configuration conf = newConfiguration(); conf.SchedulerUri = “tcp://localhost:9090/”; this.app = newGridApplication<GridThread,ThreadManager>(conf); ……… } } public class ThreadSample { privateIList<GridThread> workers = newList<GridThread>(); public void Start() { for(int i=0; i<100; i++) { MyThreadWorker worker = newMyThreadWorker(); GridThread thread = newGridThread(new ThreadStart(worker.DoWork), this.app); thread.Start(); this.workers.Add(thread); } ……… } } public class ThreadSample { … public void Join() { foreach(GridThread thread inthis.workers) { thread.Join(); } ……… } } Programming Models • Thread Programming Model • Developing Thread based Applications • Define a worker method • Create a GridApplication instance • Create and start GridThread instances • Use the common thread APIs Aneka a Platform for Enterprise Grid/Cloud Computing

  31. Programming Models • Thread Programming Model • Thread Model Samples • Mandelbrot • Warholizer (ThreadDemo) DEMO Aneka a Platform for Enterprise Grid/Cloud Computing

  32. Programming Models • MapReduce Programming Model • Based on the MapReduce framework from Google • Functional-style like primitives: • A distributed application becomes a collection of map and reduce operations. Aneka a Platform for Enterprise Grid/Cloud Computing

  33. Programming Models • Thread Programming Model • Aneka Support • User API: • Mapper<key1,value1> (base class) • Defines the map function • Reduder<key2,value2> (base class) • Defines the reduce function • Middleware: • MapReduce Scheduling Service • Coordinates and sequences the execution of Mapper and Reducer instances • MapReduce Execution Service • Provides the execution environment for the Mapper and Reducer instances • MapReduce Application Manager • MapReduce specific submission component (client component) Aneka a Platform for Enterprise Grid/Cloud Computing

  34. Programming Models • Thread Programming Model • Developing MapReduce based Applications • Define map and reduce operations • Provide the data • Run the MapReduce engine • Execution: • File staging • Task scheduling • Failed task resubmission • Replication and Fault tolerance • Collection of result Input data MapReduce engine Map & Reduce network map & reduce Aneka a Platform for Enterprise Grid/Cloud Computing

  35. Programming Models • MapReduce Programming Model • MapReduce Model Samples • Estimation of Pi • Word Counter DEMO Aneka a Platform for Enterprise Grid/Cloud Computing

  36. Programming Models • Parameter Sweeping Model • Based on the Task Programming Model • Provides a set of facilities to run applications where • ..all the tasks are homogeneous (same task) • ..the specific instance of tasks is specialized by parameters • ..all the possible combination of parameters are explored by generating a task instance for each of the combinations • These are also called: • Embarrassingly parallel applications. • Ideal for running legacy applications [B,2,#] [A,1,#] [A,2,#] Aneka a Platform for Enterprise Grid/Cloud Computing

  37. Programming Models • Parameter Sweeping Model • Aneka Support • User APIs • Additional APIs that customize the task model for Parameter Sweeping application • IParameterizedTask, RangeParameter, RandomParameter • ….. • A set of ready to use micro-tasks to compose the PSM task • CopyTask, ExecuteTask, DeleteTask • … • Middleware: • (The same as Task Model) • PSMManager (client submission component for PSM application) Aneka a Platform for Enterprise Grid/Cloud Computing

  38. Programming Models • Parameter Sweeping Model • Developing Parameter Sweeping based Applications • Use the APIs: • Implement IParameterizedTask • Compose your task with micro-tasks • Use the PSMManager to submit the execution • Use the Design Explorer • Wizard based interaction for task templates • Execution monitoring and statistics DEMO Aneka a Platform for Enterprise Grid/Cloud Computing

  39. Programming Models • Observations • Task Programming Model • Simple, Intuitive, and General Model • Can be used to run legacy tasks or C# code • “Submit and Forget” approach • Rapid prototyping with the PSM • Thread Programming Model • Low level programming model that allows for more control • Suitable for porting local multi-thread apps to a distributed middleware Aneka a Platform for Enterprise Grid/Cloud Computing

  40. Programming Models • Observations • MapReduce Programming Model • Particularly suitable for data intensive application • Task and Thread models are task-based: • the user defines the tasks that will be executed • the user submits the task to the middleware • MapReduce model is function-based: • the user defines the functions operating on the data • the user configures the middleware with these functions • the user provides the data • the middleware automatically generates the tasks required to execute the functions on the data Aneka a Platform for Enterprise Grid/Cloud Computing

  41. Programming Models • Extensibility • Implementing a new Programming Model • Defines the user abstractions • Provide an implementation for.. • Execution Service (middleware) • Scheduling Service (middleware) • Application Manager (client component) • Where to start: • Aneka.Entity.*, Aneka.Data.Entity*; • Aneka.Execution.*; • Aneka.Runtime.Scheudling.*; Aneka a Platform for Enterprise Grid/Cloud Computing

  42. Hands on Aneka Aneka a Platform for Enterprise Grid/Cloud Computing

  43. Hands on Aneka • Work In Progress • Developing application with Aneka • Choosing the programming Model • Defining the application logic • Implementing the code • Setup and running.. Aneka a Platform for Enterprise Grid/Cloud Computing

  44. Projects Aneka a Platform for Enterprise Grid/Cloud Computing

  45. RESERVATION (ALL MODELS) MARKET ORIENTED SHEDULING VIRTUAL MACHINES INDIPENDENT BAG OF TASKS DISTRIBUTED THREADS MAPREDUCE WORKFLOW/DATAFLOW? ACTORS? Projects • Contributing to Aneka • Infrastructure: • services • scheduling algorithms • resource allocation policies • security • programming/deployment models • End users: • Aneka-based application • integration Aneka a Platform for Enterprise Grid/Cloud Computing

  46. Projects • Research • Cooperative Scheduling • Virtual Execution Environment • Advanced Quality of Service • Resource Pricing • Cloud shift (happens) • Development • Programming and Deployment Models • Dataflow • MPI • Workflow implementation • Platform Porting Aneka a Platform for Enterprise Grid/Cloud Computing

  47. Uhm….. Projects • Opportunities… • Looking for a student project? • Contact us: • A/Prof. Rajkumar Buyya –raj@csse.unimelb.edu.au • Dr.Christian Vecchiola –csve@unimelb.edu.au • Mr. Xingchen Chu –xchu@csse.unimelb.edu.au Aneka a Platform for Enterprise Grid/Cloud Computing

  48. Projects • References • Manual and tutorials (Manjrasoft web: http://www.manjrasoft.com) • C. Vecchiola and X. Chu – Aneka Tutorial Series: Developing Task Model Applications • C. Vecchiola and X. Chu – Aneka Tutorial Series: Developing Thread Model Applications • C. Jin and C. Vecchiola – Aneka Tutorial Series: Developing MapReduce Applications • X. Chu and C. Vecchiola – Aneka Installation Guide • Papers • C. Vecchiola, M. Kirley, and R. Buyya, Multiobjective Problem Solving with Offspring on Enterprise Clouds, Proc. of HPC ASIA 2009, Mar. 2-5, 2009, Kaohsiung, Taiwan. • C. Jin, C. Vecchiola, and R. Buyya, MRPGA: An Extension of MapReduce for Parallelizing Genetic Algorithms, Proc. of the 4th IEEE Int. Conf. on e-Science, (e-Science 2008, IEEE CS Press, Los Alamitos, CA, USA), Dec. 10-12, 2008, Indianapolis, Indiana, USA. • X. Chu, K. Nadiminti, C. Jin, S. Venugopal, and R. Buyya, Aneka: Next-Generation Enterprise Grid Platform for e-Science and e-Business Applications, Proc. 3rd IEEE Int. Conf. on e-Science and Grid Computing (e-Science 2007, IEEE CS Press, Los Alamitos, CA, USA), Dec. 10-13, 2007, Bangalore, India. Aneka a Platform for Enterprise Grid/Cloud Computing

  49. The End.. …questions? ? ? ? ? ? ? ? Aneka a Platform for Enterprise Grid/Cloud Computing

More Related