Dynamic Cloud Deployment of a MapReduce Architecture

1. Dynamic Cloud Deployment of a MapReduce Architecture Name : AreenAmjadRabadi ID : 20123173027 Supervised By: Dr. AmerBadarnah .

2. Outline • Introduction. • Limitations are Contributed to Build this model. • Aim of Paper. • Approach to doing this. • MapReduce Programming Model. • Infrastructure as a Service.

3. Outline…Cont • Managing Large Datasets with MapReduce. • Implementation • Performance Statistics. • Conclusion .

4. Introduction • The Map­Reduce programming model addresses these problems in many scenarios. • The main challenge associated with processing large datasets is the infrastructure required. • In this new model, elastic infrastructures can dynamically adapt to the consumer’s data processing requirements.

5. Limitations are Contributed to Build this model. • cloud providers offer such services in a ready to use fashion and don’t provide any details about implementation or how the services work internally . • clients can’t control the MapReduce software stack and its configuration, which can lead to optimization,performance, and compatibility problems. • these services are always vendor specific, preventing clients from using multiple cloud providers or their own private cloud infrastructure, or even from offering their own elastic cloud­basedMapReduce service.

6. Aim of Paper • framework enables the dynamic deployment of a MapReduce service in virtual infrastructures from either public or private cloud providers.

7. Approach to doing This • MapReduce service is customizable and deployed using SmartFrog • a configuration management software that hides the complexity involved in service provisioning from users • while letting them retain full control over the service’s individual aspects. • Use the HadoopMapReduce implementation to validate our architecture.

8. Map Reduce Model

9. Infrastructure as a Service

10. Managing Large Datasets with MapReduce

11. Automatic Deployment layer

12. Implementation • Our implementation has two Components: • RESTful Web service offering functionalities from the MapReduce job management layer • front­end Web­based application. • install this application in both servlet and portlet containers. • implemented the elastic MapReduce service itself as a war artifact that developers can deploy in an existing Web application server. • The boot volume comprises a clean installation of Ubuntu 9.04 and SmartFrog v3.17. • added additional components on which SmartFrog relies to the image, such as apt-get for installing the Linu x packages Hadoop requires.

13. Performance Statistics • To evaluate our framework’s performance, we conducted an experiment that aimed to measure: • the relationship between the time for creating the virtual infrastru-cture and for booting the OS (infrastructure creation time). • the time for provisioning and starting the infrastr-ucture with the MapReduce implementations (provisioning time). • the time for executing the MapReduce job including data uploading and downloading (MapReduce execution time).

14. Performance Statistics…cont

15. Conclusion • The MapReduce program m ing model has shown immense potential for processing large and unstructured datasets.

16. Any Question???