1 / 16

Dynamic Cloud Deployment of a MapReduce Architecture

Dynamic Cloud Deployment of a MapReduce Architecture. Name : Areen Amjad Rabadi ID : 20123173027 Supervised By: Dr. Amer Badarnah. Outline. Introduction. Limitations are Contributed to Build this model. Aim of Paper. Approach to doing this. MapReduce Programming Model.

mirra
Download Presentation

Dynamic Cloud Deployment of a MapReduce Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Dynamic Cloud Deployment of a MapReduce Architecture Name : AreenAmjadRabadi ID : 20123173027 Supervised By: Dr. AmerBadarnah .

  2. Outline • Introduction. • Limitations are Contributed to Build this model. • Aim of Paper. • Approach to doing this. • MapReduce Programming Model. • Infrastructure as a Service.

  3. Outline…Cont • Managing Large Datasets with MapReduce. • Implementation • Performance Statistics. • Conclusion .

  4. Introduction • The Map­Reduce programming model addresses these problems in many scenarios. • The main challenge associated with processing large datasets is the infrastructure required. • In this new model, elastic infrastructures can dynamically adapt to the consumer’s data processing requirements.

  5. Limitations are Contributed to Build this model. • cloud providers offer such services in a ready to use fashion and don’t provide any details about implementation or how the services work internally . • clients can’t control the MapReduce software stack and its configuration, which can lead to optimization,performance, and compatibility problems. • these services are always vendor specific, preventing clients from using multiple cloud providers or their own private cloud infrastructure, or even from offering their own elastic cloud­basedMapReduce service.

  6. Aim of Paper • framework enables the dynamic deployment of a MapReduce service in virtual infrastructures from either public or private cloud providers.

  7. Approach to doing This • MapReduce service is customizable and deployed using SmartFrog • a configuration management software that hides the complexity involved in service provisioning from users • while letting them retain full control over the service’s individual aspects. • Use the HadoopMapReduce implementation to validate our architecture.

  8. Map Reduce Model

  9. Infrastructure as a Service

  10. Managing Large Datasets with MapReduce

  11. Automatic Deployment layer

  12. Implementation • Our implementation has two Components: • RESTful Web service offering functionalities from the MapReduce job management layer • front­end Web­based application. • install this application in both servlet and portlet containers. • implemented the elastic MapReduce service itself as a war artifact that developers can deploy in an existing Web application server. • The boot volume comprises a clean installation of Ubuntu 9.04 and SmartFrog v3.17. • added additional components on which SmartFrog relies to the image, such as apt-get for installing the Linu x packages Hadoop requires.

  13. Performance Statistics • To evaluate our framework’s performance, we conducted an experiment that aimed to measure: • the relationship between the time for creating the virtual infrastru-cture and for booting the OS (infrastructure creation time). • the time for provisioning and starting the infrastr-ucture with the MapReduce implementations (provisioning time). • the time for executing the MapReduce job including data uploading and downloading (MapReduce execution time).

  14. Performance Statistics…cont

  15. Conclusion • The MapReduce program m ing model has shown immense potential for processing large and unstructured datasets.

  16. Any Question???

More Related