1 / 44

Iterative MapReduce E nabling HPC-Cloud Interoperability

Iterative MapReduce E nabling HPC-Cloud Interoperability. Workshop on Petascale Data Analytics: Challenges, and Opportunities, SC11. S A L S A HPC Group http://salsahpc.indiana.edu School of Informatics and Computing Indiana University. SALSA HPC Group. Intel’s Application Stack.

shakti
Download Presentation

Iterative MapReduce E nabling HPC-Cloud Interoperability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Iterative MapReduceEnabling HPC-Cloud Interoperability Workshop on Petascale Data Analytics: Challenges, and Opportunities, SC11 SALSAHPC Group http://salsahpc.indiana.edu School of Informatics and Computing Indiana University

  2. SALSA HPC Group

  3. Intel’s Application Stack

  4. (Iterative) MapReduce in Context Support Scientific Simulations (Data Mining and Data Analysis) Kernels, Genomics, Proteomics, Information Retrieval, Polar Science, Scientific Simulation Data Analysis and Management, Dissimilarity Computation, Clustering, Multidimensional Scaling, Generative Topological Mapping Applications Security, Provenance, Portal Services and Workflow Programming Model High Level Language Cross Platform Iterative MapReduce (Collectives, Fault Tolerance, Scheduling) Runtime Distributed File Systems Object Store Data Parallel File System Storage Windows Server HPC Bare-system Amazon Cloud Azure Cloud Grid Appliance Linux HPC Bare-system Infrastructure Virtualization Virtualization CPU Nodes GPU Nodes Hardware

  5. What are the challenges? • Providing both cost effectiveness and powerful parallel programming paradigms that is capable of handling the incredible increases in dataset sizes. • (large-scale data analysis for Data Intensive applications ) • Research issues • portability between HPC and Cloud systems • scaling performance • fault tolerance • These challenges must be met for both computation and storage. If computation and storage are separated, it’s not possible to bring computing to data. • Data locality • its impact on performance; • the factors that affect data locality; • the maximum degree of data locality that can be achieved. • Factors beyond data locality to improve performance • To achieve the best data locality is not always the optimal scheduling decision. For instance, if the node where input data of a task are stored is overloaded, to run the task on it will result in performance degradation. • Task granularity and load balance • In MapReduce, task granularity is fixed. • This mechanism has two drawbacks • limited degree of concurrency • load unbalancing resulting from the variation of task execution time.

  6. Programming Models and Tools MapReduce in Heterogeneous Environment MICROSOFT 8

  7. Motivation Data Deluge MapReduce Classic Parallel Runtimes (MPI) Input map Data Centered, QoS Efficient and Proven techniques iterations Experiencing in many domains Input Input map map Output Pij reduce reduce Expand the Applicability of MapReduce to more classes of Applications Iterative MapReduce More Extensions Map-Only MapReduce

  8. Twister v0.9 New Infrastructure for Iterative MapReduce Programming • Distinction on static and variable data • Configurable long running (cacheable) map/reduce tasks • Pub/sub messaging based communication/data transfers • Broker Network for facilitating communication

  9. runMapReduce(..) Iterations Main program’s process space Worker Nodes configureMaps(..) Local Disk configureReduce(..) Cacheable map/reduce tasks while(condition){ May send <Key,Value> pairs directly Map() Reduce() Combine() operation Communications/data transfers via the pub-sub broker network & direct TCP updateCondition() } //end while close() Main program may contain many MapReduce invocations or iterative MapReduce invocations

  10. Master Node Pub/sub Broker Network B B B B Twister Driver Main Program One broker serves several Twister daemons Twister Daemon Twister Daemon map reduce Cacheable tasks Worker Pool Worker Pool Local Disk Local Disk Scripts perform: Data distribution, data collection, and partition file creation Worker Node Worker Node

  11. Components of Twister

  12. TwisterR4Azure Azure Queues for scheduling, Tables to store meta-data and monitoring data, Blobs for input/output/intermediate data storage.

  13. Iterative MapReduce for Azure • Programming model extensions to support broadcast data • Merge Step • In-Memory Caching of static data • Cache aware hybrid scheduling using Queues, bulletin board (special table) and execution histories • Hybrid intermediate data transfer

  14. Twister4Azure Distributed, highly scalable and highly available cloud services as the building blocks. Utilize eventually-consistent , high-latency cloud services effectively to deliver performance comparable to traditional MapReduce runtimes. Decentralized architecture with global queue based dynamic task scheduling Minimal management and maintenance overhead Supports dynamically scaling up and down of the compute resources. MapReduce fault tolerance

  15. Performance Comparisons BLAST Sequence Search Smith Waterman Sequence Alignment Cap3 Sequence Assembly

  16. Performance – Kmeans Clustering Performance with/without data caching Speedup gained using data cache Task Execution Time Histogram Number of Executing Map Task Histogram Scaling speedup Increasing number of iterations Strong Scaling with 128M Data Points Weak Scaling

  17. Performance – Multi Dimensional Scaling Performance with/without data caching Speedup gained using data cache Data Size Scaling Weak Scaling Task Execution Time Histogram Scaling speedup Increasing number of iterations Azure Instance Type Study Number of Executing Map Task Histogram

  18. Twister-MDS Demo This demo is for real time visualization of the process of multidimensional scaling(MDS) calculation. We use Twister to do parallel calculation inside the cluster, and use PlotViz to show the intermediate results at the user client computer. The process of computation and monitoring is automated by the program.

  19. Twister-MDS Output MDS projection of 100,000 protein sequences showing a few experimentally identified clusters in preliminary work with Seattle Children’s Research Institute

  20. Twister-MDS Work Flow Twister Driver MDS Monitor Client Node II. Send intermediate results Master Node ActiveMQ Broker Twister-MDS I. Send message to start the job PlotViz

  21. Twister-MDS Structure Master Node MDS Output Monitoring Interface Twister Driver Twister-MDS Pub/Sub Broker Network Twister Daemon Twister Daemon map map calculateBC reduce reduce Worker Pool Worker Pool calculateStress Worker Node Worker Node

  22. Iterations User Program Input map Map-Collective Model Initial Collective StepNetwork of Brokers reduce Final Collective StepNetwork of Brokers

  23. New Network of Brokers Twister Daemon Node B. Hierarchical Sending ActiveMQ Broker Node Twister Driver Node 7 Brokers and 32 Computing Nodes in total A. Full Mesh Network 5Brokers and 4 Computing Nodes in total Broker-Driver Connection Broker-Daemon Connection Broker-Broker Connection C. Streaming

  24. Performance Improvement

  25. Broadcasting on 40 Nodes(In Method C, centroids are split to 160 blocks, sent through 40 brokers in 4 rounds)

  26. Twister New Architecture Master Node Worker Node Worker Node Broker Broker Broker Configure Mapper map broadcasting chain Add to MemCache map Map Cacheable tasks merge reduce Reduce reduce collection chain Twister Daemon Twister Driver Twister Daemon

  27. Chain/Ring Broadcasting Twister Daemon Node • Driver sender: • send broadcasting data • get acknowledgement • send next broadcasting data • … • Daemon sender: • receive data from the last daemon (or driver) • cache data to daemon • Send data to next daemon (waits for ACK) • send acknowledgement to the last daemon Twister Driver Node

  28. Chain Broadcasting Protocol Daemon 0 Daemon 1 Daemon 2 Driver send receive I know this is the end of Daemon Chain handle data send receive handle data get ack ack send receive send receive handle data ack handle data ack get ack send receive handle data get ack ack get ack send receive ack handle data ack I know this is the end of Cache Block get ack ack get ack get ack ack

  29. Broadcasting Time Comparison

  30. Applications & Different Interconnection Patterns Input map iterations Input Input map map Output Pij reduce reduce Domain of MapReduce and Iterative Extensions MPI

  31. Scheduling vs. Computation of Dryad in a Heterogeneous Environment

  32. Runtime Issues

  33. Twister Futures • Development of library of Collectives to use at Reduce phase • Broadcast and Gather needed by current applications • Discover other important ones • Implement efficiently on each platform – especially Azure • Better software message routing with broker networks using asynchronous I/O with communication fault tolerance • Support nearby location of data and computing using data parallel file systems • Clearer application fault tolerance model based on implicit synchronizations points at iteration end points • Later: Investigate GPU support • Later: run time for data parallel languages like Sawzall, Pig Latin, LINQ

  34. Convergence is Happening Data intensive application (three basic activities): capture, curation, and analysis (visualization) Data Intensive Paradigms Cloud infrastructure and runtime Parallel threading and processes

  35. FutureGrid: a Grid Testbed NID: Network Impairment Device PrivatePublic FG Network IU Cray operational, IU IBM (iDataPlex) completed stability test May 6 UCSD IBM operational, UF IBM stability test completes ~ May 12 Network, NID and PU HTC system operational UC IBM stability test completes ~ May 27; TACC Dell awaiting delivery of components

  36. SALSAHPC Dynamic Virtual Cluster on FutureGrid --  Demo at SC09 Demonstrate the concept of Science on Clouds on FutureGrid Monitoring & Control Infrastructure Monitoring Interface Monitoring Infrastructure Dynamic Cluster Architecture Pub/Sub Broker Network SW-G Using Hadoop SW-G Using Hadoop SW-G Using DryadLINQ Virtual/Physical Clusters Linux Bare-system Linux on Xen Windows Server 2008 Bare-system XCAT Infrastructure Summarizer iDataplex Bare-metal Nodes (32 nodes) • Switchable clusters on the same hardware (~5 minutes between different OS such as Linux+Xen to Windows+HPCS) • Support for virtual clusters • SW-G : Smith Waterman Gotoh Dissimilarity Computation as an pleasingly parallel problem suitable for MapReduce style applications XCAT Infrastructure Switcher iDataplex Bare-metal Nodes

  37. SALSAHPC Dynamic Virtual Cluster on FutureGrid --  Demo at SC09 Demonstrate the concept of Science on Clouds using a FutureGrid cluster • Top: 3 clusters are switching applications on fixed environment. Takes approximately 30 seconds. • Bottom: Cluster is switching between environments: Linux; Linux +Xen; Windows + HPCS. • Takes approxomately 7 minutes • SALSAHPC Demo at SC09. This demonstrates the concept of Science on Clouds using a FutureGrid iDataPlex.

  38. Education and Broader Impact We devote a lot to guide students who are interested in computing

  39. Education We offer classes with emerging new topics Together with tutorials on the most popular cloud computing tools

  40. Broader Impact Hosting workshops and spreading our technology across the nation Giving students unforgettable research experience

  41. Acknowledgement SALSAHPC Group Indiana University http://salsahpc.indiana.edu

More Related