1 / 77

Data-Intensive Scalable Science: Beyond MapReduce

Data-Intensive Scalable Science: Beyond MapReduce. Bill Howe, UW. …plus a bunch of people. http://escience.washington.edu. . Science is reducing to a database problem. Old model: “Query the world” (Data acquisition coupled to a specific hypothesis)

penda
Download Presentation

Data-Intensive Scalable Science: Beyond MapReduce

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data-Intensive Scalable Science: Beyond MapReduce Bill Howe, UW …plus a bunch of people

  2. http://escience.washington.edu

  3. \ Bill Howe, UW

  4. Bill Howe, UW

  5. Science is reducing to a database problem Old model: “Query the world” (Data acquisition coupled to a specific hypothesis) New model: “Download the world” (Data acquired en masse, in support of many hypotheses) • Astronomy: High-resolution, high-frequency sky surveys (SDSS, LSST, PanSTARRS) • Oceanography: high-resolution models, cheap sensors, satellites • Biology: lab automation, high-throughput sequencing, Bill Howe, UW

  6. Astronomy Oceanography Biology Two dimensions LSST PanSTARRS OOI SDSS # of bytes LANL HIV Galaxy IOOS Pathway Commons BioMart GEO # of apps Bill Howe, UW

  7. Roadmap • Introduction • Context: RDBMS, MapReduce, etc. • New Extensions for Science • Spatial Clustering • Recursive MapReduce • Skew Handling Bill Howe, UW

  8. What Does Scalable Mean? • In the past: Out-of-core • “Works even if data doesn’t fit in main memory” • Now: Parallel • “Can make use of 1000s of independent computers” Bill Howe, UW

  9. Scales to 1000s of computers Taxonomy of Parallel Architectures Easiest to program, but $$$$ Bill Howe, UW

  10. Design Space Grid Internet This talk Data- parallel Dryad DISC Search Shared memory Private data center Transaction HPC Latency Throughput Bill Howe, UW 11 slide src: Michael Isard, MSR

  11. Map (Shuffle) Reduce Some distributed algorithm… Bill Howe, UW

  12. MapReduce Programming Model • Input & Output: each a set of key/value pairs • Programmer specifies two functions: • Processes input key/value pair • Produces set of intermediate pairs • Combines all intermediate values for a particular key • Produces a set of merged output values (usually just one) map (in_key, in_value) -> list(out_key, intermediate_value) reduce (out_key, list(intermediate_value)) -> list(out_value) Inspired by primitives from functional programming languages such as Lisp, Scheme, and Haskell slide source: Google, Inc. Bill Howe, UW

  13. Example: What does this do? map(String input_key, String input_value): // input_key: document name // input_value: document contents for each word w in input_value: EmitIntermediate(w, 1); reduce(String output_key, Iterator intermediate_values): // output_key: word // output_values: ???? int result = 0; for each v in intermediate_values: result += v; Emit(result); slide source: Google, Inc. Bill Howe, UW

  14. Example: Rendering Bronson et al. Vis 2010 (submitted) Bill Howe, UW

  15. Example: Isosurface Extraction Bronson et al. Vis 2010 (submitted) Bill Howe, UW

  16. Large-Scale Data Processing • Many tasks process big data, produce big data • Want to use hundreds or thousands of CPUs • ... but this needs to be easy • Parallel databases exist, but they are expensive, difficult to set up, and do not necessarily scale to hundreds of nodes. • MapReduce is a lightweight framework, providing: • Automatic parallelization and distribution • Fault-tolerance • I/O scheduling • Status and monitoring Bill Howe, UW

  17. What’s wrong with MapReduce? • Literally Map then Reduce and that’s it… • Reducers write to replicated storage • Complex jobs pipeline multiple stages • No fault tolerance between stages • Map assumes its data is always available: simple! • What else? Bill Howe, UW

  18. Realistic Job = Directed Acyclic Graph Outputs Processing vertices Channels (file, pipe, shared memory) slide credit: Michael Isard, MSR Inputs Bill Howe, UW

  19. Relational Database History Pre-Relational: if your data changed, your application broke. Early RDBMS were buggy and slow (and often reviled), but required only 5% of the application code. “Activities of users at terminals and most application programs should remain unaffected when the internal representation of data is changed and even when some aspects of the external representation are changed.” -- Codd 1979 Key Ideas: Programs that manipulate tabular data exhibit an algebraic structure allowing reasoning and manipulation independently of physical data representation Bill Howe, UW

  20. Key Idea: Data Independence SELECT * FROM my_sequences views SELECT seq FROM ncbi_sequences WHERE seq = ‘GATTACGATATTA’; logical data independence relations physical data independence f = fopen(‘table_file’); fseek(10030440); while (True) { fread(&buf, 1, 8192, f); if (buf == GATTACGATATTA) { . . . files and pointers Bill Howe, UW

  21. Key Idea: Indexes • Databases are especially, but exclusively, effective at “Needle in Haystack” problems: • Extracting small results from big datasets • Transparently provide “old style” scalability • Your query will always* finish, regardless of dataset size. • Indexes are easily built and automatically used when appropriate CREATE INDEX seq_idx ON sequence(seq); SELECT seq FROM sequence WHERE seq = ‘GATTACGATATTA’; *almost Bill Howe, UW

  22. Key Idea: An Algebra of Tables select project join join Other operators: aggregate, union, difference, cross product Bill Howe, UW

  23. Key Idea: Algebraic Optimization N = ((z*2)+((z*3)+0))/1 Algebraic Laws: 1. (+) identity: x+0 = x 2. (/) identity: x/1 = x 3. (*) distributes: (n*x+n*y) = n*(x+y) 4. (*) commutes: x*y = y*x Apply rules 1, 3, 4, 2: N = (2+3)*z two operations instead of five, no division operator Same idea works with the Relational Algebra! Bill Howe, UW

  24. Key Idea: Declarative Languages Find all orders from today, along with the items ordered SELECT * FROM Order o, Item i WHERE o.item = i.item AND o.date = today() join o.item = i.item select date = today() scan scan Item i Order o Bill Howe, UW

  25. Shared Nothing Parallel Databases • Teradata • Greenplum • Netezza • Aster Data Systems • Datallegro • Vertica • MonetDB Microsoft Recently commercialized as “Vectorwise” Bill Howe, UW

  26. Example System: Teradata AMP = unit of parallelism Bill Howe, UW

  27. Example System: Teradata Find all orders from today, along with the items ordered SELECT * FROM Orders o, Lines i WHERE o.item = i.item AND o.date = today() join o.item = i.item select date = today() scan scan Item i Order o Bill Howe, UW

  28. AMP 6 AMP 4 AMP 5 hash hash hash h(item) h(item) h(item) select select select date=today() date=today() date=today() scan scan scan Order o Order o Order o Example System: Teradata AMP 3 AMP 1 AMP 2 Bill Howe, UW

  29. Example System: Teradata AMP 6 AMP 4 AMP 5 hash hash hash h(item) h(item) h(item) scan scan scan Item i Item i Item i AMP 3 AMP 1 AMP 2 Bill Howe, UW

  30. Example System: Teradata join join join o.item = i.item o.item = i.item o.item = i.item AMP 6 AMP 4 AMP 5 contains all orders and all lines where hash(item) = 3 contains all orders and all lines where hash(item) = 2 contains all orders and all lines where hash(item) = 1 Bill Howe, UW

  31. MapReduce Contemporaries • Dryad (Microsoft) • Relational Algebra • Pig (Yahoo) • Near Relational Algebra over MapReduce • HIVE (Facebook) • SQL over MapReduce • Cascading • Relational Algebra • Clustera • U of Wisconsin • Hbase • Indexing on HDFS Bill Howe, UW

  32. Example System: Yahoo Pig Pig Latin program Bill Howe, UW

  33. MapReduce vs RDBMS • RDBMS • Declarative query languages • Schemas • Logical Data Independence • Indexing • Algebraic Optimization • Caching/Materialized Views • ACID/Transactions • MapReduce • High Scalability • Fault-tolerance • “One-person deployment” Dryad, Pig, HIVE HIVE, Pig, Dryad Hbase Pig, (Dryad, HIVE) Bill Howe, UW

  34. Comparison Bill Howe, UW

  35. Roadmap • Introduction • Context: RDBMS, MapReduce, etc. • New Extensions for Science • Recursive MapReduce • Skew Handling Bill Howe, UW

  36. PageRank [Bu et al. VLDB 2010 (submitted)] Rank Table R0 Linkage Table L Ri+1 π(url_dest, γurl_destSUM(rank)) Ri.rank = Ri.rank/γurlCOUNT(url_dest) Rank Table R3 Ri.url = L.url_src Ri L Bill Howe, UW

  37. MapReduce Implementation Join & compute rank Aggregate fixpoint evaluation Ri mi1 r01 mi4 r03 mi6 ri5 L-split0 mi2 r02 mi5 r04 mi7 ri6 L-split1 mi3 i=i+1 Converged? Not done ! Client done ! Bill Howe, UW [Bu et al. VLDB 2010 (submitted)]

  38. What’s the problem? Ri m01 r01 L-split0 m02 r02 L-split1 m03 Bill Howe, UW L is loaded and shuffled in each iteration L never changes [Bu et al. VLDB 2010 (submitted)]

  39. HaLoop: Loop-aware Hadoop [Bu et al. VLDB 2010 (submitted)] Hadoop HaLoop Bill Howe, UW Hadoop: Loop control in client program HaLoop: Loop control in master node

  40. Feature: Inter-iteration Locality Bill Howe, UW • Mapper Output Cache • K-means • Neural network analysis • Reducer Input Cache • Recursive join • PageRank • HITs • Social network analysis • Reducer Output Cache • Fixpiont evaluation [Bu et al. VLDB 2010 (submitted)]

  41. HaLoop Architecture Bill Howe, UW [Bu et al. VLDB 2010 (submitted)]

  42. Experiments Bill Howe, UW • Amazon EC2 • 20, 50, 90 default small instances • Datasets • Billions of Triples (120GB) • Freebase (12GB) • Livejournal social network (18GB) • Queries • Transitive Closure • PageRank • k-means [Bu et al. VLDB 2010 (submitted)]

  43. Application Run Time [Bu et al. VLDB 2010 (submitted)] Bill Howe, UW Transitive Closure PageRank

  44. Join Time [Bu et al. VLDB 2010 (submitted)] Bill Howe, UW Transitive Closure PageRank

  45. Run Time Distribution [Bu et al. VLDB 2010 (submitted)] Bill Howe, UW Transitive Closure PageRank

  46. Fixpoint Evaluation [Bu et al. VLDB 2010 (submitted)] Bill Howe, UW PageRank

  47. Roadmap • Introduction • Context: RDBMS, MapReduce, etc. • New Extensions for Science • Recursive MapReduce • Skew Handling Bill Howe, UW

  48. N-body Astrophysics Simulation • 15 years in dev • 109 particles • Months to run • 7.5 million CPU hours • 500 timesteps • Big Bang to now Simulations from Tom Quinn’s Lab, work by Sarah Loebman, YongChul Kwon, Bill Howe, Jeff Gardner, Magda Balazinska Bill Howe, UW

  49. Q1: Find Hot Gas SELECT id FROM gas WHERE temp > 150000 Bill Howe, UW

More Related