1 / 17

Parallel Sorting

Parallel Sorting. Sort. Sorting operation is frequently used for database processing. For example sorting may be requested by users through the use of Distinct, Order By and Group By clauses in SQL.

zelig
Download Presentation

Parallel Sorting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Sorting Parallel Sorting

  2. Sort • Sorting operation is frequently used for database processing. • For example sorting may be requested by users through the use of Distinct, Order By and Group By clauses in SQL. • In general, parallel database sorting consists of multiple rounds of local sort, performed in each PN in parallel, followed by movement of tuples among PNs. • A dynamic load balancing approach has been proposed, recently. Parallel Sorting

  3. Parallel Sorting • Traditional Technologies • Parallel Merge-All Sort • Parallel Binary-Merge Sort • Parallel Partitioned Sort • Dynamic Load Balancing Approaches • Partitioned with Redistribution Sort (PRS) • Load Balancing Optimization Sort (LBOS) • Load Balancing Optimization Sort with Fragment Feature (LBOS-F) Parallel Sorting

  4. Parallel Merge-All Sort Parallel Sorting

  5. Parallel Binary-Merge Sort Parallel Sorting

  6. Parallel Partitioned Sort Parallel Sorting

  7. Partitioned with Redistribution Sort(PRS) • Partition Phase: each PN first scans its local tuples and distributes them as buckets according to range partitioning. • Optimization Phase: a coordinator PN is used to monitor the load for every PN. If every PN has the similar amount of data to process, Go to Sorting Phase. Otherwise, the imbalanced work load is detected, thus, a redistributed partitioning is needed. • Redistribution Phase: data redistribution. • Sorting Phase: local sort is carried out in each PN. Parallel Sorting

  8. PRS Parallel Sorting

  9. Load Balancing Optimization Sort (LBOS) • Phase 1. Bucket Sizes Counting : Each PN reads its local data and computes each bucket size according to range distribution. Thereafter, each PN transfers the bucket information to a coordinator. • Phase 2. Load Balancing Optimization: coordinator decides the distribution strategy for assigning each bucket to PNs according to their bucket IDs sequentially. Whether coordinator assigns one more next bucket (size B) to current PNi will be decided on: if (Pi+B)-Pavg< d, it will assign next bucket to PNi. Otherwise, the bucket will be assigned to next PNi+1. This phase only decides the distribution strategy without any data transferred. Parallel Sorting

  10. Phase 3. Data Partitioning :Each PN transfers and receives data by using the distribution strategy decided in Phase 2. • Phase 4. Sorting :Each PN performs local sort. Parallel Sorting

  11. LBOS Parallel Sorting

  12. 25.23 (秒) 15.93 (秒) 9.87 (秒) Bucket skew=1 PPS：PRS → 37.1% PPS：LOBS → 60.9% Effect of Bucket Skew Parallel Sorting

  13. 24.72 (秒) 19.32 (秒) 14.9 (秒) 4.41 (秒) 3.45 (秒) 1.77 (秒) Effect of No. of Processing Nodes PNs=16 PNs=256 PRS： PPS → 21.8% PRS ： PPS → 21.8% LOBS ： PPS → 59.9% LOBS ：PPS → 39.7% Parallel Sorting

  14. Load Balancing Optimization Sort with Fragment Feature (LBOS-F) • When the skew condition is severe, LBOS performs not very well. • LBSO-F is used to address this drawback and it also consists of four phases. • Its Phase 1, 3, and 4 are exactly the same as in LBOS except the Phase 2 in which LBOS-F can fragment oversize buckets and distribute them to different PNs. Parallel Sorting

  15. Phase 2 for LBOS-F • Phase 2. Load Balancing Optimization with Fragment Feature : coordinator decides the distribution strategy for assigning each bucket to PNs according to their bucket IDs sequentially. Suppose that the next bucket (size B andPi+B>Pavg), with the interval of data values from rs to re ( rs < re ), is being considered for the assignment. The coordinator will fragment this bucket into two portions. Suppose that PNi already has total size Piof tuples which is d tuples less than Pavg (Pi+d=Pavg). The first portion which consists of those data values within the range of rs to rs+(re-rs)d/B of the bucket will be assigned to current PNi. The second portion which consists of the remaining data of the bucket will be assigned to next PNi+1. Again, this phase is without any data transferred. Parallel Sorting

  16. Effect of Bucket Skew Parallel Sorting

  17. Effect of Number of PNs Parallel Sorting

More Related