Gputerasort high performance graphics coprocessor sorting for large database management
Download
1 / 29

GPUTeraSort: High Performance Graphics Coprocessor Sorting for Large Database Management - PowerPoint PPT Presentation


  • 142 Views
  • Uploaded on

GPUTeraSort: High Performance Graphics Coprocessor Sorting for Large Database Management. Naga K. Govindaraju Jim Gray Ritesh Kumar Dinesh Manocha Presented by: Rita McCue. We will discuss:. Recent sorting history Why we use a gpu

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'GPUTeraSort: High Performance Graphics Coprocessor Sorting for Large Database Management' - lathrop


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Gputerasort high performance graphics coprocessor sorting for large database management l.jpg

GPUTeraSort: High Performance Graphics CoprocessorSorting for Large Database Management

Naga K. Govindaraju Jim Gray

Ritesh Kumar Dinesh Manocha

Presented by: Rita McCue


We will discuss l.jpg
We will discuss:

  • Recent sorting history

  • Why we use a gpu

  • Previous sorting algorithms using an external co-processor (gpu)

  • Limitations of those sorting algorithms

  • The GPU-TeraSort algorithm


Why use a gpu l.jpg
Why Use A GPU?

  • GPUs are commodity processors. A high performance mid-range GPU costs less than $300

  • Current GPUs have 10x higher main memory bandwidth and use data parallelism to achieve 10x more operations per second than CPUs.

  • GPU performance has improved faster than Moore’s Law over the last decade - so the GPU-CPU performance gap is widening

  • GPUs are increasingly used as a co-processor for non-graphics applications, and some work has already been done on using them to speed up database queries and data streaming


Slide4 l.jpg
But…

  • previous GPU-based sorting algorithms were not able to handle gigabyte-sized databases with wide keys and could not keep up with modern disk IO systems


External memory sorting algorithms l.jpg
External Memory Sorting Algorithms

Typically come in two categories:

  • Distribution-Based Sorting: elements in one bucket precede the elements in the remaining buckets..

  • Merge-Based Sorting: input data partitioned into data chunks of approximately equal size

Phase 1: produces a set of files

Phase 2: processes that set into 1 file


External memory sorting limitations and speed l.jpg
External Memory Sorting Limitations and Speed

  • Large files don’t fit in RAM so data is sorted in >= 2 passes. Each pass reads & writes to the disk, so the 2-pass sort throughput is at most 1/4 the throughput of the disks. Thus a p-pass algorithm has a throughput of:

    (disk bandwidth)/2p


External memory sorting 2 pass sorting algorithms cont l.jpg
External Memory Sorting 2 Pass Sorting Algorithms (cont)

  • a two-pass sort’s RAM requirements (M) increase as the square root of the input file size, so multi-GB RAM machines can two-pass sort terabyte files

  • But current algorithms cannot achieve high sorting performance on such large partitions because of cache misses, I/O performance, and the bandwidth limitations of current memory interfaces


Sorting with a graphics processor l.jpg
Sorting with a Graphics Processor

  • High Data Parallelism: fragment (concurrent vector operations) processors for partition parallelism

  • Instruction-level Parallelism: each fragment processor can also exploit instruction-level parallelism.

  • Dedicated Memory Interface: high bandwidth data streaming between main memory and the GPU’s onboard memory.

  • Low Memory Latency: data accessed sequentially thereby allowing prefetch and pipelining.


Previous gpu sorting algorithms limitations l.jpg
Previous GPU Sorting Algorithms Limitations

  • Database size: Previous (single pass) algorithms were limited to databases that fit in GPU memory (i.e. 512MB on current GPUs).

  • Limit on key size: The sort keys were limited to 32-bit floating point operands.

  • Efficiency: Previous algorithms were not fast enough to match the disk array IO bandwidth.


Gputerasort l.jpg
GPUTeraSort:

  • A hybrid sorting algorithm for billion-record wide-key databases using a graphics processing unit (GPU) as a co-processor

  • GPU is used to perform memory-intensive and compute-intensive tasks to achieve higher I/O performance and better main memory performance.

  • CPU is used to perform I/O and resource management.


Slide11 l.jpg

This figure highlights the high data parallelism and memory bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.


Gputerasort hybrid sorting architecture l.jpg
GPUTeraSort/ bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.Hybrid Sorting Architecture

GPUTeraSort has five stages that can be executed sequentially, but some stages can be executed using multi-buffered pipeline-parallel independent threads:

  • 1) Reader

  • 2) Key-Generator

  • 3) Sorter

  • 4) Reorder

  • 5) Writer


Large sorts using gpus l.jpg
LARGE SORTS USING GPUS bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

Bitonic Sorting on GPUs

This figure illustrates a bitonic sorting network on 8 data values. The sorting algorithm proceeds in 3 stages. The output of each stage is the input to the next stage. In each stage, the array is conceptually divided into sorted data chunks or regions highlighted in green and red. Elements of adjacent chunks are merged as indicated by arrows. The minimum element is moved to the green region and the maximum is stored in the red colored regions producing larger sorted chunk.


Gputerasort s improved algorithm l.jpg
GPUTeraSort’s Improved Algorithm bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

  • GPUTeraSort represents the bytes of the keys as 32-bit floating point 2D arrays (or textures). Each element in the texture is associated with four color channels - red, blue, green and alpha. Given a texture of width W and height H, there are two possible representations for designing a data-parallel bitonic sorting algorithm

  • Single-array representation

  • Four-array representation

  • In each of these representations, the keys and the pointers associated with the keys are stored in two separate textures: a key texture and a pointer texture


Single array representation advantages l.jpg
Single Array Representation Advantages bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

  • Mapping: The data transfer directly maps

  • Efficient sorting: better performance as it reduces the memory accesses in early algorithm steps.


The gputerasort algorithm l.jpg
The GPUTeraSort Algorithm bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

  • GPUTeraSort(n)

  • 1) b = number of bytes in key

  • 2) W = width(tex) = 2 (log n)/2, H = height(tex) = 2 (log n)/2

  • 3) sorted = false

  • 4) currBytes = Most significant four bytes of keys

  • 5) SortLists = Input Array of Key-Pointers

  • 6) While (!sorted)

  • 7) For each array in sortedLists

  • 8) Transfer the pointerTexture and keyTexture of currBytes to the GPU

  • 9) Perform HybridBitonicSort(pointerTexture,keyTexture,n)

  • 10) Readback the pointers

  • 11) SortedLists = Lists of contiguous arrays with equal keys in currBytes

  • 12) If (sizeof(SortedLists)=0 or currBytes = b-4) sorted =true

  • 13) else currBytes = nextfourBytes

  • 14) end for


More simply l.jpg
More Simply: bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

  • GPU orders the key-pointer array based on the first few bytes

  • GPUTeraSort scans the array to identify contiguous portions of the array with equal keys

  • It sorts these portions based on the second set of bytes on the GPU.

  • It repeats the process till the entire array is sorted or all the key bytes have been compared.


Cache efficient sorting on gpus l.jpg
Cache-Efficient Sorting on GPUs bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

  • Width of the input array = W, height = H, block size = B × B. In each step, each pixel is compared against one other pixel. Therefore, there are at least ncompulsory = W×H/B2 cache misses. The goal was thus to reduce the total number of cache misses to a value close to ncompulsory in each step.

  • Case 1: h > B. In this case, cache misses in rendering the quad are unavoidable.

  • Case 2: h <= B. In this case, many cache misses occur if nblocks do not fit in the cache because the cache blocks fetched at the beginning of the quad are mostly evicted by the end of the quad. We use a technique similar to blocking to minimize cache misses. We decompose each quad into multiple quads of width B and height h. We then perform computation on all the quads lying within the B ×B block


Gputerasort s performance l.jpg
GPUTeraSort’s Performance bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

Figure 6: Comparison of GPUTeraSort’s bitonic sort with other GPU-based algorithms indicates a 3-7 fold performance advantage over prior GPU-based bitonic sort and PBSN algorithms (Kipfer et al, Govindaraju et al, and Purcell et al.).


Analysis l.jpg
Analysis bandwidth inside a GPU. GPUTeraSort uses the vector processing functionalities to implement a highly parallel bitonic sorting network. It outperforms prior CPU-based and GPU-based algorithms by 3-10 times.

  • Computational Performance- Theoretically, GPUTeraSort has a computational complexity of O( (n log^2 n)/2 ). In practice, on a NVIDIA 7900 GTX GPU GPUTeraSort achieves 14 GOPs/sec processor performance and a memory bandwidth of 49 GB/s (peak bandwidth on this card is 56GB/s)

  • Bandwidth Requirements - relatively low. Given n data values, it transfers the data to and from the GPU once. The overall main memory bandwidth requirement of O(n) is significantly lower than the total computational cost i.e., O(nlog2n). In practice, we observed that the data transfer time is less than 10% of the total sorting time, indicating that GPUTeraSort is not bandwidth limited

  • Performance and Memory Growth – as it uses therasterization capabilities of the GPU to sort, this is directly related to the performance growth rate of the GPUs.


Slide21 l.jpg

The time taken by GPUTeraSort on an NVIDIA 7800 GTX GPU to sort key-pointers with 32-bit floating point keys including the data transfer overhead. Observe that the algorithm has very low data transfer overhead. In practice, the data transfer cost is less than 10% of the total computational time.


Performance l.jpg
PERFORMANCE sort key-pointers with 32-bit floating point keys including the data transfer overhead. Observe that the algorithm has very low data transfer overhead. In practice, the data transfer cost is less than 10% of the total computational time.

  • SortBenchmark - We measured GPUTeraSort’s performance on a 3 GHz Pentium IV processor, 2 GB PC5400 main memory, a SuperMicro Marvel MV8 SATA controller connecting 8 Western Digital Caviar SATA disks running Windows XP SP2, and a wide range of commodity GPUs costting less than $300 and used OpenGL to program the GPUs.


Analysis23 l.jpg
ANALYSIS sort key-pointers with 32-bit floating point keys including the data transfer overhead. Observe that the algorithm has very low data transfer overhead. In practice, the data transfer cost is less than 10% of the total computational time.

  • GPUTeraSort’s performance depends on:

  • Hardware performance

  • Load Balancing: I/O, memory, and sorting operations are performed efficiently and concurrently.

  • Key sizes: worst-case, it may need to perform multiple sorting operations on the entire array based on the distribution of the keys

  • Database size: data transfer time is significantly lower than the computation time with the exception of very small database

  • Partition sizes: performance varies as a function of the Phase 1 run sizes


The experiment l.jpg
The Experiment: sort key-pointers with 32-bit floating point keys including the data transfer overhead. Observe that the algorithm has very low data transfer overhead. In practice, the data transfer cost is less than 10% of the total computational time.

  • We implemented GPUTeraSort on an inexpensive 3 GHz Pentium IV EE CPU with a $265 NVIDIA 7800 GT GPU.

  • GPUTeraSort running the SortBenchmark on this inexpensive computer has performance comparable to an “expensive” $2,200 3.6 GHz Dual Xeon server. Our experimental results show a 4 times performance improvement over the 2005 Daytona PennySort benchmark record and 1.4 times improvement over the 2003 Indy PennySort benchmark record.


Results condensed l.jpg
Results? (Condensed) sort key-pointers with 32-bit floating point keys including the data transfer overhead. Observe that the algorithm has very low data transfer overhead. In practice, the data transfer cost is less than 10% of the total computational time.

  • nearly 50 giga-byte per second memory band-width and 14 giga-operations per second on a current GPU.

  • 10x what we can achieve on the CPU.


Slide26 l.jpg

GPUTeraSort and nsort (solid lines) Phase 1 performance vs file size. GPUTeraSort runs are 100MB while nsort runs are less than 25MB in order to fit the key-pointers in the CPU L2 cache. GPUTeraSort on a 7800 GT GPU has performance comparable to an expensive 3.6 GHz Dual Xeon server.


Slide27 l.jpg

Total sort time for GPUTeraSort and nsort (Pentium 4) shows the performance of GPUTeraSort on different mid-range GPUs and nsort on high-end dual Xeon processors. Observe that GPU-assisted sorts are competitive with CPU-only sorts for larger databases – yet the hardware is much less expensive.


Limitations l.jpg
Limitations the performance of GPUTeraSort on different mid-range GPUs and nsort on high-end dual Xeon processors. Observe that GPU-assisted sorts are competitive with CPU-only sorts for larger databases – yet the hardware is much less expensive.

  • Variable-sized keys: GPUTeraSort’s is based on radix sort and works well for keys with fixed number of bytes. It works correctly on variable-sized keys.

  • Almost sorted databases: GPUTeraSort does not benefit if the input data file is almost sorted

  • Programmable GPUs: GPUTeraSort requires programmable GPUs with fragment processors

  • Memory sizes: Due to the pipelined design of Phase 1 of GPUTeraSort’s run size of Phase 1 is limited to 1/5 of the total main memory size


Questions l.jpg
Questions? the performance of GPUTeraSort on different mid-range GPUs and nsort on high-end dual Xeon processors. Observe that GPU-assisted sorts are competitive with CPU-only sorts for larger databases – yet the hardware is much less expensive.


ad