Efficient Data Processing Algorithms with Sample Sorting for Parallel Systems

4. Adding numbers n data items, p processors ts = O(n) tp = O(n/p) if data on each proc => S=ts/tp=O(p) tp = O(n + n/p) if data needs broadcasting => S=ts/tp=o(1)

5. Sequential Recursion

7. Parallel Recursion tcomm = O(n/2 +n/4 + ..+ n/p) = O(n) S=o(1) tcomp = O(n/2 +n/4 + ..+ n/p) = O(n)

8. tcomm = O(1 +1 + ..+ 1) = O(log p) S=O(n / log p) tcomp = O(1 +1 + ..+ 1) = O(log p)

9. Parallel Bucket Sort

10. Sequential m buckets , n numbers ts = O(n + m((n/m) log (n/m))) = O(n log(n/m))

11. m buckets , n numbers, p=m processors tp = O(n + (n/p) log (n/p))

13. tp = O(n/p + (n/p) log (n/p)) = O( (n/p) log (n/p)) => S=O(p)

15. Det. Sample Sort

16. Det. Sample Sort • sort locally and create p-sample

17. Det. Sample Sort • send all p-samples to processor 1

18. Det. Sample Sort • proc.1: sort all received samples and compute global p-sample

19. Det. Sample Sort • broadcast global p-sample • bucket locally according to global p-sample • send bucket i to proc.i • resort locally

20. Det. Sample Sort Lemma: Each proc. receives at most 2 n/p data items n/p2 n/p2 global sample global sample

21. Det. Sample Sort Post-Processing: “Array Balancing” n/p n/p n/p n/p n/p n/p n/p n/p 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 2 Rounds: • Each proc. sends rec. data size to all other proc. • Move data to right location via one h-relation

22. Det. Sample Sort • 5 MPI_AlltoAllv for n/p > p2 • O(n/p log n) local comp. • Goodrich (FOCS'98): O(1) rounds for n/p > pe

23. Performance: Det. Sample Sort

24. Numerical Integration

25. static assignment of processors to segments of [a,b] area = d (f(p)+f(q))/2

26. Problem: precision depends on curve’s shape

27. Adaptive Quadrature Terminate when C is sufficiently small Problem: different parts of the curve need different resolution

28. segment 1 segment 3 segment 4 segment 2 segment 5

29. Gravitational N-Body Problem

32. ts = O(n2) per time step

39. for each time step: for each object: traverse tree to determine its forces Problem:traversals have different lengths

40. object 1 object 3 object 5 object 2 object 4