1 / 37

Parallel ODETLAP for Terrain Compression and Reconstruction

Parallel ODETLAP for Terrain Compression and Reconstruction. Jared Stookey Zhongyi Xie W. Randolph Franklin. Dan Tracy Barbara Cutler Marcus V. A. Andrade. Outline. Quick Overview of our research ODETLAP (Non-Patch) Motivation for Parallelization Our Approach MPI Implementation

arthurdixon
Download Presentation

Parallel ODETLAP for Terrain Compression and Reconstruction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Parallel ODETLAP for Terrain Compression and Reconstruction Jared Stookey ZhongyiXie W. Randolph Franklin Dan Tracy Barbara Cutler Marcus V. A. Andrade RPI GeoStar Group

  2. Outline • Quick Overview of our research • ODETLAP (Non-Patch) • Motivation for Parallelization • Our Approach • MPI Implementation • Results • Current and Future Work RPI GeoStar Group

  3. Quick Overview • Our research • Terrain compression • Compress terrain by selecting subset of points • Reconstruct the terrain by solving a system of equations to fill in missing points • The method we use to reconstruct the terrain is slow for large datasets • We came up with a method for reconstructing very large datasets quickly using MPI RPI GeoStar Group

  4. ODETLAP • Over-Determined Laplacian • Two Equations: • 4zij = zi-1,j + zi+1,j + zi,j-1 + zi,j+1 • zij = hij • Multiple values for some points • Require a smooth parameter R to interpolate when multiple values exist • Reconstruct an approximated surface from {hij} (Red points) RPI GeoStar Group

  5. Example • ODETLAP can be used to fill in the unknown data from a from a sparse point cloud: 13,000,000 points 10,000*10,000 RPI GeoStar Group

  6. ODETLAP Compression • Lossily compress image by selecting subset of points • ODETLAP reconstruction solves for the whole terrain 2) Store 1) Compress 3) Reconstruct (ODETLAP) RPI GeoStar Group

  7. ODETLAP Performance Issue RPI GeoStar Group

  8. Motivation for Parallelization • ODETLAP prohibitively slow for large datasets • We need a scalable implementation • Only a small neighborhood of points will affect a particular elevation. • 1 pixel only affectedan area of 62x62 RPI GeoStar Group

  9. Our Approach • Divide the terrain into individual patches • Run ODETLAP on each patch separately 3) Reconstruct each patch 1) Compressed terrain 2) Divide it into patches 4) Merge the patches RPI GeoStar Group

  10. There is a problem! • Points near the edges of patches have incomplete data which causes errors Pixels in red show erroneous results Pixels in blue show correct results RPI GeoStar Group

  11. There is a problem! (continued) • We get discontinuity if we naively merge the patches Errors: Naively reconstructed terrain: RPI GeoStar Group

  12. Solution • Use overlapping layers of patches RPI GeoStar Group

  13. Solution • Use overlapping layers of patches RPI GeoStar Group

  14. Solution • Use overlapping layers of patches RPI GeoStar Group

  15. Solution • Use overlapping layers of patches RPI GeoStar Group

  16. Solution • Use overlapping layers of patches RPI GeoStar Group

  17. Solution • Use overlapping layers of patches RPI GeoStar Group

  18. Solution • Use overlapping layers of patches RPI GeoStar Group

  19. Solution • Use overlapping layers of patches RPI GeoStar Group

  20. Solution • Use overlapping layers of patches RPI GeoStar Group

  21. Solution • Use overlapping layers of patches • Then merge the results RPI GeoStar Group

  22. Solution • Use overlapping layers of patches • Then merge the results RPI GeoStar Group

  23. Problem: Averaging the patches • A simple averaging of the patches incorporates the border error into the reconstructed terrain: Terrain reconstructed using averaged patches Errors: RPI GeoStar Group

  24. Solution: Bilinear Interpolation • Use bilinear interpolation to do a weighted average such that border values fall off to zero: Naively averaging results Bilinear interpolation results Error (avg: 0.1m, max: 2m): Elevation Range of the Original: 1105m..1610mUsing DTED Level 2 (30m spacing) RPI GeoStar Group

  25. Weighting Pattern for Bilinear Interpolation vs. Simple Averaging Bilinear Interpolation Simple Averaging RPI GeoStar Group

  26. MPI Implementation 1) Each processor (except central process) is pre-assigned one or more patches 2) Every MPI process does the following for each patch assigned to it: • Load patch • Run ODETLAP on the patch • MPI_send the patch to the central process 3) When all of the patches have been received by the central process, merge them using bilinear interpolation. RPI GeoStar Group

  27. Results • 16,000*16,000 Central USA terrain data • Use 128 2.6 GHz processors on RPI CCNI cluster • Divide into 101,761 patches of 100x100 size • Completed in 28 minutes and 32 seconds • Non-patch ODETLAP would have taken 179 days RPI GeoStar Group

  28. Results(cont.) • Size: 16K*16K • STD: 217 • Range: 1013 • Mean Error: 1.96 • Max Error: 50 • RMS Error: 2.76 The terrain was compressed by a factor of 100, with a mean error within 0.2% of the range. RPI GeoStar Group

  29. Original and reconstructed Terrain Original Terrain (1000 * 1000) Reconstruction Result (1000 * 1000) RPI GeoStar Group

  30. Patch Size vs. Time & Error These results come from an 8-processor machine RPI GeoStar Group

  31. Serialized vs. Parallel • Serialized: A single worker processor runs each patch sequentially (speedup of 9.5 in the test) • Parallel: Several processors run on many patches in parallel (additional speedup of 5.6 in the test) Test data: 800 x 800 size with mean elevation of 107 RPI GeoStar Group

  32. Running Time Comparison Test data: 800 x 800 size with mean elevation of 107, run on 8 processors. Parallel ODETLAP is 50 times faster, while introducing only 0.1% additional error. RPI GeoStar Group

  33. RPI GeoStar Group

  34. RPI GeoStar Group

  35. Current and Future Work • Improvements to our implementation • Reduce data size – regular grid can be more compact • Each process should grab the next available patch • Optimize for the Blue Gene/L system (see next slide) • Reduce errors from the patch method • Improve the method for merging patches RPI GeoStar Group

  36. Blue Gene/L System • Computational Center for Nanotechnology Innovations (CCNI) at RPI • 32,768 CPU’s @ 700 Mhz • 512-1024MB memory/CPU (non-shared) • Opportunity to run very large data sets quickly • New method • Source, Sink, Workers, and Coordinator • DEM size is not limited by process memory size • Use processors as cache instead of the disk • On the BG, disk is slow, network and memory is very fast • We must reduce the overhead to take advantage of all CPU’s RPI GeoStar Group

  37. Summary Original Reconstructed The terrain is divided into overlapping patches and ODETLAP is solved separately on individual patches, which will be merged into a whole patch using bilinear interpolation at the end We experiment on a 16K*16K test data. We show two 1K*1K pieces from the original and reconstructed terrain to illustrate the accuracy of the reconstruction RPI GeoStar Group

More Related