1 / 41

Progressive Transmission and Rendering of Foveated Volume Data

Progressive Transmission and Rendering of Foveated Volume Data. Chen Chen, Zhiyong Huang, Hang Yu, Ee-Chien Chang School of Computing National University of Singapore. Introduction - motivation. Problem Large volume data provided by modern scanners like CT and MRI

roxannea
Download Presentation

Progressive Transmission and Rendering of Foveated Volume Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Progressive Transmission and Rendering of FoveatedVolume Data Chen Chen, Zhiyong Huang, Hang Yu, Ee-Chien Chang School of Computing National University of Singapore

  2. Introduction- motivation • Problem • Large volume data provided by modern scanners like CT and MRI • Slow network speed comparing to the large size of volume data • Solution • Wavelet foveation • Progressive transmission and rendering • coarse-to-fine • region-based grapp 2006

  3. Introduction- wavelet foveated volume • Foveated volume is a non-uniform sampled volume whose resolution is highest at the fovea (user specified ROI) and falls off as the distance to the fovea increases– priority selection • Wavelet foveated volume is constructed by select ROI at each level of details in the volume’s wavelet form - compression grapp 2006

  4. Proposed method- overview • Transform to Wavelet Form • Divide into 83 block, run-length encode (RLE) each block • Client request = ROI {(x,y,z),s} • Generate Fw • Reorder data blocks of Fw • Progressive transmission • Progressive Rendering grapp 2006

  5. Proposed method- overview • Client-server volume rendering system • Server Side: • Input: Cw & ROI{(x,y,z),s} • Output: Fw • Client Side: • Input: Fw • Output: Iw1 -> Iw2 -> … -> Iwfinal • Two transmission and rendering schemes grapp 2006

  6. Proposed method- overview Server Site Client Site Wavelet Transform Cw request Generate Fw from Cw and ROI User Input ROI={(x,y,z),r} Progressive Render RB or CTF Reorder data blocks in Fw according to the schemes Progressive transmission Progressive rendering Display grapp 2006

  7. Wavelet transform • Divide the volume into blocks of size 2k3 • Apply generic Haar wavelet transform on each block to obtain 7 detail blocks and 1 average block • Group 8 adjacent blocks to again obtain a block of size 2k3 • Repeat this procedure until the average volume reaches size m3 grapp 2006

  8. Wavelet transform- run-length encoding (RLE) • A more compact representation • RLE is performed block by block • Compressed block will be a list of value(v) and length(l) pairs • v= Integer value of a coefficient l = Number of successive v • Example: • L=1,1,1,1,0,0,5,0,0,0,0,0,0,0,0 • R=(1,4),(0,2),(5,1),(0,8) grapp 2006

  9. Derive foveated volume • Cw, ROI{(x,y,z),s} => Fw • Layer 0 image = average image of Cw • Layer i image = IWT(layer i-1 image, 7 layer i-1 detail images) • ROI in ith layer = ROI in Layer i image grapp 2006

  10. Derive foveated volume • Center of ROI in ith layer is • Size of ROI in ith layer is s3 • Boundary of ROI in ith layer can be defined as grapp 2006

  11. Derive foveated volume • 4th ROI = {(9,9),6} = {[6,11],[6,11]} • Layer 3 details (i,j) =([3,5],[3,5]) grapp 2006

  12. Progressive transmission- motivation • Data size = 10243, Fovea size = 483, Average image size = 163 • Maxlayer=Log2(1024/16)=6 • Total amount of data = 243*(5*7+1)=497,664 coefficients • Reorder data blocks • Progressive transfer grapp 2006

  13. Progressive transmission- two schemes • Coarse-to-fine • Description: Transmit a rough average image first and refine the fovea from outer ROIi to inner ROIi+1 • Implementation: Send the wavelet foveated data from inner layer to outer layer progressively • Region-based • Description: Transmit full resolution ROImaxlayer first and expand the image from ROImaxlayer-1 to ROI0 with decreasing resolution • Implementation: Data blocks and parent data blocks of ROImaxlayer-i+1 will be sent in ith iteration grapp 2006

  14. Progressive transmission- illustration in 2D • 2D image size= 128*128, m=16, k=8, ROI={(86,86),45}, maxlayer=3 • ROI0={(10,10),45}=[0,15] blk_index=[0,1] • ROI1={(21,21),45}=[0,31] blk_index=[0,1] • ROI2={(43,43),45}=[20,63] blk_index=[1,3] • ROI3={(86,86),45}=[64,107] blk_index=[4,6] grapp 2006

  15. Progressive transmission- illustration in 2D grapp 2006

  16. Progressive transmission- coarse-to-fine grapp 2006

  17. Progressive transmission- coarse-to-fine grapp 2006

  18. Progressive transmission- region-based grapp 2006

  19. Progressive rendering- rendering equation • For each voxel: • intensity value • opacity value • The final intensity result reaches the viewer of each pixel will be Eulerian Sum over accumulated opacity • Rendering equation to render a subvolume with same intensity and opacity value grapp 2006

  20. Progressive rendering- rendering equation • For 2 subvolumes Sa and Sb with size na and nb, voxels in Sa and Sb have , , and , respectively. Sa is in front of Sb, that is, Sa is “over” Sb. The intensity of Sa and Sb and Sfinal can be defined as grapp 2006

  21. Region-based- partial volume reconstruction • Starting coefficient: • Ending coefficient: • Wavelet coefficients at ith layer detail image • begin: • end: • Reconstruction starts from layer 0, that is, the average image and move out to higher level of details until the desired resolution is reached grapp 2006

  22. Region-based • Center fovea is reconstructed using partial volume reconstruction and rendered at iteration 1 • 6 subvolumes are rendered: Top, Bottom, Left, Right, Front, Back with resolution scale 2(1-i), i is the number of iteration • 6 sub volumes are rendered separately and combine the rendered result to the previous rendering image grapp 2006

  23. Region-based- image composition • Top, bottom, left and right are added to the previous rendering result image at their right position • Front and back image needs image composition with the previous rendering result image: (front) over (inner) over (back) • “over” operator can be defined as: given image pixel A(da, αa) and B(db, αb) • C = A over B • dc = da + αa*db • αc = αa* αb grapp 2006

  24. Region-based- image composition grapp 2006

  25. Coarse-to-fine • Start rendering from ROI0 to ROImaxlayer • Each ROIi except ROImaxlayer can be divided into 7 subvolumes: top, bottom, left, right, front, back and center • Center subvolume is the average image of the inner ROI and its rendering result, accumulated density value and opacity value will be kept for next iteration grapp 2006

  26. Coarse-to-fine • ROIi is constructed by center part of ROIi-1 and level i detail coefficients • Number of voxels each coefficient in ROIi represent is 2maxlayer-i • Rendering equation for each voxel in ROIi is • Rendering result of front and back subvolume RFi and RBi will be take down for next iteration • Rendering result of iteration i+1 will be combined with RFi and RBi in iteration i+1 grapp 2006

  27. Results - direct rendering grapp 2006

  28. Results - region-based grapp 2006

  29. Results - region-based grapp 2006

  30. Results - coarse-to-fine grapp 2006

  31. Results - coarse-to-fine grapp 2006

  32. Results - time grapp 2006

  33. Results - coarse-to-fine grapp 2006

  34. Results - coarse-to-fine grapp 2006

  35. Results - region-based grapp 2006

  36. Results - region-based grapp 2006

  37. Results - coarse-to-fine grapp 2006

  38. Results - coarse-to-fine grapp 2006

  39. Results - region-based grapp 2006

  40. Results - region-based grapp 2006

  41. Conclusion and future work • Two progressive transmission and rendering schemes, region-based and coarse-to-fine • Blockwise RLE used in wavelet compression • Wavelet foveation • Image composition • Large memory is still needed • cut the data set exceeding a certain amount into smaller data sets and build location map to indicate actual position of each data segments • Cache can be used to store frequently used blocks, for example, those high level details grapp 2006

More Related