1 / 26

Sparse Matrix Dense Vector Multiplication

Sparse Matrix Dense Vector Multiplication. by Pedro A. Escallon Parallel Processing Class Florida Institute of Technology April 2002. The Problem. Improve the speed of sparse matrix - dense vector multiplication using MPI in a beowolf parallel computer. What To Improve.

latona
Download Presentation

Sparse Matrix Dense Vector Multiplication

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sparse Matrix Dense Vector Multiplication by Pedro A. Escallon Parallel Processing Class Florida Institute of Technology April 2002

  2. The Problem • Improve the speed of sparse matrix - dense vector multiplication using MPI in a beowolf parallel computer.

  3. What To Improve • Current algorithms use excessive indirect addressing • Current optimizations depend on the structure of the matrix (distribution of the nonzero elements)

  4. Sparse Matrix Representations • Coordinate format • Compressed Sparse Row (CSR) • Compressed Sparse Column (CSC) • Modified Sparse Row (MSR)

  5. Compressed Sparse Row (CSR) rS ndx val

  6. CSR Code void sparseMul(int m, double *val, int *ndx, int *rS, double *x, double *y) { int i,j; for(i=0;i<m;i++) { for(j=rowStart[i];j<rS[i+1];j++) { y[i]+=(*val++)*x[*ndx++]; } } }

  7. Goals • Eliminate indirect addressing • Remove the dependency on the distribution of the nonzero elements • Further compress the matrix storage • Most of all, to speed up the operation

  8. Proposed Solution A =

  9. Data Structure typedef struct { int rCol; double val; } dSparS_t; {rCol,val}

  10. hdr.size Process A local_size residual < p local_size – hdr.size / p residual = hdr.size % p

  11. Scatter A local_A local_size

  12. Multiplication Code if( (index=local_A[0].rCol) > 0 ) local_Y[0].val = local_A[0].val * X[index]; else local_Y[0].val = local_A[0].val * X[0]; local_Y[0].rCol = -1; k=1; h=0; while(k<local_size) { while((0<(index=local_A[k].rCol)) && (k<local_size)) local_Y[h].val += local_A[k++].val * X[index]; if(k<local_size) { local_Y[h++].rCol = -index-1; local_Y[h].val = local_A[k++].val * X[0]; } } local_Y[h].rCol = local_Y[-1+h++].rCol+1; while(h < stride) local_Y[h++].rCol = -1;

  13. Multiplication local_Y stride doamin Range local_A * = local_size X

  14. Algorithm local_A X Y.val Y.rCol

  15. Gather stride split element range local_Y gatherBuffer residual

  16. Consolidation of Split Rows nCols gatherBuffer += Y residual

  17. Results (vavasis3) vavasis3.rua - Total non-zero values: 1,683,902 - p = 10

  18. Results (vavasis3) vavasis3.rua - Total non-zero values: 1,683,902 - p = 8 vavasis3.rua - Total non-zero values: 1,683,902 - p = 1

  19. Results (vavasis3) vavasis3.rua - Total non-zero values: 1,683,902 - p = 4 vavasis3.rua - Total non-zero values: 1,683,902 - p = 2

  20. Results (vavasis3) vavasis3.rua - Calculated Results

  21. Results (bayer02) bayer02.rua - Total non-zero values: 63,679 - p = 10

  22. Results (bayer02) bayer02.rua - Total non-zero values: 63,679 - p = 8 bayer02.rua - Total non-zero values: 63,679 - p = 1

  23. Results (bayer02) bayer02.rua - Total non-zero values: 63,679 - p = 4 bayer02.rua - Total non-zero values: 63,679 - p = 2

  24. Results (bayer02) bayer02.rua - Calculated Results

  25. Conclusions • The proposed representation speeds up the matrix calculation • Data mismatch solution before gather should be improved • There seems to be a communication penalty for using moving structured data

  26. Bibliography • “Optimizing the Performance of Sparse Matrix-Vector Multiplication” dissertation by Eun-Jin Im. • “Iterative Methods for Sparse Linear Systems” by Yousef Saad • “Users’ Guide for the Harwell-Boeing Sparse Matrix Collection” by Iain S. Duff

More Related