Matrix Multiplication

1 / 14

# Matrix Multiplication - PowerPoint PPT Presentation

Matrix Multiplication. To make this discussion easier we will assume square matrices The product of two n by n matrices A and B is given by Note that all valid products are of the form. Sequential Matrix Multiplication. MODULE matrix1; CONST n = 3; TYPE

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Matrix Multiplication' - star

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Matrix Multiplication
• To make this discussion easier we will assume square matrices
• The product of two n by n matrices A and B is given by
• Note that all valid products are of the form

ICSS571 - Matrix Multiplication

Sequential Matrix Multiplication

MODULE matrix1;

CONST

n = 3;

TYPE

matrix = ARRAY [0..n-1],[0..n-1] OF INTEGER;

VAR

a,b,c : matrix;

i,j,k : INTEGER;

BEGIN

FOR i := 1 TO n - 1 DO

FOR j := 0 TO n - 1 DO

c[i,j] := 0;

FOR k := 0 TO n-1 DO

c[i,j] := c[i,j] + a[i,k] * b[k,j];

END

END

END

END matrix1.

The complexity of this algorithm is clearly (n3).

For the 3 by 3 matrix case this requires 27 multiplications

Gee would’nt it be neat to do this in parallel 

ICSS571 - Matrix Multiplication

Dissection Time

a00 a01 a02

a10 a11 a12

a20 a21 a22

b00 b01 b02

b10 b11 b12

b20 b21 b22

x

=

a00*b00+a01*b10+a02*b20 a00*b01+a01*b11+a02*b21 a00*b02+a01*b12+a02*b22

a10*b00+a11*b10+a12*b20 a10*b01+a11*b11+a12*b21 a10*b02+a11*b12+a12*b22

a20*b00+a21*b10+a22*b20 a20*b01+a21*b11+a22*b21 a20*b02+a21*b12+a22*b22

ICSS571 - Matrix Multiplication

Parallelize
• Organize the PE grid as a N x N x N cube
• Place the data in the processors so that each computes a sum for one of the Cij’s so the multiplication can be done in one step
• All that is left to sum the products

ICSS571 - Matrix Multiplication

Parallelize

a02*b20 a02*b21 a02*b22

a12*b20 a12*b21 a12*b22

a22*b20 a22*b21 a22*b22

Sum Reduction

a01*b10 a01*b11 a01*b12

a11*b10 a11*b11 a11*b12

a21*b10 a21*b11 a21*b12

a00*b00 a00*b01 a00*b02

a10*b00 a10*b01 a10*b02

a20*b00 a20*b01 a20*b02

ICSS571 - Matrix Multiplication

The Algorithm

The algorithm for parallel matrix multiplication

• Load the arrays into the cube
• Everyone multiplies
• Do a REDUCE.SUM from back to front
• Result is in the front 3x3 plane of the cube

ICSS571 - Matrix Multiplication

Sequential Matrix Multiplication

MODULE matrix2;

CONST

n = 3;

TYPE

matrix = ARRAY [0..n-1],[0..n-1] OF INTEGER;

CONFIGURATION

grid [0..n-1],[0..n-1],[0..n-1];

CONNECTION

front: grid[i,j,k] -> grid[0,j,k];

VAR

a,b,c : grid OF INTEGER;

i,j,k : INTEGER;

BEGIN

(* load the processor planes *)

c:=a*b;

SEND.front:#SUM(c,c);

(* retrieve the result *)

END matrix2.

The complexity of this algorithm is clearly O(log2n).

However, the number of processors required is O(n3)

ICSS571 - Matrix Multiplication

Using Fewer Processors

b22

b12

b02

b21

b11

b01

b20

b10

b00

a02 a01 a00

a12 a11 a10

a22 a21 a20

ICSS571 - Matrix Multiplication

Using Fewer Processors

b22

b12

b02

b21

b11

b01

b20

b10

a02 a01

a12 a11 a10

a22 a21 a20

ICSS571 - Matrix Multiplication

Using Fewer Processors

b22

b12

b02

b21

b11

b20

a02

a12 a11

a22 a21 a20

ICSS571 - Matrix Multiplication

Using Fewer Processors

b22

b12

b21

a12

a22 a21

ICSS571 - Matrix Multiplication

Using Fewer Processors

b22

a22

ICSS571 - Matrix Multiplication

Improving Efficiency

a00*b00+a01*b10+a02*b20 a00*b01+a01*b11+a02*b21 a00*b02+a01*b12+a02*b22

a10*b00+a11*b10+a12*b20 a10*b01+a11*b11+a12*b21 a10*b02+a11*b12+a12*b22

a20*b00+a21*b10+a22*b20 a20*b01+a21*b11+a22*b21 a20*b02+a21*b12+a22*b22

b22

b12

b02

b21

b11

b01

b20

b10

b00

a02 a01 a00

a12 a11 a10

a22 a21 a20

ICSS571 - Matrix Multiplication

Improving Efficiency

ICSS571 - Matrix Multiplication