1 / 50

Chapter 5

Chapter 5. Numeric Algorithms. Chapter Outline. Calculating polynomials Matrix multiplication Linear equations. Prerequisites. Before beginning this chapter, you should be able to: Do simple algebra Evaluate polynomials Describe growth rates and order. Goals.

devon
Download Presentation

Chapter 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 5 Numeric Algorithms

  2. Chapter Outline • Calculating polynomials • Matrix multiplication • Linear equations

  3. Prerequisites • Before beginning this chapter, you should be able to: • Do simple algebra • Evaluate polynomials • Describe growth rates and order

  4. Goals • At the end of this chapter you should be able to: • Evaluate a polynomial by Horner’s method • Evaluate a polynomial by preprocessing its coefficients • Explain the analysis of preprocessed coefficients • Explain matrix multiplication

  5. Goals (continued) • Trace Winograd’s matrix multiplication • Explain the analysis of Winograd’s matrix multiplication • Use Strassen’s matrix multiplication • Explain the Gauss-Jordan method for solving systems of linear equations

  6. Numeric Algorithm Efficiency • Numeric programs will do a particular calculation a large number of times • Small improvements can result in significant time savings because of how many times the calculation is done

  7. Numeric Analysis Basics • We count additions and multiplications • Because additions are typically much faster than multiplications, reducing the multiplications by increasing the additions can still result in an improvement • The analysis will be based on the highest power in a polynomial or the size of the matrices we are working with

  8. Calculating Polynomials • We will use a generic polynomial form of:where the coefficient values are known constants • The value of x will be the input and the result is the value of the polynomial using this x value

  9. Standard Evaluation Algorithm result = a[0] + a[1]*x xPower = x for i = 2 to n do xPower = xPower * x result = result + a[i]*xPower end for return result

  10. Analysis • Before the loop, there is • One multiplication • One addition • The for loop is done N-1 times • There are two multiplications in the loop • There is one addition in the loop • There are a total of • 2N – 1 multiplications • N additions

  11. Horner’s Method • Based on the factorization of a polynomial • The generic equation is factored as: • For example, the equation: • would be factored as:

  12. Horner’s Method Algorithm result = a[n] for i = n - 1 down to 0 do result = result * x result = result + a[i] end for return result

  13. Analysis • The for loop is done N times • There is one multiplication in the loop • There is one addition in the loop • There are a total of • N multiplications • N additions • Saves N – 1 multiplications over the standard algorithm

  14. Preprocessed Coefficients • Uses a factorization of a polynomial based on polynomials of about half the original polynomial’s degree • For example, where the standard algorithm would do 255 multiplications to calculate x256, we could square x and then square the result seven more times to get the same result

  15. Preprocessed Coefficients • Used with polynomials that are monic (an=1) and have a highest power that is one less than a power of 2 • If the polynomial has highest power that is 2k– 1, we can factor it as:where j = 2k-1

  16. Preprocessed Coefficients • If we choose b so that it is aj-1 – 1, q(x) and r(x) will both be monic, and so the process can be recursively applied to them as well

  17. Preprocessed Coefficients Example • For the example equation of:because the highest degree is 3 = 22 –1, j would be 21 = 2, and b would have a value of a1 – 1 = 6 • This makes one factor x2 + 6, and we divide p(x) by this polynomial to find q(x) and r(x)

  18. Preprocessed Coefficients Example • The division is:which gives

  19. Analysis • We analyze preprocessed coefficients by developing recurrence relations for the number of multiplications and additions • In the factorization, we break the polynomial up into two smaller polynomials and do one additional multiplication and two additional additions

  20. Analysis • Because we required that N = 2k– 1, we get: M(1) = 0 A(1) = 0M(k) = 2M(k–1) + 1 A(k) = 2A(k–1) + 2 • Solving these equations gives about N/2 multiplications and (3N – 1)/2 additions • We need to include the k – 1 (or lg N) multiplications needed to calculate the series of values x2, x4, x8, …

  21. Polynomial Algorithm Comparison • For the example polynomial: • Standard algorithm: • 5 multiplications and 3 additions • Horner’s method • 3 multiplications and 3 additions • Preprocessed coefficients • 2 multiplications and 4 additions

  22. Polynomial Algorithm Comparison • In general, for a polynomial of degree N: • Standard algorithm: • 2N – 1 multiplications and N additions • Horner’s method • N multiplications and N additions • Preprocessed coefficients • N/2 + lg N multiplications and (3N – 1)/2 additions

  23. Matrix Multiplication • Two matrices can be multiplied if the number of columns in the first is the same as the number of rows in the second • Each row of the first matrix is multiplied by each column of the second matrix • The value at location i, j of the result is from the addition of the products when row i of the first matrix is multiplied by row j of the second matrix

  24. Matrix Multiplication • An example of this process is:

  25. Matrix Multiplication Algorithm for i = 1 to a do for j = 1 to c do R[i,j] = G[i,1] * H[1,j] for k = 2 to b R[i,j] = R[i,j] + G[i,k] * H[k,j] end for k end for j end for i

  26. Analysis • The “for k” loop runs b – 1 times and does one multiplication and addition each time • The “for j” loop runs c times and does b multiplications and b – 1 additions each time • The “for i” loop runs a times and does b*c multiplications and (b – 1)*c additions each time • Therefore, the algorithm does a*b*c multiplications and a*(b – 1)*c additions

  27. Winograd’s Matrix Multiplication • We can consider the row and column as vectors V = (v1, v2, v3, v4) and W = (w1, w2, w3, w4) • Each element of the result becomes a dot product of these two vectors:V•W = v1*w1 + v2*w2 + v3*w3 + v4*w4

  28. Winograd’s Matrix Multiplication • We can factor this dot product into: • Though this looks like more work, the last two lines can be done once for each row of the first matrix, and each column of the second matrix

  29. Winograd’s AlgorithmPreprocessing Step d = b/2 // calculate rowFactors for G for i = 1 to a do rowFactor[i] = G[i,1] * G[i,2] for j = 2 to d do rowFactor[i] = rowFactor[i] + G[i,2j-1] * G[i,2j] end for end for // calculate columnFactors for H for i = 1 to c do columnFactor[i] = H[1,i] * H[2,i] for j = 2 to d do columnFactor[i] = columnFactor[i] + H[2j-1,i] * H[2j,i] end for end for

  30. Winograd’s AlgorithmCalculation Step for i = 1 to a do for j = 1 to c do R[i,j] = -rowFactor[i] - columnFactor[j] for k = 1 to d do R[i,j] = R[i,j] + (G[i,2k-1] + H[2k,j]) * (G[i,2k] + H[2k-1,j]) end for end for end for // add in terms for odd shared dimension if (2 * (b / 2) ≠ b) then for i = 1 to a do for j = 1 to c do R[i,j] = R[i,j] + G[i,b] * H[b,j] end for end for end if

  31. Analysis • Preprocessing step: • The “for j” loop runs d – 1 times and does one multiplication and addition • The “for i” loop runs a (or c) times and does d multiplications and d – 1 additions

  32. Analysis • Calculation step for an even shared dimension (b) of the matrices: • The “for k” loop runs d times and does one multiplication and three additions • The “for j” loop runs c times and does d multiplications and 3d + 1 additions • The “for i” loop runs a times and does c*d multiplications and c*(3d + 1) additions

  33. Analysis • The entire algorithm does:a*d + c*d + a*c*d multiplicationsa*(d–1) + c*(d–1) + a*c*(3d+1) additionswhen b is even and d = b/2

  34. Strassen’s Matrix Mutiplication • Uses a set of seven formulas to multiply two 2 X 2 matrices • Does not rely on elements being commutative under multiplication • It can be applied recursively, in other words, two 4 X 4 matrices can be multiplied by treating each as a 2 X 2 matrix of 2 X 2 matrices

  35. Strassen’s Formulas • First we calculate a set of temporary values: x1 = (G[1,1] + G[2,2]) * (H[1,1] + H[2,2]) x2 = (G[2,1] + G[2,2]) * H[1,1] x3 = G[1,1] * (H[1,2] – H[2,2]) x4 = G[2,2] * (H[2,1] – H[1,1]) x5 = (G[1,1] + G[1,2]) * H[2,2] x6 = (G[2,1] – G[1,1]) * (H[1,1] + H[1,2]) x7 = (G[1,2] – G[2,2]) * (H[2,1] + H[2,2])

  36. Strassen’s Formulas • The result is then calculated by: R[1,1] = x1 + x4 – x5 + x7 R[2,1] = x2 + x4 R[1,2] = x3 + x5 R[2,2] = x1 + x3 – x2 + x6

  37. Analysis • These formulas require 7 multiplications and 18 additions to multiply two 2 X 2 matrices • The real savings occur when this is applied recursively and we do approximately N2.81 multiplications and 6N2.81 – 6N2 additions • Though not used in practice, Strassen’s method is important because it was the first algorithm that is faster than O(N3)

  38. Linear Equations • A system of linear equations is a set of N equations in N unknowns: a11x1 + a12x2 + a13x3 + … + a1NxN = b1 a21x1 + a22x2 + aa3x3 + … + a2NxN = b2  aNx1 + aN2x2 + aN3x3 + … + aNNxN = bN

  39. Linear Equations • When these equations are used, the coefficients (a values) are constants and the results (b values) are typically input values • We want to determine the x values that satisfy these equations and produce the indicated results

  40. Linear Equation Example • An example set of linear equations with three unknowns is:2x1 – 4x2 + 6x3 = 146x1 – 6x2 + 6x3 = 244x1 + 2x2 + 2x3 = 18

  41. Solving Linear Equations • One method to determine a solution would be to substitute one equation into another • For example, we solve the first equation for x1 and then substitute this into the rest of the equations • This substitution reduces the number of unknowns and equations

  42. Solving Linear Equations • If we repeat this, we get to one unknown and one equation and can then back up to get the values of the rest • If there are many equations, this process can be time consuming, prone to error, and is not easily computerized

  43. Gauss-Jordan Method • This method is based on the previous idea • We store the equation constants in a matrix with N rows and N+1 columns • For the example, we would get:

  44. Gauss-Jordan Method • We perform operations on the rows until we eventually have the identity matrix in the first N columns and then the unknown values will be in the final column:

  45. Gauss-Jordan Method • On each pass, we pick a new row and divide it by the first element that is not zero • We then subtract multiples of this row from all of the others to create all zeros in a column except in this row • When we have done this N times, each row will have one value of 1 and the last column will have the unknown values

  46. Example • Consider the example again: • We begin by dividing the first row by 2, and then subtract 6 times it from the second row and 4 times it from the third row

  47. Example • Now, we divide the second row by 6, and then subtract -2 times it from the first row and 10 times it from the third row

  48. Example • Now, we divide the third row by 10, and then subtract -1 times it from the first row and -2 times it from the second row

  49. Example • This gives the result of: • And we have that x1 = 3, x2 = 1, and x3 = 2

  50. Concerns • In practice, computer round-off error give inaccurate results • If the matrix is singular, one row will be just a multiple of another row and we will get a divide by zero error • There are other algorithms that can also handle these problems but they are more appropriate for a numerical analysis course

More Related