Mac 2103
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

MAC 2103 PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on
  • Presentation posted in: General

MAC 2103. Module 11 lnner Product Spaces II. Learning Objectives. Upon completing this module, you should be able to: Construct an orthonormal set of vectors from an orthogonal set of vectors . Find the coordinate vector with respect to a given orthonormal basis.

Download Presentation

MAC 2103

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Mac 2103

MAC 2103

Module 11

lnner Product Spaces II


Learning objectives

Learning Objectives

Upon completing this module, you should be able to:

  • Construct an orthonormal set of vectors from an orthogonal set of vectors.

  • Find the coordinate vector with respect to a given orthonormal basis.

  • Construct an orthogonal basis from a nonstandard basis in ℜⁿ using the Gram-Schmidt process.

  • Find the least squares solution to a linear system Ax = b.

  • Find the orthogonal projection on col(A).

  • Obtain the best approximation.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


General vector spaces ii

General Vector Spaces II

The major topics in this module:

Orthogonal Bases, Gram-Schmidt Process,

Least Squares and Best Approximation

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.09


How to construct an orthonormal set of vectors from an orthogonal set of vectors

How to Construct an Orthonormal Set of Vectorsfrom an Orthogonal Set of Vectors?

We have learned from the previous module that two vectors u and v in an inner product space V are orthogonal to each other iff <u,v> = 0.

To obtain an orthonormal set, we will normalize each of the vectors in the orthogonal set.

How to normalize the vectors? This can be done by dividing each of them by their respective norm and making each of them a unit vector.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to construct an orthonormal set of vectors from an orthogonal set of vectors cont

How to Construct an Orthonormal Set of Vectorsfrom an Orthogonal Set of Vectors? (Cont.)

Example 1: Find the orthonormal set of vectors from the following set of vectors: Let S ={v1, v2} where v1 = (5,0) and v2= (0,-3).

Step 1: Verify that the set of vectors are mutually orthogonal with respect to the Euclidean inner product on ℜ².

Step 2: Find the norm for both vectors.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to construct an orthonormal set of vectors from an orthogonal set of vectors cont1

How to Construct an Orthonormal Set of Vectorsfrom an Orthogonal Set of Vectors? (Cont.)

Step 3: Normalize the vectors in the orthogonal set.

Step 4: Verify that the set S is orthonormal by showing that

and

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Orthonormal set orthonormal basis and orthogonal basis

Orthonormal Set, Orthonormal Basis, and Orthogonal Basis

Orthonormal Set: An orthogonal set in which each vector is a unit vector.

Orthonormal basis: A basis consisting of orthonormal vectors in an inner product space. Example: The standard basis for ℜⁿ.

Orthogonal basis: A basis consisting of orthogonal vectors in an inner product space.

Note that if S is an orthogonal set, then S is a linearly independent set.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Orthonormal set orthonormal basis and orthogonal basis cont

Orthonormal Set, Orthonormal Basis, and Orthogonal Basis (Cont.)

If S ={v1, v2 , … , vn} is an orthogonal basis of W, then for any w∈ W,

where

are called the Fourier coefficients.

So the coordinate vector of w,

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Orthonormal set orthonormal basis and orthogonal basis cont1

Orthonormal Set, Orthonormal Basis, and Orthogonal Basis (Cont.)

If S ={q1, q2 , … , qn} is an orthonormal basis of W, then for any w∈ W,

where are called the Fourier coefficients.

So the coordinate vector of w,

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the coordinate vector with respect to a given orthogonal basis

How to Find the Coordinate Vector with Respect to a Given Orthogonal Basis?

Example 2: Compute the coefficients and determine the coordinate vectors in Example 1 for u = (10,3).

From Example 1, we have v1 = (5,0), v2= (0,-3) and

In this case, the coefficients are:

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the coordinate vector with respect to a given orthogonal basis cont

How to Find the Coordinate Vector with Respect to a Given Orthogonal Basis? (Cont.)

So the coordinate vector of u,

We can see that a nice advantage of working with an orthogonal basis is that the coefficients in any basis representation for a vector are immediately known; they are called Fourier coefficients.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the coordinate vector with respect to a given orthonormal basis

How to Find the Coordinate Vector with Respect to a Given Orthonormal Basis?

Example 3: Find the coordinates of w = (2,3) relative to the orthonormal basis for ℜ², B = {v1, v2}, where

Since B is orthonormal, we have

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


The gram schmidt process

The Gram-Schmidt Process

Let S = {u1, u2, …, um} with nonzero ui∈ℜⁿ for i = 1, 2, … , m. S does not have to be a linearly independent set. It might be that A = [u1u2 … um] is an n x m matrix and the source of S.

The Gram-Schmidt Algorithm: S = {u1, u2, …, um}

Let

For k = 2, 3, … , m, let

If we discard it since it is linearly dependent.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


The gram schmidt process cont

The Gram-Schmidt Process (Cont.)

We then have r ≤ m orthogonal and linearly independent vectors in B = {v1, v2, …, vr}.

span(S) = span(B) = W, a r dimensional subspace of ℜⁿ and B is an orthogonal basis for W. W = col(A) if A = [u1u2 … um].

An orthogonal basis for W is {q1, q2, …, qr} where

for i =1, 2, … , r.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to construct an orthonormal basis from a nonstandard basis in using the gram schmidt process

How to Construct an Orthonormal Basis from a Nonstandard Basis in ℜⁿ using the Gram-Schmidt Process?

Let ℜⁿ be the usual Euclidean inner product space of dimension n. Let {u1, u2, …, un} be a nonstandard basis in ℜⁿ.

Step 1: Use the Gram-Schmidt method to construct an orthogonal basis {v1, v2, …, vn} from the basis vectors {u1, u2, …, un}.

Step 2: Normalize the orthogonal basis vectors to obtain the orthonormal basis {q1, q2,…, qn}.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Mac 2103

How to Construct an Orthonormal Basis from a Nonstandard Basis in ℜⁿ using the Gram-Schmidt Process? (Cont.)

Example 4:

The set is a nonstandard basis in ℜ³.

Step 1:

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Mac 2103

How to Construct an Orthonormal Basis from a Nonstandard Basis in ℜⁿ using the Gram-Schmidt Process? (Cont.)

form an orthogonal basis for ℜ³.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Mac 2103

How to Construct an Orthonormal Basis from a Nonstandard Basis in ℜⁿ using the Gram-Schmidt Process? (Cont.)

Step 2: Normalize the orthogonal basis vectors to obtain the orthonormal basis B = {q1, q2, q3}.

The norms of these vectors are:

So an orthonormal basis is B where

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the least squares solution

How to Find the Least Squares Solution?

The linear system Ax = b always has the associated normal system ATAx = ATb which is consistent and has one or more solutions. Any solution of this system is a least squares solution of Ax = b. Moreover, the orthogonal projection of b on W = col(A) is where x is a least squares solution.

Example 5: Find the least squares solution of the linear system Ax = b given by

Observe that A has two linearly independent column vectors, so ATA is invertible, and there is a unique solution to ATAx = ATb, which will be our least squares solution to Ax = b.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the least squares solution cont

How to Find the Least Squares Solution? (Cont.)

We have

So the normal system ATAx = ATb in this case is

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the least squares solution cont1

How to Find the Least Squares Solution? (Cont.)

By Gauss Elimination, the row-echelon form of

is

The solution to the normal system ATAx = ATb in this case is

is our unique least square solution to Ax = b.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


How to find the orthogonal projection and obtain the best approximation

How to Find the Orthogonal Projection and Obtain the Best Approximation?

is the orthogonal projection of b on W = col(A). The is the Best Approximation to b in W since the distance between b and is the minimum for all vectors in W,

Example 6: From the previous example, the orthogonal projection of b on W = col(A) is

Thus, we have obtained the best approximation to b in W.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


What have we learned

What have we learned?

We have learned to :

  • Construct an orthonormal set of vectors from an orthogonal set of vectors.

  • Find the coordinate vector with respect to a given orthonormal basis.

  • Construct an orthogonal basis from a nonstandard basis in ℜⁿ using the Gram-Schmidt process.

  • Find the least squares solution to a linear system Ax = b.

  • Find the orthogonal projection on col(A).

  • Obtain the best approximation.

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


Credit

Credit

Some of these slides have been adapted/modified in part/whole from the following textbook:

  • Anton, Howard: Elementary Linear Algebra with Applications, 9th Edition

http://faculty.valenciacc.edu/ashaw/

Click link to download other modules.

Rev.F09


  • Login