Dan witzner hansen email witzner@itu dk
1 / 48

Linear algbra - PowerPoint PPT Presentation

  • Uploaded on

Dan Witzner Hansen Email : [email protected] Linear algbra. Last week?. Groups? Improvements – what is missing?. Misc. The goal is to be able to solve linear equations Continue with linear algebra Linear mappings Basis vectors & independence Solving linear equations & Determinants

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Linear algbra' - shina

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Dan witzner hansen email witzner@itu dk

Dan Witzner Hansen


[email protected]

Linear algbra


Improvements – what is missing?



  • The goal is to be able to solve linear equations

    • Continue with linear algebra

    • Linear mappings

    • Basis vectors & independence

    • Solving linear equations & Determinants

    • Inverse & Least squares

    • SVD

  • Lot’s of stuff. Don’t despair – you will be greatly rewarded in the future


What is a linear equation
What is a linear equation?

  • A linear equation is an equation of the form,

    anxn+ an-1xn-1+ . . . + a1x1 = b.

What is a system of linear equations
What is a system of linear equations?

  • A system of linear equations is simply a set of linear equations. i.e.

    a1,1x1+ a1,2x2+ . . . + a1,nxn = b1

    a2,1x1+ a2,2x2+ . . . + a2,nxn = b2

    . . .

    am,1x1+ am,2x2+ . . . + am,nxn = bm

Matrix form of linear system
Matrix Form of Linear System

Compact notation Ax=b

Linear mappings
Linear Mappings

Affine mapping


Species 1: eats 5 units of A and 3 of B.

Species 2: eats 2 units of A and 4 of B.

Everyday a total of 900 units of A and 960 units of B are eaten. How many animals of each species are there?



Matlab code
Matlab code

A = [5 2; 3 4];

b = [900 960];

x = linspace(0,150,100);

y1 = (-A(1,1)*x+b(1))/A(1,2); %made for clarity

y2 = (-A(2,1)*x+b(2))/A(2,2);

Plot(x,y1,'r-','LineWidth',3); hold on

Plot(x,y2,'b-','LineWidth',3); hold off

title('Linear equations and their solution')

An now for some formalism


Independent vectors

Basis vectors / Orthonomal basis

An now for some formalism


A subspaceis a vector space contained in another vector space


Independent vectors
Independent vectors

Can it happen that y=0 if x is nonzero?

  • If y is non-zero for all non-zero x, then the column vectors of A are said to be linear independent.

  • These vectors form a set of basis vectors

  • Orthonormal basis when the vectors are unit size and orthogonal.

Basis vectors example
Basis vectors - example

Change of basis

Show that it is an orthonormal basis
Show that it is an Orthonormal basis

What happens with this one
What happens with this one?

A = [1 4 2;2 8 6; 3 124];

[X,Y,Z] = meshgrid(-10:10,-10:10,-10:10);

x = [X(:),Y(:),Z(:)]’;

p = A*x;


Solutions of linear equations

A solution to a system of equations is simply an assignment of values to the variables that satisfies (is a solution to) all of the equations in the system.

If a system of equations has at least one solution, we say it is consistent.

If a system does not have any solutions we say that it is inconsistent.

Solutions of linear equations



Solving systems algebraically
Solving systems algebraically

Which solution(s)?

Can we always do this?

How many solutions are there?


  • For A (2x2 matrix)

  • When det A ≠0 a unique solution exists (nonsingular)

  • When det A =0 the matrix is singular (lines same slope) and are therefore the columns are linear dependent

    • Coincident (infinitely many solutions)

    • Parallel (no solutions)

  • Determinant can be used when solving linear equations (Cramers’ rule), but not useful in practice



What if

What to do when the dimension and the number of data points is large?

How many data points are needed to solve for the unknown parameters in x?

What if?

Matrix inverse

Solve simple linear equation

Matrix inverse:

A (unique) inverse exist if det(A) ≠ 0 (NxN matrices)

Matlab: >>invA =inv(A)

Matrix Inverse

Solving linear systems
Solving Linear Systems

  • If m=n (A is a square matrix & Det(A)!=0), then we can obtain the solution by simple inversion (:

  • If m>n, then the system is over-constrained and Ais not invertible

  • If n>m then under constrained.

Notice implementation

Don’tuse for solvingthe linear system. It is mostlymeant for notationalconvenience.

It is faster and more accurate (numerically) to write (solve)x=A\bthaninv(A)*b:

Notice: implementation

Simple inversion of some matrices

Diagonal matrices

Orthogonal matrices

Simple inversion of (some) matrices

Fitting lines
Fitting Lines

  • A 2-D point x = (x, y) is on a line with slope m and intercept b if and only if y =mx + b

  • Equivalently,

  • So the line defined by two points x1, x2 is the solution to the following system of equations:

Example fitting a line
Example: Fitting a Line

  • Suppose we have points (2, 1), (5, 2), (7, 3), and (8, 3)?????

Fitting lines1

With more than two points, there is no guarantee that they will all be on the same line

Fitting Lines

courtesy of

Vanderbilt U.

Least squares
Least squares


Find the vector Fx in the column range of F, which is closest to the right-hand side vector y.

The residual r=y-Fx

Fitting lines2
Fitting Lines

Solution: Use the pseudoinverse

A+ =(ATA)-1AT to obtain least-squares solutionx=A+b

courtesy of

Vanderbilt U.

Example fitting a line1

and x=A+b=(0.3571, 0.2857)T

Example: Fitting a Line

  • Suppose we have points (2, 1), (5, 2), (7, 3), and (8, 3)

  • Then????

Example fitting a line2
Example: Fitting a Line

(2, 1), (5, 2), (7, 3), and (8, 3)

Homogeneous systems of equations
Homogeneous Systems of Equations

  • Suppose we want to solve Ax = 0

  • There is a trivial solution x = 0, but we don’t want this. For what other values of x is Ax close to 0?

  • This is satisfied by computing the singular value decomposition (SVD) A = UDVT (a non-negative diagonal matrix between two orthogonal matrices) and taking x as the last column of V

    • In Matlab[U, D, V] = svd(A)

Properties of svd

When the columns of A =UDV are independent then all

Tells how close to singular A is.

Inverse and pseudoinverse

The columns of U corresponding to nonzeros singular values span the range of A, the columns of V corresponding to zero singular values the nullspace.

Properties of SVD

Example line fitting as a homogeneous system
example: Line-Fitting as a Homogeneous System

A 2-D homogeneous point x = (x, y, 1)T is on the line l = (a, b, c)T only when

ax+ by + c = 0

We can write this equation with a dot product:

x.l= 0,and hence the following system is implied for multiple points x1, x2, ..., xn:

Example homogeneous line fitting
Example: Homogeneous Line-Fitting

Again we have 4 points, but now in homogeneous form:

(2, 1, 1), (5, 2, 1), (7, 3, 1), and (8, 3, 1)

  • The system of equations is:

  • Taking the SVD of A, we get:

compare tox =(0.3571, 0.2857)T

Robust methods
Robust methods

  • So what about outliers

  • Other metrics such as other norms

  • More about this later