This presentation is the property of its rightful owner.
1 / 32

# Foundations II: Data Structures and Algorithms PowerPoint PPT Presentation

Foundations II: Data Structures and Algorithms. Instructor : Yusu Wang Lecture 1 : Introduction and Asymptotic notation. Course Information. Course webpage http://www.cse.ohio-state.edu/~yusu/courses/2331 Office hours TuTh 11:00 – 11:30 am Grading policy:

Foundations II: Data Structures and Algorithms

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Foundations II: Data Structures and Algorithms

Instructor : Yusu Wang

Lecture 1 : Introduction and

Asymptotic notation

CSE 2331 / 5331

### Course Information

• Course webpage

http://www.cse.ohio-state.edu/~yusu/courses/2331

• Office hours

• TuTh 11:00 – 11:30 am

• homework: 20%,

• two midterms: 40%,

• final: 40%

CSE 2331 / 5331

### Exams

• Midterm 1

• Sept 30th in class.

• Midterm 2

• Nov. 4th in class.

• Final exam:

• Dec. 12th, 10:0am--11:45am, Room DL0264

Mark you calendar.

Contact me within first two full weeks if any schedule conflict for midterms

CSE 2331 / 5331

### Notes

• All homework should be submitted before or in class on the due date. Late homework during the same day receives a 10% penalty. Late homework submitted the next day receives a 25% penalty.

• You may discuss homework with your classmates. It is very important that you write up your solutions individually, and acknowledge any collaboration / discussion with others.

CSE 2331 / 5331

• Textbook

• Introduction to Algorithms

byCormen, Leiserson, Rivest and Stein

(third edition)

• Others

• CSE 2331 course notes

• By Dr. Rephael Wenger (SBX)

• The Art of Computer Programming

By Donald Knuth

CSE 2331 / 5331

### Introduction

• What is an algorithm?

• Step by step strategy to solve problems

• Should terminate

• Should return correct answer for any input instance

CSE 2331 / 5331

### This Course

• Various issues involved in designing good algorithms

• What do we mean by “good”?

• Algorithm analysis, language used to measure performance

• How to design good algorithms

• Fundamental problems

• Graph traversal, shortest path etc.

CSE 2331 / 5331

Asymptotic Notation

CSE 2331 / 5331

A = < , , …. >

a

a

a

n

2

1

### Sorting problem

• Input:

• Output: permutation of A that is sorted

• Example:

• Input: < 3, 11, 6, 4, 2, 7, 9 >

• Output: < 2, 3, 4, 6, 7, 9, 11 >

CSE 2331 / 5331

3, 6, 11, 4, 2, 7, 9

3, 4, 6, 11, 2, 7, 9

2, 3, 4, 6, 11, 7, 9

### Insertion Sort

1

i

i+1

n

3, 11, 6, 4, 2, 7, 9

CSE 2331 / 5331

### Pseudo-code

InsertionSort(A, n)

for i = 1 to n-1 do

key = A[i+1]

j = i

while j > 0 and A[j] > key do

A[j+1] = A[j]

j = j - 1

A[j+1] = key

CSE 2331 / 5331

### Analysis

• Termination

• Correctness

• beginning of for-loop: if A[1.. i] sorted, then

• end of for-loop: A[1.. i+1] sorted.

• when i = n-1, A[1.. i+1] is sorted.

• Efficiency: time/space

• Depends on input size n

• Space: roughly n

CSE 2331 / 5331

### Running Time

• Depends on input size

• Depends on particular input

• Best case

• Worst case

First Idea:

Provide upper bound (as a guanrantee)

Make it a function of n : T(n)

CSE 2331 / 5331

### Running Time

• Worst case

• T(n) = max time of algorithm on any input of size n

• Average case

• T(n) = expected time of algorithm over all possible inputs of size n

• Need a statistical distribution model for input

• E.g, uniform distribution

CSE 2331 / 5331

c

c

c

c

c

c

c

1

2

4

2

1

3

5

n

T(n) =  ( + i ) = + n +

i=2

2

n

### Insertion Sort

InsertionSort(A, n)

for i = 1 ton-1do

key = A[i]

j = i

while j > 0 and A[j] > key do

A[j+1] = A[j]

j = j - 1

A[j+1] = key

CSE 2331 / 5331

### Asymptotic Complexity

• Why shall we care about constants and lower order terms?

• Should focus on the relative growth w.r.t. n

• Especially when n becomes large !

• E.g: 0.2n1.5 v.s 500n

Second Idea:

Ignore all constants and lower order terms

Only order of growth

--- Asymptotic time complexity

CSE 2331 / 5331

f(n)

lim ≤ c

g(n)

n

### Big O Notation

• f(n)  O(g(n))if and only if c > 0, and n0 , so that

0 ≤ f(n) ≤ cg(n) for all n ≥ n0

• f(n) = O(g(n)) means f(n)  O(g(n))(i.e, at most)

• We say that g(n) is an asymptotic upper bound for f(n).

• It also means:

• E.g, f(n) = O(n2) if there exists c, n0 > 0 such that f(n) ≤ cn2, for all n ≥ n0.

CSE 2331 / 5331

CSE 2331 / 5331

• ?

• ?

• ?

• ?

• ?

• ?

• ?

CSE 2331 / 5331

f(n)

lim ≥ c

g(n)

n

### Omega  Notation

• f(n)  (g(n)) if and only if  c > 0, and n0 , so that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0

• f(n) =  (g(n)) means f(n)  (g(n)) (i.e, at least)

• We say g(n) is an asymptotic lower bound of f(n).

• It also means:

CSE 2331 / 5331

CSE 2331 / 5331

• ?

• ?

• ?

• ?

• ?

• ?

• ?

CSE 2331 / 5331

### Theta  Notation

• Combine lower and upper bound

• Means tight: of the same order

• if and only if , and, such that for any

• means

• We say g(n) is an asymptotically tight bound for f(n).

• It also means:

CSE 2331 / 5331

CSE 2331 / 5331

### Remarks -- 1

• if and only if , and .

• Insertion sort algorithm as described earlier has time complexity .

• Transpose symmetry:

• If , if and only if

• Transitivity:

• If and , then .

• Same for and

CSE 2331 / 5331

### Remarks -- 2

• It is convenient to think that

• Big-O is smaller than or equal to

• Big- is larger than or equal to

• Big- is equal

• However,

• These relations are modulo constant factor scaling

• Not every pair of functions have such relations

If , this does not imply that .

CSE 2331 / 5331

n

n

2

2

### Remarks -- 3

• Provide a unified language to measure the performance of algorithms

• Give us intuitive idea how fast we shall expect the alg.

• Can now compare various algorithms for the same problem

• Constants hidden!

• O( n lg lg n ), 2n lg n

• T(n) =+ O (n)

means T(n) = + h(n), and h(n) = O(n)

CSE 2331 / 5331

### Pitfalls

OK

• O(n) + O(n) = O(n)

• T(n) = n +  O(n)

= n + O(n)

k

?

i = 1

k should not depend on n !

It can be an arbitrarily large constant …

CSE 2331 / 5331

f(n)

lim = 0

g(n)

n

### Yet Another Notation

• Small o notation:

• f(n) = o(g(n))means for any s.t. for all In other words,

• Examples:

• Is n - lg n = o(n) ?

• Is n = o (n lg n) ?

CSE 2331 / 5331

f(n)

g(n)

lim =

lim = 0

g(n)

f(n)

n

n

### Yet Another Notation

• Small ω notation:

• f(n) = ω(g(n)) means for any s.t. for all In other words,

or

• Examples:

• Is n - lg n = ω(n) ?

• Is n = ω(n / lgn) ?

CSE 2331 / 5331

• Review some basics

• Chapter 3.2 of CLRS

• See the board

CSE 2331 / 5331

### Summary

• Algorithms are useful

• Example: sorting problem

• Insertion sort

• Asymptotic complexity

• Focus on the growth of complexity w.r.t input size

CSE 2331 / 5331