Introduction
Download
1 / 88

By Jeff Edmonds - PowerPoint PPT Presentation


  • 508 Views
  • Updated On :

Thinking about Algorithms Abstractly Introduction So you want to be a computer scientist? Grade School Revisited: How To Multiply Two Numbers By Jeff Edmonds York University Lecture 1 COSC 3101 So you want to be a computer scientist? Is your goal to be a mundane programmer?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'By Jeff Edmonds' - emily


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Introduction l.jpg

Thinking about Algorithms Abstractly

Introduction

  • So you want to be a computer scientist?

  • Grade School Revisited: How To Multiply Two Numbers

By Jeff Edmonds

York University

Lecture1

COSC 3101



Is your goal to be a mundane programmer l.jpg
Is your goal to be a mundane programmer?




Boss assigns task l.jpg
Boss assigns task:

  • Given today’s prices of pork, grain, sawdust, …

  • Given constraints on what constitutes a hotdog.

  • Make the cheapest hotdog.

Everyday industry asks these questions.


Your answer l.jpg
Your answer:

  • Um? Tell me what to code.

With more suffocated software engineering systems,the demand for mundane programmers will diminish.


Your answer8 l.jpg
Your answer:

  • I learned this great algorithm that will work.

Soon all known algorithms will be available in libraries.


Your answer9 l.jpg
Your answer:

  • I can develop a new algorithm for you.

Great thinkers will always be needed.


The future belongs to the computer scientist who has l.jpg
The future belongs to the computer scientist who has

  • Content: An up to date grasp of fundamental problems and solutions

  • Method: Principles and techniques to solve the vast array of unfamiliar problems that arise in a rapidly changing field

  • Rudich www.discretemath.com


Course content l.jpg
Course Content

  • A list of algoirthms.

    • Learn their code.

    • Trace them until you are convenced that they work.

    • Impliment them.

class InsertionSortAlgorithm extends SortAlgorithm {

void sort(int a[]) throws Exception {

for (int i = 1; i < a.length; i++) {

int j = i;

int B = a[i];

while ((j > 0) && (a[j-1] > B)) {

a[j] = a[j-1];

j--;}

a[j] = B;

}}


Course content12 l.jpg
Course Content

  • A survey of algorithmic design techniques.

  • Abstract thinking.

  • How to develop new algorithms for any problem that may arise.


Study l.jpg
Study:

  • Many experienced programmers were asked to code up binary search.


Study14 l.jpg
Study:

  • Many experienced programmers were asked to code up binary search.

80% got it wrong

Good thing is was not for a nuclear power plant.



What did they lack16 l.jpg
What did they lack?

  • Formal proof methods?


What did they lack17 l.jpg
What did they lack?

  • Formal proof methods?

Yes, likely

Industry is starting to realize that formal methods are important.

But even without formal methods …. ?


What did they lack18 l.jpg
What did they lack?

  • Fundamental understanding of the algorithmic design techniques.

  • Abstract thinking.


Course content19 l.jpg
Course Content

Notations, analogies, and abstractions

for developing,

thinking about,

and describing algorithms

so correctness is transparent



Start with some math l.jpg

Classifying Functions techniques

Time Complexity

t(n) = Q(n2)

f(i) = nQ(n)

Time

Input Size

Recurrence Relations

Adding Made Easy

T(n) = a T(n/b) + f(n)

∑i=1 f(i).

Start With Some Math


Iterative algorithms loop invariants l.jpg

0 techniques

i-1

i

T+1

i

i

9 km

5 km

Iterative Algorithms Loop Invariants

<preCond>

codeA

loop

<loop-invariant>

exit when <exit Cond>

codeB

codeC

<postCond>

One step at a time

Code

Relay Race


Recursive algorithms l.jpg

? techniques

?

Recursive Algorithms



Network flows l.jpg
Network Flows techniques


Greedy algorithms l.jpg
Greedy Algorithms techniques


Dynamic programing l.jpg
Dynamic Programing techniques


Reduction l.jpg
Reduction techniques

=

  • Rudich www.discretemath.com



Read ahead l.jpg
Read Ahead techniques

You are expected to read the lecture notesbefore the lecture.

This will facilitate more productive discussionduringclass.

Also please proof readassignments & tests.

Like in an English class


Explaining l.jpg

We are going to test you on your techniquesability to explain the material.

Hence, the best way of studying is to explain the material over and over again out loud toyourself, to each other, and to your stuffed bear.

Explaining


Day dream l.jpg

W techniqueshile going along with your day

Day Dream

Mathematics is not all linear thinking.

Allow the essence of the materialto seepinto your subconscious

Pursue ideas that percolate up and flashes of inspiration that appear.


Be creative l.jpg

Be Creative techniques

Ask questions.

Why is it done this way and not thatway?


Guesses and counter examples l.jpg
Guesses and Counter Examples techniques

  • Guess at potential algorithms for solving a problem.

  • Look forinput instances for which your algorithm gives the wrong answer.

  • Treat it as a game between these two players.


Slide35 l.jpg
Refinement: techniquesThe best solution comes from a process of repeatedly refining and inventing alternative solutions

  • Rudich www.discretemath.com


Grade school revisited how to multiply two numbers l.jpg

2 X 2 = 5 techniques

A Few Example Algorithms

Grade School Revisited:How To Multiply Two Numbers

  • Rudich www.discretemath.com


Slides in this next section produced by l.jpg
Slides in this next section techniques produced by

Steven Rudich

from Carnegie Mellon University

Individual Slides

will be marked

  • Rudich www.discretemath.com


Complex numbers l.jpg
Complex Numbers techniques

  • Remember how to multiply 2 complex numbers?

  • (a+bi)(c+di) = [ac –bd] + [ad + bc] i

  • Input: a,b,c,d Output: ac-bd, ad+bc

  • If a real multiplication costs $1 and an addition cost a penny. What is the cheapest way to obtain the output from the input?

  • Can you do better than $4.02?


Gauss 3 05 method input a b c d output ac bd ad bc l.jpg
Gauss’ $3.05 Method: techniquesInput: a,b,c,d Output: ac-bd, ad+bc

  • m1 = ac

  • m2 = bd

  • A1 = m1 – m2 = ac-bd

  • m3 = (a+b)(c+d) = ac + ad + bc + bd

  • A2 = m3 – m1 – m2 = ad+bc


Question l.jpg
Question: techniques

  • The Gauss “hack” saves one multiplication out of four. It requires 25% less work.

  • Could there be a context where performing 3 multiplications for every 4 provides a more dramatic savings?


Slide41 l.jpg

Odette techniques

Bonzo


How to add 2 n bit numbers l.jpg
How to add 2 n-bit numbers. techniques

**

**

**

**

**

**

**

**

**

**

**

+


How to add 2 n bit numbers43 l.jpg
How to add 2 n-bit numbers. techniques

**

**

**

**

**

**

**

**

**

* **

**

*

+


How to add 2 n bit numbers44 l.jpg
How to add 2 n-bit numbers. techniques

**

**

**

**

**

**

**

**

* **

* **

*

**

*

+


How to add 2 n bit numbers45 l.jpg
How to add 2 n-bit numbers. techniques

**

**

**

**

**

**

**

* **

* **

*

* **

*

**

*

+


How to add 2 n bit numbers46 l.jpg
How to add 2 n-bit numbers. techniques

**

**

**

**

**

**

* **

* **

*

* **

*

* **

*

**

*

+


How to add 2 n bit numbers47 l.jpg
How to add 2 n-bit numbers. techniques

*

*

* **

*

* **

*

* **

*

* **

*

* **

*

***

*

* **

*

* **

*

* **

*

* **

*

**

*

+


Time complexity of grade school addition l.jpg

* techniques

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

**

*

+

Time complexity of grade school addition

On any reasonable computer adding 3 bits can be done in constant time.

T(n) = The amount of time grade school addition uses to add two n-bit numbers

= θ(n) = linear time.


Slide49 l.jpg

f = techniquesθ(n) means that f can be sandwiched between two lines

time

# of bits in numbers


Slide50 l.jpg

Please feel free techniques

to ask questions!

  • Rudich www.discretemath.com


Is there a faster way to add l.jpg
Is there a faster way to add? techniques

  • QUESTION: Is there an algorithm to add two n-bit numbers whose time grows sub-linearly in n?


Any algorithm for addition must read all of the input bits l.jpg
Any algorithm for addition must read all of the input bits techniques

  • Suppose there is a mystery algorithm that does not examine each bit

  • Give the algorithm a pair of numbers. There must be some unexamined bit position i in one of the numbers

  • If the algorithm is not correct on the numbers, we found a bug

  • If the algorithm is correct, flip the bit at position i and give the algorithm the new pair of numbers. It give the same answer as before so it must be wrong since the sum has changed


Slide53 l.jpg

So techniquesany algorithm for addition must use time at least linear in the size of the numbers.

Grade school addition is essentially as good as it can be.


How to multiply 2 n bit numbers l.jpg

n techniques2

How to multiply 2 n-bit numbers.

* * * * * * * *

X

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * * * * * * * * * *



Grade school addition linear time grade school multiplication quadratic time l.jpg
Grade School Addition: Linear time techniquesGrade School Multiplication: Quadratic time

  • No matter how dramatic the difference in the constants the quadratic curve will eventually dominate the linear curve

time

# of bits in numbers


Slide57 l.jpg

Neat! We have demonstrated that as things scale multiplication is a harder problem than addition.Mathematical confirmation of our common sense.


Slide58 l.jpg

Don’t jump to conclusions! multiplication is a harder problem than addition.We have argued that grade school multiplication uses more time than grade school addition. This is a comparison of the complexity of two algorithms.

To argue that multiplication is an inherently harder problem than addition we would have to show that no possible multiplication algorithm runs in linear time.


Grade school addition n time grade school multiplication n 2 time l.jpg
Grade School Addition: multiplication is a harder problem than addition.θ(n) timeGrade School Multiplication: θ(n2) time

Is there a clever algorithm to multiply two numbers in linear time?




Divide and conquer an approach to faster algorithms l.jpg
Divide And Conquer you learned in grade school?(an approach to faster algorithms)

  • DIVIDE my instance to the problem into smaller instances to the same problem.

  • Have a friend (recursively) solve them.Do not worry about it yourself.

  • GLUEthe answers together so as to obtain the answer to your larger instance.


Multiplication of 2 n bit numbers l.jpg
Multiplication of 2 n-bit numbers you learned in grade school?

a

b

  • X =

  • Y =

  • X = a 2n/2 + b Y = c 2n/2 + d

  • XY = ac 2n + (ad+bc) 2n/2 + bd

c

d


Multiplication of 2 n bit numbers64 l.jpg
Multiplication of 2 n-bit numbers you learned in grade school?

a

b

c

d

  • X =

  • Y =

  • XY = ac 2n + (ad+bc) 2n/2 + bd

MULT(X,Y):

If |X| = |Y| = 1 then RETURN XY

Break X into a;b and Y into c;d

RETURN

MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)


Time required by mult l.jpg
Time required by MULT you learned in grade school?

  • T(n) = time taken by MULT on two n-bit numbers

  • What is T(n)? What is its growth rate? Is it θ(n2)?


Recurrence relation l.jpg
Recurrence Relation you learned in grade school?

  • T(1) = k for some constant k

  • T(n) = 4 T(n/2) + k’ n + k’’ for some constants k’ and k’’

MULT(X,Y):

If |X| = |Y| = 1 then RETURN XY

Break X into a;b and Y into c;d

RETURN

MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)


Let s be concrete l.jpg
Let’s be concrete you learned in grade school?

  • T(1) = 1

  • T(n) = 4 T(n/2) + n

  • How do we unravel T(n) so that we can determine its growth rate?


Technique 1 guess and verify l.jpg
Technique 1 you learned in grade school?Guess and Verify

  • Recurrence Relation:

    T(1) = 1 & T(n) = 4T(n/2) + n

  • Guess: G(n) = 2n2 – n

  • Verify:


Technique 2 decorate the tree l.jpg

Technique 2: Decorate The Tree

T(1)

=

1

  • T(n) = n + 4 T(n/2)

  • T(n) = n + 4 T(n/2)

T(n)

T(n)

n

n

=

=

T(n/2)

T(n/2)

T(n/2)

T(n/2)

T(n/2)

T(n/2)

T(n/2)

T(n/2)


Slide70 l.jpg

T(n) you learned in grade school?

n

=

T(n/2)

T(n/2)

T(n/2)

T(n/2)


Slide71 l.jpg

n/2 you learned in grade school?

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n)

n

=

T(n/2)

T(n/2)

T(n/2)


Slide72 l.jpg

n/2 you learned in grade school?

n/2

n/2

n/2

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n)

n

=


Slide73 l.jpg

T(n) you learned in grade school?

n

=

n/2

n/2

n/2

n/2

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

n/4

11111111111111111111111111111111 . . . . . . 111111111111111111111111111111111


Slide74 l.jpg

n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4

0

1

2

i


Slide75 l.jpg

=1 n/4 + n/4 + n/4 + n/4 + n/4 + n/4×n

= 4×n/2

= 16×n/4

= 4i ×n/2i

= 4logn×n/2logn =nlog4×1

Total: θ(nlog4) = θ(n2)


Divide and conquer mult n 2 time grade school multiplication n 2 time l.jpg
Divide and Conquer MULT: n/4 + n/4 + n/4 + n/4 + n/4 + n/4θ(n2) time Grade School Multiplication: θ(n2) time

All that work for nothing!


Mult revisited l.jpg
MULT revisited n/4 + n/4 + n/4 + n/4 + n/4 + n/4

MULT(X,Y):

If |X| = |Y| = 1 then RETURN XY

Break X into a;b and Y into c;d

RETURN

MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)

  • MULT calls itself 4 times. Can you see a way to reduce the number of calls?


Gauss hack input a b c d output ac ad bc bd l.jpg
Gauss’ Hack: n/4 + n/4 + n/4 + n/4 + n/4 + n/4Input: a,b,c,d Output: ac, ad+bc, bd

  • A1 = ac

  • A3 = bd

  • m3 = (a+b)(c+d) = ac + ad + bc + bd

  • A2 = m3– A1- A3 = ad + bc


Gaussified mult karatsuba 1962 l.jpg
Gaussified MULT n/4 + n/4 + n/4 + n/4 + n/4 + n/4(Karatsuba 1962)

MULT(X,Y):

If |X| = |Y| = 1 then RETURN XY

Break X into a;b and Y into c;d

e = MULT(a,c) and f =MULT(b,d)

RETURN e2n + (MULT(a+b, c+d) – e - f) 2n/2 + f

  • T(n) = 3 T(n/2) + n

  • Actually: T(n) = 2 T(n/2) + T(n/2 + 1) + kn


Slide80 l.jpg

n/2 n/4 + n/4 + n/4 + n/4 + n/4 + n/4

n/2

n/2

n/2

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n)

n

=


Slide81 l.jpg

T(n) n/4 + n/4 + n/4 + n/4 + n/4 + n/4

n

=

T(n/2)

T(n/2)

T(n/2)


Slide82 l.jpg

n/2 n/4 + n/4 + n/4 + n/4 + n/4 + n/4

T(n/4)

T(n/4)

T(n/4)

T(n)

n

=

T(n/2)

T(n/2)


Slide83 l.jpg

n/2 n/4 + n/4 + n/4 + n/4 + n/4 + n/4

n/2

n/2

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n/4)

T(n)

n

=


Slide84 l.jpg

0 n/4 + n/4 + n/4 + n/4 + n/4 + n/4

1

2

n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4

i


Slide85 l.jpg

=1 n/4 + n/4 + n/4 + n/4 + n/4 + n/4×n

= 3×n/2

= 9×n/4

= 3i ×n/2i

= 3logn×n/2logn =nlog3×1

Total: θ(nlog3) = θ(n1.58..)


Dramatic improvement for large n l.jpg
Dramatic improvement for large n n/4 + n/4 + n/4 + n/4 + n/4 + n/4

Not just a 25% savings!

θ(n2) vs θ(n1.58..)


Multiplication algorithms l.jpg

Homework n/4 + n/4 + n/4 + n/4 + n/4 + n/4

Multiplication Algorithms

3*4=3+3+3+3


Slide88 l.jpg

You’re cool! Are you free sometime this weekend? n/4 + n/4 + n/4 + n/4 + n/4 + n/4

Not interested, Bonzo. I took the initiative and asked out a guy in my 3101 class.

Studying done in groupsAssignments are done in pairs.


ad