1 / 55

Algorithms

Algorithms. Readings: [SG] Ch. 2 & 3 Chapter Outline: Chapter Goals What are Algorithms Real Life Examples (origami, recipes) Definition Example: A = B + C Expressing Algorithms – Pseudo-Code Simple Algorithms Recursive Algorithms Time Complexity of Algorithms. Algorithms.

reed-boyd
Download Presentation

Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Algorithms • Readings: [SG] Ch. 2 & 3 • Chapter Outline: • Chapter Goals • What are Algorithms • Real Life Examples (origami, recipes) • Definition • Example: A = B + C • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms

  2. Algorithms • Computing devices are dumb • How to explain to a dumb mechanical / computing device how to solve a problem • How to solve a problem using “a small, basic set of primitive instructions”. • Complexity of Solving Problems.

  3. 1. Goals of Algorithm Study • To develop framework for instructing computer to perform tasks • To introduce notion of algorithm as means of specifying how to solve a problem • To introduce and appreciate approaches for defining and solving very complex tasks in terms of simpler tasks;

  4. Chapter Outline: • Chapter Goals • What are Algorithms • Real Life Examples (origami, recipes) • Definition of Algorithm • Example: A = B + C • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms

  5. 2. Computer Science and Algorithms… • Computer Science can be considered… as study of algorithms including • their formal properties • Their hardware and software realisations • Their applications to diverse areas

  6. Algorithms: Real Life Examples • Many Real-Life Examples • Cooking: Recipe for preparing a dish • Origami: The Art of Paper Folding • Directions: How to go to Changi Airport • Remember: • Framework: How to give instructions; • Alg: The actual step-by-step instructions; • Abstraction: Decomposing / Simplifying

  7. Real Life Example: Cooking • Framework: “Cooking or Recipe language” • Algorithm: “Recipe for a Dish” (step by step instructions on how to cook a dish) • Problem Decomposition • Preparing Ingredients; • Preparing Sauce;

  8. Real Life Example: Origami • Framework: “Origami or Paper-Folding language” • Algorithm: “Sequence of Paper-Folding Instructions” (step by step instructions for each fold) • Problem Decomposition • Start with a Bird Base; … • Finish the Head; • Finish the Legs; Finish the Tail;

  9. Real Life Examples: Issues • Problems/Difficulties: • Imprecise Instructions; • Job can often be done even if instructions are not followed precisely • Modifications may be done by the person following the instructions; • But, NOT for a Computer • Needs to told PRECISELY what to do; • Instructions must be PRECISE; • Cannot be vague or ambiguous

  10. Definition of Algorithm: • An algorithm for solving a problem “a finite sequence of unambiguous, executable steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”. • Note the keywords: • Finite set of steps; • Unambiguous; • Executable; • Terminates;

  11. Algorithm Examples? • Problem 1: What is the largest integer INPUT: All the integers { … -2, -1, 0, 1, 2, … } OUTPUT: The largest integer Algorithm: • Arrange all the integers in a list in decreasing order; • MAX = first number in the list; • Print out MAX; • WHY is the above NOT an Algorithm? • (Hint: How many integers are there?) • Problem 2: Who is the tallest women in the world? • Algorithm: Tutorial...

  12. Example: Adding two (n-digit) numbers • Input: Two positive m-digit decimal numbers (a and b) am, am-1, …., a1 bm, bm-1, …., b1 • Output: The sum c = a + b cm+1, cm, cm-1, …., c1 (Note: In the textbook, it was am-1,…,a1,a0 …)

  13. How to “derive” the algorithm • Adding is something we all know • done it a thousand times, know it “by heart” • How do we give the algorithm? • A step-by-step instruction • to a dumb machine • Try an example: 3 4 9 2 8 1 5 7 “Imagine you looking at yourself solving it”

  14. Step 1: Set the value of carry to 0 Step 2: Set the value of i to 1. Step 3: Repeat steps 4 through 6 until the value of i is > m. Step 4: Add ai and bi to the current value of carry, to get xi. Step 5: If xi < 10 then let ci=xi and reset carry to 0. Else (* i.e. xi 10 *) let ci=xi - 10 and reset carry to 1. Step 6: Increase the value of i by 1. Step 7: Set cm+1 to the value of carry. Step 8: Print the final answer cm+1, cm, …., c1 Step 9: Stop. Algorithm: Finding sum of A & B

  15. Chapter Outline: • Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Communicating Alg to computer • Pseudo-Code • Primitive Operations and examples • Variables and Arrays • Algorithm C=A+B in pseudo-code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms

  16. 3. Expressing Algorithms to Computer • To communicate algorithm to computer • Need way to “represent” the algorithm • Cannot use English • Can use computer language • machine language and • programming languages (Java, Pascal, C) • But, these are too tedious (&technical) • Use Pseudo-Code instead…

  17. Pseudo-Code to express Algorithms • Pseudo-Code • Mixture of computer language and English • Somewhere in between • precise enough to describe what is meant without being too tediuos • Examples: • Let c be 0; • Sort the list of numbers in increasing order; • Need to know both • syntax – representation • semantics – meaning

  18. Primitive Operations • To describe an algorithm, we need some well-defined programming primitives • Assignment primitive: • assignment statement, input/output statements • Conditional primitive: • if statement • case statement • Looping (iterative) primitive: • for loop, • while loop, • Statements are executed one-by-one

  19. true false condition? Step A Step B Conditional Primitives… • if statement • to take different actions based on condition • Syntax if (condition) then (Step A) else (Step B) endif if (condition) then (Step A) endif • Semantics

  20. Examples -- (if statement)… Let mark be the total-mark obtained if (mark < 40) then (print “Student fail”) else (print “Student pass”) endif … read in mark (*from the terminal*) if (mark < 40) then (Grade  “F”) else if (mark < 50) then (Grade  “D”) else if (mark < 60) then (Grade  “C”) else if (mark < 70) then (Grade  “B”) else if (mark < 80) then (Grade  “A”); endif print “Student grade is”, Grade …

  21. the while-loop loop a “variable” number of times Syntax while (condition) do (some sequence of statements) endwhile Semantics… false condition? true Some sequence of statements; Looping Primitive – while-loop

  22. First, the for-loop loop a “fixed” or (pre-determined) number of times Syntax for j  a to b do (some sequence of statements) endfor Semantics… j  a; false (j <= b)? true Some sequence of statements; j  j+1; Looping Primitive – for-loop

  23. “Exercising the alg”: for and while for j  1 to 4 do print 2*j; endfor print “--- Done ---” j  1; while (j <= 4) do print 2*j; j  j + 1; endwhile print “--- Done ---” Output: 2 4 6 8 --- Done --- Output: 2 4 6 8 --- Done ---

  24. Variables and Arrays… • In the computer, each variable is assigned a storage “box” • can store one number at any time • eg: sum, j, carry • Arrays: • Often deal with many numbers • Such as A1, A2, A3, … , A100 • Store as an “array” A[1], A[2], … , A[100] • we treat each of them as a variable, • each is assigned a storage “box”

  25. We can re-write the C=A+B algorithm as follows: Alg. to Compute C = A + B: (*sum two big numbers*) carry  0; for i  1 to m do x[i]  a[i] + b[i] + carry ; if (x[i] < 10) then ( c[i] x[i]; carry  0; ) else ( c[i]  x[i] – 10; carry  1; ) endfor; c[m+1] carry; Print c[m+1], c[m], …., c[1] Algorithm: A = B + C (in pseudo-code)

  26. Chapter Outline: • Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Simple iterative algorithms Computing Sum, Find Max/Min • Modular Program Design • Divisibility and Prime Numbers • Recursive Algorithms • Time Complexity of Algorithms

  27. Simple iterative algorithm: Sum • Given: List of numbers: A1, A2, A3, …., An • Output: To compute the sum of the numbers Note: Store numbers in array A[1], A[2], … , A[n] Sum(A, n); begin Sum_sf  0; k  1; while (k <= n) do Sum_sf  Sum_sf + A[k]; k  k + 1; endwhile Sum  Sum_sf; Print “Sum is”, Sum end;

  28. Exercising Algorithm Sum: A[1] A[2] A[3] A[4] A[5] A[6] n=6 2 5 10 3 12 24 Input: k Sum-sf Sum ? 0 ? 1 2 ? 2 7 ? 3 17 ? 4 20 ? 5 32 ? 6 56 ? 6 56 56 Processing: Output: Sum is 56

  29. Algorithm for Sum (with for-loop) • We can also use a while-loop instead of a for loop. Sum(A, n); (* Find the sum of A1, A2,…, An. *) begin Sum_sf  0; for k  1 to n do Sum_sf  Sum_sf + A[k]; endfor Sum  Sum_sf; Print “Sum is”, Sum end; • HW: (a) Note the differences… (b) Modify it to compute the average?

  30. Remarks about the iterative algorithm… • Note the three stages: • Initialization • Set some values at the beginning • Iteration • This is the KEY STEP • Where most of work is done • Post-Processing or Cleanup • Can use this setup for other problems • Calculating average, sum-of-squares • Finding max, min; Searching for a number,

  31. Modular Program Design • Software are complex • HUGE (millions of lines of code) eg: Linux, Outlook • COMPLEX; eg: Flight simulator • Idea: Divide-and-Conquer Method • Complex tasks can be divided and each part solved separately and combined later. • Modular Program Design • Divide big programs into smaller modules • The smaller parts are • called modules, subroutines, or procedures • Design, implement, and test separately • Modularity, Abstraction, Division of Labour • Simplifies process of writing alg/programs

  32. Simple Algorithms: Prime Numbers • Algorithm to determine if n is prime Given: A positive integer n Question: Is n a prime number? • What do we know? “A number n is prime iffn has no positive factors except 1 and n” • Idea: Express it “algorithmically” “n is not divisible by any positive number k such that 1 < k < n.” or k=2,3,4,…,(n-1) • What we already have: • A module Divisible(m,n) to check divisibility • We can use it as a primitive

  33. Pseudo-Code for Prime: • Note: Prime uses the module Divisible(k,n) • Exercise it with: • Prime (5); Prime (4); • Prime (55); Prime (41); Prime(n) (* To determine if n is prime *) begin for k  2 to (n-1) do if Divisible(k,n) then (Print “False” & exit) else (* Do nothing *) endfor Print “True” & Stop end;

  34. A Simple Module: Divisibility • A module (procedure) for divisibility Given: Two positive integers m and n Question: Is n divisible by m? or Algorithm to compute Divisible(m,n) • Algorithm Idea: “A positive integer n is divisible by m iff for some positive integer k  n, m*k = n.”

  35. Module for Divisibility: • Algorithm (in pseudo-code) • Exercise it with: • Divisible (3, 9); Divisible (3, 5); Divisible (3,101); Divisible(m,n) (* To compute if n is divisible by m *) begin D  false; (* assume false, first *) for k  1 to n do if (m*k = n) then (Dtrue; exit-loop) else (* Do nothing *) endfor Divisible  D; (* true or false *) end;

  36. Chapter Outline: • Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms • Sequential search algorithm • Binary search algorithm • Analysis of Time Complexity • Summary…

  37. Search: sequential search algorithm • Given: List of numbers: A1, A2, A3, …., An and a query number x; • Question: Search for x in the list; Sequential-Search(A, n, x); (* Search for x in A1, A2,…, An. *) begin for k  1 to n do if (x = A[k]) then (Print “Yes”; Stop) else (* do nothing *) endfor Print “No”; Stop end;

  38. Remarks on Sequential Search Alg • Analogy: House-to-house search… • How fast is it to search for x? • How many comparisons do we need? • Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55 • When x=14, need 6 comparisons • When x=13, need only 1 comparison  BEST CASE • When x=55, need 10 comparisons  WORST CASE • When x=5, need 10 comparisons  WORST CASE • In general, given n numbers, A1,…,An • Best Case: 1 comparison • Worst Case: n comparisons • Average Case: (n+1)/2 comparisons

  39. Binary Search • If the List is sorted, that is A1 A2  A3  ….  An • Then, we can do better, • actually a lot better….

  40. 1, 4, 9, 11, 14, 43, 78 Binary Search

  41. Binary Search (How fast is it?) • Imagine n=100 • After 1 step, size is  50 • After 2 steps, size is  25 • After 3 steps, size is  12 • After 4 steps, size is  6 • After 5 steps, size is  3 • After 6 steps, size is  1 • DONE!! • What if n=1000? 1,000,000?

  42. Input: Sorted List A1 A2  A3  ….  An A number x. Question: Is x in the list? Binary-Search(A,n,x); 1. First  1 Last  n 2. While ( First  Last ) do mid  (first+last) / 2  If ( x = A[mid] ) then (output Yes and Stop) else If ( x < A[mid] ) then Last  mid-1 else If ( x > A[mid] ) then First mid+1 EndWhile 3. Output False and Stop 4. End Binary Search Algorithm

  43. Time Complexity Analysis • Sequential Search (Alg): • Worst Case: n comparisons • Best Case: 1 comparison • Avg Case: n/2 comparisons • Binary Search (Alg): • Worst Case: log2 n comparisons • Best Case: 1 comparison • Avg Case: about log2 n comparisons • How to get the Average Case? • using mathematical analysis…

  44. Complexity of Algorithm… • Logarithmic Time Algorithm • Binary Search • A Linear Time Algorithm • Algorithm Sum(A,n) -- O(n) time • Algorithm Sequential-Search(A,n,x) – O(n) time • A Quadratic Time Algorithm • Simple Median-Find (T2-Q3) • An Exponential Time Algorithm • All-Subsets(A,n)

  45. Chapter Outline: • Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Recursion – the idea • Fibonacci sequence • Tower of Hanoi • Time Complexity of Algorithms

  46. 5. Recursion • A problem solving method of “decomposing bigger problems into smaller sub-problems that are identical to itself.” • General Idea: • Solve simplest (smallest) cases DIRECTLY • usually these are very easy to solve • Solve bigger problems using smaller sub-problems • that are identical to itself (but smaller and simpler) • Abstraction: • To solve a given problem, we first assume that we ALREADY know how to solve it for smaller instances!! • Dictionary definition: recursion see recursion

  47. Example: Fibonacci Numbers… • Definition of Fibonacci numbers • F1 = 1, • F2 = 1, • for n>2, Fn = Fn-1 + Fn-2 • Problem: Compute Fn for any n. • The above is a recursive definition. • Fn is computed in-terms of itself • actually, smaller copies of itself – Fn-1 and Fn-2 • Actually, Not difficult: F3 = 1 + 1 = 2 F6 = 5 + 3 = 8 F9 = 21 + 13 = 34 F4 = 2 + 1 = 3 F7 = 8 + 5 = 13 F10 = 34 + 21 = 55 F5 = 3 + 2 = 5 F8 = 13 + 8 = 21 F11 = 55 + 34 = 89 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …

  48. Fibonacci Numbers: Recursive alg • The above is a recursive algorithm • It is simple to understand and elegant! • But, very SLOW Fibonacci(n) (* Recursive, SLOW *) begin if (n=1) or (n=2) then Fibonacci(n)  1 (*simple case*) else Fibonacci(n)  Fibonacci(n-1) + Fibonacci(n-2) endif end;

  49. F(6) F(5) F(4) F(4) F(3) F(3) F(2) F(2) F(3) F(2) F(2) F(1) F(1) F(2) F(1) Recursive Fibonacci Alg -- Remarks • How slow is it? • Eg: To compute F(6)… • HW: Can we compute it faster?

  50. Given: Three Pegs A, B and C Peg A initially has n disks, different size, stacked up, larger disks are below smaller disks Problem: to move the n disks to Peg C, subject to Can move only one disk at a time Smaller disk should be above larger disk Can use other peg as intermediate B B A A C C Example: Tower of Hanoi

More Related