theory of algorithms introduction
Skip this Video
Download Presentation
Theory of Algorithms: Introduction

Loading in 2 Seconds...

play fullscreen
1 / 24

Theory of Algorithms: Introduction - PowerPoint PPT Presentation

  • Uploaded on

Theory of Algorithms: Introduction. James Gain and Edwin Blake {jgain | edwin} Department of Computer Science University of Cape Town August - October 2004. Objectives. To define an algorithm To introduce: Problem types The Process of Algorithm Design Solution strategies

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Theory of Algorithms: Introduction' - clea

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
theory of algorithms introduction

Theory of Algorithms:Introduction

James Gain and Edwin Blake

{jgain | edwin}

Department of Computer Science

University of Cape Town

August - October 2004

  • To define an algorithm
  • To introduce:
    • Problem types
    • The Process of Algorithm Design
    • Solution strategies
    • Ways of Analysing Algorithms
  • To cover the structure of the course, including practicals
  • An algorithm is a sequence of unambiguous instructions for solving a problem
    • For obtaining a required output for any legitimate input in a finite amount of time
    • Does not require implementation in software
    • Not an answer but a method for deriving an answer
  • Historical Perspective:
    • Named after Muhammad ibn Musa al-Khwarizmi – 9th century mathematician
notion of algorithm
Notion of algorithm






  • Each step of the algorithm must be unambiguous
  • The range of inputs must be specified carefully
  • The same algorithm can be represented in different ways
  • Several algorithms for solving the same problem may exist - with different properties
what is an algorithm
What is an algorithm?
  • Recipe, process, method, technique, procedure, routine,… with following requirements:
  • Finiteness
    • Terminates after a finite number of steps
  • Definiteness
    • Rigorously and unambiguously specified
  • Input
    • Valid inputs are clearly specified
  • Output
    • Can be proved to produce the correct output given a valid input
  • Effectiveness
    • Steps are sufficiently simple and basic
example sorting
Example: Sorting
  • Statement of problem:
    • Input: A sequence of n numbers <a1, a2, …, an>
    • Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤ a´j whenever i < j
  • Instance: The sequence <5, 3, 2, 8, 3>
  • Algorithms:
    • Selection sort
    • Insertion sort
    • Merge sort
    • (many others)
selection sort

for i=1 to n

swap a[i] with smallest of a[i],…,a[n]

Selection Sort
  • Input: array a[1],..,a[n]
  • Output: array a sorted in non-decreasing order
  • Algorithm:
  • for i1 to ndo
    • min  i
    • for j  i+1 to n do
      • if a[j] < a[min] min  j
    • swap a[i] and a[min]
exercise bridge puzzle
Exercise: Bridge Puzzle
  • Problem:
    • 4 People want to cross a bridge. You have 17 minutes to get them across
  • Constraints:
    • It is night and you have 1 flashlight. Max of 2 on the bridge at one time. All start on the same side
    • Those crossing must have the flashlight with them. The flashlight must be walked back and forth (no throwing)
    • People walk at different speeds: person A = 1 minute to cross, person B = 2 minutes, person C = 5 minutes, person D = 10 minutes
    • A pair walks at the speed of the slower person’s pace
  • Rumour: this problem is given to Microsoft interviewees
solution bridge puzzle
Solution: Bridge Puzzle
  • Start (0 min): A B C D
  • AB Across (2 min): A B C D
  • A Back (1 min): B A C D
  • CD Across (10 min): B C D A
  • B Back (2 min): C D A B
  • AB Across (2 min): A B C D
  • Total Time = 17 minutes
extension exercise
Extension Exercise
  • This is an instance of a problem. How would you generalise it?
  • Can you derive an algorithm to solve this generalised problem?
    • Must show the sequence of moves
    • Must output the minimum time required for crossing
    • Are there any special cases to watch out for?
    • Are there any constraints on the input?
extension solution
Extension Solution
  • Input: a list a of crossing times for n people, numbered 1, …, n
  • Output: total time to cross
  • Strategy: use 1 & 2 as shuttles and send the others across in pairs
  • for i 2 to n/2do
    • t  a[2] // 1 & 2 across
    • t  t + a[1] // 1 back
    • t  t + a[i*2] // i*2 & (i*2)-1 across
    • t  t + a[2] // 2 back
  • t  a[2] // 1 & 2 across
  • return t
extension problems
Extension Problems
  • This is an inadequate solution
  • It falsely assumes certain inputs
  • List may not be sorted in ascending order
    • Sort a
  • n may not be even numbered
    • Alter final iteration of loop
  • n > 3 not guaranteed
    • Special case for n = 1, 2, 3
  • Is not optimal for all inputs, e.g. 1, 20, 21, 22
    • Can you quantify the nature of these inputs? Suggest an alternative.
  • Final solution is left as an exercise. Attempt to make your solution elegant
fundamentals of algorithmic problem solving
Fundamentals of Algorithmic Problem Solving
  • Understanding the Problem
    • Make sure you are solving the correct problem and for all legitimate inputs
  • Ascertaining the Capabilities of a Computational Device
    • Sequential vs. Parallel.
    • What are the speed and memory limits?
  • Choosing between exact and approximate Problem Solving
    • Is absolute precision required? Sometimes this may not be possible
  • Deciding on Appropriate Data Structures
    • Algorithms often rely on carefully structuring the data
    • Fundamental Data Structures: array, linked list, stacks, queues, heaps, graphs, trees, sets
fundamentals of algorithm design
Fundamentals of Algorithm Design
  • Applying an Algorithm Design Technique
    • Using a general approach to problem solving that is applicable to a variety of problems
  • Specifying the Algorithm
    • Pseudocode is a mixture of natural language and programming constructs that has replaced flowcharts
  • Proving an Algorithms Correctness
    • Prove that an algorithm yields a required result for legitimate inputs in finite time
  • Analyzing an Algorithm
    • Consider time efficiency, space efficiency, simplicity, generality, optimality
    • Analysis can be empirical or theoretical
  • Coding an Algorithm
well known computational problems
Well known Computational Problems
  • Sorting
  • Searching
  • String Processing
    • String Matching
  • Graph Problems
    • Graph Traversal, Shortest Path, Graph Colouring
  • Combinatorial Problems
    • Find a combinatorial object - permutation, combination, subset - subject to constraints
  • Geometric Problems
    • Closest-Pair, Convex-Hull
  • Numerical Problems
    • Solving systems of equations, computing definite integrals, evaluating functions, etc.
algorithm design strategies
Algorithm Design Strategies
  • Brute force
    • A straightforward approach to solving a problem, usually directly based on the problem’s statement
  • Divide and conquer
    • Divide a problem into smaller instances, solve smaller instances (perhaps recursively), combine
  • Decrease and conquer
    • Exploit relationship between the problem and a smaller instance reduced by some factor (often 1)
  • Transform and conquer
    • Transform the problem to a simpler instance, another representation or an instance with a known solution
more algorithm design strategies
More Algorithm Design Strategies
  • Greedy approach
    • Make locally optimal steps which (hopefully) lead to a globally optimal solution for an optimization problem
  • Dynamic programming
    • Technique for solving problems with overlapping sub-domains
  • Backtracking and Branch and bound
    • A way of tackling difficult optimization and combinatorial problems without exploring all state-space
  • Space and time tradeoffs
    • Preprocess the input and store additional information to accelerate solving the problem
how to solve it understanding the problem
How to Solve It: Understanding the Problem
  • Taken from G. Polya, “How to Solve It”, 2nd edition. A classic textbook on problem solving for mathematics
  • You have to understand the problem.
    • What is the unknown? What are the data? Is the problem statement sufficient, redundant, contradictory
    • Draw a figure. Introduce suitable notation
    • Separate the various parts of the problem. Can you write them down?
devising a plan
Devising a Plan
  • Find the connection between the data and the unknown. You may be obliged to consider auxiliary problems if an immediate connection cannot be found. You should obtain eventually a plan of the solution.
  • Have you seen it before? Or have you seen the same problem in a slightly different form?
  • Do you know a related problem? Do you know a theorem that could be useful?
  • Look at the unknown! And try to think of a familiar problem having the same or a similar unknown.
  • Could you restate the problem? Could you restate it still differently? Go back to definitions.
  • If you cannot solve the proposed problem try to solve first some related problem. Are the unknown and the new data nearer to each other?
  • Did you use all the data? Did you use the whole condition? Have you taken into account all essential notions involved in the problem?
carrying it through
Carrying it Through
  • Carry out the Plan
    • Carrying out your plan of the solution, check each step. Can you see clearly that the step is correct? Can you prove that it is correct?
  • Looking Back
    • Can you check the result? Can you check the argument?
    • Can you derive the solution differently? Can you see it at a glance?
    • Can you use the result, or the method, for some other problem?
analysis of algorithms
Analysis of Algorithms
  • How good is the algorithm?
    • Correctness
    • Time efficiency
    • Space efficiency
    • Simplicity
  • Does there exist a better algorithm?
    • Lower bounds
    • Optimality
why study algorithms
Why Study Algorithms?
  • Theoretical importance
    • The core of computer science
  • Practical importance
    • A practitioner’s toolkit of known algorithms
    • Framework for designing and analyzing algorithms for new problems
    • Useful mindset
course structure
Course Structure
  • Fundamentals of the Analysis of Algorithms (Ch. 2)
    • Asymptotic notations, analysis of recursive and non-recursive algorithms, empirical analysis
  • Algorithmic Strategies (Ch. 3-9)
    • Brute force, Divide-and-Conquer, Decrease-and-Conquer, Transform-and-Conquer, Space and Time Tradeoffs, Greedy Techniques, Biologically-inspired techniques, Dynamic Programming
  • Limitations of Algorithms (Ch. 10 + handouts)
    • Turing Machines, Computability, Problem Classification
  • Coping with Limitations on Algorithms (Ch. 11)
    • Backtracking and Branch and Bound
  • Anany Levitin, “Introduction to the Design and Analysis of Algorithms”, International Edition, Addison-Wesley, 2003
  • Weekly mini prac exams
  • Given a problem specification that is solvable using the algorithm design strategies presented in the course
    • Design Algorithm
    • Code it in C++
    • Submit it for automatic marking
  • After the 3-hour lab session will be asked to do a short analysis of the solution