Theory of Algorithms: Introduction

1 / 24

# Theory of Algorithms: Introduction - PowerPoint PPT Presentation

Theory of Algorithms: Introduction. James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004. Objectives. To define an algorithm To introduce: Problem types The Process of Algorithm Design Solution strategies

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Theory of Algorithms: Introduction' - clea

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Theory of Algorithms:Introduction

James Gain and Edwin Blake

{jgain | edwin} @cs.uct.ac.za

Department of Computer Science

University of Cape Town

August - October 2004

Objectives
• To define an algorithm
• To introduce:
• Problem types
• The Process of Algorithm Design
• Solution strategies
• Ways of Analysing Algorithms
• To cover the structure of the course, including practicals
Definitions
• An algorithm is a sequence of unambiguous instructions for solving a problem
• For obtaining a required output for any legitimate input in a finite amount of time
• Does not require implementation in software
• Historical Perspective:
• Named after Muhammad ibn Musa al-Khwarizmi – 9th century mathematician
• www.lib.virginia.edu/science/parshall/khwariz.html
Notion of algorithm

problem

algorithm

“computer”

input

output

• Each step of the algorithm must be unambiguous
• The range of inputs must be specified carefully
• The same algorithm can be represented in different ways
• Several algorithms for solving the same problem may exist - with different properties
What is an algorithm?
• Recipe, process, method, technique, procedure, routine,… with following requirements:
• Finiteness
• Terminates after a finite number of steps
• Definiteness
• Rigorously and unambiguously specified
• Input
• Valid inputs are clearly specified
• Output
• Can be proved to produce the correct output given a valid input
• Effectiveness
• Steps are sufficiently simple and basic
Example: Sorting
• Statement of problem:
• Input: A sequence of n numbers <a1, a2, …, an>
• Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤ a´j whenever i < j
• Instance: The sequence <5, 3, 2, 8, 3>
• Algorithms:
• Selection sort
• Insertion sort
• Merge sort
• (many others)

for i=1 to n

swap a[i] with smallest of a[i],…,a[n]

Selection Sort
• Input: array a[1],..,a[n]
• Output: array a sorted in non-decreasing order
• Algorithm:
• for i1 to ndo
• min  i
• for j  i+1 to n do
• if a[j] < a[min] min  j
• swap a[i] and a[min]
Exercise: Bridge Puzzle
• Problem:
• 4 People want to cross a bridge. You have 17 minutes to get them across
• Constraints:
• It is night and you have 1 flashlight. Max of 2 on the bridge at one time. All start on the same side
• Those crossing must have the flashlight with them. The flashlight must be walked back and forth (no throwing)
• People walk at different speeds: person A = 1 minute to cross, person B = 2 minutes, person C = 5 minutes, person D = 10 minutes
• A pair walks at the speed of the slower person’s pace
• Rumour: this problem is given to Microsoft interviewees
Solution: Bridge Puzzle
• Start (0 min): A B C D
• AB Across (2 min): A B C D
• A Back (1 min): B A C D
• CD Across (10 min): B C D A
• B Back (2 min): C D A B
• AB Across (2 min): A B C D
• Total Time = 17 minutes
Extension Exercise
• This is an instance of a problem. How would you generalise it?
• Can you derive an algorithm to solve this generalised problem?
• Must show the sequence of moves
• Must output the minimum time required for crossing
• Are there any special cases to watch out for?
• Are there any constraints on the input?
Extension Solution
• Input: a list a of crossing times for n people, numbered 1, …, n
• Output: total time to cross
• Strategy: use 1 & 2 as shuttles and send the others across in pairs
• for i 2 to n/2do
• t  a[2] // 1 & 2 across
• t  t + a[1] // 1 back
• t  t + a[i*2] // i*2 & (i*2)-1 across
• t  t + a[2] // 2 back
• t  a[2] // 1 & 2 across
• return t
Extension Problems
• This is an inadequate solution
• It falsely assumes certain inputs
• List may not be sorted in ascending order
• Sort a
• n may not be even numbered
• Alter final iteration of loop
• n > 3 not guaranteed
• Special case for n = 1, 2, 3
• Is not optimal for all inputs, e.g. 1, 20, 21, 22
• Can you quantify the nature of these inputs? Suggest an alternative.
• Final solution is left as an exercise. Attempt to make your solution elegant
Fundamentals of Algorithmic Problem Solving
• Understanding the Problem
• Make sure you are solving the correct problem and for all legitimate inputs
• Ascertaining the Capabilities of a Computational Device
• Sequential vs. Parallel.
• What are the speed and memory limits?
• Choosing between exact and approximate Problem Solving
• Is absolute precision required? Sometimes this may not be possible
• Deciding on Appropriate Data Structures
• Algorithms often rely on carefully structuring the data
• Fundamental Data Structures: array, linked list, stacks, queues, heaps, graphs, trees, sets
Fundamentals of Algorithm Design
• Applying an Algorithm Design Technique
• Using a general approach to problem solving that is applicable to a variety of problems
• Specifying the Algorithm
• Pseudocode is a mixture of natural language and programming constructs that has replaced flowcharts
• Proving an Algorithms Correctness
• Prove that an algorithm yields a required result for legitimate inputs in finite time
• Analyzing an Algorithm
• Consider time efficiency, space efficiency, simplicity, generality, optimality
• Analysis can be empirical or theoretical
• Coding an Algorithm
Well known Computational Problems
• Sorting
• Searching
• String Processing
• String Matching
• Graph Problems
• Graph Traversal, Shortest Path, Graph Colouring
• Combinatorial Problems
• Find a combinatorial object - permutation, combination, subset - subject to constraints
• Geometric Problems
• Closest-Pair, Convex-Hull
• Numerical Problems
• Solving systems of equations, computing definite integrals, evaluating functions, etc.
Algorithm Design Strategies
• Brute force
• A straightforward approach to solving a problem, usually directly based on the problem’s statement
• Divide and conquer
• Divide a problem into smaller instances, solve smaller instances (perhaps recursively), combine
• Decrease and conquer
• Exploit relationship between the problem and a smaller instance reduced by some factor (often 1)
• Transform and conquer
• Transform the problem to a simpler instance, another representation or an instance with a known solution
More Algorithm Design Strategies
• Greedy approach
• Make locally optimal steps which (hopefully) lead to a globally optimal solution for an optimization problem
• Dynamic programming
• Technique for solving problems with overlapping sub-domains
• Backtracking and Branch and bound
• A way of tackling difficult optimization and combinatorial problems without exploring all state-space
• Preprocess the input and store additional information to accelerate solving the problem
How to Solve It: Understanding the Problem
• Taken from G. Polya, “How to Solve It”, 2nd edition. A classic textbook on problem solving for mathematics
• You have to understand the problem.
• What is the unknown? What are the data? Is the problem statement sufficient, redundant, contradictory
• Draw a figure. Introduce suitable notation
• Separate the various parts of the problem. Can you write them down?
Devising a Plan
• Find the connection between the data and the unknown. You may be obliged to consider auxiliary problems if an immediate connection cannot be found. You should obtain eventually a plan of the solution.
• Have you seen it before? Or have you seen the same problem in a slightly different form?
• Do you know a related problem? Do you know a theorem that could be useful?
• Look at the unknown! And try to think of a familiar problem having the same or a similar unknown.
• Could you restate the problem? Could you restate it still differently? Go back to definitions.
• If you cannot solve the proposed problem try to solve first some related problem. Are the unknown and the new data nearer to each other?
• Did you use all the data? Did you use the whole condition? Have you taken into account all essential notions involved in the problem?
Carrying it Through
• Carry out the Plan
• Carrying out your plan of the solution, check each step. Can you see clearly that the step is correct? Can you prove that it is correct?
• Looking Back
• Can you check the result? Can you check the argument?
• Can you derive the solution differently? Can you see it at a glance?
• Can you use the result, or the method, for some other problem?
Analysis of Algorithms
• How good is the algorithm?
• Correctness
• Time efficiency
• Space efficiency
• Simplicity
• Does there exist a better algorithm?
• Lower bounds
• Optimality
Why Study Algorithms?
• Theoretical importance
• The core of computer science
• Practical importance
• A practitioner’s toolkit of known algorithms
• Framework for designing and analyzing algorithms for new problems
• Useful mindset
Course Structure
• Fundamentals of the Analysis of Algorithms (Ch. 2)
• Asymptotic notations, analysis of recursive and non-recursive algorithms, empirical analysis
• Algorithmic Strategies (Ch. 3-9)
• Brute force, Divide-and-Conquer, Decrease-and-Conquer, Transform-and-Conquer, Space and Time Tradeoffs, Greedy Techniques, Biologically-inspired techniques, Dynamic Programming
• Limitations of Algorithms (Ch. 10 + handouts)
• Turing Machines, Computability, Problem Classification
• Coping with Limitations on Algorithms (Ch. 11)
• Backtracking and Branch and Bound
• Anany Levitin, “Introduction to the Design and Analysis of Algorithms”, International Edition, Addison-Wesley, 2003
Practicals
• Weekly mini prac exams
• Given a problem specification that is solvable using the algorithm design strategies presented in the course
• Design Algorithm
• Code it in C++
• Submit it for automatic marking
• After the 3-hour lab session will be asked to do a short analysis of the solution