- 66 Views
- Uploaded on
- Presentation posted in: General

Theory of Algorithms: Introduction

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Theory of Algorithms:Introduction

James Gain and Edwin Blake

{jgain | edwin} @cs.uct.ac.za

Department of Computer Science

University of Cape Town

August - October 2004

- To define an algorithm
- To introduce:
- Problem types
- The Process of Algorithm Design
- Solution strategies
- Ways of Analysing Algorithms

- To cover the structure of the course, including practicals

- An algorithm is a sequence of unambiguous instructions for solving a problem
- For obtaining a required output for any legitimate input in a finite amount of time
- Does not require implementation in software
- Not an answer but a method for deriving an answer

- Historical Perspective:
- Named after Muhammad ibn Musa al-Khwarizmi – 9th century mathematician
- www.lib.virginia.edu/science/parshall/khwariz.html

problem

algorithm

“computer”

input

output

- Each step of the algorithm must be unambiguous
- The range of inputs must be specified carefully
- The same algorithm can be represented in different ways
- Several algorithms for solving the same problem may exist - with different properties

- Recipe, process, method, technique, procedure, routine,… with following requirements:
- Finiteness
- Terminates after a finite number of steps

- Definiteness
- Rigorously and unambiguously specified

- Input
- Valid inputs are clearly specified

- Output
- Can be proved to produce the correct output given a valid input

- Effectiveness
- Steps are sufficiently simple and basic

- Statement of problem:
- Input: A sequence of n numbers <a1, a2, …, an>
- Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤ a´j whenever i < j

- Instance: The sequence <5, 3, 2, 8, 3>
- Algorithms:
- Selection sort
- Insertion sort
- Merge sort
- (many others)

for i=1 to n

swap a[i] with smallest of a[i],…,a[n]

- Input: array a[1],..,a[n]
- Output: array a sorted in non-decreasing order
- Algorithm:

- for i1 to ndo
- min i
- for j i+1 to n do
- if a[j] < a[min] min j

- swap a[i] and a[min]

- Problem:
- 4 People want to cross a bridge. You have 17 minutes to get them across

- Constraints:
- It is night and you have 1 flashlight. Max of 2 on the bridge at one time. All start on the same side
- Those crossing must have the flashlight with them. The flashlight must be walked back and forth (no throwing)
- People walk at different speeds: person A = 1 minute to cross, person B = 2 minutes, person C = 5 minutes, person D = 10 minutes
- A pair walks at the speed of the slower person’s pace

- Rumour: this problem is given to Microsoft interviewees

- Start (0 min): A B C D
- AB Across (2 min): A B C D
- A Back (1 min): B A C D
- CD Across (10 min): B C D A
- B Back (2 min): C D A B
- AB Across (2 min): A B C D
- Total Time = 17 minutes

- This is an instance of a problem. How would you generalise it?
- Can you derive an algorithm to solve this generalised problem?
- Must show the sequence of moves
- Must output the minimum time required for crossing
- Are there any special cases to watch out for?
- Are there any constraints on the input?

- Input: a list a of crossing times for n people, numbered 1, …, n
- Output: total time to cross
- Strategy: use 1 & 2 as shuttles and send the others across in pairs

- for i 2 to n/2do
- t a[2] // 1 & 2 across
- t t + a[1] // 1 back
- t t + a[i*2] // i*2 & (i*2)-1 across
- t t + a[2] // 2 back

- t a[2] // 1 & 2 across
- return t

- This is an inadequate solution
- It falsely assumes certain inputs
- List may not be sorted in ascending order
- Sort a

- n may not be even numbered
- Alter final iteration of loop

- n > 3 not guaranteed
- Special case for n = 1, 2, 3

- Is not optimal for all inputs, e.g. 1, 20, 21, 22
- Can you quantify the nature of these inputs? Suggest an alternative.

- Final solution is left as an exercise. Attempt to make your solution elegant

- Understanding the Problem
- Make sure you are solving the correct problem and for all legitimate inputs

- Ascertaining the Capabilities of a Computational Device
- Sequential vs. Parallel.
- What are the speed and memory limits?

- Choosing between exact and approximate Problem Solving
- Is absolute precision required? Sometimes this may not be possible

- Deciding on Appropriate Data Structures
- Algorithms often rely on carefully structuring the data
- Fundamental Data Structures: array, linked list, stacks, queues, heaps, graphs, trees, sets

- Applying an Algorithm Design Technique
- Using a general approach to problem solving that is applicable to a variety of problems

- Specifying the Algorithm
- Pseudocode is a mixture of natural language and programming constructs that has replaced flowcharts

- Proving an Algorithms Correctness
- Prove that an algorithm yields a required result for legitimate inputs in finite time

- Analyzing an Algorithm
- Consider time efficiency, space efficiency, simplicity, generality, optimality
- Analysis can be empirical or theoretical

- Coding an Algorithm

- Sorting
- Searching
- String Processing
- String Matching

- Graph Problems
- Graph Traversal, Shortest Path, Graph Colouring

- Combinatorial Problems
- Find a combinatorial object - permutation, combination, subset - subject to constraints

- Geometric Problems
- Closest-Pair, Convex-Hull

- Numerical Problems
- Solving systems of equations, computing definite integrals, evaluating functions, etc.

- Brute force
- A straightforward approach to solving a problem, usually directly based on the problem’s statement

- Divide and conquer
- Divide a problem into smaller instances, solve smaller instances (perhaps recursively), combine

- Decrease and conquer
- Exploit relationship between the problem and a smaller instance reduced by some factor (often 1)

- Transform and conquer
- Transform the problem to a simpler instance, another representation or an instance with a known solution

- Greedy approach
- Make locally optimal steps which (hopefully) lead to a globally optimal solution for an optimization problem

- Dynamic programming
- Technique for solving problems with overlapping sub-domains

- Backtracking and Branch and bound
- A way of tackling difficult optimization and combinatorial problems without exploring all state-space

- Space and time tradeoffs
- Preprocess the input and store additional information to accelerate solving the problem

- Taken from G. Polya, “How to Solve It”, 2nd edition. A classic textbook on problem solving for mathematics
- You have to understand the problem.
- What is the unknown? What are the data? Is the problem statement sufficient, redundant, contradictory
- Draw a figure. Introduce suitable notation
- Separate the various parts of the problem. Can you write them down?

- Find the connection between the data and the unknown. You may be obliged to consider auxiliary problems if an immediate connection cannot be found. You should obtain eventually a plan of the solution.
- Have you seen it before? Or have you seen the same problem in a slightly different form?
- Do you know a related problem? Do you know a theorem that could be useful?
- Look at the unknown! And try to think of a familiar problem having the same or a similar unknown.
- Could you restate the problem? Could you restate it still differently? Go back to definitions.
- If you cannot solve the proposed problem try to solve first some related problem. Are the unknown and the new data nearer to each other?
- Did you use all the data? Did you use the whole condition? Have you taken into account all essential notions involved in the problem?

- Carry out the Plan
- Carrying out your plan of the solution, check each step. Can you see clearly that the step is correct? Can you prove that it is correct?

- Looking Back
- Can you check the result? Can you check the argument?
- Can you derive the solution differently? Can you see it at a glance?
- Can you use the result, or the method, for some other problem?

- How good is the algorithm?
- Correctness
- Time efficiency
- Space efficiency
- Simplicity

- Does there exist a better algorithm?
- Lower bounds
- Optimality

- Theoretical importance
- The core of computer science

- Practical importance
- A practitioner’s toolkit of known algorithms
- Framework for designing and analyzing algorithms for new problems
- Useful mindset

- Fundamentals of the Analysis of Algorithms (Ch. 2)
- Asymptotic notations, analysis of recursive and non-recursive algorithms, empirical analysis

- Algorithmic Strategies (Ch. 3-9)
- Brute force, Divide-and-Conquer, Decrease-and-Conquer, Transform-and-Conquer, Space and Time Tradeoffs, Greedy Techniques, Biologically-inspired techniques, Dynamic Programming

- Limitations of Algorithms (Ch. 10 + handouts)
- Turing Machines, Computability, Problem Classification

- Coping with Limitations on Algorithms (Ch. 11)
- Backtracking and Branch and Bound

- Anany Levitin, “Introduction to the Design and Analysis of Algorithms”, International Edition, Addison-Wesley, 2003

- Weekly mini prac exams
- Given a problem specification that is solvable using the algorithm design strategies presented in the course
- Design Algorithm
- Code it in C++
- Submit it for automatic marking

- After the 3-hour lab session will be asked to do a short analysis of the solution