Design & Analysis of Algorithms Introduction

1. Design & Analysis of AlgorithmsIntroduction

2. Introduction • Algorithms are the ideas behind computer programs. • An algorithm is the thing which stays the same whether the program is in C++ running on a PC in New York or is in Android running on a Smartphone in Lahore!

3. What is Algorithm? Algorithm Input output • Sequence of steps that can be taken to solve a problem • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. • An Algorithm is well defined computational procedure that takes some value, or set of values, as inputand produces some value, or set of values as output. • More generally, an Algorithm is any well defined computational procedure that takes collection of elements as inputand produces a collection of elements as output.

4. Properties of Algorithms It must be correct. It must be composed of a series of concrete steps. The execution sequence of instructions should not be ambiguous. It must have finite number of instructions and steps. It mustterminate. 4

5. Syntax & Semantics WARNINGS: 1. An algo. can be syntactically correct, yet semantically incorrect – very dangerous situation! 2. Syntactic correctness is easier to check as compared with semantic An algo. is “correct” if its: • Semantics are correct • Syntax is correct Semantics: The concept embedded in an algorithm (the soul!) Syntax: The actual representation of an algorithm (the body!)

6. Solving Problems (1) When faced with a problem: • We first clearly define the problem • Think of possible solutions • Select the one that we think is the best under the prevailing circumstances • And then apply that solution • If the solution woks as desired, fine; else we go back to step 2

7. Early History:Search for a Generic Algorithm • The study of algorithms began withmathematicians and was a significant area of work in the early years • The goal of those early studies was to find a single, general algorithm that could solve all problems of a single type

8. Origin of the Term “Algorithm” • The name derives from the title of a Latin book: Algoritmi de numero Indorum • That book was a translation of an Arabic book: Al-Khwarizmi Concerning the Hindu Art of Reckoning • That book was written by the famous 9-th century Muslim mathematician, Muhammad ibn Musa al-Khwarizmi

9. Al-Khwarzmi • Al-Khwarizmi lived in Baghdad, where he worked at the Dar al-Hikma • Dar al-Hikma acquired and translated books on science and philosophy, particularly those in Greek, as well as publishing original research • The word Algebra has its origins in the title of another Latin book which was a translation of yet another book written by Al-Khwarzmi: Kitab al-Mukhtasar fi Hisab al-Jabr wa'l-Muqabala

10. Al-Khwarizmi’s Golden Principle All complex problems can be and must be solved using the following simple steps: • Break down the problem into small, simple sub-problems • Arrange the sub-problems in such an order that each of them can be solved without effecting any other • Solve them separately, in the correct order • Combine the solutions of the sub-problems to form the solution of the original problem

11. Algorithms to Programs Algorithm: A sequence of instructions describing how to do a task Problem High Level Language Program 11

12. Difference b/w Algorithm and a program • Algorithm • It is a method followed to solve a problem. • A mapping of input to output. As mentioned on slide no. 2. • A problem can have many algorithms • Computer program • It is a concrete representation of an algorithm in some programming language • A set of instructions which the computer will follow to solve a problem 12

13. Popular Algorithms, Factors of Dependence • Most basic and popular algorithms are • Sorting algorithms • Searching algorithms Which algorithm is best? • Mainly, it depends upon various factors, for example in case of sorting • The number of items to be sorted • The extent to which the items are already sorted • Possible restrictions on the item values • The kind of storage device to be used etc.

14. One Problem, Many Algorithms Problem • The statement of the problem specifies, in general terms, the desired input/output relationship. Algorithm • The algorithm describes a specific computational procedure for achieving input/output relationship. Example • One might need to sort a sequence of numbers into non-decreasing order. Algorithms • Various algorithms e.g. merge sort, quick sort, heap sorts etc.

15. Important Designing Techniques • Brute Force • Straightforward, naive approach • Mostly expensive • Divide-and-Conquer • Divide into smaller sub-problems • Iterative Improvement • Improve one change at a time • Decrease-and-Conquer • Decrease instance size • Transform-and-Conquer • Modify problem first and then solve it

16. Important Designing Techniques • Greedy Approach • Locally optimal decisions, can not change once made. • Efficient • Easy to implement • The solution is expected to be optimal • Every problem may not have greedy solution • Dynamic programming • Decompose into sub-problems like divide and conquer • Sub-problems are dependant • Record results of smaller sub-problems • Re-use it for further occurrence • Mostly reduces complexity exponential to polynomial There is a Tradeoffs b/w Space and Time • Use more space now to save time later

17. Problem Solving Process • Problem • Strategy • Algorithm • Input • Output • Steps • Analysis • Correctness • Time & Space • Optimality • Implementation • Verification