Data Structures & Algorithms

1. Data Structures & Algorithms Lecture 1 Dr. AdilYousif

2. Course Objectives • On completion of this module student should: • Understand the importance of algorithm analysis for computer science; • Have an understanding of the informal big- oh. • Have had experience in the analysis of time complexity of several types of algorithms;

3. Course Objectives Cont • Understand the operation of recursive algorithms; • Have an appreciation of the importance of data structures and their implementation in the consideration of algorithm analysis;

4. Course Outlines • Algorithms and their analysis • Worst case Analysis of Algorithms • Iterative Algorithm • Recursive Algorithm • Sorting Algorithms • Selection Sort • Merge Sort • Quick Sort • Searching and Ordering Table • Arrays • Lists • Stacks • Queues • Heaps

5. Algorithms and their analysis • An algorithm is a step-by-step procedure for solving a problem in a finite amount of time. Input Algorithm Output

6. Algorithm Descriptions Methods • Nature languages: Arabic, English, etc. • Flowcharts. • Flowcharts • Pseudo-code: codes very close to computer languages, e.g., C programming language. • Programs: C programs, C++ programs, Java programs.

7. Properties of Algorithms • Finiteness • Absence of Ambiguity • Definition of Sequence • Feasibility • Input • Output

8. Algorithm Analysis • The same problem can frequently be solved with algorithms that differ in efficiency. • The differences between the algorithms may be immaterial for processing a small number of data items, but these differences grow with the amount of data. • To compare the efficiency of algorithms, a measure of the degree of difficulty of an algorithm called computational complexity is developed.

9. Computational complexity • Computational complexity indicates how much effort is needed to apply an algorithm or how costly it is. • This cost can be measured in a variety of ways, and the particular context determines its meaning. • The main efficiency criteria: time and space.

10. Algorithm Analysis • Space complexity • How much space is required • Time complexity • How much time does it take to run the algorithm

11. Space Complexity • Space complexity = The amount of memory required by an algorithm to run to completion • [The most often encountered cause is “memory leaks” – the amount of memory required larger than the memory available on a given system] • Some algorithms may be more efficient if data completely loaded into memory • Need to look also at system limitations • E.g. Classify 2GB of text in various categories [politics, tourism, sport, natural disasters, etc.] – can I afford to load the entire collection?

12. Space Complexity (cont’d) • Fixed part: The size required to store certain data/variables, that is independent of the size of the problem: - e.g. name of the data collection - same size for classifying 2GB or 1MB of texts • Variable part: Space needed by variables, whose size is dependent on the size of the problem: - e.g. actual text - load 2GB of text VS. load 1MB of text

13. Time Complexity • Often more important than space complexity • space available (for computer programs!) tends to be larger and larger • time is still a problem for all of us • 3-4GHz processors on the market • still … • researchers estimate that the computation of various transformations for 1 single DNA chain for one single protein on 1 TerraHZ computer would take about 1 year to run to completion • Algorithms running time is an important issue

14. Time Complexity Cont. • For example, to compare 100 algorithms, all of them would have to be run on the same machine. • Furthermore, the results of run time tests depend on the language in which a given algorithm is written, even if the tests are performed on the same machine.

15. Time Complexity Cont. • If programs are compiled, they execute much faster than when they are interpreted. • A program written in C or Ada may be 20 times faster than the same program encoded in BASIC or LISP.

16. Time Complexity Cont. • To evaluate an algorithm’s efficiency, real-time units such as microseconds and nanoseconds should not be used. • Rather, logical units that express a relationship between the size n of a file or an array and the amount of time t required to process the data should be used. I

17. Running Time • Depends on the input size. • The running time of an algorithm typically grows with the input size.

18. Rules • We assume an arbitrary time unit. • Execution of one of the following operations takes time 1: • assignment operation • single I/O operations • single Boolean operations, numeric comparisons • single arithmetic operations • function return • array index operations, pointer dereferences

19. More Rules • Running time of a selection statement (if, switch) is the time for the condition evaluation + the maximum of the running times for the individual clauses in the selection. • Loop execution time is the sum, over the number of times the loop is executed, of the body time + time for the loop check and update operations, + time for the loop setup. † Always assume that the loop executes the maximum number of iterations possible • Running time of a function call is 1 for setup + the time for any parameter calculations + the time required for the execution of the function body.

20. The Execution Time of Algorithms • Each operation in an algorithm (or a program) has a cost.  Each operation takes a certain of time. count = count + 1;  take a certain amount of time, but it is constant A sequence of operations: count = count + 1; Cost: c1 sum = sum + count; Cost: c2  Total Cost = c1 + c2

21. The Execution Time of Algorithms (cont.) Example: Simple If-Statement CostTimes if (n < 0) c1 1 absval = -n c2 1 else absval = n; c3 1 Total Cost <= c1 + max(c2,c3)

22. The Execution Time of Algorithms (cont.) Example: Simple Loop CostTimes i = 1; c1 1 sum = 0; c2 1 while (i <= n) { c3 n+1 i = i + 1; c4 n sum = sum + i; c5 n } Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*c5  The time required for this algorithm is proportional to n

23. By inspecting the pseudocode, we can determine the maximum number of primitive operations executed by an algorithm, as a function of the input size AlgorithmarrayMax(A, n) # operations currentMaxA[0] 2 for (i =1; i<n; i++) 2n (i=1 once, i<n n times, i++ (n-1) times) ifA[i]  currentMaxthen 2(n 1) currentMaxA[i] 2(n 1) returncurrentMax 1 Total 6n1 Counting Primitive Operations

24. Complexity • In general, we are not so much interested in the time and space complexity for small inputs. • For example, while the difference in time complexity between linear and binary search is meaningless for a sequence with n = 10, it is gigantic for n = 230. CMSC 203 - Discrete Structures

25. Complexity Cont. • For example, let us assume two algorithms A and B that solve the same class of problems. • The time complexity of A is 5,000n, the one for B is 1.1n for an input with n elements. • For n = 10, A requires 50,000 steps, but B only 3, so B seems to be superior to A. • For n = 1000, however, A requires 5,000,000 steps, while B requires 2.51041 steps. CMSC 203 - Discrete Structures

26. Complexity Cont. • This means that algorithm B cannot be used for large inputs, while algorithm A is still feasible. • So what is important is the growthof the complexity functions. • The growth of time and space complexity with increasing input size n is a suitable measure for the comparison of algorithms. CMSC 203 - Discrete Structures

27. The Growth Rate

28. Common Growth Rates Algorithms

29. The Growth of Functions • “Popular” functions g(n) are • n log n, 1, 2n, n2, n!, n, n3, log n • Listed from slowest to fastest growth: • 1 • log n • n • n log n • n2 • n3 • 2n • n! CMSC 203 - Discrete Structures

30. Questions