Algorithm Analysis: Understanding Big-O Notation

1. Thought for the Day “Sense shines with a double luster when it is set in humility. An able and yet humble man is a jewel worth a kingdom.”– William Penn

2. table key key key ... value value value External Hash Table: Iterators • Slightly more complicated: • need to work through array • and work through linked lists

3. The HashTableIterator Class private class HashTableIterator ... { private int index; private EntryNode<K, V> nextEntry; public HashTableIterator () { for (index = 0; index < table.length; index++) if (table[index] != null) break; // First non-empty bucket if (index < table.length) // Have data nextEntry = table[index]; } // constructor public Pair<K, V> get () { return nextEntry; } // get ... } // class HashTableIterator

4. public void next () { nextEntry = nextEntry.next; if (nextEntry == null) // Look for next non-empty bucket while (++index < table.length) { if (table[index] != null) // Found more data { nextEntry = table[index]; break; } } } // next public boolean atEnd () { return index >= table.length; } // atEnd

5. Uses of the Hash Table Classes • Same interface (Dictionary) as the ListDictionary class • Similar applications • more efficient • But:unordered

6. Dictionary ADT Summary

7. O Section 3Algorithm Analysis Big-O

8. Chapter 8: Big-O • Objectives • Introduce algorithm analysis • Study methods for measuring algorithm efficiency and thus comparing algorithms • Consider some common results • Show some simple applications of the techniques of algorithm analysis

9. Analysing Algorithms • Many algorithms may appear simple and efficient • This may not be true in fact!

10. Example • Solving simultaneous equations • Cramer’s Rule • Calculate the determinant • For n equations, it takes n! operations

11. Time taken = 1016years Longer than the estimated life of the universe! Example (cont.) • If n = 30 (a very small set of equations) n! = 30! = 3 × 1032 • Computers are very fast • Assume 109 operations per second (1GHz)

12. Example (cont.) • Another approach is needed! • The tri-diagonal method • Needs about 10n operations If n = 30, time taken = 10-7seconds

13. Implications • Need to choose algorithms very carefully • This example focussed on time • other resources (memory, disk space, etc.) may also be important

14. Algorithmic Complexity • Not how difficult it is to understand! • How it behaves with respect to time (or other resources) • Example: we say that Cramer’s Rule has a complexity of n!

15. Algorithmic Complexity • Need to measure complexity in a way that is independent of external factors • compiler optimisations, speed of computers, etc. • Find some important operation(s) • Often comparisons, or data exchanges • Count them

16.   • On Apple II (1MHz) • 30s • On Pentium 4 (3GHz) • 3s Use to Compare Algorithms • Algorithm 1 • 20 comparisons • 30 exchanges • Algorithm 2 • 100 comparisons • 150 exchanges

17. The Impact of the Input Data • Vitally important • Must compare “apples with apples” • But we don’t want to get into specific details! • Use some abstract measure of data • Example: • Cramer’s Rule: n equations

18. Input Data (cont.) • Often the number of data items • But not always! • Example: text searching algorithms • length of search string x length of document

19. Input Data (cont.) • Often three cases we need to consider: • average case • best case • worst case

20. Dominant term Big-O Notation • Or order notation • Assume we can find a function giving the number of operations: Order n2 or more simply: O(n2) f(n) = 3.5n2 + 120n + 45