1 / 31

Lecture 1 Preliminaries

Lecture 1 Preliminaries. COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski. Announcements. Textbook: J. Kleinberg, E. Tardos. Algorithm Design . Addison-Wesley, 2005 Lectures: Wednesday 9-11, Friday 15-16 in 3.10 Tutorials: Thursday 11-12 in 1.01 Ashton Bldg

mura
Download Presentation

Lecture 1 Preliminaries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lecture 1Preliminaries COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski Lecture 1: Preliminaries

  2. Announcements • Textbook: J. Kleinberg, E. Tardos. Algorithm Design. Addison-Wesley, 2005 • Lectures: Wednesday 9-11, Friday 15-16 in 3.10 • Tutorials: Thursday 11-12 in 1.01 Ashton Bldg • Eleni Akrida (akridel@hotmail com) • Office hours (room 3.11, Ashton Bldg): • Wednesday 12-13 • Friday 14-15 • Info and contact: • http://www.csc.liv.ac.uk/~darek/comp523.html • darek@liv.ac.uk Lecture 1: Preliminaries

  3. Assessment and feedback • Assessment: Exam (75%) and 2 tasks (25% total) • Feedback: • Presentation and discussion of solutions after assignment deadline • In case of specific questions, individual appointments Lecture 1: Preliminaries

  4. Algorithms - what does it mean? Algorithm - a list/structure of instructions, which are carried out in a fixed order • to find the answer to a question • to calculate Al-Khwarizmi (Muhammed Ibn Musa) - Arabian mathematician who introduced the decimal positional number system to the Western world, the author of Algebra This module: techniques to design algorithms for problems concerning data commonly processed by computers (written in hard discs, propagated in networks, entered by user or external device, etc.) Lecture 1: Preliminaries

  5. Types of algorithms • Constructive vs. Non-constructive • Discrete vs. Numerical • Deterministic vs. non-deterministic (e.g., Randomized,Quantum) • Sequential vs. Concurrent vs. DNA vs. … • Exact vs. Approximate • and many others Lecture 1: Preliminaries

  6. Example Problem: Find if a given word occurs in a given text Input: Text and word represented as lists Output: Word found or not o n c e u p o n a t o n a Lecture 1: Preliminaries

  7. Example - algorithm Naïve solution (exhaustive search): Intuition: • Check letter after letter, starting from the beginning of the lists, if the corresponding letters are equal • If some corresponding letters are not equal, re-start comparing from the second letter of the text (and the first letter of the word) • Etc. from the 3rd letter, 4th letter, until the end of text or successful comparison Implementation: • Initiate three pointers: • blue_pointer and black_pointer at the beginning of blue_list • yellow_pointer at the beginning of yellow_list • Initiate Boolean variable successful into false Lecture 1: Preliminaries

  8. Naïve solution cont. • Repeat • stop := false • Move blue_pointer to the next element • Set black_pointer to blue_pointer • Move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true until successful or blue_pointer is at the end of blue_list Output: successful Lecture 1: Preliminaries

  9. Example - how it works o n c e u p o n a t o n a • Repeat • stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true • until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true • until successful or blue_pointer is at the end of blue_list Lecture 1: Preliminaries

  10. What we expect from algorithms? • Correctness: computes desired output • Termination: stops eventually (or with high probability) • Efficiency: with respect to • Performance measure • Time (or total number of computation steps) • Size of memory used • Number of messages sent • … • Methods of measuring • Worst-case • Average-case • Smoothed (a subset of possible inputs, specific distribution of inputs, etc.) • Competitive (comparing to the best solution for particular input) • Expected • … Analysis depends on Model! Lecture 1: Preliminaries

  11. Example - correctness • Repeat • stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true • until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true • until successful or blue_pointer is at the end of blue_list • Successfulis set to true iff after some execution of the internal loop the yellow_pointer is at the end of the yellow_list • yellow_pointer is at the end of the yellow_list iff it came through the whole yellow_list without coming back to the beginning • it happens if the values pointed by black_pointer and yellow_pointer have been the same during all checks of internal loop in the current run (thus a copy of the word exists in the text) Lecture 1: Preliminaries

  12. Example - termination • Repeat • stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true • until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true • until successful or blue_pointer is at the end of blue_list • External Loop Invariant: At the beginning of the external loop, blue_pointer advances, and it has a finite number of advances to take before reaching the end of the text • Internal Loop Invariant: Each run of the internal loop finishes eventually; It happens because yellow_pointer keeps advancing every iteration (first line of the loop), unless stop condition becomes true or it reaches the end of the word (or black_ pointer reaches the end of the text) Lecture 1: Preliminaries

  13. Example - efficiency • Complexity measures: time, size of additional memory • Size of additional memory: • 3 single pointers and 2 binary variables • Time complexity: • Worst-case: if the sequence is not included in the text then time is (almost) proportional to the multiplication of sizes of blue and yellow lists, e.g., text: ‘aab’ repeated ntimes word: aaa 1, 2 or 3 internal loop runs, 3n times, gives about 6nruns Lecture 1: Preliminaries

  14. Example - efficiency cont. • Time complexity (cont.): • Average case: text or sequence are randomly selected; average short word should be find quickly in the beginning of an average long text, in time proportional to the squared length of the word (exercise) • Smoothed analysis: text must be randomly selected from “reasonable” set of texts (usually complicated analysis, depends on the family of texts - dictionary) Lecture 1: Preliminaries

  15. Efficiency - asymptotic notation • n - integer, size of data, f() - function over integers • Complexity O(f(n)): there is a positive constant csuch that for every positive integer ncomplexity is at mostcf(n) • Complexity (f(n)) : there is a positive constant csuch that for every positive integer ncomplexity is at leastcf(n) • Complexity o(f(n)): complexity divided by f(n)comes to0 with ngoing to infinity (strictly smaller than f(n)) • Complexity (f(n)) : complexity divided by f(n) comes towith ngoing to infinity (strictly bigger than f(n)) Lecture 1: Preliminaries

  16. Example - asymptotic complexity • Worst case time: • If size of text is n and size of word is n/2: O((n/2)(n/2)) = O(n2) • If size of text is m and size of sequence is k : O((m-k)k) Lecture 1: Preliminaries

  17. Examples 5n3+100 = O(n3) , 5n3+100 ≠ O(n2) 5n3+100=(n3) , 5n3+100 ≠ (n4) log n = o(na) for any positive constanta na = o(cn) for any positive constantsa,c log (4n)= log n + log 4 = O(log n) log(n4)= 4 log n = O(log n) (4n)3 = 64n3= O(n3) (n4)3 = n12 = (n4) (3n)4= 81n = (3n) Logarithms are to the base 2 Lecture 1: Preliminaries

  18. Symmetric properties and tight bound Iff(n) = O(g(n)) theng(n) = (f(n)) and vice versa Iff(n) = o(g(n)) theng(n) = (f(n)) and vice versa Definition: If f(n) = O(g(n)) and f(n) =(g(n)) then f(n) = (g(n))andg(n) = (f(n)) Example: 9n2 + n +7 = (3n2) = (n2) Lecture 1: Preliminaries

  19. Transitive properties (order) Iff(n) = O(g(n)) and g(n) = O(h(n)) then f(n) = O(h(n)) Iff(n) = (g(n)) and g(n) = (h(n)) then f(n) = (h(n)) Iff(n) = (g(n)) and g(n) = (h(n)) then f(n) = (h(n)) Lecture 1: Preliminaries

  20. Sum and maximum f1(n) + … + fa(n) = (max(f1(n), … , fa(n))) for any constant positive integer a Example: • 3n2 + n +7 = (3n2) = (n2) If the range of the summation index is not constant: • in i = n(n+1)/2 = (n2)  (max{1, … , n}) = (n) Lecture 1: Preliminaries

  21. Logarithmic time O(log n) • Reduce input to any fraction of it in constant time • Example: searching if a given element xis in a sorted array Lecture 1: Preliminaries

  22. Linear time O(n) • It is sufficient to look at each element of the input constant number of times • Example: finding maximum in an (unsorted) array Lecture 1: Preliminaries

  23. Time O(n log n) • Split the input into two pieces of the similar size and merge both solutions in linear time • Example: sorting an array (split into halves, sort each part separately, merge sorted parts in linear time) Lecture 1: Preliminaries

  24. Quadratic time O(n2) • It is enough to consider pairs of elements • Examples: • finding the closest pair of points located in a plane • sorting by comparison of subsequent elements (insertion sort) Lecture 1: Preliminaries

  25. Polynomial time O(na) • Algorithm has many nested loops • Example: are there any two disjoint sets among given family of nsets, each of at most nelements (time O(n3)) Lecture 1: Preliminaries

  26. Exponential time O(cn) • Algorithm considers many subsets of the input • Example: exhaustive search to find the largest clique contained in a given graph (time O(2n)) Lecture 1: Preliminaries

  27. Beyond exponential time: O(n!), O(nn), … • Algorithm searching in a large space • Example: search performed in the set of all permutations (time O(n!)) Lecture 1: Preliminaries

  28. Graphs • Set of nodes |V| = n • Set of edges |E| = m • Undirected edges/graph: pairs of nodes {v,w} • Directed edges/graph: pairs of nodes (v,w) • Set of neighbors of v : set of nodes connected by an edge with v (directed: in-neighbors, out-neighbors) • Degree of a node: number of its neighbors • Path: a sequence of (non-repeating) nodes such that every two consecutive nodes constitute an edge • Length of a path: number of nodes minus 1 • Distance between two nodes: the length of the shortest path between these nodes • Diameter: the longest distance in the graph Lecture 1: Preliminaries

  29. Lines, cycles, trees, cliques Line Cycle Tree Clique Lecture 1: Preliminaries

  30. Conclusions • Algorithm: list/structure of instructions • Algorithmic methods for problems arising from computer and communication applications • Guarantee correctness, termination and efficiency • Model is important! • Asymptotic analysis of efficiency Lecture 1: Preliminaries

  31. Textbook and exercises READING: • Chapter 2, up to section 2.4 OBLIGATORY EXERCISES: • Exercises 1,2,3 page 67 • Solved exercises 1,2 pages 65,66 OPTIONAL: • Exercises 3,4,5,6 pages 67,68,69 • Exercises 7,8 pages 69,70 Lecture 1: Preliminaries

More Related