CpE602: Applied Discrete Mathematics

1. CpE602: Applied Discrete Mathematics The Fundamentals: Algorithms Growth of functions Complexity of algorithms Integers and division Primes and greatest common divisors

2. 4/4/2012 1:05:35 AM Algorithms Definition: Algorithm is a finite set of precise instructions for performing a computation or for solving a problem. Example: Describe an algorithm for finding the maximum (largest) value in a finite sequence of integers. Algorithm: Finding Maximum Element. procedure max (a1, a2, � ,an: integers) max := a1 for i := 2 to n if (max < ai) then max := ai { max is the largest value }

3. 4/4/2012 1:05:35 AM Algorithms Properties that algorithms generally share Input. An algorithm has input values from a specified set. Output. The algorithms produce output values. Definiteness. The steps of the algorithm must be defined precisely. Correctness. The output values should be correct solutions. Finiteness. An algorithm should output a value after a finite number of steps. Effectiveness. It must be possible to perform each step of the algorithm exactly and in a finite amount of time. Generality. The procedure should be applicable for all problems of a given form. Example: Show that the algorithm for fining maximum value has the properties listed above.

4. 4/4/2012 1:05:35 AM Algorithms Searching Problems: Locate an element x in a list of distinct elements a1, a2, � , an or determine that it is not in the list. Algorithm: The Linear Search Algorithm. procedure linear search (x: integer, a1, a2, � , an: distinct integers) i := 1 while (i = n and x ? ai) i := i + 1 if i = n then location := i else location := 0 {location is the subscript of the term that is equal to x, or is 0 if x is not found}

5. 4/4/2012 1:05:35 AM Algorithms Algorithm: The Binary Search Algorithm. procedure binary search (x: integer, a1, a2, � , an: increasing integers) i := 1 {i is the left endpoint of the search interval} j := n {j is the right endpoint of the search interval} while ( i < j ) begin m := [(i + j)/2] if x > am then i:= m + 1 else j := m end if x = ai then location := i else location := 0 {location is the subscript of the term equal to x, or 0 if x is not found} Example: Find 19 in 1,2,3,5,6,7,8,10,12,13,15,16,18,19,20,22.

6. 4/4/2012 1:05:35 AM Algorithms Sorting. Suppose that we have a list of elements and a way to order the elements. Sorting is putting this elements into a list in which the elements are in increasing order. Algorithm: The Bubble Sort. procedure bubble sort (a1, a2, � ,an: real numbers with n > 1) for i := 1 to n - 1 for j := 1 to n - i if (aj > aj+1) then interchange aj+1 and aj {a1, a2, � ,an is in increasing order } Example: Use bubble sort to put 3, 2, 4, 1, 5 in increasing order.

7. 4/4/2012 1:05:35 AM Algorithms Algorithm: The Insertion Sort. procedure insertion sort (a1, a2, � ,an: real numbers with n > 1) for j := 2 to n begin i := 1 while ( aj > ai) i := i + 1 m := aj for k := 0 to j - i � 1 aj-k := aj-k-1 ai := m end {a1, a2, � ,an is in increasing order } Example: Use insertion sort to put 3, 2, 4, 1, 5 in increasing order.

8. 4/4/2012 1:05:35 AM Algorithms Optimization problems. � The goal is to find a solution that either maximizes or minimizes the value of some parameter. Greedy algorithms. � Instead of considering all sequences of steps, the best choice is made at each step. Often lead to optimal solutions. Example: The problem is to make n cents change using quarters, dimes, nickels and pennies. The goal is to use least total number of coins. Greedy approach: At each step we choose the coin with the largest possible denomination.

9. 4/4/2012 1:05:35 AM Algorithms Algorithm: Greedy Change-Making Algorithm. procedure change (c1, c2, � ,cr: denominations in decreasing order, n: a positive integer) for j := 1 to r begin i := 1 while ( n = ci ) begin add a coin with value ci to the change n := n - ci end Lemma. In an optimal solution, there can be at most 2 dimes, at most 1 nickel, at most four pennies, and the amount of change in dimes nickels and pennies cannot exceed 24 cents. Theorem. The Greedy Change-Making Algorithm is optimal.

10. 4/4/2012 1:05:35 AM The Growth of functions The Big-O Notation Definition. Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say f(x) is O(g(x)), denoted f(x) = O(g(x)), if there are constants C and k such that |f(x)| = C |g(x)| whenever x > k. C and k are the witnesses of the relationship. Example: Show that f(x) = x2+2x+1 is O(x2). Is x2 big-O of f(x)? Example: Show that 7x2 is O(x3). Example: Show that n2 is not O(n). Example: Is it also true that x3 is O(7x2)?

11. 4/4/2012 1:05:35 AM The Growth of functions Some Important Big-O Results Theorem. Let f(x) = anxn + � + a1x + a0, where a0, a1, � , an are real numbers. Then f(x) is O(xn). Example: Use big-O notation to estimate 1 + 2 + � + n. Example: Use big-O estimates of the factorial function and the logarithm of the factorial function. Example: Use the inequality n < 2n for all positive integers n to show that log n is O(n).

12. 4/4/2012 1:05:35 AM The Growth of functions Growth of combinations of functions Theorem. Suppose the f1(x) = O(g1(x)) and f2(x) = O(g2(x)). Then, (f1+f2)(x) is O( max(|g1(x)|, |g2(x)|) ). Corollary. Suppose the f1(x) = O(g(x)) and f2(x) = O(g(x)). Then, (f1+f2)(x) is O(g(x)). Theorem. Suppose the f1(x) = O(g1(x)) and f2(x) = O(g2(x)). Then, (f1f2)(x) is O(g1(x)g2(x)). Example: Give big-O estimates of: 3nlog(n!) + (n2+3)log n (x + 1)log(x2+1) + 3x2

13. 4/4/2012 1:05:35 AM The Growth of functions Big-Omega and Big-Theta Notation Definition. Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say f(x) is O(g(x)), denoted f(x) = O(g(x)), if there are constants C and k such that |f(x)| = C |g(x)| whenever x > k. Definition. Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say f(x) is T(g(x)), or f(x) is of order g(x), if f(x) is both O(g(x)) and O(g(x)).

14. 4/4/2012 1:05:35 AM The Growth of functions Example: Show that f(x) = 8x3 +5x2 + 7 is O(g(x)) where g(x) = x3. Example: Show that f(n) = 1 + 2 + � + n is T(n2). One can show that f(x) is T(g(x)) by finding constants C1, C2 and k s.t. C1|g(x)| = f(x) = C2|g(x)| for x > k. Example: Show that 3x2 + 8logx is T(x2). Theorem. Let f(x) = anxn + � + a1x + a0, where a0, a1, � , an are real numbers with an ? 0. Then f(x) is T(xn).

15. 4/4/2012 1:05:35 AM Complexity of Algorithms Computational complexity Time complexity � the time required to solve the problem Space complexity � the memory required to solve the problem Example: What is the time complexity of the algorithm for finding maximum element? Example: What is the time complexity of the binary search algorithm? Example: What is the average-case complexity of the linear search algorithm assuming that the element x is in the list? Example: What is the worst-case complexity of the bubble sort in terms of the number of comparisons made?

16. 4/4/2012 1:05:35 AM Complexity of Algorithms Complexity Classes - Constant complexity - Logarithmic complexity - Linear complexity - Polynomial complexity - Exponential complexity - Factorial complexity A problem that can be solved with a worst-case polynomial time algorithm is said to be tractable. Otherwise, it is intractable. Problems that cannot be solved by any algorithm are called unsolvable. Problems for which a solution can be checked in polynomial time are said to belong to the NP (nondeterministic polynomial) class. NP-complete problems are problems in NP with the property that if any of these problems can be solved in polynomial time, then all problems in NP can be solved in poly time.

17. 4/4/2012 1:05:35 AM The Integers and Division Division Definition. If a and b are integers with a ? 0, we say a divides b, denoted a|b, if there is an integer c such that b = ac. When a divides b we say that a is a factor of b and that b is a multiple of a. Example: Determine whether 3|7 and whether 3|12. Example: Let n and d be positive integers. How many positive integers not exceeding n are divisible by d? Theorem. Let a, b and c be integers. Then (i) if a|b and a|c, then a|(b+c) (ii) if a|b, then a|bc for any integer c (iii) if a|b and b|c, then a|c

18. 4/4/2012 1:05:35 AM The Integers and Division Corollary. If a, b and c are integers s.t. a|b and a|c, then a|(mb+nc) for any integers m and n. Theorem (The Division Algorithm). Let a be an integer and d a positive integer. Then there are unique integers q and r, with 0 = r < q, s.t. a = dq+r. Here, d is called the divisor, a is called the dividend, q is called the quotient and r is called the remainder. One can also use the following notation to express the quotient and remainder: q = a div d, r = a mod d. Example: What are the quotient and remainder when: (i) 101 is divided by 11, (ii) -11 is divided by 3.

19. 4/4/2012 1:05:35 AM The Integers and Division Definition. Let a and b be integers and let m be a positive integer. We say that a is congruent to b modulo m, denoted a = b (mod m), if m divides (a-b). Theorem. Let a and b be integers and let m be a positive integer. Then a = b (mod m) if and only if a mod m = b mod m. Theorem. Let m be a positive integer. Then a = b (mod m) if and only if there is an integer k s.t. a = b + km. Theorem. Let m be a positive integer. If a = b (mod m) and c = d (mod m), then a+c = b+d (mod m) and ac = bd (mod m). Corollary. (a+b) mod m = ((a mod m) + (b mod m)) mod m (ab) mod m = ((a mod m)(b mod m)) mod m

20. 4/4/2012 1:05:35 AM The Integers and Division Applications of Congruences � Hashing The records in databases are identified using a key (e.g., the social security number of a person). Hash tables are data structures that are used to store and search for records. To store a record with a key k in a hash table consisting of m �rows�, one computes a hash h = h(k) of the key k and stores the record in the h-th �row�. One of the most common hash functions is the function h(k) = k mod m. Example: Where will the record with a key k = 111111375 be stored in a hash table with m = 111 memory locations?

21. 4/4/2012 1:05:35 AM The Integers and Division Applications of Congruences � Pseudorandom Numbers The linear congruential method. To generate a sequence {xn} of pseudorandom numbers, we select four integers: the modulus m, multiplier a, increment c and seed x0, with 1 < a < m, 0 = c < m and 0 = x0 < m. The numbers in the sequence are generated using the congruence: xn+1 = (axn + c) mod m. When c = 0, we say the generator is pure multiplicative generator. Example: Find the sequence when m = 9, a = 7, c = 4 and x0 = 3.

22. 4/4/2012 1:05:35 AM The Integers and Division Applications of Congruences � Cryptology Caesar�s cipher. Each letter is substituted with a letter that is three letters forward in the alphabet. Mathematically the encryption transformation of a single letter can be expressed as f(p) = (p + 3) mod 26. Example: What is the encryption of �MEET ME IN THE PARK�? Shift ciphers. A generalization of the Caesar�s cipher. The letter substitution transformation is f(p) = (p + k) mod 26. Example (More generalization): What letter is the substitute of K when the function f(p) = (7p + 3) mod 26 is used for encryption?

23. 4/4/2012 1:05:35 AM Primes and Greatest Common Divisors Definition. A positive integer p greater than 1 is prime if the only positive factors of p are 1 and p. A positive integer that is greater than 1 and is not prime is called composite. Example: Are 7 and 9 primes? Theorem (The Fundamental Theorem of Arithmetic). Every positive integer greater than 1 can be written uniquely as a prime or a product of primes where the prime factors are listed in non-decreasing order. Example: Find the prime factorization of 100, 641, 999, 1024. Theorem. If n is a composite integer, then n has a prime divisor less than or equal to the square root of n. Example: Show that 101 is prime.

24. 4/4/2012 1:05:35 AM Primes and Greatest Common Divisors Theorem. There are infinitely many primes. Mersenne primes. Mersenne primes are primes of the special form 2p-1 where p is a prime too. Examples: 22-1 = 3, 23-1 = 7 and 25 � 1 = 31. Distribution of primes Theorem (The Prime Number Theorem). The ratio of the number of primes less than or equal to n and n/lnn approaches 1 when n grows without bound.

25. 4/4/2012 1:05:35 AM Primes and Greatest Common Divisors Conjectures and open problems. Example: It is nice to have a strictly increasing function f(n) such that f(n) is prime for every positive integer n. Is f(n) = n2 � n + 41 such funtion? Goldbach�s Conjecture. Every even integer n > 2 can be expressed as a sum of two primes. Example: There are many conjectures about primes of a special form. One such conjecture is that there are infinitely many primes of the form n2 + 1. The Twin Prime Conjecture. Twin primes are primes that differ by 2. The twin prime conjecture asserts that there are infinitely many twin primes.

26. 4/4/2012 1:05:35 AM Primes and Greatest Common Divisors Greatest Common Divisor Definition. Let a and b be integers, not both zero. The greatest common divisor of a and b, denoted by gcd(a, b), is the largest integer d that divides both a and b. Example: What is the greatest common divisor of 24 and 36? Example: What is the greatest common divisor of 17 and 22? Definition. The integers a and b are relatively prime if their greatest common divisor is 1. Definition. The integers a1, a2, � , an are pairwise relatively prime if gcd(ai, aj) = 1 for all 0 < i < j = n. Example: Are 10, 17 and 21 pairwise relatively prime? Example: One can use factorization to find gcd of two numbers. Find the gcd of 120 and 500.

27. 4/4/2012 1:05:35 AM Primes and Greatest Common Divisors Least Common Multiple Definition. The least common multiple of the two positive integers a and b is the smallest positive integer that is divisible by both a and b. The least common multiple of a and b is denoted by lcm(a, b). Example: What is the least common multiple of 233572 and 2433? How can we use factorization to find lcm in general? Theorem. If a and b be positive integers, then ab = gcd(a, b) � lcm(a, b).