Optimization Problems. Optimization problem: a problem of finding the best solution from all feasible solutions. Two common techniques: Greedy Algorithms (local) Dynamic Programming (global). Greedy Algorithms. Greedy algorithms typically consist of A set of candidate solutions
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Greedy algorithms typically consist of
Cost ?
O(V+E)
2
3
3
2
2
3
1
S
S
S
2
1
1
2
3
2
3
Finished
Discovered
Undiscovered
Implementation?
r
s
t
u
0
v
w
y
x
Q: s
0
r
s
t
u
0
1
1
v
w
y
x
Q: w r
1 1
r
s
t
u
2
0
1
2
1
v
w
y
x
Q: r t x
1 2 2
r
s
t
u
2
0
1
2
2
1
v
w
y
x
Q: t x v
2 2 2
r
s
t
u
2
0
3
1
2
2
1
v
w
y
x
Q: x v u
2 2 3
r
s
t
u
2
0
3
1
2
3
2
1
v
w
y
x
Q: v u y
2 3 3
r
s
t
u
2
0
3
1
2
3
2
1
v
w
y
x
Q: u y
3 3
r
s
t
u
2
0
3
1
2
3
2
1
v
w
y
x
Q: y
3
r
s
t
u
2
0
3
1
2
3
2
1
v
w
y
x
Q:
r
s
t
u
2
0
3
1
2
3
2
1
v
w
y
x
Breath First Tree
r
r
s
s
t
t
u
u
2
0
0
3
1
2
3
2
1
v
v
w
w
y
y
x
x
1
10
9
2
3
s
4
6
7
5
2
y
x
An adaptation of BFS
u
v
1
10
9
2
3
0
s
4
6
7
5
2
y
x
u
v
1
10
10
9
2
3
0
s
4
6
7
5
5
5
2
y
x
u
v
1
8
14
10
9
2
3
0
s
4
6
7
5
7
7
5
2
y
x
u
v
1
8
8
13
10
9
2
3
0
s
4
6
7
5
7
5
2
y
x
u
v
1
9
8
9
10
9
2
3
0
s
4
6
7
5
7
5
2
y
x
u
v
1
8
9
10
9
2
3
0
s
4
6
7
5
7
5
2
y
x
priority queue Q.
Cost:
?
O(V2)
Minimal Spanning Tree (MST) Problem:
Input: An undirected, connected graph G.
Output: The subgraph of G that
(the sum of the values of the edges in the subset is at the minimum)
A greedy algorithm.
http://wwwb2.is.tokushimau.ac.jp/~ikeda/
suuri/dijkstra/PrimApp.shtml?demo1
Huffman codes
– very effective technique for compressing data,
saving 20%  90%.
Problem:
a,e,i,o,u, blanks, newlines, few q, x, z’s
Solution:
(Can take advantage of relative freq of letters to save space)
000
360
001
126
010
126
011
111
100
96
101
72
110
21
111
6
918
3
Char
Frequency
Code
Total Bits
E
120
L
42
D
42
U
37
C
32
M
24
K
7
Z
2
37
E:120
L:42
D:42
U:37
C:32
M:24
K:7
Z:2
Char
Code
0
1
0
1
1
0
0
1
0
1
0
1
0
1
Complete binary tree
(Can take advantage of relative freq of letters to save space)
 Huffman codes
Char
Code
M
char
Freq
C
32
128
D
42
126
E
120
120
M
24
120
K
7
42
L
42
126
U
37
111
Z
2
12
785
Compare with: 918
~15% less
Code
Bits
1110
101
0
11111
111101
110
100
111100
Char
Code
010000000010
101011100110
Char
Code
10100101
111111001110111101
A set of codes is said to meet the prefix property if no code in the set is the prefix of another. Such codes are called prefix codes.
Huffman codes are prefix codes.
Char
Code
Coin Changing
Sort coins denominations by value: c1 < c2 < … < cn.
S
while (x 0) {
let k be largest integer such that ck x
if (k = 0)
return "no solution found"
x x  ck
S S {k}
}
return S
coins selected
Greedy algorithm
failed!
k
ck
All optimal solutionsmust satisfy
Max value of coins1, 2, …, k1 in any OPT
1
1
P 4

2
5
N 1
4
3
10
N + D2
4 + 5 = 9
4
25
Q 3
20 + 4 = 24
5
100
no limit
75 + 24 = 99
k
k
ck
ck
All optimal solutionsmust satisfy
All optimal solutionsmust satisfy
Max value of coins1, 2, …, k1 in any OPT
Max value of coins1, 2, …, k1 in any OPT
1
1
1
1
P 4
P 9


2
5
N 1
4
2
10
P + D8
9
3
10
N + D2
4 + 5 = 9
3
25
Q 3
40 + 4 = 44
4
25
Q 3
20 + 4 = 24
4
100
no limit
75 + 44 = 119
5
100
no limit
75 + 24 = 99
Kevin’s problem
Knapsack Problem
Dynamic programming
Item
Value
Weight
1
1
1
2
6
2
Let W=11
3
18
5
4
22
6
5
28
7
Answer for 01 knapsack problem:
OPT: { 4, 3 }
value = 22 + 18 = 40
> 45
Activityselection Problem
a
b
c
d
e
f
g
h
Time
0
1
2
3
4
5
6
7
8
9
10
11
Greedy template. Consider jobs in some order. Take each job provided it's compatible with the ones already taken.
Earliest start time:
Consider jobs in ascending order of start time sj.
Earliest finish time:
Consider jobs in ascending order of finish time fj.
Shortest interval:
Consider jobs in ascending order of interval length fj  sj.
Fewest conflicts:
For each job, count the number of conflicting jobs cj. Schedule in ascending order of conflicts cj.
…
Sort jobs by finish times so that f1 f2 ... fn.
A
for j = 1 to n {
if (job j compatible with A)
A A {j}
}
return A
jobs selected
Theorem. Greedy algorithm is optimal.
Proof: (by contradiction)
job ir+1 finishes before jr+1
Greedy:
i1
i1
ir
ir+1
ir+1
j1
j2
jr
jr+1
. . .
OPT:
Still optimal with a bigger value than r : ir+1=jr+1 contradiction!
why not replace job jr+1 with job ir+1?
a
b
c
d
e
f
g
h
Time
0
1
2
3
4
5
6
7
8
9
10
11
Greedy algorithm?
For each town t, St = {towns that are within 30 miles of it}
 a Walmart center at t will cover all towns in St .
while (not all covered)
Pick Si with largest uncovered elements
Proof: Let nt be # of elements not covered after t iterations. There must be a set with ≥ nt/k elements:
nt+1 ≤ nt  nt/k ≤ n0(11/k)t+1 ≤ n0(e1/k)t+1
nt < 1 when t=klnn.