Lecture 1 the greedy method
Sponsored Links
This presentation is the property of its rightful owner.
1 / 81

Lecture 1: The Greedy Method PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

Lecture 1: The Greedy Method. 主講人 : 虞台文. Content. What is it? Activity Selection Problem Fractional Knapsack Problem Minimum Spanning Tree Kruskal’s Algorithm Prim’s Algorithm Shortest Path Problem Dijkstra’s Algorithm Huffman Codes. Lecture 1: The Greedy Method. What is it?.

Download Presentation

Lecture 1: The Greedy Method

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Lecture 1: The Greedy Method

主講人:虞台文


Content

  • What is it?

  • Activity Selection Problem

  • Fractional Knapsack Problem

  • Minimum Spanning Tree

    • Kruskal’s Algorithm

    • Prim’s Algorithm

  • Shortest Path Problem

    • Dijkstra’s Algorithm

  • Huffman Codes


Lecture 1: The Greedy Method

What is it?


The Greedy Method

  • A greedy algorithm always makes the choice that looksbest at the moment

  • For some problems, it always give aglobally optimal solution.

  • For others, it may only give a locally optimal one.


Main Components

  • Configurations

    • differentchoices, collections, or values to find

  • Objective function

    • a score assigned to configurations, which we want to either maximize or minimize


Is the solution alwaysoptimal?

Example: Making Change

  • Problem

    • A dollar amount to reach and a collection of coin amounts to use to get there.

  • Configuration

    • A dollar amount yet to return to a customer plus the coins already returned

  • Objective function

    • Minimizenumber of coins returned.

  • Greedy solution

    • Always return the largest coin you can


Example: Largest k-out-of-n Sum

  • Problem

    • Pick k numbers out of n numbers such that the sum of these k numbers is the largest.

  • Exhaustive solution

    • There are choices.

    • Choose the one with subset sum being the largest

  • Greedy Solution

    FOR i = 1 to k

    pick out the largest number and

    delete this number from the input.

    ENDFOR

Is the greedy solution alwaysoptimal?


Example:Shortest Paths on a Special Graph

  • Problem

    • Find a shortest path from v0 to v3

  • Greedy Solution


Is the solution optimal?

Example:Shortest Paths on a Special Graph

  • Problem

    • Find a shortest path from v0 to v3

  • Greedy Solution


Is the greedy solution optimal?

Example:Shortest Paths on a Multi-stage Graph

  • Problem

    • Find a shortest path from v0 to v3


Is the greedy solution optimal?

Example:Shortest Paths on a Multi-stage Graph

  • Problem

    • Find a shortest path from v0 to v3

The optimal path


Is the greedy solution optimal?

Example:Shortest Paths on a Multi-stage Graph

  • Problem

    • Find a shortest path from v0 to v3

What algorithm can be used to find the optimum?

The optimal path


Advantage and Disadvantageof the Greedy Method

  • Advantage

    • Simple

    • Work fast when they work

  • Disadvantage

    • Not always work  Short term solutions can be disastrous in the long term

    • Hard to prove correct


Lecture 1: The Greedy Method

Activity Selection Problem


Activity Selection Problem(Conference Scheduling Problem)

  • Input: A set of activities S = {a1,…, an}

    • Each activity has a start time and a finish time

      ai = [si, fi)

    • Two activities are compatible if and only if their interval does notoverlap

  • Output: a maximum-size subset of mutually compatible activities


Example:Activity Selection Problem

Assume thatfi’s are sorted.


1

2

3

4

5

6

7

8

9

10

11

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Example:Activity Selection Problem


0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Example:Activity Selection Problem

Is the solution optimal?

1

2

3

4

5

6

7

8

9

10

11


0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Example:Activity Selection Problem

Is the solution optimal?

1

2

3

4

5

6

7

8

9

10

11


Activity Selection Algorithm

Greedy-Activity-Selector (s, f)

// Assume that f1 f2   ...  fn

n length [s]

A { 1 }

j 1

for i 2 to n

   if sifj then

AA{ i }

ji

return A

Is the algorithm optimal?


Proof of Optimality

  • Suppose A  S is an optimal solution and the first activity is k 1.

  • If k 1, one can easily show that B =A – {k}  {1} is also optimal. (why?)

  • This reveals that greedy-choice can be applied to the first choice.

  • Now, the problem is reduced to activity selection on S’ = {2, …, n}, which are all compatible with 1.

  • By the same argument, we can show that, to retain optimality, greedy-choice can also be applied for next choices.


Lecture 1: The Greedy Method

Fractional Knapsack Problem


The Fractional Knapsack Problem

  • Given: A set S of n items, with each item i having

    • bi - a positive benefit

    • wi - a positive weight

  • Goal: Choose items, allowing fractional amounts, to maximizetotal benefit but with weight at most W.


1

2

3

4

5

Value:

($ per ml)

10 ml

The Fractional Knapsack Problem

“knapsack”

  • Solution:

  • 1 ml of 5

  • 2 ml of 3

  • 6 ml of 4

  • 1 ml of 2

Items:

wi:

4 ml

8 ml

2 ml

6 ml

1 ml

bi:

$12

$32

$40

$30

$50

3

4

20

5

50


The Fractional Knapsack Algorithm

  • Greedy choice: Keep taking item with highest value

AlgorithmfractionalKnapsack(S,W)

Input:set S of items w/ benefit biand weight wi; max. weight W

Output:amount xi of each item i to maximize benefit w/ weight at most W

for each item i in S

xi 0

vi bi / wi{value}

w 0{total weight}

whilew < W

remove item i with highest vi

xi min{wi , W  w}

w  w + min{wi , W  w}

Does the algorithm always gives an optimum?


Proof of Optimality

  • Suppose there is a better solution

  • Then, there is an item i with higher value than a chosen itemj, but xi < wi, xj > 0 and vi > vj

  • Substituting some i with j, we’ll get a better solution

  • How much of i: min{wi xi, xj}

  • Thus, there is no better solution than the greedy one


Recall: 0-1 Knapsack Problem

Which boxes should be chosen to maximize the amount of money while still keeping the overall weight under 15 kg ?

Is the fractional knapsack algorithm applicable?


Exercise

  • Construct an example show that the fractional knapsack algorithm doesn’t give the optimal solution when applying it to the 0-1 knapsack problem.


Lecture 1: The Greedy Method

Minimum

Spanning Tree


What is a Spanning Tree?

  • A tree is a connected undirected graph that contains nocycles

  • A spanning tree of a graph G is a subgraph of G that is a tree and contains all the vertices of G


B

B

B

A

A

A

C

C

C

D

D

D

E

E

E

Properties of a Spanning Tree

  • The spanning tree of a n-vertex undirected graph has exactly n – 1 edges

  • It connects all the vertices in the graph

  • A spanning tree has no cycles

Undirected Graph

Some Spanning Trees


What is a Minimum Spanning Tree?

  • A spanning tree of a graph G is a subgraph of G that is a tree and contains all the vertices of G

  • A minimumspanning tree is the one among all the spanning trees with the lowest cost


Applications of MSTs

  • Computer Networks

    • To find how to connect a set of computers using the minimum amount of wire

  • Shipping/Airplane Lines

    • To find the fastest way between locations


Two Greedy Algorithms for MST

  • Kruskal’s Algorithm

    • merges forests into tree by adding small-cost edges repeatedly

  • Prim’s Algorithm

    • attaches vertices to a partially built tree by adding small-cost edges repeatedly


8

7

4

9

2

14

11

4

7

16

b

c

d

8

10

1

2

a

i

e

b

c

d

h

g

f

a

i

e

h

g

f

Kruskal’s Algorithm


b

c

d

a

i

e

h

g

f

Kruskal’s Algorithm

8

7

4

9

2

14

11

4

7

16

8

10

1

2

b

c

d

a

i

e

h

g

f


Kruskal’s Algorithm

G = (V, E) – Graph

w: ER+– Weight

T Tree

MST-Kruksal(G)

T← Ø

for each vertex vV[G]

Make-Set(v) // Make separate sets for vertices

sort the edges by increasing weight w

for each edge (u, v)E, in sorted order

if Find-Set(u) ≠ Find-Set(v) // If no cycles are formed

T ← T {(u, v)}// Add edge to Tree

Union(u, v) // Combine Sets

return T


O(|E|log|E|)

Time Complexity

G = (V, E) – Graph

w: ER+– Weight

T Tree

MST-Kruksal(G , w)

T← Ø

for each vertex vV[G]

Make-Set(v) // Make separate sets for vertices

sort the edges by increasing weight w

for each edge (u, v)E, in sorted order

if Find-Set(u) ≠ Find-Set(v) // If no cycles are formed

T ← T {(u, v)}// Add edge to Tree

Union(u, v) // Combine Sets

return T

O(1)

O(|V|)

O(|E|log|E|)

O(|E|)

O(|V|)

O(1)


8

7

4

9

2

14

11

4

7

16

b

c

d

8

10

1

2

a

i

e

b

c

d

h

g

f

a

i

e

h

g

f

Prim’s Algorithm


Prim’s Algorithm

8

7

d

b

b

c

d

c

4

9

2

a

a

i

e

e

i

14

11

4

7

16

8

10

h

g

f

h

g

f

1

2

b

b

c

c

d

d

a

i

e

a

i

e

h

g

f

h

g

f


Prim’s Algorithm

G = (V, E) – Graph

w: ER+– Weight

r – Starting vertex

Q – Priority Queue

Key[v] – Key of Vertex v

π[v] –Parent of Vertex v

Adj[v] – Adjacency List of v

MST-Prim(G, w, r)

Q← V[G]// Initially Q holds all vertices

for each uQ

Key[u] ← ∞// Initialize all Keys to ∞

Key[r] ← 0 // r is the first tree node

π[r] ← Nil

while Q ≠ Ø

u ← Extract_min(Q) // Get the min key node

for each v Adj[u]

if vQ and w(u, v) < Key[v]// If the weight is less than the Key

π[v] ← u

Key[v] ← w(u, v)


O(|E|log|V|)

Time Complexity

G = (V, E) – Graph

w: ER+– Weight

r – Starting vertex

Q – Priority Queue

Key[v] – Key of Vertex v

π[v] –Parent of Vertex v

Adj[v] – Adjacency List of v

MST-Prim(G, r)

Q← V[G]// Initially Q holds all vertices

for each uQ

Key[u] ← ∞// Initialize all Keys to ∞

Key[r] ← 0 // r is the first tree node

π[r] ← Nil

while Q ≠ Ø

u ← Extract_min(Q) // Get the min key node

for each v Adj[u]

if vQ and w(u, v) < Key[v]// If the weight is less than the Key

π[v] ← u

Key[v] ← w(u, v)


Are the algorithms optimal?

Optimality

Yes

  • Kruskal’s Algorithm

    • merges forests into tree by adding small-cost edges repeatedly

  • Prim’s Algorithm

    • attaches vertices to a partially built tree by adding small-cost edges repeatedly


Lecture 1: The Greedy Method

Shortest Path Problem


Shortest Path Problem (SPP)

  • Single-Source SPP

    • Given a graph G = (V, E), and weight w: ER+, find the shortest path from a source node s  V to any other node, say, v  V.

  • All-Pairs SPP

    • Given a graph G = (V, E), and weight w: ER+, find the shortest path between each pair of nodes in G.


Dijkstra's Algorithm

  • Dijkstra's algorithm, named after its discoverer, Dutch computer scientist Edsger Dijkstra, is an algorithm that solves the single-source shortest path problem for a directed graph with nonnegative edge weights.


Dijkstra's Algorithm

  • Start from the source vertex, s

  • Take the adjacent nodes and update the current shortest distance

  • Select the vertex with the shortest distance, from the remaining vertices

  • Update the current shortest distance of the Adjacent Vertices where necessary,

    • i.e. when the new distance is less than the existing value

  • Stop when all the vertices are checked


Dijkstra's Algorithm


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

0


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

0

5


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

0

5

5


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

0

5


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

0

7

5


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

0

7

5

7


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

0

5

7


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

13

0

5

7


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

8

14

13

0

5

7


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

8

14

13

0

5

7


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

8

14

9

13

0

5

7


u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

8

13

14

9

9

0

5

7


Dijkstra's Algorithm

G = (V, E) – Graph

w: ER+– Weight

s – Source

d[v] – Current shortest distance from s to v

S – Set of nodes whose shortest distance is known

Q – Set of nodes whose shortest distance is unknown

Dijkstra(G, w ,s)

for each vertex vV[G]

d[v] // Initialize all distances to 

π[v]  Nil

d[s]  0// Set distance of source to 0

S

QV[G]

while Q≠ 

u Extract_Min(Q) // Get the min in Q

SS {u}// Add it to the already known list

for each vertex v Adj[u]

if d[v] > d[u] + w(u, v)// If the new distance is shorter

d[v] d[u] + w(u, v)

π[v] u


Lecture 1: The Greedy Method

Huffman Codes


Huffman Codes

  • Huffman code is a technique for compressing  data.

    • Variable-Length code

  • Huffman's greedy algorithm look at the occurrence of each character and it as a binary string in an optimal way.


Example

Suppose we have a data consists of 100,000 characters with following frequencies.


Fixed vs. Variable Length Codes

Suppose we have a data consists of 100,000 characters with following frequencies.

Total Bits:

Fixed Length Code

345,000 + 313,000 + 312,000 + 316,000 + 39,000 + 35,000= 300,000

Variable Length Code

145,000 + 313,000 + 312,000 + 316,000 + 49,000 + 45,000= 224,000


0

1

a:45

0

1

0

1

0

1

b:13

c:12

d:16

0

1

f:5

e:9

Prefix Codes

In which no codeword is a prefix of other codeword.

Encode

aceabfd

=0100110101011100111

Decode

0100110101011100111

a

c

e

a

b

f

d


0

1

a:45

0

1

0

1

0

1

b:13

c:12

d:16

0

1

f:5

e:9

Huffman-Code Algorithm

100

55

25

30

14


14

0

1

f:5

e:9

Huffman-Code Algorithm

f:5

e:9

c:12

b:13

d:16

a:45


14

14

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

f:5

e:9

c:12

b:13

d:16

a:45

c:12

b:13

d:16

a:45


25

0

1

14

b:13

c:12

0

1

f:5

e:9

Huffman-Code Algorithm

c:12

b:13

d:16

a:45


25

25

0

0

1

1

14

14

b:13

b:13

c:12

c:12

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

a:45

d:16

c:12

b:13

d:16

a:45


25

30

0

1

0

1

14

14

b:13

c:12

d:16

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

a:45

d:16


25

25

30

30

0

0

1

1

0

0

1

1

14

14

14

b:13

b:13

c:12

c:12

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

a:45

a:45

d:16


55

0

1

25

25

30

30

0

0

1

1

0

0

1

1

14

14

b:13

b:13

c:12

c:12

d:16

d:16

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

a:45


55

55

0

0

1

1

25

25

25

30

30

30

0

0

0

1

1

1

0

0

0

1

1

1

14

14

14

b:13

b:13

b:13

c:12

c:12

c:12

d:16

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

a:45

a:45


100

0

1

55

55

a:45

0

0

1

1

25

25

25

30

30

30

0

0

0

1

1

1

0

0

0

1

1

1

14

14

14

b:13

b:13

b:13

c:12

c:12

c:12

d:16

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

a:45


100

0

1

55

55

a:45

0

0

1

1

25

25

25

30

30

30

0

0

0

1

1

1

0

0

0

1

1

1

14

14

14

b:13

b:13

b:13

c:12

c:12

c:12

d:16

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

Huffman tree built

a:45


Huffman-Code Algorithm

Huffman (C)

n |C|

QC

for i 1 to n 1

z Allocate-Node ()

x left[z]  Extract-Min (Q) // least frequent

y right[z]  Extract-Min (Q)// next least

f[z] f[x] + f[y] // update frequency

      Insert ( Q, z )

return Extract-Min (Q)


Optimality

Exercise


  • Login