Lecture 1 the greedy method
This presentation is the property of its rightful owner.
Sponsored Links
1 / 81

Lecture 1: The Greedy Method PowerPoint PPT Presentation


  • 73 Views
  • Uploaded on
  • Presentation posted in: General

Lecture 1: The Greedy Method. 主講人 : 虞台文. Content. What is it? Activity Selection Problem Fractional Knapsack Problem Minimum Spanning Tree Kruskal’s Algorithm Prim’s Algorithm Shortest Path Problem Dijkstra’s Algorithm Huffman Codes. Lecture 1: The Greedy Method. What is it?.

Download Presentation

Lecture 1: The Greedy Method

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Lecture 1 the greedy method

Lecture 1: The Greedy Method

主講人:虞台文


Content

Content

  • What is it?

  • Activity Selection Problem

  • Fractional Knapsack Problem

  • Minimum Spanning Tree

    • Kruskal’s Algorithm

    • Prim’s Algorithm

  • Shortest Path Problem

    • Dijkstra’s Algorithm

  • Huffman Codes


Lecture 1 the greedy method1

Lecture 1: The Greedy Method

What is it?


The greedy method

The Greedy Method

  • A greedy algorithm always makes the choice that looksbest at the moment

  • For some problems, it always give aglobally optimal solution.

  • For others, it may only give a locally optimal one.


Main components

Main Components

  • Configurations

    • differentchoices, collections, or values to find

  • Objective function

    • a score assigned to configurations, which we want to either maximize or minimize


Example making change

Is the solution alwaysoptimal?

Example: Making Change

  • Problem

    • A dollar amount to reach and a collection of coin amounts to use to get there.

  • Configuration

    • A dollar amount yet to return to a customer plus the coins already returned

  • Objective function

    • Minimizenumber of coins returned.

  • Greedy solution

    • Always return the largest coin you can


Example largest k out of n sum

Example: Largest k-out-of-n Sum

  • Problem

    • Pick k numbers out of n numbers such that the sum of these k numbers is the largest.

  • Exhaustive solution

    • There are choices.

    • Choose the one with subset sum being the largest

  • Greedy Solution

    FOR i = 1 to k

    pick out the largest number and

    delete this number from the input.

    ENDFOR

Is the greedy solution alwaysoptimal?


Example shortest paths on a special graph

Example:Shortest Paths on a Special Graph

  • Problem

    • Find a shortest path from v0 to v3

  • Greedy Solution


Example shortest paths on a special graph1

Is the solution optimal?

Example:Shortest Paths on a Special Graph

  • Problem

    • Find a shortest path from v0 to v3

  • Greedy Solution


Example shortest paths on a multi stage graph

Is the greedy solution optimal?

Example:Shortest Paths on a Multi-stage Graph

  • Problem

    • Find a shortest path from v0 to v3


Example shortest paths on a multi stage graph1

Is the greedy solution optimal?

Example:Shortest Paths on a Multi-stage Graph

  • Problem

    • Find a shortest path from v0 to v3

The optimal path


Example shortest paths on a multi stage graph2

Is the greedy solution optimal?

Example:Shortest Paths on a Multi-stage Graph

  • Problem

    • Find a shortest path from v0 to v3

What algorithm can be used to find the optimum?

The optimal path


Advantage and disadvantage of the greedy method

Advantage and Disadvantageof the Greedy Method

  • Advantage

    • Simple

    • Work fast when they work

  • Disadvantage

    • Not always work  Short term solutions can be disastrous in the long term

    • Hard to prove correct


Lecture 1 the greedy method2

Lecture 1: The Greedy Method

Activity Selection Problem


Activity selection problem conference scheduling problem

Activity Selection Problem(Conference Scheduling Problem)

  • Input: A set of activities S = {a1,…, an}

    • Each activity has a start time and a finish time

      ai = [si, fi)

    • Two activities are compatible if and only if their interval does notoverlap

  • Output: a maximum-size subset of mutually compatible activities


Example activity selection problem

Example:Activity Selection Problem

Assume thatfi’s are sorted.


Example activity selection problem1

1

2

3

4

5

6

7

8

9

10

11

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Example:Activity Selection Problem


Example activity selection problem2

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Example:Activity Selection Problem

Is the solution optimal?

1

2

3

4

5

6

7

8

9

10

11


Example activity selection problem3

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Example:Activity Selection Problem

Is the solution optimal?

1

2

3

4

5

6

7

8

9

10

11


Activity selection algorithm

Activity Selection Algorithm

Greedy-Activity-Selector (s, f)

// Assume that f1 f2   ...  fn

n length [s]

A { 1 }

j 1

for i 2 to n

   if sifj then

AA{ i }

ji

return A

Is the algorithm optimal?


Proof of optimality

Proof of Optimality

  • Suppose A  S is an optimal solution and the first activity is k 1.

  • If k 1, one can easily show that B =A – {k}  {1} is also optimal. (why?)

  • This reveals that greedy-choice can be applied to the first choice.

  • Now, the problem is reduced to activity selection on S’ = {2, …, n}, which are all compatible with 1.

  • By the same argument, we can show that, to retain optimality, greedy-choice can also be applied for next choices.


Lecture 1 the greedy method3

Lecture 1: The Greedy Method

Fractional Knapsack Problem


The fractional knapsack problem

The Fractional Knapsack Problem

  • Given: A set S of n items, with each item i having

    • bi - a positive benefit

    • wi - a positive weight

  • Goal: Choose items, allowing fractional amounts, to maximizetotal benefit but with weight at most W.


The fractional knapsack problem1

1

2

3

4

5

Value:

($ per ml)

10 ml

The Fractional Knapsack Problem

“knapsack”

  • Solution:

  • 1 ml of 5

  • 2 ml of 3

  • 6 ml of 4

  • 1 ml of 2

Items:

wi:

4 ml

8 ml

2 ml

6 ml

1 ml

bi:

$12

$32

$40

$30

$50

3

4

20

5

50


The fractional knapsack algorithm

The Fractional Knapsack Algorithm

  • Greedy choice: Keep taking item with highest value

AlgorithmfractionalKnapsack(S,W)

Input:set S of items w/ benefit biand weight wi; max. weight W

Output:amount xi of each item i to maximize benefit w/ weight at most W

for each item i in S

xi 0

vi bi / wi{value}

w 0{total weight}

whilew < W

remove item i with highest vi

xi min{wi , W  w}

w  w + min{wi , W  w}

Does the algorithm always gives an optimum?


Proof of optimality1

Proof of Optimality

  • Suppose there is a better solution

  • Then, there is an item i with higher value than a chosen itemj, but xi < wi, xj > 0 and vi > vj

  • Substituting some i with j, we’ll get a better solution

  • How much of i: min{wi xi, xj}

  • Thus, there is no better solution than the greedy one


Recall 0 1 knapsack problem

Recall: 0-1 Knapsack Problem

Which boxes should be chosen to maximize the amount of money while still keeping the overall weight under 15 kg ?

Is the fractional knapsack algorithm applicable?


Exercise

Exercise

  • Construct an example show that the fractional knapsack algorithm doesn’t give the optimal solution when applying it to the 0-1 knapsack problem.


Lecture 1 the greedy method4

Lecture 1: The Greedy Method

Minimum

Spanning Tree


What is a spanning tree

What is a Spanning Tree?

  • A tree is a connected undirected graph that contains nocycles

  • A spanning tree of a graph G is a subgraph of G that is a tree and contains all the vertices of G


Properties of a spanning tree

B

B

B

A

A

A

C

C

C

D

D

D

E

E

E

Properties of a Spanning Tree

  • The spanning tree of a n-vertex undirected graph has exactly n – 1 edges

  • It connects all the vertices in the graph

  • A spanning tree has no cycles

Undirected Graph

Some Spanning Trees


What is a minimum spanning tree

What is a Minimum Spanning Tree?

  • A spanning tree of a graph G is a subgraph of G that is a tree and contains all the vertices of G

  • A minimumspanning tree is the one among all the spanning trees with the lowest cost


Applications of msts

Applications of MSTs

  • Computer Networks

    • To find how to connect a set of computers using the minimum amount of wire

  • Shipping/Airplane Lines

    • To find the fastest way between locations


Two greedy algorithms for mst

Two Greedy Algorithms for MST

  • Kruskal’s Algorithm

    • merges forests into tree by adding small-cost edges repeatedly

  • Prim’s Algorithm

    • attaches vertices to a partially built tree by adding small-cost edges repeatedly


Kruskal s algorithm

8

7

4

9

2

14

11

4

7

16

b

c

d

8

10

1

2

a

i

e

b

c

d

h

g

f

a

i

e

h

g

f

Kruskal’s Algorithm


Kruskal s algorithm1

b

c

d

a

i

e

h

g

f

Kruskal’s Algorithm

8

7

4

9

2

14

11

4

7

16

8

10

1

2

b

c

d

a

i

e

h

g

f


Kruskal s algorithm2

Kruskal’s Algorithm

G = (V, E) – Graph

w: ER+– Weight

T Tree

MST-Kruksal(G)

T← Ø

for each vertex vV[G]

Make-Set(v) // Make separate sets for vertices

sort the edges by increasing weight w

for each edge (u, v)E, in sorted order

if Find-Set(u) ≠ Find-Set(v) // If no cycles are formed

T ← T {(u, v)}// Add edge to Tree

Union(u, v) // Combine Sets

return T


Time complexity

O(|E|log|E|)

Time Complexity

G = (V, E) – Graph

w: ER+– Weight

T Tree

MST-Kruksal(G , w)

T← Ø

for each vertex vV[G]

Make-Set(v) // Make separate sets for vertices

sort the edges by increasing weight w

for each edge (u, v)E, in sorted order

if Find-Set(u) ≠ Find-Set(v) // If no cycles are formed

T ← T {(u, v)}// Add edge to Tree

Union(u, v) // Combine Sets

return T

O(1)

O(|V|)

O(|E|log|E|)

O(|E|)

O(|V|)

O(1)


Prim s algorithm

8

7

4

9

2

14

11

4

7

16

b

c

d

8

10

1

2

a

i

e

b

c

d

h

g

f

a

i

e

h

g

f

Prim’s Algorithm


Prim s algorithm1

Prim’s Algorithm

8

7

d

b

b

c

d

c

4

9

2

a

a

i

e

e

i

14

11

4

7

16

8

10

h

g

f

h

g

f

1

2

b

b

c

c

d

d

a

i

e

a

i

e

h

g

f

h

g

f


Prim s algorithm2

Prim’s Algorithm

G = (V, E) – Graph

w: ER+– Weight

r – Starting vertex

Q – Priority Queue

Key[v] – Key of Vertex v

π[v] –Parent of Vertex v

Adj[v] – Adjacency List of v

MST-Prim(G, w, r)

Q← V[G]// Initially Q holds all vertices

for each uQ

Key[u] ← ∞// Initialize all Keys to ∞

Key[r] ← 0 // r is the first tree node

π[r] ← Nil

while Q ≠ Ø

u ← Extract_min(Q) // Get the min key node

for each v Adj[u]

if vQ and w(u, v) < Key[v]// If the weight is less than the Key

π[v] ← u

Key[v] ← w(u, v)


Time complexity1

O(|E|log|V|)

Time Complexity

G = (V, E) – Graph

w: ER+– Weight

r – Starting vertex

Q – Priority Queue

Key[v] – Key of Vertex v

π[v] –Parent of Vertex v

Adj[v] – Adjacency List of v

MST-Prim(G, r)

Q← V[G]// Initially Q holds all vertices

for each uQ

Key[u] ← ∞// Initialize all Keys to ∞

Key[r] ← 0 // r is the first tree node

π[r] ← Nil

while Q ≠ Ø

u ← Extract_min(Q) // Get the min key node

for each v Adj[u]

if vQ and w(u, v) < Key[v]// If the weight is less than the Key

π[v] ← u

Key[v] ← w(u, v)


Optimality

Are the algorithms optimal?

Optimality

Yes

  • Kruskal’s Algorithm

    • merges forests into tree by adding small-cost edges repeatedly

  • Prim’s Algorithm

    • attaches vertices to a partially built tree by adding small-cost edges repeatedly


Lecture 1 the greedy method5

Lecture 1: The Greedy Method

Shortest Path Problem


Shortest path problem spp

Shortest Path Problem (SPP)

  • Single-Source SPP

    • Given a graph G = (V, E), and weight w: ER+, find the shortest path from a source node s  V to any other node, say, v  V.

  • All-Pairs SPP

    • Given a graph G = (V, E), and weight w: ER+, find the shortest path between each pair of nodes in G.


Dijkstra s algorithm

Dijkstra's Algorithm

  • Dijkstra's algorithm, named after its discoverer, Dutch computer scientist Edsger Dijkstra, is an algorithm that solves the single-source shortest path problem for a directed graph with nonnegative edge weights.


Dijkstra s algorithm1

Dijkstra's Algorithm

  • Start from the source vertex, s

  • Take the adjacent nodes and update the current shortest distance

  • Select the vertex with the shortest distance, from the remaining vertices

  • Update the current shortest distance of the Adjacent Vertices where necessary,

    • i.e. when the new distance is less than the existing value

  • Stop when all the vertices are checked


Dijkstra s algorithm2

Dijkstra's Algorithm


Dijkstra s algorithm3

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm


Dijkstra s algorithm4

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm


Dijkstra s algorithm5

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

0


Dijkstra s algorithm6

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

0

5


Dijkstra s algorithm7

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

0

5

5


Dijkstra s algorithm8

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

0

5


Dijkstra s algorithm9

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

0

7

5


Dijkstra s algorithm10

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

0

7

5

7


Dijkstra s algorithm11

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

0

5

7


Dijkstra s algorithm12

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

14

13

0

5

7


Dijkstra s algorithm13

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

9

8

8

14

13

0

5

7


Dijkstra s algorithm14

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

8

14

13

0

5

7


Dijkstra s algorithm15

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

8

14

9

13

0

5

7


Dijkstra s algorithm16

u

v

1

9

2

3

9

0

s

4

6

7

5

2

y

x

Dijkstra's Algorithm

8

13

14

9

9

0

5

7


Dijkstra s algorithm17

Dijkstra's Algorithm

G = (V, E) – Graph

w: ER+– Weight

s – Source

d[v] – Current shortest distance from s to v

S – Set of nodes whose shortest distance is known

Q – Set of nodes whose shortest distance is unknown

Dijkstra(G, w ,s)

for each vertex vV[G]

d[v] // Initialize all distances to 

π[v]  Nil

d[s]  0// Set distance of source to 0

S

QV[G]

while Q≠ 

u Extract_Min(Q) // Get the min in Q

SS {u}// Add it to the already known list

for each vertex v Adj[u]

if d[v] > d[u] + w(u, v)// If the new distance is shorter

d[v] d[u] + w(u, v)

π[v] u


Lecture 1 the greedy method6

Lecture 1: The Greedy Method

Huffman Codes


Huffman codes

Huffman Codes

  • Huffman code is a technique for compressing  data.

    • Variable-Length code

  • Huffman's greedy algorithm look at the occurrence of each character and it as a binary string in an optimal way.


Example

Example

Suppose we have a data consists of 100,000 characters with following frequencies.


Fixed vs variable length codes

Fixed vs. Variable Length Codes

Suppose we have a data consists of 100,000 characters with following frequencies.

Total Bits:

Fixed Length Code

345,000 + 313,000 + 312,000 + 316,000 + 39,000 + 35,000= 300,000

Variable Length Code

145,000 + 313,000 + 312,000 + 316,000 + 49,000 + 45,000= 224,000


Prefix codes

0

1

a:45

0

1

0

1

0

1

b:13

c:12

d:16

0

1

f:5

e:9

Prefix Codes

In which no codeword is a prefix of other codeword.

Encode

aceabfd

=0100110101011100111

Decode

0100110101011100111

a

c

e

a

b

f

d


Huffman code algorithm

0

1

a:45

0

1

0

1

0

1

b:13

c:12

d:16

0

1

f:5

e:9

Huffman-Code Algorithm

100

55

25

30

14


Huffman code algorithm1

14

0

1

f:5

e:9

Huffman-Code Algorithm

f:5

e:9

c:12

b:13

d:16

a:45


Huffman code algorithm2

14

14

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

f:5

e:9

c:12

b:13

d:16

a:45

c:12

b:13

d:16

a:45


Huffman code algorithm3

25

0

1

14

b:13

c:12

0

1

f:5

e:9

Huffman-Code Algorithm

c:12

b:13

d:16

a:45


Huffman code algorithm4

25

25

0

0

1

1

14

14

b:13

b:13

c:12

c:12

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

a:45

d:16

c:12

b:13

d:16

a:45


Huffman code algorithm5

25

30

0

1

0

1

14

14

b:13

c:12

d:16

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

a:45

d:16


Huffman code algorithm6

25

25

30

30

0

0

1

1

0

0

1

1

14

14

14

b:13

b:13

c:12

c:12

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

a:45

a:45

d:16


Huffman code algorithm7

55

0

1

25

25

30

30

0

0

1

1

0

0

1

1

14

14

b:13

b:13

c:12

c:12

d:16

d:16

0

0

1

1

f:5

f:5

e:9

e:9

Huffman-Code Algorithm

a:45


Huffman code algorithm8

55

55

0

0

1

1

25

25

25

30

30

30

0

0

0

1

1

1

0

0

0

1

1

1

14

14

14

b:13

b:13

b:13

c:12

c:12

c:12

d:16

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

a:45

a:45


Huffman code algorithm9

100

0

1

55

55

a:45

0

0

1

1

25

25

25

30

30

30

0

0

0

1

1

1

0

0

0

1

1

1

14

14

14

b:13

b:13

b:13

c:12

c:12

c:12

d:16

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

a:45


Huffman code algorithm10

100

0

1

55

55

a:45

0

0

1

1

25

25

25

30

30

30

0

0

0

1

1

1

0

0

0

1

1

1

14

14

14

b:13

b:13

b:13

c:12

c:12

c:12

d:16

d:16

d:16

0

0

0

1

1

1

f:5

f:5

f:5

e:9

e:9

e:9

Huffman-Code Algorithm

Huffman tree built

a:45


Huffman code algorithm11

Huffman-Code Algorithm

Huffman (C)

n |C|

QC

for i 1 to n 1

z Allocate-Node ()

x left[z]  Extract-Min (Q) // least frequent

y right[z]  Extract-Min (Q)// next least

f[z] f[x] + f[y] // update frequency

      Insert ( Q, z )

return Extract-Min (Q)


Optimality1

Optimality

Exercise


  • Login