UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2008 - PowerPoint PPT Presentation

1 / 32

UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2008. Lecture 2 Tuesday, 9/16/08 Design Patterns for Optimization Problems Greedy Algorithms. Algorithmic Paradigm Context. Solve subproblem(s), then make choice. Make choice, then solve subproblem(s).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2008

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

UMass Lowell Computer Science 91.503Analysis of AlgorithmsProf. Karen DanielsFall, 2008

Lecture 2

Tuesday, 9/16/08

Design Patterns for Optimization Problems

Greedy Algorithms

Solve subproblem(s), then make choice

Make choice, then solve subproblem(s)

Subproblem solution order

Greedy Algorithms

What is a Greedy Algorithm?

• Solves an optimization problem

• Optimal Substructure:

• optimal solution contains in it optimal solutions to subproblems

• Greedy Strategy:

• At each decision point, do what looks best “locally”

• Choice does not depend on evaluating potential future choices or presolving overlapping subproblems

• Top-down algorithmic structure

• With each step, reduce problem to a smaller problem

• Greedy Choice Property:

• “locally best” = globally best

Greedy Strategy Approach

• Determine the optimal substructure of the problem.

• Develop a recursive solution.

• Prove that, at any stage of the recursion, one of the optimal choices is the greedy choice.

• Show that all but one of the subproblems caused by making the greedy choice are empty.

• Develop a recursive greedy algorithm.

• Convert it to an iterative algorithm.

source: 91.503 textbook Cormen, et al.

Examples

• Activity Selection

• Minimum Spanning Tree

• Dijkstra Shortest Path

• Huffman Codes

• Fractional Knapsack

Activity Selection

Activity Selection Optimization Problem

• Problem Instance:

• Set S = {1,2,...,n} of n activities

• Each activity i has:

• start time: si

• finish time: fi

• Activities i, j are compatible iff non-overlapping:

• Objective:

• select a maximum-sized set of mutually compatible activities

source: 91.404 textbook Cormen, et al.

1

2

4

5

9

3

12

16

6

10

14

15

11

7

13

8

1

2

3

4

5

6

7

8

Activity Selection

Activity Time Duration

Activity Number

Algorithmic Progression

• “Brute-Force”

• (board work)

• Dynamic Programming #1

• Exponential number of subproblems

• (board work)

• Dynamic Programming #2

• Greedy Algorithm

Activity Selection

Solution to Sij including ak produces 2 subproblems:

1) Sik (start after ai finishes; finish before ak starts)

2) Skj (start after ak finishes; finish before aj starts)

c[i,j]=size of maximum-size subset of mutually compatible activities in Sij.

source: 91.404 textbook Cormen, et al.

Recursive Activity Selection

High-level call: RECURSIVE-ACTIVITY-SELECTOR(s,f,0,n)

Returns an optimal solution for

Errors from earlier printing are corrected in red.

source: web site accompanying 91.503 textbook Cormen, et al.

source: web site accompanying 91.503 textbook Cormen, et al.

Running time?

Greedy Algorithm

source: 91.503 textbook Cormen, et al.

• Algorithm:

• S’ = presort activities in Sby nondecreasing finish time

• and renumber

• GREEDY-ACTIVITY-SELECTOR(S’)

• n length[S’]

• A {1}

• j1

• for i 2 to n

• do if

• then

• ji

• return A

Streamlined Greedy Strategy Approach

• View optimization problem as one in which making choice leaves one subproblem to solve.

• Prove there always exists an optimal solution that makes the greedy choice.

• Show that greedy choice + optimal solution to subproblem optimal solution to problem.

Greedy Choice Property: “locally best” = globally best

source: 91.503 textbook Cormen, et al.

Minimum Spanning Tree

Invariant: Minimum weight spanning forest

Becomes single tree at end

Invariant: Minimum weight tree

2

A

B

4

3

G

6

Spans all vertices at end

5

1

1

E

7

6

F

8

4

D

C

2

Minimum Spanning Tree

Time: O(|E|lg|E|) given fast FIND-SET, UNION

• Produces minimum weight tree of edges that includes every vertex.

Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue

• forUndirected, Connected, Weighted Graph

• G=(V,E)

source: 91.503 textbook Cormen et al.

Dijkstra Shortest Path

2

A

4

3

6

5

1

1

7

B

6

F

8

G

4

C

2

E

D

Single Source Shortest Paths: Dijkstra’s Algorithm

for (nonnegative)weighted,directed graph G=(V,E)

source: 91.503 textbook Cormen et al.

Huffman Codes

Huffman Codes

source: 91.503 textbook Cormen, et al.

source: web site accompanying 91.503 textbook Cormen, et al.

source: web site accompanying 91.503 textbook Cormen, et al.

source: web site accompanying 91.503 textbook Cormen, et al.

source: web site accompanying 91.503 textbook Cormen, et al.

source: web site accompanying 91.503 textbook Cormen, et al.

Fractional Knapsack

Knapsack

Each item has value and weight.

Goal: maximize total value of items chosen, subject to weight limit.

50

fractional: can take part of an item

0-1: take all or none of an item

30

20

10

item1

item2

item3

“knapsack”

Value: \$60 \$100\$120

source: web site accompanying 91.503 textbook Cormen, et al.

On course web site under Miscellaneous Docs

• Patriotic Tree

• 404 review handout

• Tree Vertex Cover

• 91.503 midterm

Greedy Heuristic

• If optimization problem does not have “greedy choice property”, greedy approach may still be useful as a heuristic in bounding the optimal solution

• Example: minimization problem

Upper Bound (heuristic)

Solution Values

Optimal (unknown value)

Lower Bound

Homework

HW#AssignedDueContent

1 T 9/9 T 9/16 91.404 review & Chapter 15

2 T 9/16 T 9/23 Chapter 16