cpsc 319
Download
Skip this Video
Download Presentation
CPSC 319

Loading in 2 Seconds...

play fullscreen
1 / 31

CPSC 319 - PowerPoint PPT Presentation


  • 196 Views
  • Uploaded on

Greedy Methods and Backtracking Dr. Marina Gavrilova Computer Science University of Calgary Canada. CPSC 319. Outline. Greedy Strategy Backtracking Technique Dynamic Programming. Goal.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' CPSC 319' - summer


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cpsc 319

Greedy Methods and Backtracking

Dr. Marina Gavrilova

Computer Science

University of Calgary

Canada

CPSC 319

outline
Outline
  • Greedy Strategy
  • Backtracking Technique
  • Dynamic Programming
slide3
Goal
  • In this lecture we will learn about backtracking and greedy algorithms, as well as the concept of dynamic programming and optimization.
slide4

Dynamic programming

  • Optimal structure: an optimal solution to a problem
  • consists of optimal solutions to sub-problems
  • Overlapping sub-problems: few sub-problems in total,
  • many recurring instances of each
  • Solve in “near linear” time through recursion, using a table to show state of problem at any given moment of time
  • Reduces computation time from exponential to linear in some cases
slide5

Greedy methods

Greedy algorithm always makes the choice that is the locally optimal (best) at the moment

It can reduce complex problem to linearly solvable time, and can be much easier to code

slide6

Greedy methods

  • Shortest-path problem and multiple instance knapsack problems can be solved using greedy approach
  • Activity selection is another example
slide7

Greedy methods

Problem: Stampede midway problem

» Buy a wristband that lets you onto any ride

» Lots of rides, each starting and ending at different times

» Your goal: ride as many rides as possible

• Alternative goal that we don’t solve here: maximize

time spent on rides

slide8

Greedy methods

Let A be an optimal solution of S and let k be the

minimum activity in A (i.e., the one with the

earliest finish time). Then A - {k} is an optimal

solution to S’ = {i ∈ S: si ≥ fk}

In words: once activity #1 is selected, the problem

reduces to finding an optimal solution for activity selection over activities in S compatible with #1

slide9

Greedy methods

The algorithm is following:

» Sort the activities by finish time

» Schedule the first activity

» Then schedule the next activity in sorted list which

starts after previous activity finishes

» Repeat until no more activities

Idea:

Always pick the shortest ride available at the time

Greedy-choice Property.

» A globally optimal solution can be arrived at by

making a locally optimal (greedy) choice.

slide10

Backtracking

  • Backtracking can reduce a NP compete problem to linear problem by only going through selected branches of the global solution.
slide11

Backtracking

  • Graph-coloring problem
  • 8 queens problem
  • 4 knight problem
  • Perfect hash function problems

are examples of backtracking method

slide12

Backtracking

  • The goal is to color vertices in a graph G={V,E} so that no 2 adjacent vertices have the same color. Partial 3-coloring problem means only 3 colors are considered.
  • Direct approach builds the tree of ALL possibilities in exponential time.
slide13

Backtracking

  • Partial 3-coloring (3 colors) is solved by the following method:
  • Color first vertex with 1st color, color next vertex with the next color, check if those two vertices are adjacent, if not - coloring is legal, proceed to next vertex, if yes and color is the same – coloring is illegal, try next color for second vertex. If all colors tried and all colorings are illegal, backtrack, try next color for previous vertex etc.
  • Note: sometimes solution is impossible.
  • Exponential O(3^n) complexity is reduced to O(n) on average.
slide14

Job scheduling problem

  • Finally, job scheduling problem can be solved by converting the algebraic relationship xi-xj>=c to directed graph with vertices xi, xj, direction from xj to xi, and edge cost (or time required to complete job xj) is c.
  • The problem of finding time (minimum) when the last activity can commence (i.e. all preceding activities has been completed) is then converted to longest path problem on the graph.
slide15

Dynamic programming

  • Optimal structure: optimal solution to problem
  • consists of optimal solutions to subproblems
  • Overlapping subproblems: few subproblems in total,
  • many recurring instances of each
  • Solve in “near linear” time through recursion, using a table to show state of problem at any given moment of time
  • Reduces computation time from exponential to linear in some cases
slide16

Dynamic programming

  • Longest Common Subsequence
  • Problem: Given 2 sequences, A = 〈a1,...,an〉 and B = 〈b1,...,bm〉, find a common subsequence of characters not necessarily adjacent whose length is maximum.
  • Subsequence must be in order.
slide17

Dynamic programming

  • Straight-forward solution finds all permutations of substrings in string A in exponential time 2^n, then checks for every beginning position in string B – another m times, O(m2^n)
slide18

Dynamic programming

  • Application: comparison of two DNA strings of lengths m and n
  • Ex: X= {A B C B D A B }, Y= {B D C A B A}
  • Longest Common Subsequence:
  • X = A B C BD AB
  • Y = BD CA B A
slide19

Dynamic programming

  • Dynamic programming defines an algorithmic technique to solve the class of problems through recursion on small subproblems and noticing patterns.
slide20

Dynamic programming

We determine the length of Longest Common Substring and at the same time record the substring as well. A has length m, B has lengths n

Define Ai, Bj to be the prefixes of A and B of length i and j respectively

Define L[i,j] to be the length of LCS of Ai and Aj

Then the length of LCS of A and B will be L[n,m]

We start with i = j = 0 (empty substrings of A and B)

LCS of empty string and any other string is empty, so for every i and j: c[0, j] = L[i,0] = 0

First case: A[i]=B[j]: one more symbol in strings A and B matches, so the length of LCS Ai and Aj equals to the length of LCS L [AXi-1, Bi-1]+1

Second case: As symbols don’t match, solution is not improved, and the length of L(Ai , Bj) is the same as before (i.e. maximum of L(Ai, Bj-1) and L(Ai-1,Bj))

slide21

Dynamic programming

  • LCS algorithm calculates the values of each entry of the array ARRAY[n,m]

in O(m*n) since each c[i,j] is calculated in

constant time, and there are m*n

elements in the array

slide22

Knapsack problem

Given some items, pack the knapsack to get the maximum total value. Each item has some size and some value. Total size of the knapsack is is no more than some constant C.

We must consider sizes of items as well as their value.

Item# Size Value

1 2 5

2 3 7

3 5 6

slide23

Knapsack problem

Given a knapsack with maximum capacity C, and

a set Uconsisting of n items {U1,U2,Un}

Each item j has some size sjand value vj

Problem: How to pack the knapsack to achieve

maximum total value of packed items?

slide24

Knapsack problem

There are two versions of the problem:

(1) “0-1 knapsack problem” and

(2) “Fractional knapsack problem”

(1) Items are single; you either take an item

or not. Solved with dynamic programming

(2) Items can be taken a number of times. Solved with a greedy algorithm

slide25

Knapsack problem

  • Let’s first solve this problem with a straightforward algorithm
  • Since there are n items, there are 2n possible combinations of items. We go through all combinations and find the one with the most total value and with total size less or equal to C
  • Running time will be O(2n)
knapsack problem
Knapsack problem
  • “Fractional knapsack problem” – multiple instances of the same item are possible
  • Divide knapsack’s value by size, get w (worth) of an item
  • Sort items by their worth in decreasing order.
  • Start filling out knapsack with as many items of higher worth as possible (as size allows)
  • Go to next highest worth item if previous does not fit anymore
  • O(n) solution!
slide27

Knapsack problem

If items are labeled 1..n, then a subproblem

would be to find an optimal solution for Ui = {items labeled 1, 2, .. i}

However, the solution for U4 might not be part

of the solution for U5, so this definition of a subproblem is flawed.

Adding another parameter: j, which represents the exact size for each subset of items U, is the solution, woth the subproblem to be computed as V[i,j]

slide28

Knapsack problem

Now, the best subset of Ujthat has

total size Jis one of the two:

1) the best subset of Ui-1 that has total size s,

or

2) the best subset of Ui-1 that has total size (C-si)plus the item i

slide29

Knapsack problem

  • Intuitively, the best subset of Uithat has the total size seither contains item i or not:

First case: si>C. Item i can’t be part of the

solution, since if it was, the total size would

be more than C, which is unacceptable

Second case: si <= C. Then the item ican be

in the solution, and we choose the case with

greater value benefit (whether it includes item i or not)

slide30

Review questions

  • What are greedy methods and how multi-instance knapsack problem can be solved through them?
  • How backtracking works for graph coloring?
  • How job scheduling problem can be solved through directed graphs?
  • What are the two types of knapsack problems?
slide31

Conclusions

  • Backtracking and Dynamic programming are useful technique of solving optimization problems
  • When the solution can be recursively described in terms of partial solutions, we can store these partial solutions and re-use them to save time
  • Running time (Dynamic Programming) is much better than straight-forward approach
  • Sources: A. Drozdek textbook
ad