Thinking about Algorithms Abstractly. Recursive Back Tracking & Dynamic Programming. Jeff Edmonds York University. Lecture 7. COSC 3101. Problems. Techniques. Optimization Problems A Sequence of Decisions The Little Bird & Friend Optimal Substructure Memoization Set of SubInstances
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Thinking about Algorithms Abstractly
Recursive Back Tracking&Dynamic Programming
Jeff Edmonds York University
Lecture7
COSC 3101
Problems
Techniques
Optimization Problems
A Sequence of Decisions
The Little Bird & Friend
Optimal Substructure
Memoization
Set of SubInstances
Tracing Dyn. Prog. Alg
Reversing
Code
Speeding Up Running Time
Multiple Opt Solutions
Review
Question for Little Bird
Review & Don'ts
Best Path
Printing Neatly
Longest Common Subsequence
Knapsack Problem
The Event Scheduling Problem
Parsing
Satisfiability
Optimization Problems
Ingredients:
Optimization Problems
Specification of an Optimization Problem
We use it because it nicely demonstrates the concepts in a graphical way.
An instance (input) consists of <G,s,t>.
G is a weighted directed layered graphs source nodet sink node
5
3
4
2
3
2
5
2
2
1
3
4
A solution for an instance is a path from s to t.
The cost of a solution is the sum of the weights.
The goal is to find a path with minimum total cost.
2+6+3+7=18
4+2+1+5=12
An instance (input) consists of <G,s,t>.
Brute Force Algorithm
Try all paths, return the best.
But there may be an exponential number of paths!
I ask a question about the solution.
“Which edge should we take first?”
Some how I decide <s,v3>.
My friend asks the next question.
“Which edge do we take second?”
Some how he decides <v3,v5>.
His friend asks the next question.
“Which edge do we take third?”
Some how he decided <v5,v8>.
Taking the best first edge.
I ask a question about the solution.
“Which edge should we take first?”
How do I decide?
The greedy algorithm?
Does not work!
a little bird to answer this little question.
I ask a question about the solution.
“Which edge should we take first?”
How do I decide?
"Little Bird" Abstraction
Recall: Nondeterministic Finite Automata
Nondeterministic Turing Machine
0
These have a higher power to tell them which way to go.
The little bird is a little higher power, answering a little question about an optimal solution.
(It is up to you whether or not to use it)
But we don’t want toworry about how our friend
solves his problem.
I ask a question about the solution.
“Which edge should we take first?”
The bird answers <s,v1>.
My friend asks the next question.
“Which edge do we take second?”
The bird answers <v1,v4>.
Our instance is <G,s,t>: Find best path from s to t.
Our friend is recursion
The bird answers <s,v1>.
If I trust the little bird, I take step along edge <s,v1>and ask my friend,
“Which is the best path from v1 to t?”
Friend answers <v1,v6,t>with weight 10.
To get my solution
I tack on the bird’s edge making the path <s,v1,v6,t>
with weight
10+3=13.
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
In reality we will try all possible first edges,
giving …..
…the best path from amongst those starting with <s,v1>.
… and the best path from amongst those starting with <s,v2>.
… and the best path from amongst those starting with <s,v3>.
… and the best path from amongst those starting with <s,v4>.
I give the best of the best as the best path.
At least one of these four paths must be an over all best path.
Bird/Friend  Best of the Best
Consider our instance I.
Consider the set of solutions
A sequence of question to a little bird about a solutionforms a tree of possible answers.
Bird/Friend  Best of the Best
Consider our instance I.
Consider the set of solutions
k
But we only care aboutthe first bird answer.
k
The answers classifiesthe possible solutions.
Solutions consistent with the kth bird' s answer.
Bird/Friend  Best of the Best
optS<I,k>
Consider our instance I.
Consider the set of solutions
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
Do this for each k.
k
Solutions consistent with the kth bird' s answer.
Bird/Friend  Best of the Best
optS<I,k>
Consider our instance I.
optS[I]
Consider the set of solutions
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
Do this for each k.
kmax
k
Let kmax be the bird' s answergiving the best optS<I,k>.
optS[I] = optS<I,k >= Bestk optS<I,k >
max
Bird/Friend  Best of the Best
Constructing optS<I,k> :
the optimum solution
for instance I
consistent with the kth bird' s answer.
Given my instance I.
I ask my little bird foran answer k.
I ask my friend for his solution.
I combine them.
Recursive backtracking
code always has thissame basic structure.
Be clear what are
Loop through the bird answers.
Be clear which is the current one being tried.
Give the bird & friend algorithmas a comment.
(Unless it is in an earlier question.)
What is the bird asked?
What does she answer?
Get help from friend
Be clear what
subinstance you give him.
Store the solution& cost he gives you.
How do you formyour solution from the friend’s and fromthe bird’s?
How do you formyour cost from the friend’s and fromthe bird’s?
optSolkis a best solutionfor our instance from amongstthose consistent with the bird'skth answer.
Take the bestof the best
Return the solutionand cost for the original instance.
Base Cases:Instances that are too small to have smaller instances to give to friends.
What are these?
What are their solutionsand costs?
i.e. for a path from s to t to be optimal,the subpath from vi to t must optimal.
In order to be able to design a recursive backtracking algorithm for a computational problem,
the problem needs to have a recursive structure,
If shorter from vi to t.
shorter to s to t.
i.e. for a path from s to t to be optimal,the subpath from vi to t must optimal.
In order to be able to design a recursive backtracking algorithm for a computational problem,
the problem needs to have an optimal substructure,
And finding such a subpath is a subinstance of the samecomputational problem.
Optimal Substructure
Don’t all problems have this optimal substructure property?
Consider the following graph:
The longest simple path (path not containing a cycle) from A to D is A B C D
However, the subpath A B is not the longest simple path from A to B (A C B is longer)
The principle of optimality is not satisfied for this problem
Hence, the longest simple path problem cannot be solved by a dynamic programming approach
B
1
2
3
1
4
C
D
A
Optimal Substructure
NPComplete
Same as Brute Force Algorithm
I try each edge out of s.
A friend tries each edge out of these.
A friend tries each edge out of these.
Time?
Same as the brute force algorithm that tries each path.
Same as Brute Force Algorithm
But there may be an exponential number of paths!
Speeding Up the Time
Why do all this work with birds & friends?
Speeding Up the Time
Sometimes entire an branch can be pruned off.
Speeding Up the Time
Memoization
Exponential Time
Redoing Work
“Which is the best path from v7 to t?”
How many friends solve this subinstance?
Exponential Time
Redoing Work
“Which is the best path from s to t?”
Exponential Time
Redoing Work
“Which is the best path from v1to t?”
Exponential Time
Redoing Work
“Which is the best path from v4to t?”
Exponential Time
Redoing Work
“Which is the best path from v7to t?”
There’s one.
Exponential Time
Redoing Work
“Which is the best path from s to t?”
Exponential Time
Redoing Work
“Which is the best path from v3to t?”
Exponential Time
Redoing Work
“Which is the best path from v5to t?”
Exponential Time
Redoing Work
“Which is the best path from v7to t?”
There’s another.
Exponential Time
Redoing Work
“Which is the best path from v7to t?”
How many friends solve this subinstance?
Once for each path to v7
Waste time redoing work
Save time by only doing once.
Depth First
Search
Drop
bread crumbs and don’t revisit.
But we need shortest path
Dynamic Programming
We allocate one friend to the job.
Having many friends solving this same subinstanceis a waste of time.
Dynamic Programming
It is my job to learn and remember
the optSol to my subInstance
i.e. the best path from v7to t
Dynamic Programming
When I need to find
the best path from v4 to t
I will ask you for
the best path from v7 to t
I will find my best pathand tell you.
Dynamic Programming
When I need to find
the best path from v2 to t
I will ask you for
the best path from v7 to t
I remember my best pathand will tell you.
Dynamic Programming
When I need to find
the best path from v5 to t
I will ask you for
the best path from v7 to t
I remember my best pathand will tell you.
Dynamic Programming
Avoid waiting.
When I need to find
the best path from v2 to t
I will ask you for
the best path from v7 to t
I will find my best pathand tell you.
But I hate to wait for you.
Recursion has a lot of overhead
Why don’t you go first?
Dynamic Programming
Before anyone asks me,I will find my best path
and remember it.
Set of SubInstances
But what subinstance need to be solved
and
in which order?
Given an instance I,
Imagine running the recursive algorithm on it.
Determine the complete set of subInstances ever given to you, your friends, their friends, …
Set of SubInstances
v21
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
v21is not a part of ouroriginal instance.
Set of SubInstances
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
“Best path from v3to v7?”
No
All paths considered end in t.
Set of SubInstances
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
“Best path from v3to v7?”
No
All paths considered end in t.
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
“Best path from v3to v7?”
No
Yes
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
Assign one friend to each subInstance.
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
Integers closed under addition
x,yI x+y I
subInstance S
subsubInstance S
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
asked of some friend, friend, …
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
asked of some friend, friend, …
A fine set of subinstances!
i
“Best path from vi to t?”
Order to complete
The complete set S of subInstances is
In what order should they go?
For this problem, the order relies on the graph being “leveled.”
i
“Best path from vi to t?”
Instance to be solved.
Order to complete
The complete set S of subInstances is
In what order should they go?
First
Base Case easy
Last
Base Case
easy
Dynamic Programming
"Which is the best path from t to t?"
Dynamic Programming
"Which is the best path from v8 to t?"
easy
Dynamic Programming
"Which is the best path from v7 to t?"
easy
Dynamic Programming
"Which is the best path from v6to t?"
easy
Dynamic Programming
"Which is the best path from v5to t?"
Harder
Dynamic Programming
"Which is the best path from v5to t?"
Little bird suggests first edge <v5,v7>
Friend gives best
path <v7,t>.
Dynamic Programming
"Which is the best path from v5 to t?"
Little bird suggests first edge <v5,v8>
Friend gives best
path <v8,t>.
Dynamic Programming
"Which is the best path from v5 to t?"
Take best of best
Dynamic Programming
"Which is the best path from v4to t?"
Dynamic Programming
"Which is the best path from v4to t?"
Little bird suggests first edge <v4,v6>
Friend gives best
path <v7,t>.
Dynamic Programming
"Which is the best path from v4to t?"
Little bird suggests first edge <v4,t>
Friend gives best
path <t,t>.
Dynamic Programming
"Which is the best path from v4to t?"
Little bird suggests first edge <v4,v7>
Friend gives best
path <v7,t>.
Dynamic Programming
"Which is the best path from v4to t?"
Take best of best
Dynamic Programming
"Which is the best path from v3to t?"
Dynamic Programming
"Which is the best path from v3to t?"
Little bird suggests first edge <v3,v5>
Friend gives best
path <v5,t>.
Dynamic Programming
"Which is the best path from v3to t?"
Little bird suggests first edge <v3,v8>
Friend gives best
path <v8,t>.
Dynamic Programming
"Which is the best path from v3to t?"
Take best of best
Dynamic Programming
"Which is the best path from v2to t?"
Dynamic Programming
"Which is the best path from v2to t?"
Little bird suggests first edge <v2,v4>
Friend gives best
path <v4,t>.
Dynamic Programming
"Which is the best path from v2to t?"
Little bird suggests first edge <v2,v7>
Friend gives best
path <v7,t>.
Dynamic Programming
"Which is the best path from v2to t?"
Take best of best
Dynamic Programming
"Which is the best path from v1to t?"
Dynamic Programming
"Which is the best path from v1to t?"
Little bird suggests first edge <v1,v3>
Friend gives best
path <v3,t>.
Dynamic Programming
"Which is the best path from v1to t?"
Little bird suggests first edge <v1,v4>
Friend gives best
path <v4,t>.
Dynamic Programming
"Which is the best path from v1to t?"
Little bird suggests first edge <v1,v5>
Friend gives best
path <v5,t>.
Dynamic Programming
"Which is the best path from v1to t?"
Take best of best
Dynamic Programming
"Which is the best path from s to t?"
Original Problem
Dynamic Programming
"Which is the best path from s to t?"
Little bird suggests first edge <s,v1>
Friend gives best
path <v1,t>.
Dynamic Programming
"Which is the best path from s to t?"
Little bird suggests first edge <s,v2>
Friend gives best
path <v2,t>.
Dynamic Programming
"Which is the best path from s to t?"
Little bird suggests first edge <s,v3>
Friend gives best
path <v3,t>.
Dynamic Programming
"Which is the best path from s to t?"
Little bird suggests first edge <s,v4>
Friend gives best
path <v4,t>.
Dynamic Programming
"Which is the best path from s to t?"
Take best of best
DONE
i
“Best path from vi to t?”
Dynamic Programming
Construct a table
Map
SubInstances
i ϵ [n], i.e. for each node vi
Indexes
Cell of table
“Which is the best path from vi to t?”
t, v8, v7, v6, v5, …., s
i
Dynamic Programming
Fill out a table containing an optimal solution for each subinstance.
“Which is the best path from vi to t?”
t, v8, v7, v6, v5, …., s
Base case
Original
Communication
k
k
i
k
i
Friend k gives friend i a best path from vk to t.
Recursive BackTracking
<optSubSol,optSubCost> = LeveledGraph(<G,vk,t>)
i
return(optSolmin,optCostmin),
Dynamic Programming
optSol[k] = optSolmin
optSubSol = optSol[k]
optSolk = <vi,vk> + optSol[k]
Dynamic Programming
code always has thissame basic structure.
Be clear what are
Dynamic Programsdo not recurse making the instance smaller and smaller.
Instead, it up frontdetermines the set Sof all subinstancesthat ever need to be solved.
Be clear what subinstances are.
Be clear what subinstances are.
How are they indexed?
Tables indexed by these subinstancesstore an optimal solution and it’s cost.
The set Sof subinstancesare solved fromsmallest to largestso that no body waits.
Base Cases:Instances that are too small to have smaller instances to give to friends.
They get solved firstand their solutions stored.
Then we iterate through the remaining subinstances.
From smallest to largest.
Each gets solvedand their solutions stored.
Consider yourselfto be a friendworking on oneof these.
Be clear which subinstance is yours.
Solve this as you did before.
Loop through the bird answers.
Be clear which is the current one being tried.
Give the bird & friend algorithmas a comment.
(Unless it is in an earlier question.)
What is the bird asked?
What does she answer?
k
k
k
k
Get help from friend
Be clear what subinstance you give your friend.
k
k
k
k
Get help from friend
Instead of recursing,we simply look in the table for the solution.
Because his instance is smaller, he has already solved it and stored sol in the table.
How do you formyour solution from the friend’s and fromthe bird’s?
How do you formyour cost from the friend’s and fromthe bird’s?
optSol<i,k>is a best solution for our instance subI[i]from amongstthose consistent with the bird'skth answer.
Take the bestof the best
Store the solution to our instance subI[i]in the table.
Base Cases:Instances that are too small to have smaller instances to give to friends.
Is this code correct?
Dynamic Programsdo not recurse making the instance smaller and smaller.Hence, lets not worry about our instance I being a base case.
But there is a tableof subinstancesthat must be solved.
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
t=
n
n
n
n
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
Then we solvethe rest.
s=
0
0
n
0
Return the solutionand cost for the original instance.
Order Feels Backwards
Path from s to t.
Path from t to t.
An Esthetically Better
Path from s to s.
Path from s to t.
Reversing
Reversing
“Which is the best path from s to vi?”
i
Determine the complete set of subinstances
Reversing
Fill out a table containing an optimal solution for each subinstance.
“Which is the best path from s to vi?”
s, v1, v2, v3, v4, …., t
Base case
Original
Running Time
?
Time =
# of SubInstances
× # of Bird Answers
× q(1)
= n
× d
Communication Time
Friend k gives best path from s to vk
to friend i, who adds the edge <vk,vi>.
Time =
q(1)
k
?
Time =
q(n)
i
optSolk = <optSol[k],vi>
Size of path =
q(n).
Running Time
# of SubInstances
× q(1)
Store path costs, not paths
= q(n)
Time =
# of SubInstances
× # of Bird Answers
× size of solution
= q(n × d × n)
Space =
Store Path Costs, not Paths
"What is cost of the best path from s to v7?"
Store Path Costs, not Paths
"What is cost of the best path from s to v7?"
Little bird suggests last edge <v4,v7> with weight 2.
8
Friend gives cost 8
of best path <s,v4>.
Best cost via <v4,v7> is 8+2=10.
Store Path Costs, not Paths
2
"What is cost of the best path from s to v7?"
Little bird suggests last edge <v2,v7> with weight 7.
Friend gives cost 2
of best path <s,v2>.
Best cost via <v2,v7> is 2+7=9.
Store Path Costs, not Paths
"What is cost of the best path from s to v7?"
Little bird suggests last edge <v5,v7> with weight 5.
6
Friend gives cost 6
of best path <s,v5>.
Best cost via <v5,v7> is 6+5=11.
Store Path Costs, not Paths
9
2
We also learn the wise little bird’s advice.
We will store this in the table too.
"What is cost of the best path from s to v7?"
Take best of best:
10, 9, 11
Running Time
birdsAdvice[i] = kmin
Leave these lines
as comments
for extra clarity for the reader
Previous algorithm gives:
We run the birdfriend algorithm again, but with a reliable bird.
The bird gives that the last edge
of the best path from s to tis<v8,t>.
The bird gives that the last edge
of the best path from s to v8is<v5,v8>.
The bird gives that the last edge
of the best path from s to v5is<v3,v5>.
The bird gives that the last edge
of the best path from s to v3is<s,v3>.
Done!
This could be done iteratively.As an exercise, design it.
I ask the bird:
“Which is the last edge?”
She could give either answer.
By giving this edge she says
“There exists an optimal solution consistent with this answer.”
Similar to greedy proof.
6
I ask the bird:
“Which is the last edge?”
We try all the bird answers.
When we try this bird answer,
we find this best solution.
When we try this bird answer,
we find this best solution.
When we take best of best,
we choose between them.
6
Review
Designing Recursive Back Tracking Algorithm
Review
Recursive Back Tracking Algorithm
Dynamic Programming Algorithm
Purpose of Little Bird:
A solution: a path
t
s
I ask the bird:
“What is the first edge in the path?”
An instance: Graph, s, and t
The Dynamic Programming reverses the recursive backtracking algorithm.
Hence, to end up with a “forward order”,we first reverse the recursive backtracking algorithm.
A solution: a path
t
s
I ask the bird:
“What is the last edge in the path?”
An instance: Graph, s, and t
The Dynamic Programming reverses the recursive backtracking algorithm.
Hence, to end up with a “forward order”,we first reverse the recursive backtracking algorithm.
A solution: a path
t
s
I ask the bird:
“What is the rest of the path?”
An instance: Graph, s, and t
“What is the last edge in the path?”
A good question for the bird leaves you with a good recursive subinstance to ask your friend.
A solution: a path
t
s
I ask the bird:
An instance: Graph, s, and t
“What is the last edge in the path?”
Giving a good follow up question for your friend to ask the bird.
“What is the second last edge in the path?”
number of edges into node t.
t
s
K
# of solutions.
Trying all is the Brute Force algorithm.
A solution: a path
t
s
I ask the bird:
An instance: Graph, s, and t
“How many edges are in the path?”
Bad Question:
I ask the bird:
A solution: a sequence of objects
Z = a b c d
I ask my friend:
An instance: ???
“What is the last object in the sequence?”
# of answersK =
# of possible last objects.
“What is the rest of the solution?”
An instance: a sequence of objects
I ask the bird:
X = a s b e f c h d a
X =asbe fchda
A solution: a subset of these objects
Z = a b c d
“What is the last object in the sequence?”
# of answersK =
# of possible last objects.
Is there a smaller question that we could ask?
An instance: a sequence of objects
I ask the bird:
X = a s b e f c h d a
A solution: a subset of these objects
Z = a b c d
“Is the last object of the instance included in the optimal solution?”
# of answersK =
2, Yes or No
I ask the bird:
A solution: a binary tree of objects
38
25
51
I ask a second friend:
I ask my friend:
17
31
42
63
4
21
28
35
40
49
55
71
An instance: ???
“What object is at the root?”
“What is the left subtree?”
“What is the right subtree?”
Previous problems had one friend given a bird ans.
I ask the bird:
38
25
51
17
31
42
63
4
21
28
35
40
49
55
71
An instance: ???
A solution: a binary tree of objects
“What object is at a leaf?”
Bad Question:
A solution: # of words to put on each line.
few blanks on the end of each line.
3 = 83 = 83 = 2163 = 27 259
2263
Love.life.. man.while.. there......as.we.be...
11
An instance: text to print neatly & # chars per line
“Love life man while there as we be”, 11
The cost: a measure of how neat,
small punishmentbig punishment
The goal is to to print it as “neatly” as possible.
Brute Force Algorithm
Try all ways to print, return the best.
But there may be an exponential number of ways to!
love.life.. man........ love.......life.man...love.......life.man...love.life.. man........
I ask the bird:
I combine
bird’s and
friend’s answers.
I ask the friend:
An instance:
“Love life man while there as we be”, 11
“How many words on the last line?”
She may answer 3.
“Which is the best way to print the remaining n3 words?”
We try the bird answers words,
1
2
3
4
5
An instance:
“Love life man while there as we be”, 11
Even if the bird was wrong, this work is not wasted.
This is best way to print from amongst those ending in 3 words.
and take best of best.
Same as Brute Force Algorithm
I try each # words on last line.
A friend tries # on next.
A friend tries # on next.
Time?
Same as the brute force algorithm that tries each path.
Memoization
i
“Which is the best path from vi to t?”
Assign one friend to each subinstance.
Set of SubInstances
“Love life man while there as we be”, 11
Determine the complete set of subInstances.
Given an instance I,
Set of SubInstances
“Love life man while there as we be”, 11
“Love life man while there”, 11
“Hi there”, 81
“man while”, 11
Guess the complete set of subInstances.
Yes
No
No
This may appear on a line,but it will never be a subInstance for a friend.
Set of SubInstances
“Love life man while there as we be”, 11
“Love life man while there”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life”, 11
“Love”, 11
“”, 11
“Love life man while there as”, 11
“Love life man while there as we”, 11
The set of subInstances is the set of prefixes.
The set S of subInstances needs to:
subInstance S
Set of SubInstances
The bird answers words,
5
3
2
1
4
“Love life man while there as we”, 11
“Love life man while there as”, 11
“Love life man while there”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life”, 11
“Love”, 11
“”, 11
The set of subInstances is the set of prefixes.
“Love life man while there as we be”, 11
The set S of subInstances needs to:
subInstance S
subsubInstance i
Set of SubInstances
“Love life man while there as we”, 11
“Love life man while there as”, 11
“Love life man while there”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life”, 11
“Love”, 11
“”, 11
asked of some friend, friend, …
The set of subInstances is the set of prefixes.
“Love life man while there as we be”, 11
The set Sof subInstances needs to:
Set of SubInstances
“Love life man while there as we be”, 11
“Love life man while there as we”, 11
“Love life man while there as”, 11
“Love life man while there”, 11
“Love life man while”, 11
The bird
answers 1.
asked of some friend, friend, …
The set of subInstances is the set of prefixes.
The set S of subInstances needs to:
A fine set of subinstances!
Set of SubInstances
“Love life man while there as we be”, 11
“Love life man while there as we”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life man while there”, 11
“Love”, 11
“”, 11
“Love life man while there as”, 11
“Love life”, 11
The set of subInstances is the set of prefixes.
In what order should they go?
First
Base Case easy
Last
Instance to be solved.
The Table
Construct a table
Map
The set of prefixes of words.
SubInstances
i ϵ [n], i.e. for each word.
Indexes
Cell of table
“Which is the best printing of first i words?”
i
Dynamic Programming
Fill out a table containing an optimal solution for each subinstance.
“Which is the best printing of first i words?”
Base case
Original
“Love life man while there as we be”, 11
The 5th subinstance is
“Love life man while there”, 11
5 words
with 4, 4, 3, 5, 5 letters.
“Love life man while there as we be”, 11
Love.life.. man.while.. there......
The 5th subinstance is
“Love life man while there”, 11
Its solution is
with 2,2,1 words on each line.
The bird’s advice is 1 word on last.
Solution’s cost is
23 + 23 +63 = 232
“Love life man while there as we be”, 11
Assume the table is filled in so far.
We will work to fill in the last line
“Love life man while there as we be”, 11
Love.life.. man.while.. there.as.we
be.........
“Love life man while there as we be”, 11
Love.life.. man.while.. there.as...
we.be......
“Love life man while there as we be”, 11
Love.life.. man.while.. there......
as.we.be...
“Love life man while there as we be”, 11
there.as.we.be
“Love life man while there as we be”, 11
Tried all bird answers.
Choose best of the best.
“Love life man while there as we be”, 11
Choose best of the best.
Dynamic Programming
code always has thissame basic structure.
Amusingly,when formatting this code, I had to fight with line breaks to get the height/width ratioPrinting Neatly.
Be clear what are
Dynamic Programsdo not recurse making the instance smaller and smaller.
Instead, it up frontdetermines the set Sof all subinstancesthat ever need to be solved.
Be clear what subinstances are.
Be clear what subinstances are.
How are they indexed?
Tables indexed by these subinstancesstore an optimal solution and it’s cost.
The set Sof subinstancesare solved fromsmallest to largestso that no body waits.
Base Cases:Instances that are too small to have smaller instances to give to friends.
They get solved firstand their solutions stored.
Then we iterate through the remaining subinstances.
From smallest to largest.
Each gets solvedand their solutions stored.
Actually, we store thebird’s advice instead of the solution.
Consider yourselfto be a friendworking on oneof these.
Be clear which subinstance is yours.
Solve this as you did before.
Loop through the bird answers.
Be clear which is the current one being tried.
Give the bird & friend algorithmas a comment.
(Unless it is in an earlier question.)
What is the bird asked?
What does she answer?
ik
ik
ik
Get help from friend
Be clear what subinstance you give your friend.
ik
ik
ik
Instead of recursing,we simply look in the table for the solution.
Because his instance is smaller, he has already solved it and stored sol in the table.
How do you formyour solution from the friend’s and fromthe bird’s?
How do you formyour cost from the friend’s and fromthe bird’s?
optSol<i,k>is a best solution for our instance subI[i]from amongstthose consistent with the bird'skth answer.
Take the bestof the best
Store the solution toour instance subI[i]in the table.
Actually, we store thebird’s advice instead of the solution.
Base Cases:Instances that are too small to have smaller instances to give to friends.
Is this code correct?
Dynamic Programsdo not recurse making the instance smaller and smaller.Hence, lets not worry about our instance I being a base case.
But there is a tableof subinstancesthat must be solved.
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
Then we solvethe rest.
Return the solutionand cost for the original instance.
But actually,we don’t have the solution.
We must rerun it, this time with advice from the bird.
# of SubInstances
× # of Bird Answers
= q(n × n)
Time =
Space =
# of SubInstances
= q(n)
We run the birdfriend algorithm again, but with a reliable bird.
Previous algorithm gives cost and bird’s advice.
“Love life man while there as we be”, 11
,1
,3>
,2
<2
Love.life..
man.while..
there……
as we be…
Longest Common Subsequence problem
X =asbe fchda
Y = r t wabg jck t fd
An instance: Two strings
X = a s b e f c h d a
Y = r t w a b g j c k t f d
A solution: A common subsequence.
Z = a b c d
Longest Common Subsequence problem
X =asbe tchda
Y = r t wabg jck t fd
The cost: The length of Z.
The goal is to find a longest common subsequence.
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
“Is the last character of either X or Yincluded in Z?”
She answers one of :
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I ask my friend:
“Is the last character of either X or Y included in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I combine
bird’s and
friend’s answersand give
“Is the last character of either Xor Y included in Z?”
She answers:
My friend answers:
Z = a b c d
X =asbe tchd
Y = r t wabg jck t fd
the sameZ.
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d a
Y = r t w a b g j c k t f
I ask the bird:
I ask my friend:
“Is the last character of either X or Yincluded in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d a
Y = r t w a b g j c k t f
I ask the bird:
I combine
bird’s and
friend’s answersand give
“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
the same Z.
Z = a b c
X =asbe tchd a
Y = r t wabg jck t f
Not as good as lastbut we need to try.
The instance:
X = a s b e t c h
Y = r t w a b g j c k d f
I ask the bird:
I ask my friend:
An instance:
X = a s b e t c h d
Y = r t w a b g j c k d f d
Last chars equal
“Is the last character of either X or Y included in Z?”
She answers:
The instance:
X = a s b e t c h
Y = r t w a b g j c k d f
I ask the bird:
I combine
bird’s and
friend’s answersand give
An instance:
X = a s b e t c h d
Y = r t w a b g j c k d f d
Last chars equal
“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
Z = a b c
X =asbe tch
Y = r t wabg jck d f
Zd = abcd.
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Last chars not equal
“Is the last character of either X or Y included in Z?”
She answers:
I politely tell her that she is wrong.
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I ask my friend:
“Is the last character of either X or Y included in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I combine
bird’s and
friend’s answersand give
“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
Z = a b c d
X =asbe tch d
Y = r t wabg jck t f d
Wrong
Za = abcda.
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w
I ask the bird:
I ask my friend:
“Is the last character of either X or Y included in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w
I ask the bird:
I combine
bird’s and
friend’s answersand give
“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
Z = t
X =a s b e t c h d
Y = r t w
Za = ta.
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Last chars not equal
“Is the last character of either X or Yincluded in Z?”
Can we eliminatesome of her answers?
She answers one of :
Given any optSolshe needs to havea valid answer.
?
?
?
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Last chars not equal
“Is the last character of either X or Y included in Z?”
She answers one of :
3
# of answersK =
Same as Brute Force Algorithm
Time?
I try each of 3 bird ans.
My friends try 3
His friends try 3
Same as the brute force algorithm that tries each solution.
Memorization
“Which is the best path from vito t?”
i
Assign one friend to each subinstance.
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
X = a s b e t c h d a
Y = r t w a b g j c k t f d
Is this a subInstance?
X’ = a s b e t c
Y’ = r t w a b g j c k
Yes
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
X = a s b e t c h d a
Y = r t w a b g j c k t f d
friends, …
Is this a subInstance?
No
X’ = b e t
Y’ = a b g j c k
Set of SubInstances
i [0..X]
j [0..Y]
Determine the complete set of subInstances.
Given an instance I,
X = a s b e t c h d a
Y = r t w a b g j c k t f d
friends, …
Is this a subInstance?
Yes
X’ = x1,…xiY’ = y1,…,yj
X × Y of these.
Set of SubInstances
Xi = x1,…xiYj = y1,…,yj
i [0..X]
j [0..Y]
Guess the complete set S of subInstances.
The set S of subInstances needs to:
Yes: i = X & j = Y
Set of SubInstances
Xi = x1,…xiYj = y1,…,yj
Xi1 = x1,…xi1Yj = y1,…,yj
Xi = x1,…xiYj = y1,…,yj
S
Xi = x1,…xiYj1 = y1,…,yj1
Xi1 = x1,…xi1Yj1 = y1,…,yj1
i [0..X]
j [0..Y]
Guess the complete set S of subInstances.
The set S of subInstances needs to:
subInstance S
subsubInstance S
S
Set of SubInstances
Xi = x1,…xiYj = y1,…,yj
asked of some friend, friend, …
i [0..X]
j [0..Y]
Guess the complete set S of subInstances.
The set S of subInstances needs to:
subInstance S
subsubInstance S
We showed this.
This is a fine set of subInstances!
The Table
i [0..X]
j [0..Y]
Construct a table
Map
Xi = x1,…xiYj= y1,…,yj
SubInstances
Indexes
“LCS of x1,…xiand y1,…,yj ?”
Cell of table
j
i
The Table
Y
X
Original instance
I = <X,Y>
subInstancei,j=
Yj
j=
Xi = x1,…xi
Yj = y1,…,yj
Xi
i=
Optimal Solution
= Longest Common
Subsequence
Cost = length of LCS.
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Optimal Solution
= Longest Common
Subsequence
Bird’s Advice
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Optimal Solution
= Longest Common
Subsequence
Bird’s Advice
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Optimal Solution
= Longest Common
Subsequence
Bird’s Advice
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
5
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
5
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost +1
6
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
6
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
4
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
3
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Yj
Xi
Fill in box
Sorry bird is wrong.
Our cost
= 

subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
4
Yj
Xi
Fill in box
Fill in Box
Fill in Box
Order to Fill in Table
Order to fill table:
This guy waits for
Order to Fill in Table
(later)
Base Cases
Base Cases:
does not work
Base Cases
Base Cases:
does not work
Base Cases
With Advice
With Advice
Done
Get as much value
as you can
into the knapsack
Ingredients:
v=4,p=4
v=4,p=4
V=8
Greedy Criteria:
Most valuable pi
v=4,p=4
v=4,p=4
v=7,p=5
V=8
gives 8
Optimal
Greedy give 5
Most dense in value
pi
vi
v=7,p=5
V=7
Greedy Criteria:
v=4,p=4
v=4,p=4
v=7,p=5
V=8
V=7
gives 5
Optimal
Greedy give 4
Most dense in value
pi
vi
¾ of
v=4,p=4
v=7,p=5
V=7
gives 5
Optimal
Greedy Criteria:
If fractional solutions are allowed.
Works
Often an Integersolution is MUCH
harder to find.
v=4,p=4
V=7
+ ¾ × 4 = 7
Greedy give 4
= Optimal
I ask the bird:
v=4,p=4
v=4,p=4
V=9
v=7,p=5
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
A solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
“What is the last object to take?”
n
# of answers K =
I ask the bird:
v=4,p=4
v=4,p=4
V=9
v=7,p=5
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
A solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
“Do we keep the last object?”
2 Yes & No
# of answers K =
I ask my friend:
Bird says, Yes keep the last object.
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
v=4,p=4
v=4,p=4
V=9
v=7,p=5
Trust her and put it into your knapsack.
To fill the rest of the knapsack.
But what instance do I give him?
His solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
<<v5,p5>,<v9,p9>,...........,<v82,p82>,<vn,pn>>
His instance:
<Vvn:<v1,p1>,<v2,p2>,.........<vn1,pn1>,<vn,pn>>.
v=4,p=4
v=4,p=4
V=94
v=7,p=5
My solution:
same+ pn
My cost:
v=4,p=4
v=4,p=4
V=9
v=7,p=5
<<v5,p5>,<v9,p9>,...........,<v82,p82>,<vn,pn>>
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
If we trust the bird and friend,
this is valid and optimal.
My solution:
same +pn
My cost:
I ask my friend:
Bird says, No do not keep the last object.
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
v=4,p=4
v=4,p=4
V=9
v=7,p=5
Trust her and delete it.
To fill the knapsack with the rest.
What instance do I give him?
His solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
His instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
v=4,p=4
v=4,p=4
V=9
v=7,p=5
My solution:
same
If we trust the bird and friend,
this is valid and optimal.
same
My cost:
Same as Brute Force Algorithm
Time?
I try each of 2 bird ans.
My friends tries 2
His friends tries 2
Same as the brute force algorithm that tries each solution.
Memoization
“Which is the best path from vito t?”
i
Assign one friend to each subinstance.
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Is this a subInstance?
<V:<v1,p1>,<v2,p2>,<v3,p3>>.
Yes, if the bird keeps saying “No”.
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Is this a subInstance?
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
No, the set of objects is always a prefix of the original set.
Set of SubInstances
Determine the complete set of subInstances.
Is this a subInstance?
<V’:<v1,p1>,<v2,p2>,<v3,p3>>.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Quite possibly, if V’V.
It is easier to solve than to determine if it is a subinstance.
Set of SubInstances
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
No
S
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
Guess the complete set S of subInstances.
The set S of subInstances needs to:
Yes: V’=V & i = n
subInstance S
subsubInstance S
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>> S
<V’ :<v1,p1>,<v2,p2>,...,<vi1,pi1>>
<V’vi:<v1,p1>,<v2,p2>,...,<vi1,pi1>>
Yes
The Table
Construct a table
Map
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
SubInstances
i [0..n]
V’ [0..V]
Indexes
“Which of first i objects to put in a knapsack of size v’?”
Cell of table
v’
i
The Table
The complete setS of subInstances.
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
same +pi
same
No
1
0
2
V’vi
V’
V
OptSol Cost & Bird’s Advicefor this
subInstance
0
1
2
i1
Yes
i
Take best of best.
Our cost?
n
The Table
The complete set S of subInstances.
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
1
0
2
V’vi
V’
V
OptSol Cost & Bird’s Advicefor this
subInstance
0
1
2
i1
Order to fill
so nobody
waits?
i
n
The Code
Running Time
The complete set S of subInstances is
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
No
Yes
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
Running time
= ( # of subinstances × # bird answers )
= (Vn× 2 )
= ( 2#bits in V× n )
Polynomial?
Exponential in “size” in instance!
The Knapsack Problem
The Knapsack Problem
If there is a polytime algorithmfor the Knapsack Problem
For EVERY optimization problem there is a polytime algorithm.
The Knapsack Problem
Likely there is not a polytime algorithmfor the Knapsack Problem.
Likely there is not a polytime algorithmfor EVERY optimization problem.
The Knapsack Problem
done
Schedule as
many events
in your room
as possible
The Job/Event Scheduling Problem
Ingredients:
Greedy Criteria:
Earliest Finishing Time
Motivation:
Schedule the event which will free up your room for someone else as soon as possible.
Weighted Event Scheduling
Ingredients:
100
1
1
Greedy Criteria:
Earliest Finishing Time
Motivation:
Schedule the event which will free up your room for someone else as soon as possible.
I ask the bird:
An instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
A solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
“What is the last event to take?”
n
# of answers K =
I ask the bird:
An instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
A solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
“Do we keep the last event?”
# of answers K =
2 Yes & No
I ask the bird:
“Do we keep the last event?”
I ask my friend:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
An instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
She answers: No
His solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
My solution:
same
My cost:
same
I ask the bird:
“Do we keep the last event?”
I ask my friend:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
An instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
Carefull
She answers: Yes
His solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
My solution:
same + <sn,fn,wn>.
same +wn
My cost:
Bird answers:
last event
“Yes keep the last event.”
No this solution is not valid!
Give the rest to my friend.
Here is my best subsolution.
I add to his subsolution the bird’s answer.
Bird answers:
“Yes keep the last event.”
No we trust the bird!
Then I should politely tell the bird she is wrong
Bird answers:
“Yes keep the last event.”
No we trust the bird!
You only tell her she is wrong if you really know.Eg k words don’t fit on the last lineThe bear does not fit into the knapsack
Bird answers:
“Yes keep the last event.”
No we trust the bird!
Your friend could have just as easily given you this subsolution that does not conflict with the bird’s answer.
Bird answers:
“Yes keep the last event.”
No we trust the bird!
Or maybe he needs to make a sacrifice when finding his answer in order that the overall solution is the best.
Bird answers:
“Yes keep the last event.”
No we trust the bird!
Or goal now is to find the best solution to our instance that is consistent with the bird’s answer.Then we will take the best of best.
Bird answers:
“Yes keep the last event.”
No we trust the bird!
Dude! It is your job to give me the right subinstance so that I give you a subsolution that does not conflict with the bird
Bird answers:
last event
“Yes keep the last event.”
My instance:
Cant keep any events that overlap with it.
Bird answers:
last event
“Yes keep the last event.”
I ask my friend:
My instance:
Bird answers:
“Yes keep the last event.”
I ask my friend:
His instance:
His solution
Bird answers:
“Yes keep the last event.”
I ask my friend:
My instance:
My solution:
My solution:
same + <sn,fn,wn>.
Yes
Valid?
Bird answers:
“Yes keep the last event.”
I ask my friend:
My instance:
My solution:
My solution:
same + <sn,fn,wn>.
same+wn
My cost:
Same as Brute Force Algorithm
Time?
I try each of 2 bird ans.
My friends tries 2
His friends tries 2
Same as the brute force algorithm that tries each solution.
Memorization
“Which is the best path from vi to t?”
i
Assign one friend to each subinstance.
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
Every subset of {1,…,9}is a possible subInstance.
I.e. could be an exponentialnumber of them.
Hence, running time is exponential.
Greedy algorithm sortedjobs by finishing time.Let us do that too.
Each subInstance is prefix.
I.e. only n of them.
Hence, running time is polynomial!
Set of SubInstances
i [0..n]
<<s1,f1,w1>,<s2,f2,w2>,… ,<si,fi,wi>>
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................… ,<sn,fn,wn>>.
Guess the complete set S of subInstances.
The set S of subInstances needs to:
Yes: i = n
?
subInstance S
subsubInstance S
asked of some friend, friend…
Only nsubInstances
Good enough.
Set of SubInstances
subInstance =
last event
<<s1,f1,w1>,<s2,f2,w2>,................................,<si,fi,wi>>
Show closed under “friend” operation
subInstance S
subsubInstance S
Events sorted by finishing time.
Set of SubInstances
Bird answers:
subInstance =
last event
<<s1,f1,w1>,<s2,f2,w2>,................................,<si,fi,wi>>
“Yes keep the last event.”
Show closed under “friend” operation
subInstance S
subsubInstance S
Delete overlapping events for friend.
Set of SubInstances
Bird answers:
subsubInstance =
<<s1,f1,w1>,<s2,f2,w2>,.....,<sj,fj,wj>>
“Yes keep the last event.”
Show closed under “friend” operation
subInstance S
subsubInstance S
Delete overlapping events for friend.
Set of SubInstances
subsubInstance =
<<s1,f1,w1>,<s2,f2,w2>,.....,<sj,fj,wj>>
Event j is kept fjsi
Show closed under “friend” operation
subInstance S
subsubInstance S
typical kept job
typical deleted job
set of kept jobs is a prefix of events.
subsubInstance S
Set of SubInstances
i [0..n]
<<s1,f1,w1>,<s2,f2,w2>,… ,<si,fi,wi>>
0, 1, 2, 3, 4, …. n
Base case
Original
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................… ,<sn,fn,wn>>.
The complete set S of subInstances is
Table:
Set of SubInstances
i [0..n]
<<s1,f1,w1>,<s2,f2,w2>,… ,<si,fi,wi>>
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................… ,<sn,fn,wn>>.
The complete set S of subInstances is
Running time
= # of subinstances × # bird answers
= n× 2
But to find your friend’s “yes” subinstanceyou must know how many events overlapwith your last event. This takes time:
O(logn) using binary searchfor a total of O(nlogn) time.
Done
Input:
s=6*8+((2+42)*(5+12)+987*7*123+15*54)
Output:
Recursive Alg:
Parsing
C CB
AC
c
d
B TA
BC
b
e
AAA
BT
a
TAB
CA
TT
Context Free Grammar
(Not look ahead one)
For ease, we will assume every nonterminal
either goes to two nonterminals
or to one terminal
Parsing
T
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Input:
start nonterminal = T
string to parse
= a1a2a3 ..... an
= baeaadbda
T a1a2a3 ..... an
Output: A parsing
Parsing
Input:
T a1a2a3 ..... an
T
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Recursive Algorithm:
GetT does not know whether to call
GetA, GetC, or GetT.
Parsing
Input:
T a1a2a3 ..... an
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b a e a
CCB
AC
c
d
B TA
BC
b
e
A AA
BT
a
T AB
CA
TT
T
Ask Little Bird:
Ask Friend
Ask Another Friend
b d a
Parsing
Input:
T a1a2a3 ..... an
A
C
a d b
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
T
Ask Little Bird:
Instance to give Friend
b d a
Parsing
Input:
T a1a2a3 ..... an
T
A
C
A
B
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
C CB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Ask Little Bird:
Want from Friend:
Instance to give him:
C
Parsing
Input:
T a1a2a3 ..... an
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b a e a
C CB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
T
Ask Little Bird:
How can we know split?
b d a
Parsing
Input:
T a1a2a3 ..... an
A
C
a d b
b a e a
C CB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
T
Ask Little Bird:
# of ans K =
mT = # of rules for T.
n = # chars in string.
# of ans K =
mT×n.
Total # of ans K =
b d a
Parsing
Input:
T a1a2a3 ..... an
T
A
C
A
B
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Ask left friend:
C
Parsing
Input:
T a1a2a3 ..... an
T
C
T
B
A
C
a d b
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
A
Ask right friend:
b d a
Parsing
Input:
T a1a2a3 ..... an
T
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Combine:
Same as Brute Force Algorithm
Time?
I try each of 2 bird ans.
My friends tries 2
His friends tries 2
Same as the bruteforce algorithm that tries each solution.
Memoization
“Which is the best path from vi to t?”
i
Assign one friend to each subinstance.
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
Set of SubInstances
Determine the complete set of subInstances.
A
gives:
gives:
A
My instance I:
T a1a2a3 ..... an
T
C
C
My left subInstance.
His right subInstance.
a d b
b d a
b a e a
Set of SubInstances
Determine the complete set of subInstances.
T’=
C
a1...ai1
aiai+1...aj
aj+1...an
a d b
My instance I:
T a1a2a3 ..... an
subInstances:
T’ aiai+1 ..... aj
# of nonterminals ×n2
# of subInstances =
The Table
Construct a table
Map
SubInstances
T’ aiai+1 ..... aj
T’ i,j [1,n]
Indexes
i
T’
Cell of table
j
Running Time
gives: First rule and split
subInstances:
T’ aiai+1 ..... aj nonterminals T’
& i,j [1,n]
Running time
= ( # of subinstances × # bird answers )
= (# of nonterminals × n2
× # of rules · n )
Done
Find a Satisfying Assignment
An instance (input) consists of a circuit:
c =(x3orx5orx6) and (x2orx5orx7) and (x3orx4)
true
true
false
true
true
true
true
A solution is an assignment of the variables.
true
x1= 0,x2= 1,x3= 0,x4= 0,x5= 1,x6= 0,x7= 1
The cost of a solution is
The goal is to find satisfying assignment.
Find a Satisfying Assignment
Ask the little bird
Instance:
c = (x3orx5orx6) and (x2orx5orx7) and (x3orx4)
Value of x1 in an optimal solution
or even better
Value of x3 in an optimal solution
We will have to try both x3 = 0 and x3 = 1.
For now, suppose she answered x3 = 0.
Find a Satisfying Assignment
true
false
true
SubInstance:
c =(x2orx5orx7) and x4
Friend gives SubSolution:
x1= 0,x2= 1,x4= 0,x5= 1,x6= 0,x7= 1
Our Solution:
x1= 0,x2= 1,x3= 0,x4= 0,x5= 1,x6= 0,x7= 1
Instance:
c =(x3orx5orx6) and (x2orx5orx7) and (x3orx4)
Commit to x3 = 0 and simplify
x3
1
0
x2
x1
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
In the end, some friend looks at each of the 2n assignments,
Memoization
“Which is the best path from vi to t?”
i
Assign one friend to each subinstance.
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
Set of SubInstances
Determine the complete set of subInstances.
Given an instance I,
c =(x1 or y1) and (x2 or y2) and (x3 or y3) and (x4 or y4)
Is this a subInstance?
c = (x1) and (x3)
Commit to
y1=0, y2=1, y3=0, y4=1,
and simplify
Yes
True for any subset of the xi.
could be an exponential # of different subInstances.
running time is exponential.
x3
1
0
x2
x1
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
In the end, some friend looks at each of the 2n assignments,
x3
1
0
x2
x1
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
But sometimes we can prune off branches.
Find a Satisfying Assignment
x3 is forced to x3 = 0
x3
1
0
x2
x1
c =(x2orx5orx7) and x3
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Instance:
Find a Satisfying Assignment
This is trivially unsatisfiable because x3 can’t be both 0 and 1.
x3
1
0
x2
x1
c =(x2orx5orx7) and x3and x3
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Instance:
Review
Designing Recursive Back Tracking Algorithm
Review
Recursive Back Tracking Algorithm
Dynamic Programming Algorithm
find the solution to your instance.
38
25
51
17
31
42
63
4
21
28
35
40
49
55
71
Matrix Multiplication
i
“Best path from vi to t?”
Dynamic Programming
Construct a table
Map
SubInstances
i ϵ [n], i.e. for each node vi
Index
Cell of table
“Which is the best path from vi to t?”
t, v8, v7, v6, v5, …., s
i