Thinking about Algorithms Abstractly. Recursive Back Tracking & Dynamic Programming. Jeff Edmonds York University. Lecture 7. COSC 3101. Problems. Techniques. Optimization Problems A Sequence of Decisions The Little Bird & Friend Optimal Substructure Memoization Set of SubInstances
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Thinking about Algorithms Abstractly
Jeff Edmonds York University
Lecture7
COSC 3101
Techniques
Optimization Problems
A Sequence of Decisions
The Little Bird & Friend
Optimal Substructure
Memoization
Set of SubInstances
Tracing Dyn. Prog. Alg
Reversing
Code
Speeding Up Running Time
Multiple Opt Solutions
Review
Question for Little Bird
Review & Don'ts
Best Path
Printing Neatly
Longest Common Subsequence
Knapsack Problem
The Event Scheduling Problem
Parsing
Satisfiability
Ingredients:
Specification of an Optimization Problem
We use it because it nicely demonstrates the concepts in a graphical way.
An instance (input) consists of <G,s,t>.
G is a weighted directed layered graphs source nodet sink node
5
3
4
2
3
2
5
2
2
1
3
4
Search Graph For Best PathA solution for an instance is a path from s to t.
The cost of a solution is the sum of the weights.
The goal is to find a path with minimum total cost.
2+6+3+7=18
4+2+1+5=12
Search Graph For Best PathAn instance (input) consists of <G,s,t>.
Try all paths, return the best.
But there may be an exponential number of paths!
I ask a question about the solution.
“Which edge should we take first?”
Some how I decide <s,v3>.
My friend asks the next question.
“Which edge do we take second?”
Some how he decides <v3,v5>.
His friend asks the next question.
“Which edge do we take third?”
Some how he decided <v5,v8>.
I ask a question about the solution.
“Which edge should we take first?”
How do I decide?
The greedy algorithm?
Does not work!
a little bird to answer this little question.
I ask a question about the solution.
“Which edge should we take first?”
How do I decide?
Recall: Nondeterministic Finite Automata
Nondeterministic Turing Machine
0
These have a higher power to tell them which way to go.
The little bird is a little higher power, answering a little question about an optimal solution.
(It is up to you whether or not to use it)
But we don’t want toworry about how our friend
solves his problem.
Little Bird & Friend AlgI ask a question about the solution.
“Which edge should we take first?”
The bird answers <s,v1>.
My friend asks the next question.
“Which edge do we take second?”
The bird answers <v1,v4>.
Our instance is <G,s,t>: Find best path from s to t.
Our friend is recursion
The bird answers <s,v1>.
If I trust the little bird, I take step along edge <s,v1>and ask my friend,
“Which is the best path from v1 to t?”
Friend answers <v1,v6,t>with weight 10.
To get my solution
I tack on the bird’s edge making the path <s,v1,v6,t>
with weight
10+3=13.
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
But what if we do not have a bird that we trust?
This work is not wasted, because we have found
In reality we will try all possible first edges,
giving …..
…the best path from amongst those starting with <s,v1>.
… and the best path from amongst those starting with <s,v2>.
… and the best path from amongst those starting with <s,v3>.
… and the best path from amongst those starting with <s,v4>.
I give the best of the best as the best path.
Faulty BirdAt least one of these four paths must be an over all best path.
Bird/Friend  Best of the Best
Consider our instance I.
Consider the set of solutions
A sequence of question to a little bird about a solutionforms a tree of possible answers.
Bird/Friend  Best of the Best
Consider our instance I.
Consider the set of solutions
k
But we only care aboutthe first bird answer.
k
The answers classifiesthe possible solutions.
Solutions consistent with the kth bird' s answer.
Bird/Friend  Best of the Best
optS<I,k>
Consider our instance I.
Consider the set of solutions
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
Do this for each k.
k
Solutions consistent with the kth bird' s answer.
Bird/Friend  Best of the Best
optS<I,k>
Consider our instance I.
optS[I]
Consider the set of solutions
Define optS<I,k> to be:
the optimum solution
for instance I
consistent with the kth bird' s answer.
Do this for each k.
kmax
k
Let kmax be the bird' s answergiving the best optS<I,k>.
optS[I] = optS<I,k >= Bestk optS<I,k >
max
Bird/Friend  Best of the Best
Constructing optS<I,k> :
the optimum solution
for instance I
consistent with the kth bird' s answer.
Given my instance I.
I ask my little bird foran answer k.
I ask my friend for his solution.
I combine them.
Recursive backtracking
code always has thissame basic structure.
Loop through the bird answers.
Be clear which is the current one being tried.
Give the bird & friend algorithmas a comment.
(Unless it is in an earlier question.)
What is the bird asked?
What does she answer?
Be clear what
subinstance you give him.
Store the solution& cost he gives you.
How do you formyour solution from the friend’s and fromthe bird’s?
How do you formyour cost from the friend’s and fromthe bird’s?
optSolkis a best solutionfor our instance from amongstthose consistent with the bird'skth answer.
Take the bestof the best
Return the solutionand cost for the original instance.
Base Cases:Instances that are too small to have smaller instances to give to friends.
What are these?
What are their solutionsand costs?
i.e. for a path from s to t to be optimal,the subpath from vi to t must optimal.
Optimal SubstructureIn order to be able to design a recursive backtracking algorithm for a computational problem,
the problem needs to have a recursive structure,
If shorter from vi to t.
shorter to s to t.
i.e. for a path from s to t to be optimal,the subpath from vi to t must optimal.
Optimal SubstructureIn order to be able to design a recursive backtracking algorithm for a computational problem,
the problem needs to have an optimal substructure,
And finding such a subpath is a subinstance of the samecomputational problem.
The longest simple path (path not containing a cycle) from A to D is A B C D
However, the subpath A B is not the longest simple path from A to B (A C B is longer)
The principle of optimality is not satisfied for this problem
Hence, the longest simple path problem cannot be solved by a dynamic programming approach
B
1
2
3
1
4
C
D
A
Longest simple pathOptimal Substructure
NPComplete
I try each edge out of s.
A friend tries each edge out of these.
A friend tries each edge out of these.
Time?
Same as the brute force algorithm that tries each path.
But there may be an exponential number of paths!
Why do all this work with birds & friends?
Sometimes entire an branch can be pruned off.
Memoization
Redoing Work
“Which is the best path from v7 to t?”
How many friends solve this subinstance?
Redoing Work
“Which is the best path from v7to t?”
How many friends solve this subinstance?
Once for each path to v7
Waste time redoing work
Save time by only doing once.
We allocate one friend to the job.
Having many friends solving this same subinstanceis a waste of time.
It is my job to learn and remember
the optSol to my subInstance
i.e. the best path from v7to t
When I need to find
the best path from v4 to t
I will ask you for
the best path from v7 to t
I will find my best pathand tell you.
When I need to find
the best path from v2 to t
I will ask you for
the best path from v7 to t
I remember my best pathand will tell you.
When I need to find
the best path from v5 to t
I will ask you for
the best path from v7 to t
I remember my best pathand will tell you.
Avoid waiting.
When I need to find
the best path from v2 to t
I will ask you for
the best path from v7 to t
I will find my best pathand tell you.
But I hate to wait for you.
Recursion has a lot of overhead
Why don’t you go first?
But what subinstance need to be solved
and
in which order?
Given an instance I,
Imagine running the recursive algorithm on it.
Determine the complete set of subInstances ever given to you, your friends, their friends, …
v21
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
v21is not a part of ouroriginal instance.
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
“Best path from v3to v7?”
No
All paths considered end in t.
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
“Best path from v3to v7?”
No
All paths considered end in t.
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances.
“Best path from v7to t?”
Yes
“Best path from v21to t?”
No
“Best path from v3to v7?”
No
Yes
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
Assign one friend to each subInstance.
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
Integers closed under addition
x,yI x+y I
subInstance S
subsubInstance S
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
asked of some friend, friend, …
i
“Best path from vi to t?”
Set of SubInstances
Guess the complete set S of subInstances is
The set S of subInstances needs to:
asked of some friend, friend, …
A fine set of subinstances!
i
“Best path from vi to t?”
Order to complete
The complete set S of subInstances is
In what order should they go?
For this problem, the order relies on the graph being “leveled.”
i
“Best path from vi to t?”
Instance to be solved.
Order to complete
The complete set S of subInstances is
In what order should they go?
First
Base Case easy
Last
"Which is the best path from v5to t?"
Little bird suggests first edge <v5,v7>
Friend gives best
path <v7,t>.
"Which is the best path from v5 to t?"
Little bird suggests first edge <v5,v8>
Friend gives best
path <v8,t>.
"Which is the best path from v4to t?"
"Which is the best path from v4to t?"
Little bird suggests first edge <v4,v6>
Friend gives best
path <v7,t>.
"Which is the best path from v4to t?"
Little bird suggests first edge <v4,t>
Friend gives best
path <t,t>.
"Which is the best path from v4to t?"
Little bird suggests first edge <v4,v7>
Friend gives best
path <v7,t>.
"Which is the best path from v3to t?"
"Which is the best path from v3to t?"
Little bird suggests first edge <v3,v5>
Friend gives best
path <v5,t>.
"Which is the best path from v3to t?"
Little bird suggests first edge <v3,v8>
Friend gives best
path <v8,t>.
"Which is the best path from v2to t?"
"Which is the best path from v2to t?"
Little bird suggests first edge <v2,v4>
Friend gives best
path <v4,t>.
"Which is the best path from v2to t?"
Little bird suggests first edge <v2,v7>
Friend gives best
path <v7,t>.
"Which is the best path from v1to t?"
"Which is the best path from v1to t?"
Little bird suggests first edge <v1,v3>
Friend gives best
path <v3,t>.
"Which is the best path from v1to t?"
Little bird suggests first edge <v1,v4>
Friend gives best
path <v4,t>.
"Which is the best path from v1to t?"
Little bird suggests first edge <v1,v5>
Friend gives best
path <v5,t>.
"Which is the best path from s to t?"
Little bird suggests first edge <s,v1>
Friend gives best
path <v1,t>.
"Which is the best path from s to t?"
Little bird suggests first edge <s,v2>
Friend gives best
path <v2,t>.
"Which is the best path from s to t?"
Little bird suggests first edge <s,v3>
Friend gives best
path <v3,t>.
"Which is the best path from s to t?"
Little bird suggests first edge <s,v4>
Friend gives best
path <v4,t>.
i
“Best path from vi to t?”
Dynamic Programming
Construct a table
Map
SubInstances
i ϵ [n], i.e. for each node vi
Indexes
Cell of table
“Which is the best path from vi to t?”
t, v8, v7, v6, v5, …., s
i
Fill out a table containing an optimal solution for each subinstance.
“Which is the best path from vi to t?”
t, v8, v7, v6, v5, …., s
Base case
Original
k
k
i
k
i
Friend k gives friend i a best path from vk to t.
Recursive BackTracking
<optSubSol,optSubCost> = LeveledGraph(<G,vk,t>)
i
return(optSolmin,optCostmin),
Dynamic Programming
optSol[k] = optSolmin
optSubSol = optSol[k]
optSolk = <vi,vk> + optSol[k]
Dynamic Programming
code always has thissame basic structure.
Dynamic Programsdo not recurse making the instance smaller and smaller.
Instead, it up frontdetermines the set Sof all subinstancesthat ever need to be solved.
Be clear what subinstances are.
Be clear what subinstances are.
How are they indexed?
Tables indexed by these subinstancesstore an optimal solution and it’s cost.
The set Sof subinstancesare solved fromsmallest to largestso that no body waits.
Base Cases:Instances that are too small to have smaller instances to give to friends.
They get solved firstand their solutions stored.
Then we iterate through the remaining subinstances.
From smallest to largest.
Each gets solvedand their solutions stored.
Consider yourselfto be a friendworking on oneof these.
Be clear which subinstance is yours.
Solve this as you did before.
Loop through the bird answers.
Be clear which is the current one being tried.
Give the bird & friend algorithmas a comment.
(Unless it is in an earlier question.)
What is the bird asked?
What does she answer?
k
k
k
Get help from friend
Instead of recursing,we simply look in the table for the solution.
Because his instance is smaller, he has already solved it and stored sol in the table.
How do you formyour solution from the friend’s and fromthe bird’s?
How do you formyour cost from the friend’s and fromthe bird’s?
optSol<i,k>is a best solution for our instance subI[i]from amongstthose consistent with the bird'skth answer.
Take the bestof the best
Store the solution to our instance subI[i]in the table.
Base Cases:Instances that are too small to have smaller instances to give to friends.
Is this code correct?
Dynamic Programsdo not recurse making the instance smaller and smaller.Hence, lets not worry about our instance I being a base case.
But there is a tableof subinstancesthat must be solved.
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
t=
n
n
n
n
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
Then we solvethe rest.
Fill out a table containing an optimal solution for each subinstance.
“Which is the best path from s to vi?”
s, v1, v2, v3, v4, …., t
Base case
Original
Friend k gives best path from s to vk
to friend i, who adds the edge <vk,vi>.
Time =
q(1)
k
?
Time =
q(n)
i
optSolk = <optSol[k],vi>
Size of path =
q(n).
# of SubInstances
× q(1)
Store path costs, not paths
= q(n)
Time =
# of SubInstances
× # of Bird Answers
× size of solution
= q(n × d × n)
Space =
"What is cost of the best path from s to v7?"
"What is cost of the best path from s to v7?"
Little bird suggests last edge <v4,v7> with weight 2.
8
Friend gives cost 8
of best path <s,v4>.
Best cost via <v4,v7> is 8+2=10.
2
"What is cost of the best path from s to v7?"
Little bird suggests last edge <v2,v7> with weight 7.
Friend gives cost 2
of best path <s,v2>.
Best cost via <v2,v7> is 2+7=9.
"What is cost of the best path from s to v7?"
Little bird suggests last edge <v5,v7> with weight 5.
6
Friend gives cost 6
of best path <s,v5>.
Best cost via <v5,v7> is 6+5=11.
9
2
We also learn the wise little bird’s advice.
We will store this in the table too.
"What is cost of the best path from s to v7?"
Take best of best:
10, 9, 11
birdsAdvice[i] = kmin
Previous algorithm gives:
We run the birdfriend algorithm again, but with a reliable bird.
Done!
This could be done iteratively.As an exercise, design it.
“Which is the last edge?”
Multiple Optimal SolutionsShe could give either answer.
By giving this edge she says
“There exists an optimal solution consistent with this answer.”
Similar to greedy proof.
6
“Which is the last edge?”
We try all the bird answers.
Multiple Optimal SolutionsWhen we try this bird answer,
we find this best solution.
When we try this bird answer,
we find this best solution.
When we take best of best,
we choose between them.
6
Designing Recursive Back Tracking Algorithm
Recursive Back Tracking Algorithm
Dynamic Programming Algorithm
Purpose of Little Bird:
A solution: a path
t
s
I ask the bird:
“What is the first edge in the path?”
The Question For the Little BirdAn instance: Graph, s, and t
The Dynamic Programming reverses the recursive backtracking algorithm.
Hence, to end up with a “forward order”,we first reverse the recursive backtracking algorithm.
A solution: a path
t
s
I ask the bird:
“What is the last edge in the path?”
The Question For the Little BirdAn instance: Graph, s, and t
The Dynamic Programming reverses the recursive backtracking algorithm.
Hence, to end up with a “forward order”,we first reverse the recursive backtracking algorithm.
A solution: a path
t
s
I ask the bird:
“What is the rest of the path?”
The Question For the Little BirdAn instance: Graph, s, and t
“What is the last edge in the path?”
A good question for the bird leaves you with a good recursive subinstance to ask your friend.
A solution: a path
t
s
I ask the bird:
The Question For the Little BirdAn instance: Graph, s, and t
“What is the last edge in the path?”
Giving a good follow up question for your friend to ask the bird.
“What is the second last edge in the path?”
number of edges into node t.
t
s
K
The Question For the Little Bird# of solutions.
Trying all is the Brute Force algorithm.
A solution: a path
t
s
I ask the bird:
The Question For the Little BirdAn instance: Graph, s, and t
“How many edges are in the path?”
Bad Question:
A solution: a sequence of objects
Z = a b c d
I ask my friend:
The Question For the Little BirdAn instance: ???
“What is the last object in the sequence?”
# of answersK =
# of possible last objects.
“What is the rest of the solution?”
An instance: a sequence of objects
I ask the bird:
X = a s b e f c h d a
X =asbe fchda
A solution: a subset of these objects
Z = a b c d
The Question For the Little Bird“What is the last object in the sequence?”
# of answersK =
# of possible last objects.
Is there a smaller question that we could ask?
An instance: a sequence of objects
I ask the bird:
X = a s b e f c h d a
A solution: a subset of these objects
Z = a b c d
The Question For the Little Bird“Is the last object of the instance included in the optimal solution?”
# of answersK =
2, Yes or No
A solution: a binary tree of objects
38
25
51
I ask a second friend:
I ask my friend:
17
31
42
63
4
21
28
35
40
49
55
71
The Question For the Little BirdAn instance: ???
“What object is at the root?”
“What is the left subtree?”
“What is the right subtree?”
Previous problems had one friend given a bird ans.
38
25
51
17
31
42
63
4
21
28
35
40
49
55
71
The Question For the Little BirdAn instance: ???
A solution: a binary tree of objects
“What object is at a leaf?”
Bad Question:
A solution: # of words to put on each line.
few blanks on the end of each line.
3 = 83 = 83 = 2163 = 27 259
2263
Love.life.. man.while.. there......as.we.be...
11
Printing NeatlyAn instance: text to print neatly & # chars per line
“Love life man while there as we be”, 11
The cost: a measure of how neat,
small punishmentbig punishment
The goal is to to print it as “neatly” as possible.
Try all ways to print, return the best.
But there may be an exponential number of ways to!
love.life.. man........ love.......life.man...love.......life.man...love.life.. man........
I combine
bird’s and
friend’s answers.
I ask the friend:
Bird & Friend AlgorithmAn instance:
“Love life man while there as we be”, 11
“How many words on the last line?”
She may answer 3.
“Which is the best way to print the remaining n3 words?”
We try the bird answers words,
1
2
3
4
5
Bird & Friend AlgorithmAn instance:
“Love life man while there as we be”, 11
Even if the bird was wrong, this work is not wasted.
This is best way to print from amongst those ending in 3 words.
and take best of best.
I try each # words on last line.
A friend tries # on next.
A friend tries # on next.
Time?
Same as the brute force algorithm that tries each path.
“Love life man while there as we be”, 11
Determine the complete set of subInstances.
Given an instance I,
“Love life man while there as we be”, 11
“Love life man while there”, 11
“Hi there”, 81
“man while”, 11
Guess the complete set of subInstances.
Yes
No
No
This may appear on a line,but it will never be a subInstance for a friend.
“Love life man while there as we be”, 11
“Love life man while there”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life”, 11
“Love”, 11
“”, 11
“Love life man while there as”, 11
“Love life man while there as we”, 11
The set of subInstances is the set of prefixes.
The set S of subInstances needs to:
subInstance S
The bird answers words,
5
3
2
1
4
“Love life man while there as we”, 11
“Love life man while there as”, 11
“Love life man while there”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life”, 11
“Love”, 11
“”, 11
The set of subInstances is the set of prefixes.
“Love life man while there as we be”, 11
The set S of subInstances needs to:
subInstance S
subsubInstance i
“Love life man while there as we”, 11
“Love life man while there as”, 11
“Love life man while there”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life”, 11
“Love”, 11
“”, 11
asked of some friend, friend, …
The set of subInstances is the set of prefixes.
“Love life man while there as we be”, 11
The set Sof subInstances needs to:
“Love life man while there as we be”, 11
“Love life man while there as we”, 11
“Love life man while there as”, 11
“Love life man while there”, 11
“Love life man while”, 11
The bird
answers 1.
asked of some friend, friend, …
The set of subInstances is the set of prefixes.
The set S of subInstances needs to:
A fine set of subinstances!
“Love life man while there as we be”, 11
“Love life man while there as we”, 11
“Love life man while”, 11
“Love life man”, 11
“Love life man while there”, 11
“Love”, 11
“”, 11
“Love life man while there as”, 11
“Love life”, 11
The set of subInstances is the set of prefixes.
In what order should they go?
First
Base Case easy
Last
Instance to be solved.
Construct a table
Map
The set of prefixes of words.
SubInstances
i ϵ [n], i.e. for each word.
Indexes
Cell of table
“Which is the best printing of first i words?”
i
Fill out a table containing an optimal solution for each subinstance.
“Which is the best printing of first i words?”
Base case
Original
“Love life man while there as we be”, 11
The 5th subinstance is
“Love life man while there”, 11
5 words
with 4, 4, 3, 5, 5 letters.
“Love life man while there as we be”, 11
Love.life.. man.while.. there......
The 5th subinstance is
“Love life man while there”, 11
Its solution is
with 2,2,1 words on each line.
The bird’s advice is 1 word on last.
Solution’s cost is
23 + 23 +63 = 232
“Love life man while there as we be”, 11
Assume the table is filled in so far.
We will work to fill in the last line
“Love life man while there as we be”, 11
there.as.we.be
“Love life man while there as we be”, 11
Choose best of the best.
Dynamic Programming
code always has thissame basic structure.
Amusingly,when formatting this code, I had to fight with line breaks to get the height/width ratioPrinting Neatly.
Dynamic Programsdo not recurse making the instance smaller and smaller.
Instead, it up frontdetermines the set Sof all subinstancesthat ever need to be solved.
Be clear what subinstances are.
Be clear what subinstances are.
How are they indexed?
Tables indexed by these subinstancesstore an optimal solution and it’s cost.
The set Sof subinstancesare solved fromsmallest to largestso that no body waits.
Base Cases:Instances that are too small to have smaller instances to give to friends.
They get solved firstand their solutions stored.
Then we iterate through the remaining subinstances.
From smallest to largest.
Each gets solvedand their solutions stored.
Actually, we store thebird’s advice instead of the solution.
Consider yourselfto be a friendworking on oneof these.
Be clear which subinstance is yours.
Solve this as you did before.
Loop through the bird answers.
Be clear which is the current one being tried.
Give the bird & friend algorithmas a comment.
(Unless it is in an earlier question.)
What is the bird asked?
What does she answer?
ik
ik
Instead of recursing,we simply look in the table for the solution.
Because his instance is smaller, he has already solved it and stored sol in the table.
How do you formyour solution from the friend’s and fromthe bird’s?
How do you formyour cost from the friend’s and fromthe bird’s?
optSol<i,k>is a best solution for our instance subI[i]from amongstthose consistent with the bird'skth answer.
Take the bestof the best
Store the solution toour instance subI[i]in the table.
Actually, we store thebird’s advice instead of the solution.
Base Cases:Instances that are too small to have smaller instances to give to friends.
Is this code correct?
Dynamic Programsdo not recurse making the instance smaller and smaller.Hence, lets not worry about our instance I being a base case.
But there is a tableof subinstancesthat must be solved.
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
Some of these will bebase casesand their solutionsmust be stored in the table.
But there is a tableof subinstancesthat must be solved.
Then we solvethe rest.
Return the solutionand cost for the original instance.
But actually,we don’t have the solution.
We must rerun it, this time with advice from the bird.
We run the birdfriend algorithm again, but with a reliable bird.
Find Optimal PathPrevious algorithm gives cost and bird’s advice.
An instance: Two strings
X = a s b e f c h d a
Y = r t w a b g j c k t f d
A solution: A common subsequence.
Z = a b c d
Longest Common Subsequence problem
X =asbe tchda
Y = r t wabg jck t fd
The cost: The length of Z.
The goal is to find a longest common subsequence.
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Bird & Friend Algorithm“Is the last character of either X or Yincluded in Z?”
She answers one of :
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I ask my friend:
Bird & Friend Algorithm“Is the last character of either X or Y included in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I combine
bird’s and
friend’s answersand give
Bird & Friend Algorithm“Is the last character of either Xor Y included in Z?”
She answers:
My friend answers:
Z = a b c d
X =asbe tchd
Y = r t wabg jck t fd
the sameZ.
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d a
Y = r t w a b g j c k t f
I ask the bird:
I ask my friend:
Bird & Friend Algorithm“Is the last character of either X or Yincluded in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d a
Y = r t w a b g j c k t f
I ask the bird:
I combine
bird’s and
friend’s answersand give
Bird & Friend Algorithm“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
the same Z.
Z = a b c
X =asbe tchd a
Y = r t wabg jck t f
Not as good as lastbut we need to try.
The instance:
X = a s b e t c h
Y = r t w a b g j c k d f
I ask the bird:
I ask my friend:
Bird & Friend AlgorithmAn instance:
X = a s b e t c h d
Y = r t w a b g j c k d f d
Last chars equal
“Is the last character of either X or Y included in Z?”
She answers:
The instance:
X = a s b e t c h
Y = r t w a b g j c k d f
I ask the bird:
I combine
bird’s and
friend’s answersand give
Bird & Friend AlgorithmAn instance:
X = a s b e t c h d
Y = r t w a b g j c k d f d
Last chars equal
“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
Z = a b c
X =asbe tch
Y = r t wabg jck d f
Zd = abcd.
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Bird & Friend AlgorithmLast chars not equal
“Is the last character of either X or Y included in Z?”
She answers:
I politely tell her that she is wrong.
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I ask my friend:
Bird & Friend Algorithm“Is the last character of either X or Y included in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w a b g j c k t f d
I ask the bird:
I combine
bird’s and
friend’s answersand give
Bird & Friend Algorithm“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
Z = a b c d
X =asbe tch d
Y = r t wabg jck t f d
Wrong
Za = abcda.
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w
I ask the bird:
I ask my friend:
Bird & Friend Algorithm“Is the last character of either X or Y included in Z?”
She answers:
An instance:
The instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
X = a s b e t c h d
Y = r t w
I ask the bird:
I combine
bird’s and
friend’s answersand give
Bird & Friend Algorithm“Is the last character of either X or Y included in Z?”
She answers:
My friend answers:
Z = t
X =a s b e t c h d
Y = r t w
Za = ta.
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Bird & Friend AlgorithmLast chars not equal
“Is the last character of either X or Yincluded in Z?”
Can we eliminatesome of her answers?
She answers one of :
Given any optSolshe needs to havea valid answer.
?
?
?
An instance:
X = a s b e t c h d a
Y = r t w a b g j c k t f d
I ask the bird:
Bird & Friend AlgorithmLast chars not equal
“Is the last character of either X or Y included in Z?”
She answers one of :
3
# of answersK =
Time?
I try each of 3 bird ans.
My friends try 3
His friends try 3
Same as the brute force algorithm that tries each solution.
Determine the complete set of subInstances.
Given an instance I,
X = a s b e t c h d a
Y = r t w a b g j c k t f d
Is this a subInstance?
X’ = a s b e t c
Y’ = r t w a b g j c k
Yes
Determine the complete set of subInstances.
Given an instance I,
X = a s b e t c h d a
Y = r t w a b g j c k t f d
friends, …
Is this a subInstance?
No
X’ = b e t
Y’ = a b g j c k
i [0..X]
j [0..Y]
Determine the complete set of subInstances.
Given an instance I,
X = a s b e t c h d a
Y = r t w a b g j c k t f d
friends, …
Is this a subInstance?
Yes
X’ = x1,…xiY’ = y1,…,yj
X × Y of these.
Xi = x1,…xiYj = y1,…,yj
i [0..X]
j [0..Y]
Guess the complete set S of subInstances.
The set S of subInstances needs to:
Yes: i = X & j = Y
Xi = x1,…xiYj = y1,…,yj
Xi1 = x1,…xi1Yj = y1,…,yj
Xi = x1,…xiYj = y1,…,yj
S
Xi = x1,…xiYj1 = y1,…,yj1
Xi1 = x1,…xi1Yj1 = y1,…,yj1
i [0..X]
j [0..Y]
Guess the complete set S of subInstances.
The set S of subInstances needs to:
subInstance S
subsubInstance S
S
Xi = x1,…xiYj = y1,…,yj
asked of some friend, friend, …
i [0..X]
j [0..Y]
Guess the complete set S of subInstances.
The set S of subInstances needs to:
subInstance S
subsubInstance S
We showed this.
This is a fine set of subInstances!
i [0..X]
j [0..Y]
Construct a table
Map
Xi = x1,…xiYj= y1,…,yj
SubInstances
Indexes
“LCS of x1,…xiand y1,…,yj ?”
Cell of table
j
i
subInstancei,j=
Yj
j=
Xi = x1,…xi
Yj = y1,…,yj
Xi
i=
TableOptimal Solution
= Longest Common
Subsequence
Cost = length of LCS.
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
TableYj
Xi
Optimal Solution
= Longest Common
Subsequence
Bird’s Advice
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
TableYj
Xi
Optimal Solution
= Longest Common
Subsequence
Bird’s Advice
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
TableYj
Xi
Optimal Solution
= Longest Common
Subsequence
Bird’s Advice
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
5
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
5
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost +1
6
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
6
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
4
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
Friend’s subInstance
Our cost
= friend’s cost
3
subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
Fill in BoxYj
Xi
Fill in box
Sorry bird is wrong.
Our cost
= 

subInstancei,j=
Xi = x1,…xi
Yj = y1,…,yj
4
Fill in BoxYj
Xi
Fill in box
(later)
Done
Ingredients:
v=4,p=4
v=4,p=4
V=8
Greedy AlgorithmGreedy Criteria:
Most valuable pi
v=4,p=4
v=4,p=4
v=7,p=5
V=8
gives 8
Optimal
Greedy give 5
pi
vi
v=7,p=5
V=7
Greedy AlgorithmGreedy Criteria:
v=4,p=4
v=4,p=4
v=7,p=5
V=8
V=7
gives 5
Optimal
Greedy give 4
pi
vi
¾ of
v=4,p=4
v=7,p=5
V=7
gives 5
Optimal
Greedy AlgorithmGreedy Criteria:
If fractional solutions are allowed.
Works
Often an Integersolution is MUCH
harder to find.
v=4,p=4
V=7
+ ¾ × 4 = 7
Greedy give 4
= Optimal
v=4,p=4
v=4,p=4
V=9
v=7,p=5
Bird & Friend AlgorithmMy instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
A solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
“What is the last object to take?”
n
# of answers K =
v=4,p=4
v=4,p=4
V=9
v=7,p=5
Bird & Friend AlgorithmMy instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
A solution:
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
“Do we keep the last object?”
2 Yes & No
# of answers K =
Bird says, Yes keep the last object.
Bird & Friend AlgorithmMy instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
v=4,p=4
v=4,p=4
V=9
v=7,p=5
Trust her and put it into your knapsack.
To fill the rest of the knapsack.
But what instance do I give him?
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
<<v5,p5>,<v9,p9>,...........,<v82,p82>,<vn,pn>>
Bird & Friend AlgorithmHis instance:
<Vvn:<v1,p1>,<v2,p2>,.........<vn1,pn1>,<vn,pn>>.
v=4,p=4
v=4,p=4
V=94
v=7,p=5
My solution:
same+ pn
My cost:
v=4,p=4
v=4,p=4
V=9
v=7,p=5
<<v5,p5>,<v9,p9>,...........,<v82,p82>,<vn,pn>>
Bird & Friend AlgorithmMy instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
If we trust the bird and friend,
this is valid and optimal.
My solution:
same +pn
My cost:
Bird says, No do not keep the last object.
Bird & Friend AlgorithmMy instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
v=4,p=4
v=4,p=4
V=9
v=7,p=5
Trust her and delete it.
To fill the knapsack with the rest.
What instance do I give him?
<<v5,p5>,<v9,p9>,...........,<v82,p82>>.
Bird & Friend AlgorithmHis instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
v=4,p=4
v=4,p=4
V=9
v=7,p=5
My solution:
same
If we trust the bird and friend,
this is valid and optimal.
same
My cost:
Time?
I try each of 2 bird ans.
My friends tries 2
His friends tries 2
Same as the brute force algorithm that tries each solution.
Determine the complete set of subInstances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Is this a subInstance?
<V:<v1,p1>,<v2,p2>,<v3,p3>>.
Yes, if the bird keeps saying “No”.
Determine the complete set of subInstances.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Is this a subInstance?
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
No, the set of objects is always a prefix of the original set.
Determine the complete set of subInstances.
Is this a subInstance?
<V’:<v1,p1>,<v2,p2>,<v3,p3>>.
Given an instance I,
<V:<v1,p1>,<v2,p2>,<v3,p3>,<v4,p4>,<v5,p5>,<v6,p6>>.
Quite possibly, if V’V.
It is easier to solve than to determine if it is a subinstance.
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
No
S
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
Guess the complete set S of subInstances.
The set S of subInstances needs to:
Yes: V’=V & i = n
subInstance S
subsubInstance S
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>> S
<V’ :<v1,p1>,<v2,p2>,...,<vi1,pi1>>
<V’vi:<v1,p1>,<v2,p2>,...,<vi1,pi1>>
Yes
Construct a table
Map
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
SubInstances
i [0..n]
V’ [0..V]
Indexes
“Which of first i objects to put in a knapsack of size v’?”
Cell of table
v’
i
The complete setS of subInstances.
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
same +pi
same
No
1
0
2
V’vi
V’
V
OptSol Cost & Bird’s Advicefor this
subInstance
0
1
2
i1
Yes
i
Take best of best.
Our cost?
n
The complete set S of subInstances.
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
1
0
2
V’vi
V’
V
OptSol Cost & Bird’s Advicefor this
subInstance
0
1
2
i1
Order to fill
so nobody
waits?
i
n
The complete set S of subInstances is
V’ [0..V]
<V’:<v1,p1>,<v2,p2>,......,<vi,pi>>.
i [0..n]
No
Yes
My instance:
<V:<v1,p1>,<v2,p2>,..........................,<vn,pn>>.
Running time
= ( # of subinstances × # bird answers )
= (Vn× 2 )
= ( 2#bits in V× n )
Polynomial?
Exponential in “size” in instance!
If there is a polytime algorithmfor the Knapsack Problem
For EVERY optimization problem there is a polytime algorithm.
Likely there is not a polytime algorithmfor the Knapsack Problem.
Likely there is not a polytime algorithmfor EVERY optimization problem.
done
The Job/Event Scheduling Problem
Ingredients:
Greedy Criteria:
Earliest Finishing Time
Motivation:
Schedule the event which will free up your room for someone else as soon as possible.
Ingredients:
1
1
Greedy AlgorithmGreedy Criteria:
Earliest Finishing Time
Motivation:
Schedule the event which will free up your room for someone else as soon as possible.
An instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
A solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
“What is the last event to take?”
n
# of answers K =
An instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
A solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
“Do we keep the last event?”
# of answers K =
2 Yes & No
“Do we keep the last event?”
I ask my friend:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
Bird & Friend AlgorithmAn instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
She answers: No
His solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
My solution:
same
My cost:
same
“Do we keep the last event?”
I ask my friend:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
Bird & Friend AlgorithmAn instance:
<<s1,f1,w1>,<s2,f2,w2>,… ,<sn,fn,wn>>.
Carefull
She answers: Yes
His solution:
<<s5,f5,w5>,<s9,f9,w9>,… ,<s82,f82,w82>>.
My solution:
same + <sn,fn,wn>.
same +wn
My cost:
last event
“Yes keep the last event.”
Bird & Friend AlgorithmNo this solution is not valid!
Give the rest to my friend.
Here is my best subsolution.
I add to his subsolution the bird’s answer.
“Yes keep the last event.”
Bird & Friend AlgorithmNo we trust the bird!
Then I should politely tell the bird she is wrong
“Yes keep the last event.”
Bird & Friend AlgorithmNo we trust the bird!
You only tell her she is wrong if you really know.Eg k words don’t fit on the last lineThe bear does not fit into the knapsack
“Yes keep the last event.”
Bird & Friend AlgorithmNo we trust the bird!
Your friend could have just as easily given you this subsolution that does not conflict with the bird’s answer.
“Yes keep the last event.”
Bird & Friend AlgorithmNo we trust the bird!
Or maybe he needs to make a sacrifice when finding his answer in order that the overall solution is the best.
“Yes keep the last event.”
Bird & Friend AlgorithmNo we trust the bird!
Or goal now is to find the best solution to our instance that is consistent with the bird’s answer.Then we will take the best of best.
“Yes keep the last event.”
Bird & Friend AlgorithmNo we trust the bird!
Dude! It is your job to give me the right subinstance so that I give you a subsolution that does not conflict with the bird
last event
“Yes keep the last event.”
Bird & Friend AlgorithmMy instance:
Cant keep any events that overlap with it.
last event
“Yes keep the last event.”
I ask my friend:
Bird & Friend AlgorithmMy instance:
“Yes keep the last event.”
I ask my friend:
Bird & Friend AlgorithmHis instance:
His solution
“Yes keep the last event.”
I ask my friend:
Bird & Friend AlgorithmMy instance:
My solution:
My solution:
same + <sn,fn,wn>.
Yes
Valid?
“Yes keep the last event.”
I ask my friend:
Bird & Friend AlgorithmMy instance:
My solution:
My solution:
same + <sn,fn,wn>.
same+wn
My cost:
Time?
I try each of 2 bird ans.
My friends tries 2
His friends tries 2
Same as the brute force algorithm that tries each solution.
Determine the complete set of subInstances.
Given an instance I,
Every subset of {1,…,9}is a possible subInstance.
I.e. could be an exponentialnumber of them.
Hence, running time is exponential.
Greedy algorithm sortedjobs by finishing time.Let us do that too.
i [0..n]
<<s1,f1,w1>,<s2,f2,w2>,… ,<si,fi,wi>>
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................… ,<sn,fn,wn>>.
Guess the complete set S of subInstances.
The set S of subInstances needs to:
Yes: i = n
?
subInstance S
subsubInstance S
asked of some friend, friend…
Only nsubInstances
Good enough.
subInstance =
last event
<<s1,f1,w1>,<s2,f2,w2>,................................,<si,fi,wi>>
Show closed under “friend” operation
subInstance S
subsubInstance S
Events sorted by finishing time.
Bird answers:
subInstance =
last event
<<s1,f1,w1>,<s2,f2,w2>,................................,<si,fi,wi>>
“Yes keep the last event.”
Show closed under “friend” operation
subInstance S
subsubInstance S
Delete overlapping events for friend.
Bird answers:
subsubInstance =
<<s1,f1,w1>,<s2,f2,w2>,.....,<sj,fj,wj>>
“Yes keep the last event.”
Show closed under “friend” operation
subInstance S
subsubInstance S
Delete overlapping events for friend.
subsubInstance =
<<s1,f1,w1>,<s2,f2,w2>,.....,<sj,fj,wj>>
Event j is kept fjsi
Show closed under “friend” operation
subInstance S
subsubInstance S
typical kept job
typical deleted job
set of kept jobs is a prefix of events.
subsubInstance S
i [0..n]
<<s1,f1,w1>,<s2,f2,w2>,… ,<si,fi,wi>>
0, 1, 2, 3, 4, …. n
Base case
Original
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................… ,<sn,fn,wn>>.
The complete set S of subInstances is
Table:
i [0..n]
<<s1,f1,w1>,<s2,f2,w2>,… ,<si,fi,wi>>
My instance:
<<s1,f1,w1>,<s2,f2,w2>,................… ,<sn,fn,wn>>.
The complete set S of subInstances is
Running time
= # of subinstances × # bird answers
= n× 2
But to find your friend’s “yes” subinstanceyou must know how many events overlapwith your last event. This takes time:
O(logn) using binary searchfor a total of O(nlogn) time.
Done
C CB
AC
c
d
B TA
BC
b
e
AAA
BT
a
TAB
CA
TT
Context Free Grammar
(Not look ahead one)
For ease, we will assume every nonterminal
either goes to two nonterminals
or to one terminal
T
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Input:
start nonterminal = T
string to parse
= a1a2a3 ..... an
= baeaadbda
T a1a2a3 ..... an
Output: A parsing
Input:
T a1a2a3 ..... an
T
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Recursive Algorithm:
GetT does not know whether to call
GetA, GetC, or GetT.
Input:
T a1a2a3 ..... an
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b a e a
CCB
AC
c
d
B TA
BC
b
e
A AA
BT
a
T AB
CA
TT
T
Ask Little Bird:
Ask Friend
Ask Another Friend
b d a
Input:
T a1a2a3 ..... an
A
C
a d b
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
T
Ask Little Bird:
Instance to give Friend
b d a
Input:
T a1a2a3 ..... an
T
A
C
A
B
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
C CB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Ask Little Bird:
Want from Friend:
Instance to give him:
C
Input:
T a1a2a3 ..... an
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b a e a
C CB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
T
Ask Little Bird:
How can we know split?
b d a
Input:
T a1a2a3 ..... an
A
C
a d b
b a e a
C CB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
T
Ask Little Bird:
# of ans K =
mT = # of rules for T.
n = # chars in string.
# of ans K =
mT×n.
Total # of ans K =
b d a
Input:
T a1a2a3 ..... an
T
A
C
A
B
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Ask left friend:
C
Input:
T a1a2a3 ..... an
T
C
T
B
A
C
a d b
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
A
Ask right friend:
b d a
Input:
T a1a2a3 ..... an
T
A
C
C
T
B
A
A
B
C
A
A
C
T
A
C
B
A
B
a d b
b d a
b a e a
CCB
AC
c
d
BTA
BC
b
e
AAA
BT
a
TAB
CA
TT
Combine:
Time?
I try each of 2 bird ans.
My friends tries 2
His friends tries 2
Same as the bruteforce algorithm that tries each solution.
Determine the complete set of subInstances.
Given an instance I,
Determine the complete set of subInstances.
A
gives:
gives:
A
My instance I:
T a1a2a3 ..... an
T
C
C
My left subInstance.
His right subInstance.
a d b
b d a
b a e a
Determine the complete set of subInstances.
T’=
C
a1...ai1
aiai+1...aj
aj+1...an
a d b
My instance I:
T a1a2a3 ..... an
subInstances:
T’ aiai+1 ..... aj
# of nonterminals ×n2
# of subInstances =
Construct a table
Map
SubInstances
T’ aiai+1 ..... aj
T’ i,j [1,n]
Indexes
i
T’
Cell of table
j
gives: First rule and split
subInstances:
T’ aiai+1 ..... aj nonterminals T’
& i,j [1,n]
Running time
= ( # of subinstances × # bird answers )
= (# of nonterminals × n2
× # of rules · n )
Done
An instance (input) consists of a circuit:
c =(x3orx5orx6) and (x2orx5orx7) and (x3orx4)
true
true
false
true
true
true
true
A solution is an assignment of the variables.
true
x1= 0,x2= 1,x3= 0,x4= 0,x5= 1,x6= 0,x7= 1
The cost of a solution is
The goal is to find satisfying assignment.
Ask the little bird
Instance:
c = (x3orx5orx6) and (x2orx5orx7) and (x3orx4)
Value of x1 in an optimal solution
or even better
Value of x3 in an optimal solution
We will have to try both x3 = 0 and x3 = 1.
For now, suppose she answered x3 = 0.
true
false
true
SubInstance:
c =(x2orx5orx7) and x4
Friend gives SubSolution:
x1= 0,x2= 1,x4= 0,x5= 1,x6= 0,x7= 1
Our Solution:
x1= 0,x2= 1,x3= 0,x4= 0,x5= 1,x6= 0,x7= 1
Instance:
c =(x3orx5orx6) and (x2orx5orx7) and (x3orx4)
Commit to x3 = 0 and simplify
x3
1
0
x2
x1
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Speeding Up the TimeIn the end, some friend looks at each of the 2n assignments,
Determine the complete set of subInstances.
Given an instance I,
Determine the complete set of subInstances.
Given an instance I,
c =(x1 or y1) and (x2 or y2) and (x3 or y3) and (x4 or y4)
Is this a subInstance?
c = (x1) and (x3)
Commit to
y1=0, y2=1, y3=0, y4=1,
and simplify
Yes
True for any subset of the xi.
could be an exponential # of different subInstances.
running time is exponential.
x3
1
0
x2
x1
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Speeding Up the TimeIn the end, some friend looks at each of the 2n assignments,
x3
1
0
x2
x1
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Speeding Up the TimeBut sometimes we can prune off branches.
x3 is forced to x3 = 0
x3
1
0
x2
x1
c =(x2orx5orx7) and x3
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Instance:
This is trivially unsatisfiable because x3 can’t be both 0 and 1.
x3
1
0
x2
x1
c =(x2orx5orx7) and x3and x3
1
0
1
0
x1
x2
x2
x1
1
1
1
0
0
0
1
0
Instance:
Designing Recursive Back Tracking Algorithm
Recursive Back Tracking Algorithm
Dynamic Programming Algorithm
find the solution to your instance.
38
25
51
17
31
42
63
4
21
28
35
40
49
55
71
The Question For the Little Birdi
“Best path from vi to t?”
Dynamic Programming
Construct a table
Map
SubInstances
i ϵ [n], i.e. for each node vi
Index
Cell of table
“Which is the best path from vi to t?”
t, v8, v7, v6, v5, …., s
i