Dynamic Programming

1. Dynamic Programming RahulMohare Faculty DattaMeghe Institute of Management Studies

2. Introduction • In mathematics and computer science, dynamic programming is a method for solving complex problems by breaking them down into simpler subproblems. It is applicable to problems exhibiting the properties of overlapping subproblems which are only slightly smaller

3. The key idea behind dynamic programming is quite simple. In general, to solve a given problem, we need to solve different parts of the problem (subproblems), then combine the solutions of the subproblems to reach an overall solution. The dynamic programming approach seeks to solve each subproblem only once, thus reducing the number of computations. This is especially useful when the number of repeating subproblems is exponentially large.

4. Characteristics of Dynamic Programming Applications • Characteristic 1 • The problem can be divided into stages with a decision required at each stage. • Characteristic 2 • Each stage has a number of states associated with it. • By a state, we mean the information that is needed at any stage to make an optimal decision. • Characteristic 3 • The decision chosen at any stage describes how the state at the current stage is transformed into the state at the next stage.

5. Characteristic 4 • Given the current state, the optimal decision for each of the remaining stages must not depend on previously reached states or previously chosen decisions. • This idea is known as the principle of optimality. • Characteristic 5 • If the states for the problem have been classified into on of T stages, there must be a recursion that related the cost or reward earned during stages t, t+1, …., T to the cost or reward earned from stages t+1, t+2, …. T.

6. Resource-Allocation Problems • Resource-allocation problems, in which limited resources must be allocated among several activities, are often solved by dynamic programming. • To use linear programming to do resource allocation, three assumptions must be made: • Assumption 1 : The amount of a resource assigned to an activity may be any non negative number. • Assumption 2 : The benefit obtained from each activity is proportional to the amount of the resource assigned to the activity.

7. Assumption 3: The benefit obtained from more than one activity is the sum of the benefits obtained from the individual activities. • Even if assumptions 1 and 2 do not hold, dynamic programming can be used to solve resource-allocation problems efficiently when assumption 3 is valid and when the amount of the resource allocated to each activity is a member of a finite set.

8. Generalized Resource Allocation Problem • The problem of determining the allocation of resources that maximizes total benefit subject to the limited resource availability may be written aswhere xt must be a member of {0,1,2,…}.

9. To solve this by dynamic programming, define ft(d) to be the maximum benefit that can be obtained from activities t, t+1,…, T if d unites of the resource may be allocated to activities t, t+1,…, T. • We may generalize the recursions to this situation by writing fT+1(d) = 0 for all d wherext must be a non-negative integer satisfying gt(xt)≤ d.

10. Case-1 Capital Budgeting A XYZ Corporation is entertaining proposals from its three plants for possible expansion of facilities. This corporation is budgeting $5 Million for allocation to all three plants. Each plant is requested to submit its proposals giving total cost (c) ant total revenue (R) for each proposal. Following table summarizes the cost and revenues How allocation can be done ?

11. Some thing to remember… • Stage: A stage in DP is defined as the portion of the problem that possess a set of mutually exclusive alternatives from which the best alternative is to be selected • State: A state is normally defined to reflect the status of the constraints that bind all the stages together

12. In our Capital Budgeting Example Each plant defines a stage with the first, second and third stages having 3, 4 and 2 alternatives We define the states for the stages 1,2 and 3 as follows X1= amount of capital allocated to stage 1 X2= amount of capital allocated to stage 1 & 2 X3= amount of capital allocated to stage 1,2 & 3

13. The following table summarizes the conditional decisions for stage-1

14. conditional decisions for stage-II

15. Stage 3 Computations By computing all the answers for the three stages the optimal solution calls for selecting the Proposal 2 for plant 1 Proposal 4 for plant 2 Proposal 1 for plant 3 The total cost is 5 and optimal revenue is $17 millions

16. Dynamic Vs Linear Programming

17. Thank You