1 / 16

Exploiting Graphical Structure in Decision-Making

Exploiting Graphical Structure in Decision-Making. Ben Van Roy Stanford University. Overview. Graphical models in decision-making Singly-connected  efficient computation General decision problems  intractable Sparsity  reduction in computation? Sequential decision-making

tamika
Download Presentation

Exploiting Graphical Structure in Decision-Making

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Exploiting Graphical Structure in Decision-Making Ben Van Roy Stanford University

  2. Overview • Graphical models in decision-making • Singly-connected  efficient computation • General decision problems  intractable • Sparsity  reduction in computation? • Sequential decision-making • Curse of dimensionality • Sparsity or other graphical structure  reduced computational requirements? • Structured value functions and/or policies? • Preliminary results and research directions

  3. Graphical Models in Inference • Conditional independencies simplify inference • Singly-connected graphs (trees) • General sparse graphs • Preprocessing • Approximations? x1 x2 x3 x4

  4. Graphical Models in Decision-Making • Deterministic dynamic programming • Nonserial dynamic programming • General sparse graphs • Preprocessing • Approximations? x1 x2 x3 x4

  5. Sequential Decision-Making • Bellman’s equation decision u(t) state x(t) system strategy J*(x(t)) = max E[g(x(t), u(t)) + a J*(x(t+1))|x(t)]

  6. The Curse of Dimensionality • # states is exponential in # variables • The value function encodes one value per state • Storage is intractable • Computation is intractable • Research objective: exploit sparsity and other special graphical structure to reduce computational requirements of sequential decision problems

  7. Dynamic Bayesian Networks x1(t) x1(t+1) x2(t) x2(t+1) x3(t) x3(t+1) x4(t) x4(t+1)

  8. Example Multiclass Queueing Networks u1 u2 x1 x2 x3 x4 x5

  9. Can We Exploit Proximity? • Idea: variables that are “far” from each don’t interact much • Does this allow us to decompose the problem?

  10. Yes… • The value function decomposes • N(i) = a neighborhood; i.e. a set of nodes within some “distance” of i • Complexity: O(nd)  O(dnN) • …but there’s a problem here…

  11. Optimal Decisions Depend on Global State information u1(t+1) x1(t) x1(t+1) x2(t) x2(t+1) x3(t) x3(t+1) x4(t) x4(t+1)

  12. Things Still Work Out… • Conjecture: • If decision ui influences only xi • Then near-optimal decisions can be made based only on variables “near” xi • Consequence: u1(t+1) x1(t) x1(t+1) x2(t) x2(t+1) x3(t) x3(t+1) x4(t) x4(t+1)

  13. The Underlying Problem • Which fij’s do I need to know to choose a near-optimal uk (without coordination)? x7 x2 x3 x1 x6 x4 x5

  14. A Simple Case • Let N(i) = nodes within r steps • Result: loss of optimality ~ O(1/r) • Note: amount of information required is independent of the graph size • (Rusmevichientong and Van Roy, 2000) x1 x2 x3 x4 x5 x6

  15. Future Work • Extending this result to general graphs • Exploring practical implications • Expected practical utility: reduction of complexity in approximation algorithms • Problem is no longer O(nd) • May instead be O(dnr) • Still computationally prohibitive, but not exponential in problem size • Simplification of decision-supporting information?

  16. More Future Work • Current work exploits proximity • Many graphs arising in practical problems pose additional special structure (e.g., symmetries, multiple “layers” of relationships, etc.) • Can we also exploit such structure? (e.g., are there sometimes appropriate hierarchical representations?)

More Related