1 / 19

Relational Markov Models and their Application to Adaptive Web Navigation

Relational Markov Models and their Application to Adaptive Web Navigation. Corin Anderson, Pedro Domingos, Daniel Weld. Review: Hidden Markov Models. Sequence of States Transition distribution to next state dependant on evidence variable and current state only (in first-order case).

flavian
Download Presentation

Relational Markov Models and their Application to Adaptive Web Navigation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Relational Markov Models and their Application to Adaptive Web Navigation Corin Anderson, Pedro Domingos, Daniel Weld

  2. Review: Hidden Markov Models • Sequence of States • Transition distribution to next state dependant on evidence variable and current state only (in first-order case)

  3. DBNs: A Step Up • Multiple nodes at each state in the sequence • “Object-oriented” view

  4. Complaint 1: Too General! • Absolute restriction imposed on structure of state • Could individual state structure be exploited? • Adopt “Polymorphic” view

  5. Complaint 2: Not General Enough! • Accurate probability estimation difficult with sparse data • P(Mac_instock|mainpage) = 0 • P(apple|mainpage) = 0.375 Web log for pages linked from main_page.html

  6. Solution: RMM • Represent sets of states as a relation (predicate) with instantiations of the relation defining specific states • E.g. – product_page(mac, in_stock) main_page() • Transitions can now occur from sets to sets and sets to states

  7. Ta-Da!

  8. Going further . . . • Using predicates defines a basic hierarchy over the state space • Why not impose further structure within each predicate argument? • Tree leaves correspond to unique arguements

  9. Again, Ta-Da!

  10. Sets of sets • Define the set of abstractions of a given state q = R( . . .) to be a set of states such that each argument to R in the abstraction is an ancestor to the corresponding argument in q. • More Formally (scientists love fancy math): R – The relation, Q – set of all possible instantiations of R d – possible arguments, D – the tree of a particular type of argument, delta – the arguments to the predicate for the given state q.

  11. Learning with Abstractions • Transitions between a state and an abstraction: • In practice, we can just count instances in the data. Where a is a transition matrix, q is a ground state (individual state), and alpha and beta are abstracted arguments.

  12. Defining a mixture model • Estimating the transition probability between ground states qs and qd: • Note that choosing lambda properly is crucial Where lambda is a mixing coefficient based on alpha and beta in the range [0,1] such that the sum of all lambdas is 1.

  13. Choosing Lambda: • There is a bias-variance tradeoff • Possibly high variance at lowest abstraction level • High bias at highest abstraction level • A good choice will have two properities: • Gets higher as abstraction depth increases • Gets lower as available training data decreases Where n = possible transitions from alpha to beta in the data, k is a design parameter to penalize lack of data, and rank(a) is where each d is an argument to the relation defining a.

  14. Probability Estimation Tree • With deep hierarchies and lots of arguments, considering every possible abstraction may not be feasible • Learn a decision tree over possible abstractions • To the left is the tree for the page Product_page(mac, in_stock)

  15. Empirical Evaluation • How well do the various flavors of RMM (RMM-uniform, RMM-Rank, RMM-PET) compare to traditional MMs? • Analyzed several web logs in various domains. • Tried to predict transition probabilities between pages using varied amounts of training data.

  16. Results 1: KDDCup 2000

  17. Results 2: CSE 142

  18. Related Work: The Cube • Exploiting structure within RMM states (DPRM)

  19. Fin.

More Related