Shared Features in Log-Linear Models

Representation Probabilistic Graphical Models Template Models Shared Features in Log-Linear Models

Modeling Repetition

Intelligence I(s1)‏ I(s2)‏ Grade G(s1)‏ G(s2)‏ Students s

Nested Plates Courses c Difficulty Intelligence Grade Students s D(c1)‏ D(c2)‏ I(s1,c1)‏ I(s2,c1)‏ I(s1,c2)‏ I(s2,c2)‏ G(s1,c1)‏ G(s2,c1)‏ G(s1,c1)‏ G(s2,c1)‏

Overlapping Plates Difficulty Intelligence Grade Courses c Students s D(c2)‏ D(c1)‏ I(s1)‏ I(s2)‏ G(s1,c1)‏ G(s1,c2)‏ G(s2,c1)‏ G(s2,c2)‏

Explicit Parameter Sharing D I G D(c2)‏ D(c1)‏ I(s1)‏ I(s2)‏ G(s1,c1)‏ G(s1,c2)‏ G(s2,c1)‏ G(s2,c2)‏

C Welcome to Geo101 Welcome to A low high CS101 Collective Inference easy / hard low / high

Plate Dependency Model • For a template variable A(U1,…,Uk): • Template parents B1(U1),…,Bm(Um) • CPD P(A | B1,…, Bm)

Ground Network • A(U1,…,Uk) with parents B1(U1),…,Bm(Um)

Plate Dependency Model • For a template variable A(U1,…,Uk): • Template parents B1(U1),…,Bm(Um)

Summary • Template for an infinite set of BNs, each induced by a different set of domain objects • Parameters and structure are reused within a BN and across different BNs • Models encode correlations across multiple objects, allowing collective inference • Multiple “languages”, each with different tradeoffs in expressive power

Shared Features in Log-Linear Models