1 / 35

Intelligent Systems (AI-2) Computer Science cpsc422 , Lecture 34 Apr, 7, 2014

Intelligent Systems (AI-2) Computer Science cpsc422 , Lecture 34 Apr, 7, 2014. Slide source: from Pedro Domingos UW & Markov Logic: An Interface Layer for Artificial Intelligence Pedro Domingos and Daniel Lowd University of Washington , Seattle. Lecture Overview.

coby
Download Presentation

Intelligent Systems (AI-2) Computer Science cpsc422 , Lecture 34 Apr, 7, 2014

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 7, 2014 Slide source: from Pedro Domingos UW & Markov Logic: An Interface Layer for Artificial Intelligence Pedro Domingos and Daniel LowdUniversity of Washington, Seattle CPSC 322, Lecture 34

  2. Lecture Overview • FinishInference in MLN • Probability of a formula, Conditional Probability • Markov Logic: applications • Beyond 322/422 (ML + grad courses) • AI conf. and journals • Watson…. • Final Exam (office hours, samples) • TA evaluation CPSC 322, Lecture 34

  3. Inference in MLN • MLN acts as a template for a Markov Network • We can always answer prob. queries using standard Markov network inference methods on the instantiated network • However, due to the size and complexity of the resulting network, this is often infeasible. • Instead, we combine probabilistic methods with ideas from logical inference, including satisfiabilityand resolution. • This leads to efficient methods that take full advantage of the logical structure. CPSC 322, Lecture 34

  4. MAP Inference • Find most likely state of world • Reduces to finding the pw that maximizes the sum of weights of satisfied clauses • Use weighted SAT solver(e.g., MaxWalkSAT[Kautz et al., 1997]) Probabilistic problem solved by logical inference method CPSC 322, Lecture 34

  5. Computing Probabilities P(Formula,ML,C) = ? • Brute force: Sum probs. of possible worlds where formula holds • MCMC: Sample worlds, check formula holds CPSC 322, Lecture 34

  6. Computing Cond. Probabilities Let’s look at the simplest case P(ground literal | conjuction of ground literals, ML,C) P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) ) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A) To answer this query do you need to create (ground) the whole network? CPSC 322, Lecture 34

  7. Computing Cond. Probabilities Let’s look at the simplest case P(ground literal | conjuction of ground literals, ML,C) P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) ) You do not need to create (ground) the part of the Markov Network from which the query is independent given the evidence CPSC 322, Lecture 34

  8. Computing Cond. Probabilities P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) ) The sub network is determined by the formulas (the logical structure of the problem) You can then perform Gibbs Sampling in this Sub Network CPSC 322, Lecture 34

  9. Lecture Overview • FinishInference in MLN • Probability of a formula, Conditional Probability • Markov Logic: applications • Beyond 322/422 (ML + grad courses) • AI conf. and journals • Watson…. • Final Exam (office hours, samples) • TA evaluation CPSC 322, Lecture 34

  10. Entity Resolution • Determining which observations correspond to the same real-world objects • (e.g., database records, noun phrases, video regions, etc) • Crucial importance in many areas • (e.g., data cleaning, NLP, Vision) CPSC 322, Lecture 34

  11. Entity Resolution: Example AUTHOR:H. POON & P. DOMINGOS TITLE:UNSUPERVISED SEMANTIC PARSING VENUE:EMNLP-09 SAME? AUTHOR:Hoifung Poon and Pedro Domings TITLE:Unsupervised semantic parsing VENUE:Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing AUTHOR:Poon, Hoifung and Domings, Pedro TITLE:Unsupervised ontology induction from text VENUE:Proceedings of the Forty-Eighth Annual Meeting of the Association for Computational Linguistics SAME? AUTHOR:H. Poon, P. Domings TITLE:Unsupervised ontology induction VENUE:ACL-10 CPSC 322, Lecture 34

  12. Entity Resolution (relations) Problem: Given citation database, find duplicate records Each citation has author, title, and venue fields We have 10 relations Author(bib,author) Title(bib,title) Venue(bib,venue) HasWord(author, word) HasWord(title, word) HasWord(venue, word) SameAuthor(author, author) SameTitle(title, title) SameVenue(venue, venue) SameBib(bib, bib) relate citations to their fields indicate which words are present in each field; represent field equality; represents citation equality; CPSC 322, Lecture 34

  13. Entity Resolution (formulas) Predict citation equality based on words in the fields Title(b1, t1) ∧ Title(b2, t2) ∧ HasWord(t1,+word) ∧ HasWord(t2,+word) ⇒ SameBib(b1, b2) (NOTE: +word is a shortcut notation, you actually have a rule for each word e.g., Title(b1, t1) ∧ Title(b2, t2) ∧ HasWord(t1,”bayesian”) ∧ HasWord(t2,”bayesian” ) ⇒ SameBib(b1, b2) ) Same 1000s of rules for author Same 1000s of rules for venue CPSC 322, Lecture 34

  14. Entity Resolution (formulas) Transitive closure SameBib(b1,b2) ∧ SameBib(b2,b3) ⇒ SameBib(b1,b3) SameAuthor(a1,a2) ∧ SameAuthor(a2,a3) ⇒ SameAuthor(a1,a3) Same rule fortitle Same rule for venue Link fields equivalence to citation equivalence – e.g., if two citations are the same, their authors should be the same Author(b1, a1) ∧ Author(b2, a2) ∧ SameBib(b1, b2) ⇒ SameAuthor(a1, a2) …and that citations with the same author are more likely to be the same Author(b1, a1) ∧ Author(b2, a2) ∧ SameAuthor(a1, a2) ⇒ SameBib(b1, b2) Same rules fortitle Same rules for venue CPSC 322, Lecture 34

  15. Benefits of MLN model • Standard approach: build a classifier that given two citations tells you if they are the same or not, and then apply transitive closure • New MLN approach: • performs collective entity resolution, where resolving one pair of entities helps to resolve pairs of related entities • e.g., inferring that a pair of citations are equivalent can provide evidence that the names AAAI-06 and 21st Natl. Conf. on AI refer to the same venue, even though they are superficially very different. This equivalence can then aid in resolving other entities. CPSC 322, Lecture 34

  16. Other MLN applications • Information Extraction • Co-reference Resolution (see lecture 1!) • Robot Mapping (infer the map of an indoor environment from laser range data) • Link-based Clustering (uses relationships among the objects in determining similarity) • Ontologies extraction from Text • ….. CPSC 322, Lecture 34

  17. Summary of tutorial on MLN for NLP at NA-ACL (2010) • We need to unify logical and statistical NLP • Markov logic provides a language for this • Syntax: Weighted first-order formulas • Semantics: Feature templates of Markov nets • Inference:Satisfiability, MCMC, lifted BP, etc. • Learning: Pseudo-likelihood, VP, PSCG, ILP, etc. • Growing set of NLP applications • Open-source software: Alchemy • Book: Domingos & Lowd, Markov Logic,Morgan & Claypool, 2009. alchemy.cs.washington.edu CPSC 322, Lecture 34

  18. Lecture Overview • FinishInference in MLN • Probability of a formula, Conditional Probability • Markov Logic: applications • Beyond 322/422 (ML + grad courses) • AI conf. and journals • Watson…. • Final Exam (office hours, samples) • TA evaluation CPSC 322, Lecture 34

  19. 422 big picture Hybrid: Det +Sto • Prob CFG ProbRelational Models Markov Logics Deterministic Stochastic Belief Nets Logics Approx. : Gibbs First Order Logics Markov Chains and HMMs Forward, Viterbi…. Approx. : Particle Filtering Ontologies Temporal rep. Query • Full Resolution • SAT Undirected Graphical Models Conditional Random Fields Markov Decision Processes and Partially Observable MDP Planning • Value Iteration • Approx. Inference Representation Reinforcement Learning Applications of AI Reasoning Technique CPSC 322, Lecture 34

  20. AI Conferences and Journals CPSC 322, Lecture 34

  21. Watson : analyzes natural language questions and content well enough and fast enough to compete and win against champion players at Jeopardy! “This Drug has been shown to relieve the symptoms of ADD with relatively few side effects." • 3 secs • 1000s of algorithms and KBs, Source:IBM CPSC 322, Lecture 34

  22. AI techniques in 422 / Watson • Parsing (PCFGs) • Shallow parsing (NP segmentation with CRFs) • Entity and relation Detection (NER with CRFs) • Logical Form Generation and Matching • Logical Temporal and Spatial Reasoning • Leveraging many databases, taxonomies, and ontologies • Confidence…. Probabilities (Bnetsto rank) • Strategy for playing Jeopardy…statistical models of players and games, game-theoretic analyses … .. and application of reinforcement-learning CPSC 322, Lecture 34

  23. From silly project to $1 billion investment • 2005-6 “IT’S a silly project to work on, it’s too gimmicky, it’s not a real computer-science test, and we probably can’t do it anyway.” These were reportedly the first reactions of the team of IBM researchers challenged to build a computer system capable of winning “Jeopardy! ………after 8-9 years… • On January 9th 2014, with much fanfare, the computing giant announced plans to invest $1 billion in a new division, IBM Watson Group. By the end of the year, the division expects to have a staff of 2,000 plus an army of external app developers …..Mike Rhodin, who will run the new division, calls it “one of the most significant innovations in the history of our company.” GinniRometty, IBM’s boss since early 2012, has reportedly predicted that it will be a $10 billion a year business within a decade. CPSC 322, Lecture 34

  24. More complex questions in the future… • Or something I read yesterday: “Should Europe reduce its energy dependency from Russia and what would it take?” CPSC 322, Lecture 34

  25. AI applications……. • DeepQA • Robotics • Search Engines • Games • Tutoring Systems • Medicine / Finance / ….. • ……. • Most companies are investing in AI and/or developing/adopting AI technologies CPSC 322, Lecture 34

  26. TA evaluation • KamyarArdekanikamyar.ardekany@gmail.com CPSC 322, Lecture 34

  27. Final, Tue, Apr 22, we will start at 8:30AM Location: BIO2200 How to prepare…. • Learning Goals (look at the end of the slides for each lecture) • Revise all the clicker questions, practice exercises, assignments and midterm • Will post more practice material in a few days • Office Hours – usual ones MeMon 11-12, Kamyar Thus 12:30-1:30 • – if high demand we will add a few more. • Can bring letter sized sheet of paper with anything written on it (double sided) CPSC 322, Lecture 34

  28. Example: Coreference Resolution Barack Obama, the 44th President of the United States, is the first African American to hold the office. …… CPSC 322, Lecture 34

  29. Example: Coreference Resolution Two mention constants: A and B Apposition(A,B) Head(A,“President”) Head(B,“President”) MentionOf(A,Obama) MentionOf(B,Obama) Head(A,“Obama”) Head(B,“Obama”) Apposition(B,A) In general, they represent feature templates for Markov Networks CPSC 322, Lecture 34

  30. Entity Resolution Can also resolve fields: HasToken(token,field,record) SameField(field,record,record) SameRecord(record,record) HasToken(+t,+f,r) ^ HasToken(+t,+f,r’) => SameField(f,r,r’) SameField(f,r,r’) <=> SameRecord(r,r’) SameRecord(r,r’) ^ SameRecord(r’,r”) => SameRecord(r,r”) SameField(f,r,r’) ^ SameField(f,r’,r”) => SameField(f,r,r”) More: P.Singla & P. Domingos, “Entity Resolution with Markov Logic”, in Proc. ICDM-2006. CPSC 322, Lecture 34

  31. Information Extraction Unsupervised Semantic Parsing, Hoifung Poon and Pedro Domingos. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL. UNSUPERVISED SEMANTIC PARSING. H. POON & P. DOMINGOS. EMNLP-2009. CPSC 322, Lecture 34

  32. Information Extraction Author Title Venue Unsupervised Semantic Parsing, Hoifung Poon and Pedro Domingos. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL. SAME? UNSUPERVISED SEMANTIC PARSING. H. POON & P. DOMINGOS. EMNLP-2009. CPSC 322, Lecture 34

  33. Information Extraction • Problem: Extract database from text orsemi-structured sources • Example: Extract database of publications from citation list(s) (the “CiteSeer problem”) • Two steps: • Segmentation:Use HMM to assign tokens to fields • Entity resolution:Use logistic regression and transitivity CPSC 322, Lecture 34

  34. Information Extraction Token(token, position, citation) InField(position, field!, citation) SameField(field, citation, citation) SameCit(citation, citation) Token(+t,i,c) => InField(i,+f,c) InField(i,+f,c) ^ InField(i+1,+f,c) Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’) ^ InField(i’,+f,c’) => SameField(+f,c,c’) SameField(+f,c,c’) <=> SameCit(c,c’) SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”) SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”) CPSC 322, Lecture 34

  35. Information Extraction Token(token, position, citation) InField(position, field!, citation) SameField(field, citation, citation) SameCit(citation, citation) Token(+t,i,c) => InField(i,+f,c) InField(i,+f,c) ^ !Token(“.”,i,c) ^ InField(i+1,+f,c) Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’) ^ InField(i’,+f,c’) => SameField(+f,c,c’) SameField(+f,c,c’) <=> SameCit(c,c’) SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”) SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”) More: H. Poon & P. Domingos, “Joint Inference in Information Extraction”, in Proc. AAAI-2007. CPSC 322, Lecture 34

More Related