Loading in 2 Seconds...

Intelligent Systems (AI-2) Computer Science cpsc422 , Lecture 34 Apr, 7, 2014

Loading in 2 Seconds...

- By
**coby** - Follow User

- 98 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Intelligent Systems (AI-2) Computer Science cpsc422 , Lecture 34 Apr, 7, 2014' - coby

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Computer Science cpsc422, Lecture 34

Apr, 7, 2014

Slide source: from Pedro Domingos UW & Markov Logic: An Interface Layer for Artificial Intelligence Pedro Domingos and Daniel LowdUniversity of Washington, Seattle

CPSC 322, Lecture 34

Lecture Overview

- FinishInference in MLN
- Probability of a formula, Conditional Probability
- Markov Logic: applications
- Beyond 322/422 (ML + grad courses)
- AI conf. and journals
- Watson….
- Final Exam (office hours, samples)
- TA evaluation

CPSC 322, Lecture 34

Inference in MLN

- MLN acts as a template for a Markov Network
- We can always answer prob. queries using standard Markov network inference methods on the instantiated network
- However, due to the size and complexity of the resulting network, this is often infeasible.
- Instead, we combine probabilistic methods with ideas from logical inference, including satisfiabilityand resolution.
- This leads to efficient methods that take full advantage of the logical structure.

CPSC 322, Lecture 34

MAP Inference

- Find most likely state of world

- Reduces to finding the pw that maximizes the sum of weights of satisfied clauses
- Use weighted SAT solver(e.g., MaxWalkSAT[Kautz et al., 1997])

Probabilistic problem solved by logical inference method

CPSC 322, Lecture 34

Computing Probabilities

P(Formula,ML,C) = ?

- Brute force: Sum probs. of possible worlds where formula holds
- MCMC: Sample worlds, check formula holds

CPSC 322, Lecture 34

Computing Cond. Probabilities

Let’s look at the simplest case

P(ground literal | conjuction of ground literals, ML,C)

P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) )

Friends(A,B)

Friends(A,A)

Smokes(A)

Smokes(B)

Friends(B,B)

Cancer(A)

Cancer(B)

Friends(B,A)

To answer this query do you need to create (ground) the whole network?

CPSC 322, Lecture 34

Computing Cond. Probabilities

Let’s look at the simplest case

P(ground literal | conjuction of ground literals, ML,C)

P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) )

You do not need to create (ground) the part of the Markov Network from which the query is independent given the evidence

CPSC 322, Lecture 34

Computing Cond. Probabilities

P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) )

The sub network is determined by the formulas

(the logical structure of the problem)

You can then perform Gibbs Sampling in

this Sub Network

CPSC 322, Lecture 34

Lecture Overview

- FinishInference in MLN
- Probability of a formula, Conditional Probability
- Markov Logic: applications
- Beyond 322/422 (ML + grad courses)
- AI conf. and journals
- Watson….
- Final Exam (office hours, samples)
- TA evaluation

CPSC 322, Lecture 34

Entity Resolution

- Determining which observations correspond to the same real-world objects

- (e.g., database records, noun phrases, video regions, etc)
- Crucial importance in many areas
- (e.g., data cleaning, NLP, Vision)

CPSC 322, Lecture 34

Entity Resolution: Example

AUTHOR:H. POON & P. DOMINGOS

TITLE:UNSUPERVISED SEMANTIC PARSING

VENUE:EMNLP-09

SAME?

AUTHOR:Hoifung Poon and Pedro Domings

TITLE:Unsupervised semantic parsing

VENUE:Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

AUTHOR:Poon, Hoifung and Domings, Pedro

TITLE:Unsupervised ontology induction from text

VENUE:Proceedings of the Forty-Eighth Annual Meeting of the Association for Computational Linguistics

SAME?

AUTHOR:H. Poon, P. Domings

TITLE:Unsupervised ontology induction

VENUE:ACL-10

CPSC 322, Lecture 34

Entity Resolution (relations)

Problem: Given citation database, find duplicate records

Each citation has author, title, and venue fields

We have 10 relations

Author(bib,author)

Title(bib,title)

Venue(bib,venue)

HasWord(author, word)

HasWord(title, word)

HasWord(venue, word)

SameAuthor(author, author)

SameTitle(title, title)

SameVenue(venue, venue)

SameBib(bib, bib)

relate citations to their fields

indicate which words are present in each field;

represent field equality;

represents citation equality;

CPSC 322, Lecture 34

Entity Resolution (formulas)

Predict citation equality based on words in the fields

Title(b1, t1) ∧ Title(b2, t2) ∧ HasWord(t1,+word) ∧ HasWord(t2,+word) ⇒ SameBib(b1, b2)

(NOTE: +word is a shortcut notation, you actually have a rule for each word e.g.,

Title(b1, t1) ∧ Title(b2, t2) ∧ HasWord(t1,”bayesian”) ∧ HasWord(t2,”bayesian” ) ⇒ SameBib(b1, b2) )

Same 1000s of rules for author

Same 1000s of rules for venue

CPSC 322, Lecture 34

Entity Resolution (formulas)

Transitive closure

SameBib(b1,b2) ∧ SameBib(b2,b3) ⇒ SameBib(b1,b3)

SameAuthor(a1,a2) ∧ SameAuthor(a2,a3) ⇒ SameAuthor(a1,a3)

Same rule fortitle

Same rule for venue

Link fields equivalence to citation equivalence – e.g., if two citations are the same, their authors should be the same

Author(b1, a1) ∧ Author(b2, a2) ∧ SameBib(b1, b2) ⇒ SameAuthor(a1, a2)

…and that citations with the same author are more likely to be the same

Author(b1, a1) ∧ Author(b2, a2) ∧ SameAuthor(a1, a2) ⇒ SameBib(b1, b2)

Same rules fortitle

Same rules for venue

CPSC 322, Lecture 34

Benefits of MLN model

- Standard approach: build a classifier that given two citations tells you if they are the same or not, and then apply transitive closure

- New MLN approach:
- performs collective entity resolution, where resolving one pair of entities helps to resolve pairs of related entities

- e.g., inferring that a pair of citations are equivalent can provide evidence that the names AAAI-06 and 21st Natl. Conf. on AI refer to the same venue, even though they are superficially very different. This equivalence can then aid in resolving other entities.

CPSC 322, Lecture 34

Other MLN applications

- Information Extraction
- Co-reference Resolution (see lecture 1!)
- Robot Mapping (infer the map of an indoor environment from laser range data)
- Link-based Clustering (uses relationships among the objects in determining similarity)
- Ontologies extraction from Text
- …..

CPSC 322, Lecture 34

Summary of tutorial on MLN for NLP at NA-ACL (2010)

- We need to unify logical and statistical NLP
- Markov logic provides a language for this
- Syntax: Weighted first-order formulas
- Semantics: Feature templates of Markov nets
- Inference:Satisfiability, MCMC, lifted BP, etc.
- Learning: Pseudo-likelihood, VP, PSCG, ILP, etc.
- Growing set of NLP applications
- Open-source software: Alchemy
- Book: Domingos & Lowd, Markov Logic,Morgan & Claypool, 2009.

alchemy.cs.washington.edu

CPSC 322, Lecture 34

Lecture Overview

- FinishInference in MLN
- Probability of a formula, Conditional Probability
- Markov Logic: applications
- Beyond 322/422 (ML + grad courses)
- AI conf. and journals
- Watson….
- Final Exam (office hours, samples)
- TA evaluation

CPSC 322, Lecture 34

Hybrid: Det +Sto

- Prob CFG

ProbRelational Models

Markov Logics

Deterministic

Stochastic

Belief Nets

Logics

Approx. : Gibbs

First Order Logics

Markov Chains and HMMs

Forward, Viterbi….

Approx. : Particle Filtering

Ontologies

Temporal rep.

Query

- Full Resolution
- SAT

Undirected Graphical Models

Conditional Random Fields

Markov Decision Processes and

Partially Observable MDP

Planning

- Value Iteration
- Approx. Inference

Representation

Reinforcement Learning

Applications of AI

Reasoning

Technique

CPSC 322, Lecture 34

AI Conferences and Journals

CPSC 322, Lecture 34

Watson : analyzes natural language questions and content well enough and fast enough to compete and win against champion players at Jeopardy!

“This Drug has been shown to relieve the symptoms of ADD with relatively few side effects."

- 3 secs

- 1000s of algorithms and KBs,

Source:IBM

CPSC 322, Lecture 34

AI techniques in 422 / Watson

- Parsing (PCFGs)
- Shallow parsing (NP segmentation with CRFs)
- Entity and relation Detection (NER with CRFs)
- Logical Form Generation and Matching
- Logical Temporal and Spatial Reasoning
- Leveraging many databases, taxonomies, and ontologies
- Confidence…. Probabilities (Bnetsto rank)
- Strategy for playing Jeopardy…statistical models of players and games, game-theoretic analyses … .. and application of reinforcement-learning

CPSC 322, Lecture 34

From silly project to $1 billion investment

- 2005-6 “IT’S a silly project to work on, it’s too gimmicky, it’s not a real computer-science test, and we probably can’t do it anyway.” These were reportedly the first reactions of the team of IBM researchers challenged to build a computer system capable of winning “Jeopardy!

………after 8-9 years…

- On January 9th 2014, with much fanfare, the computing giant announced plans to invest $1 billion in a new division, IBM Watson Group. By the end of the year, the division expects to have a staff of 2,000 plus an army of external app developers …..Mike Rhodin, who will run the new division, calls it “one of the most significant innovations in the history of our company.” GinniRometty, IBM’s boss since early 2012, has reportedly predicted that it will be a $10 billion a year business within a decade.

CPSC 322, Lecture 34

More complex questions in the future…

- Or something I read yesterday: “Should Europe reduce its energy dependency from Russia and what would it take?”

CPSC 322, Lecture 34

AI applications…….

- DeepQA
- Robotics
- Search Engines
- Games
- Tutoring Systems
- Medicine / Finance / …..
- …….

- Most companies are investing in AI and/or developing/adopting AI technologies

CPSC 322, Lecture 34

we will start at 8:30AM Location: BIO2200

How to prepare….

- Learning Goals (look at the end of the slides for each lecture)
- Revise all the clicker questions, practice exercises, assignments and midterm
- Will post more practice material in a few days
- Office Hours – usual ones MeMon 11-12, Kamyar Thus 12:30-1:30
- – if high demand we will add a few more.
- Can bring letter sized sheet of paper with anything written on it (double sided)

CPSC 322, Lecture 34

Example: Coreference Resolution

Barack Obama, the 44th President of the United States, is the first African American to hold the office. ……

CPSC 322, Lecture 34

Example: Coreference Resolution

Two mention constants: A and B

Apposition(A,B)

Head(A,“President”)

Head(B,“President”)

MentionOf(A,Obama)

MentionOf(B,Obama)

Head(A,“Obama”)

Head(B,“Obama”)

Apposition(B,A)

In general, they represent feature templates for Markov Networks

CPSC 322, Lecture 34

Entity Resolution

Can also resolve fields:

HasToken(token,field,record)

SameField(field,record,record)

SameRecord(record,record)

HasToken(+t,+f,r) ^ HasToken(+t,+f,r’)

=> SameField(f,r,r’)

SameField(f,r,r’) <=> SameRecord(r,r’)

SameRecord(r,r’) ^ SameRecord(r’,r”)

=> SameRecord(r,r”)

SameField(f,r,r’) ^ SameField(f,r’,r”)

=> SameField(f,r,r”)

More: P.Singla & P. Domingos, “Entity Resolution with Markov Logic”, in Proc. ICDM-2006.

CPSC 322, Lecture 34

Information Extraction

Unsupervised Semantic Parsing, Hoifung Poon and Pedro Domingos. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL.

UNSUPERVISED SEMANTIC PARSING. H. POON & P. DOMINGOS. EMNLP-2009.

CPSC 322, Lecture 34

Information Extraction

Author

Title

Venue

Unsupervised Semantic Parsing, Hoifung Poon and Pedro Domingos. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL.

SAME?

UNSUPERVISED SEMANTIC PARSING. H. POON & P. DOMINGOS. EMNLP-2009.

CPSC 322, Lecture 34

Information Extraction

- Problem: Extract database from text orsemi-structured sources
- Example: Extract database of publications from citation list(s) (the “CiteSeer problem”)
- Two steps:
- Segmentation:Use HMM to assign tokens to fields
- Entity resolution:Use logistic regression and transitivity

CPSC 322, Lecture 34

Information Extraction

Token(token, position, citation)

InField(position, field!, citation)

SameField(field, citation, citation)

SameCit(citation, citation)

Token(+t,i,c) => InField(i,+f,c)

InField(i,+f,c) ^ InField(i+1,+f,c)

Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’)

^ InField(i’,+f,c’) => SameField(+f,c,c’)

SameField(+f,c,c’) <=> SameCit(c,c’)

SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”)

SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)

CPSC 322, Lecture 34

Information Extraction

Token(token, position, citation)

InField(position, field!, citation)

SameField(field, citation, citation)

SameCit(citation, citation)

Token(+t,i,c) => InField(i,+f,c)

InField(i,+f,c) ^ !Token(“.”,i,c) ^ InField(i+1,+f,c)

Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’)

^ InField(i’,+f,c’) => SameField(+f,c,c’)

SameField(+f,c,c’) <=> SameCit(c,c’)

SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”)

SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)

More: H. Poon & P. Domingos, “Joint Inference in Information

Extraction”, in Proc. AAAI-2007.

CPSC 322, Lecture 34

Download Presentation

Connecting to Server..