1 / 82

A patient comes into a doctor’s office with a fever and a bad cough. Hypothesis space H :

Bayesian Networks Textbook: Probabilistic Reasoning, Sections 1-2, pp.168-175; Section 4.1, pp. 180-181; Section 5, pp. 188-196 S. Wooldridge, Bayesian Belief Networks (linked from course webpage). A patient comes into a doctor’s office with a fever and a bad cough. Hypothesis space H :

marnie
Download Presentation

A patient comes into a doctor’s office with a fever and a bad cough. Hypothesis space H :

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Bayesian NetworksTextbook: Probabilistic Reasoning, Sections 1-2, pp.168-175; Section 4.1, pp. 180-181; Section 5, pp. 188-196 S. Wooldridge, Bayesian Belief Networks(linked from course webpage)

  2. A patient comes into a doctor’s office with a fever and a bad cough. Hypothesis space H: h1: patient has flu h2: patient does not have flu Data D: coughing= true, fever = true, smokes = true

  3. Naive Bayes Cause cough fever flu Effects

  4. What if attributes are not independent? cough fever flu

  5. What if more than one possible cause? smokes cough fever flu

  6. Full joint probability distribution smokes Sum of all boxes is 1. In principle, the full joint distribution can be used to answer any question about probabilities of these combined parameters. However, size of full joint distribution scales exponentially with number of parameters so is expensive to store and to compute with.  smokes

  7. Full joint probability distribution smokes For example, what if we had another attribute, “allergies”? How many probabilities would we need to specify?  smokes

  8. Allergy Allergy smokes smokes Allergy Allergy smokes smokes

  9. Allergy Allergy smokes smokes Allergy Allergy smokes smokes But can reduce this if we know which variables are conditionally independent

  10. Bayesian networks • Idea is to represent dependencies (or causal relations) for all the variables so that space and computation-time requirements are minimized. Allergies smokes cough fever flu “GraphicalModels”

  11. Bayesian Networks = Bayesian Belief Networks = Bayes Nets Bayesian Network: Alternative representation for complete joint probability distribution “Useful for making probabilistic inference about models domains characterized by inherent complexity and uncertainty” Uncertainty can come from: • incomplete knowledge of domain • inherent randomness in behavior in domain

  12. Example: cough Conditional probability tables for each node flu smoke smoke flu flu smoke cough fever fever flu

  13. Inference in Bayesian networks • If network is correct, can calculate full joint probability distribution from network. where parents(Xi) denotes specific values of parents of Xi.

  14. Example • Calculate

  15. In general... • If network is correct, can calculate full joint probability distribution from network. where parents(Xi) denotes specific values of parents of Xi. But need efficient algorithms to do this (e.g., “belief propagation”, “Markov Chain Monte Carlo”).

  16. Example from the reading:

  17. What is the probability that Student A is late? What is the probability that Student B is late?

  18. What is the probability that Student A is late? What is the probability that Student B is late? Unconditional (“marginal”) probability. We don’t know if there is a train strike.

  19. Now, suppose we know that there is a train strike. How does this revise the probability that the students are late?

  20. Now, suppose we know that there is a train strike. How does this revise the probability that the students are late? Evidence: There is a train strike.

  21. Now, suppose we know that Student A is late. How does this revise the probability that there is a train strike? How does this revise the probability that Student B is late? Notion of “belief propagation”. Evidence: Student A is late.

  22. Now, suppose we know that Student A is late. How does this revise the probability that there is a train strike? How does this revise the probability that Student B is late? Notion of “belief propagation”. Evidence: Student A is late.

  23. Another example from the reading: pneumonia smoking temperature cough

  24. What is P(cough)?

  25. Three types of inference • Diagnostic: Use evidence of an effect to infer probability of a cause. • E.g., Evidence: cough=true. What is P(pneumonia| cough)? • Causal inference: Use evidence of a cause to infer probability of an effect • E.g., Evidence: pneumonia=true. What is P(cough| pneumonia)? • Inter-causal inference: “Explain away” potentially competing causes of a shared effect. • E.g., Evidence: smoking=true. What is P(pneumonia| cough and smoking)?

  26. Diagnostic: Evidence: cough=true. What is P(pneumonia| cough)? pneumonia smoking temperature cough

  27. Diagnostic: Evidence: cough=true. What is P(pneumonia| cough)? pneumonia smoking temperature cough

  28. Causal: Evidence: pneumonia=true. What is P(cough| pneumonia)? pneumonia smoking temperature cough

  29. Causal: Evidence: pneumonia=true. What is P(cough| pneumonia)? pneumonia smoking temperature cough

  30. Inter-causal: Evidence: smoking=true. What is P(pneumonia| cough and smoking)? pneumonia smoking temperature cough

  31. “Explaining away”

  32. Math we used: • Definition of conditional probability: • Bayes Theorem • Unconditional (marginal) probability • Probability inference in Bayesian networks:

  33. In-Class Exercise

  34. Complexity of Bayesian Networks For n random Boolean variables: • Full joint probability distribution: 2n entries • Bayesian network with at most k parents per node: • Each conditional probability table: at most 2kentries • Entire network: n 2k entries

  35. What are the advantages of Bayesian networks? • Intuitive, concise representation of joint probability distribution (i.e., conditional dependencies) of a set of random variables. • Represents “beliefs and knowledge” about a particular class of situations. • Efficient (approximate) inference algorithms • Efficient, effective learning algorithms

  36. Issues in Bayesian Networks • Building / learning network topology • Assigning / learning conditional probability tables • Approximate inference via sampling

  37. Real-World Example: The Lumière Project at Microsoft Research • Bayesian network approach to answering user queries about Microsoft Office. • “At the time we initiated our project in Bayesian information retrieval, managers in the Office division were finding that users were having difficulty finding assistance efficiently.” • “As an example, users working with the Excel spreadsheet might have required assistance with formatting “a graph”. Unfortunately, Excel has no knowledge about the common term, “graph,” and only considered in its keyword indexing the term “chart”.

  38. Networks were developed by experts from user modeling studies.

  39. Offspring of project was Office Assistant in Office 97, otherwise known as “clippie”. http://www.youtube.com/watch?v=bt-JXQS0zYc

  40. The famous “sprinkler” example(J. Pearl, Probabilistic Reasoning in Intelligent Systems, 1988)

  41. Recall rule for inference in Bayesian networks: Example: What is

  42. In-Class Exercise

  43. More Exercises What is P(Cloudy| Sprinkler)? What is P(Cloudy| Rain)? What is P(Cloudy| Wet Grass)?

  44. More Exercises What is P(Cloudy| Sprinkler)?

  45. More Exercises What is P(Cloudy| Rain)?

  46. More Exercises What is P(Cloudy| Wet Grass)?

More Related