1 / 28

Bayesian Networks

Bayesian Networks. Outline. Introduction to Bayesian Networks Conditional probability and Bayes ’ Theorem Analyzing a Bayesian Network Practical Uses for Bayesian Networks Conclusion Resources. Why using BN?.

muncel
Download Presentation

Bayesian Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Networks

  2. Outline • Introduction toBayesian Networks • Conditional probability and Bayes’ Theorem • Analyzing a Bayesian Network • Practical Uses for Bayesian Networks • Conclusion • Resources

  3. Why using BN? • There are countless real world examples where the probability of one event is conditional on the probability of a previous one. Decision aids, data fusion, diagnostic aids, classification, Natural language desambiguisation, data mining.

  4. What isBN? • Directed Acyclic Graph • Graphical formalism to represent dependencies between random variables • Nodes are random variables

  5. What isBN? Train strike Martin late Norman late

  6. What isBN Key features of BN: • Enable us to model and reason about uncertainty. • For example: • A train strike does not imply that Norman will definitely be late (he might leave early and drive). • but there is an increased probability that he will be late. • We model this by filling in a probability table for each node. Conditional Probability Table (CPT).

  7. What isBN? Train strike Train delay Train delay Martin late Norman late Martin late Norman late

  8. Conditional probability • Recall: P(A|B) is the probability of event A, given event B. • Lemma: P(AB) = P(A|B)P(B) • Bayes’ Theorem

  9. Example of Bayes’ Theorem • 從台北車站到世貿中心，假設目前有三條道路可行，分別為信義路，忠孝東路以及南京東路。各路線被選重的機率分別為： P(A) = 0.3，P(B) = 0.4，P(C) = 0.3。假設走信義路塞車的機率為0.4，走忠孝東路塞車的機率為 0.5，走南京東路塞車的機率為0.3。試問不塞車要走哪條路線為宜?

  10. Example of Bayes’ Theorem • Answer: P(A) = 0.3，P(B) = 0.4，P(C) = 0.3。令G表示塞車，P(G|A) = 0.4 表示走信義路塞車的機率，P(G|B) = 0.5 表示走忠孝東路塞車的機率，P(G|C) = 0.3 表示走南京東路塞車的機率。 塞車機率最小

  11. Analyzing a Bayesian Network Train strike Train strike Train strike Martin late Norman late Martinlate Normanlate Calculate the probability that Norman islate: p(Norman late) = p(Norman late | train strike) * p(train strike) + p(Norman late | no train strike) * p(no train strike) = (0.8 * 0.1) + (0.1 * 0.9) = 0.17 (marginal probability ) Similarly, the marginal probability that Martin is late = 0.51

  12. Analyzing a Bayesian Network • Revising probabilities Suppose that we do not know if there is a train strike but do know that Norman is late. Given : p(Norman late| train strike) = 0.8, p(Norman late) = 0.17, determine: a) the (revised) probability that there is a train strike; and b) the (revised) probability that Martin will be late.

  13. Analyzing a Bayesian Network • To calculate a) we use Bayes’ theorem : The observation that Norman is late significantly increases the probability that there is a train strike (up from 0.1 to 0.47). • To calculate b)p(Martin late) = p(Martin late | train strike) * p(train strike) + p(Martin late | no train strike) * p(no train strike) = (0.6 * 0.47) + (0.5 * 0.53) = 0.55 Norman is late has slightly increased the probability that Martin is late. When we enter evidence and use it to update the probabilities ,we call it propagation.

  14. Example of Bayesian Network • I'm at work, neighbor to say my is ringing, but neighbor doesn't . Sometimes it's set off by minor . Is there a ? • Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls • Network topology reflects "causal" knowledge: • A burglar can set the alarm off • An earthquake can set the alarm off • The alarm can cause Mary to call • The alarm can cause John to call John calls alarm Mary call earth quakes burglar

  15. Example contd.

  16. Example contd. • Chain rule: • P(A,B) = p(A|B) p(B) => P(A,B,C) = P(A| B,C) P(B,C) = P(A|B,C) P(B|C) P(C) • P(A1, A2, ..., An) = P(A1| A2, ..., An) P(A2| A3, ..., An) P(An-1|An) P(An) • Joint probability distribution in BBN • {A1,A2,…,An}: a set of variables in BN • parents(Ai): the set of parents of the node Ai in BN. • The joint probability distribution for {A1,A2,…,An} =

  17. Example contd. • e.g., Alarm has sounded, but neither a burglary nor an earthquake has occurred, and both John and Mary call. P(j , m , a , b , e) = P (j | a) P (m | a) P (a | b, e) P (b) P (e) = 0.90 x 0.70 x 0.001 x 0.999 x 0.998 = 0.00062

  18. P(s)=0.2 Example contd. Smoking history P(l|s)=0.003P(l|s)=0.00005 • the probability that someone has a smoking history, lung cancer but not bronchitis, suffers from fatigue and tests positive in an X-ray test is: P(b|s)=0.25P(b|s)=0.05 Bronchitis LungCancer P(f|b,l)=0.75P(f|b,l)=0.10P(f|b,l)=0.5P(f|b,l)=0.05 X-ray Fatigue P(x|l)=0.6P(x|l)=0.02

  19. Why do we need a BBN for the probability computations? • A network consisting of five variables (nodes) A,B,C,D,E. • No specification of the dependencies. • Apply chain rule, we get p(A,B,C,D,E) = p(A|B,C,D,E)*p(B|C,D,E)*p(C|D,E)*p(D|E)*p(E) • Now suppose that the dependencies are explicitly modeled in a BN as: • Then the joint probability distribution p(A,B,C,D,E) = p(A|B)*p(B|C,E)*p(C|D)*p(D)*p(E) D C E B A

  20. Dealing with large amount of variables • Tricky to work out all the probabilities and the revised probabilities. • Can BN be used to solve realistic problems? • With the introduction of software tools that implement • Algorithms • providing a graphical interface to draw the graphs and fill in the probability tables • BN tools: • Hugin (http://www.hugin.com/) • Bayesian Knowledge Discoverer (BKD) • Norsys (http://www.norsys.com/)

  21. Practical Uses for Bayesian Networks • AutoClass • Automatically interpolate raw data from interplanetary probes, and deep space explorations. • For more information, please log on to http://ic-www.arc.nasa.gov/ic/projects/bayes-group/autoclass/index.html

  22. Practical Uses for Bayesian Networks • Lumiere • Project created by Microsoft • Resulted in the "Office Assistant" with the introduction of the office 95 suite of desktop products. • Foundation: • “Inferring Informational Goals from Free-Text Queries: A Bayesian Approach.” David Heckerman and Eric HorvitzDecision Theory & Adaptive Systems GroupMicrosoft Research, Redmond, Washington 98052-6399. • Describe a Bayesian approach to modeling the relationship between words in a user's query for assistance and the informational goals of the user.

  23. Practical Uses for Bayesian Networks • 醫療影像傳輸及資料探索之系統開發(The Development of Communication and Data Mining System for Medical Image) • 對於子宮頸抹片檢查建立資料庫 • 獲取的異常抹片病人之病例資料，丟入貝氏網路中training。 • 研究發現貝氏網路可在上消化道疾病病患舌診影像分析資料獲得最佳診斷結果。 • 資料來源: 醫療影像傳輸及資料探索之系統開發 ( The Development of Communication and Data Mining System for Medical Image )中原大學醫學工程學系 范振添 著

  24. Practical Uses for Bayesian Networks • An Agent-based Bayesian Forecasting Model for Enhanced Network Security • Provides a novel application of the Bayesian forecasting technique to predict user actions. • Bayesian Intrusion Detection System • Invalid behavior is determined by comparing • User's current behavior  Their typical behavior • User’s current behavior  A set of general rules governing valid behavior formed by systems administrators Source:An agent-based Bayesian forecasting model for enhanced network securityPikoulas, J.; Buchanan, W.J.; Mannion, M.; Triantafyllopoulos, K.;Engineering of Computer Based Systems, 2001. ECBS 2001. Proceedings. Eighth Annual IEEE International Conference and Workshop on the 17-20 April 2001 Page(s):247 - 254

  25. Practical Uses for Bayesian Networks • 基於貝氏網路分析法之學生評鑑模式(Student assessment model based on bayesian method) • 元智大學資工所 學習科技實驗室 鄭乃塵撰 • 指導老師：劉晨鐘 教授 • 貝氏網路為基礎之動態軟體專案管理(Dynamic Software Project Management using Bayesian Belief Network) • 元智大學資工所 軟體工程實驗室 陳建蒝撰 • 指導教授：范金鳳 教授

  26. Conclusion • Offer assistance in a wide range of endeavors. • Support the use of probabilistic inference to update and revise belief values. • Support complex inference modeling including rational decision making systems, value of information and sensitivity analysis. • Useful for causality analysis and through statistical induction they support a form of automated learning.

  27. Resources • Papers about BBNs • A Roadmap to Research on Bayesian Networks and other Decomposable Probabilistic Models - hosted by Lonnie Chrisman, School of Computer Science, Pittsburgh. • Special Issue on Bayesian Networks: Communications of the ACM., March, 1995, vol 38, no. 3. • Special Issue on Data Mining: Communications of the ACM., November, 1996, vol 39, no. 11. • D. Heckerman. A tutorial on learning Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research, March, 1995.

  28. Resources • BBN projects • IMPRESS (IMproving the software PRocESS using bayesian nets) EPSRC Project GR/L06683. 1 Jan 1997 - 31 Dec 1999. • SERENE (SafEty and Risk Evaluation using Bayesian NEts) ESPRIT Framework IV Collaborative Project 22187. 1 June 1996 - 1 June 1999. • TRACS DERA contract for CSR, LSF/E20173. Sept 1996 - February 1999. • European Union, Project INTAS 93-725 Multivariate Statistical Analysis and Bayesian Belief Networks for the Development of Intelligent Decision Support Systems with Applications in Medecine and Agriculture

More Related