1 / 16

Neyman-Pearson Tests and ROC curves

Neyman-Pearson Tests and ROC curves. ECE 7251: Spring 2004 Lecture 21 3/3/04. Prof. Aaron D. Lanterman School of Electrical & Computer Engineering Georgia Institute of Technology AL: 404-385-2548 <lanterma@ece.gatech.edu>. The Setup. Parametric data models No prior on hypotheses

felicia-reilly
Download Presentation

Neyman-Pearson Tests and ROC curves

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neyman-Pearson Tests and ROC curves ECE 7251: Spring 2004 Lecture 21 3/3/04 Prof. Aaron D. Lanterman School of Electrical & Computer Engineering Georgia Institute of Technology AL: 404-385-2548 <lanterma@ece.gatech.edu>

  2. The Setup • Parametric data models • No prior on hypotheses • Goal as usually stated: design test which maximizes (equivalently, minimizes ) • under the constraint • This lecture covers the case of simple • hypotheses; composite hypotheses are much more complicated and will be covered later

  3. Lagrange Multiplier Approach (1)

  4. To minimize f, set • Again we see the likelihood ratio! Lagrange Multiplier Approach (2)

  5. The Constraint • If we increase , PFA goes down • and PM goes up • If we decrease , PFA goes up and PM goes down • Hence, to minimize PM, choose  so that PFAis as big as possible under the constraint

  6. Solving for the Threshold • Mission: Find the  which satisfies • Warning: we’ve been implicitly assuming PFA is a continuous function of  (we’ll return to this issue later)

  7. Receiver Operating Characteristic • Plot PD vs. PFA as we sweep the threshold • Ex: ROC curve for Gaussian hypotheses, equal var., different means (graph from Van Trees Vol. I, p. 38)

  8. MinMax Lines on ROC Curves • Recall MinMax tests equate conditional risks: • MinMax point is intersection of ROC curve with the line (graph from Van Trees Vol. I, p. 45)

  9. Example: Poisson Data (1) • Suppose we collect one data point and want to determine:

  10. We have Example: Poisson Data (2) • Note y can only take integer values, so we may as well write whereis an integer

  11. ROC Curve for Poisson Test • Only certain points on the ROC graph are achievable using a normal LRT (graph from Van Trees Vol. I, p. 42)

  12. Randomization Fills in the Gaps • Randomly choose between two adjacent LRT tests • Only needed since data took discrete values; probabilities of error were not continuous functions of the threshold (graph from Van Trees Vol. I, p. 43)

  13. Properties of ROC Curves • ROC curves should lie above the PD=PFA line • A test is unbiased if its PDis at least as big as its PFA, i.e. PD PFA • Note Neyman-Pearson LRT gives an unbiased test • ROC curves should be concave downward • Slope of the ROC curve at a particular point equals the value of the threshold required to achieve the PD and PFA at that point

  14. We want to show this = ROC Property: Threshold=Slope (♣)

  15. Showing That Thing We Want to Show • Define

  16. (♣) More on Showing That Thing

More Related