expert opinion elicitation
Download
Skip this Video
Download Presentation
Expert-Opinion Elicitation

Loading in 2 Seconds...

play fullscreen
1 / 21

Expert-Opinion Elicitation - PowerPoint PPT Presentation


  • 150 Views
  • Uploaded on

Expert-Opinion Elicitation. Robert C. Patev North Atlantic Division – Regional Technical Specialist (978) 318-8394. Expert-Opinion Elicitation. Subjective Estimation Elicitation Process Background Expert-Opinion Elicitation (EOE) Process Probability Axioms of Probability

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Expert-Opinion Elicitation' - jerry-johnson


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
expert opinion elicitation

Expert-Opinion Elicitation

Robert C. Patev

North Atlantic Division – Regional Technical Specialist

(978) 318-8394

slide2
Expert-Opinion Elicitation
  • Subjective Estimation
  • Elicitation Process
    • Background
    • Expert-Opinion Elicitation (EOE) Process
  • Probability
    • Axioms of Probability
    • Medians and Percentiles
  • Training Example
subjective estimation
Subjective Estimation
  • Uses of one or more experts to estimate a probability (qualitative or quantitative) for use in engineering risk analysis
    • Good for first estimate of probabilities
    • Quick, cost effective and efficient method
    • Problems:
      • Not a formal elicitation
      • Usually not well documented
      • Probabilities may not be repeatable or defendable
      • Probabilities may be highly subjective and biased
      • Probabilities have larger uncertainties compared to structured elicitation values
subjective estimation1
Subjective Estimation
  • How good are we at quantifying subjective estimates?
  • Let us see…..
subjective estimation2
Subjective Estimation
  • How good are we at quantifying subjective estimates?
    • Class Example:
      • How may ships passed through the Panama Canal last year?
        • Give best estimate
slide6
Expert-Opinion Elicitation
  • Background
    • Process developed by RAND Corporation in late 1950’s - early 1960’s
      • Delphi Method
      • Scenario Analysis
    • Effects of thermonuclear war
    • Civil Defense strategic planning
      • Examine if U.S. population could survive a nuclear attack
slide7
Expert-Opinion Elicitation
  • Background
    • Definition
    • A formal (protocol), heuristic (through discussion) process of obtaining information or answers to specific questions called issues
      • e.g., failure rates or probabilities, and failure consequences
slide8
Expert-Opinion Elicitation
  • Background
    • EOE is used for preliminary risk evaluation (screening) is not really intended to replace more complex reliability models
    • EOE has been used by industry and government agencies to develop failure probabilities when there is a lack of failure information
slide9
Expert-Opinion Elicitation
  • Drawbacks
    • Subjective process
      • Not consensus building
      • Inherently contains bias and dominance
    • Difficult to process result to determine reliability or hazard rates
      • Assumptions need to be made
  • Current Usage in USACE
    • Supplement to other models
      • Calculate reliability (not for critical components)
      • Event tree probabilities
      • Used in consultation with HQUSACE
slide10
Expert-Opinion Elicitation
  • EOE Process
    • Participants
      • Experts
      • Observers
      • Listeners
      • Technical Integrator and Facilitator
      • Peer Reviewers
        • ITR process and results
slide11
Expert-Opinion Elicitation
  • EOE Process
    • Identification and Selection of Experts
      • Strong relevant expertise
      • Familiarity and knowledge with issues
      • Willingness to act as impartial evaluators
      • Willingness to participate, prepare, and provide needed input
      • Strong communication skills, interpersonal skills, and ability to generalize
slide12
Expert-Opinion Elicitation
  • EOE Process
    • Inform experts of issues
      • “Read ahead” materials
      • Site visits
    • Train experts
    • Elicitation
      • First opinion
      • Discussion among experts
      • Second opinion
slide13
Expert-Opinion Elicitation
  • Probability
    • General expressions
      • Percent (1% probability of failure)
      • Fraction (1/100)
      • Relative frequency (1 out of 1000)
    • Axioms of Probability
      • 0 < Pf < 1
      • Sum of probabilities over all possible outcomes must equal 1.
        • This assume events are independent.
slide14
Expert-Opinion Elicitation
  • Statistics
    • Median
      • e.g., Median income, median age
      • Rank value
      • For odd n, value with rank of (n+1)/2
      • For even n, average of value with rank n/2 or (n/2) + 1
      • Used to limit extreme values
    • Average
      • Sum of Xi divided by sample size
median vs average
Sample 1

100

100

200

300

400

Median = 200

Average = 220

Sample 2

100

100

200

300

2000

Median = 200

Average = 540

Median Vs. Average
slide16
Expert-Opinion Elicitation
  • Percentiles
    • A p-percentile value (Xp) based on a sample is the value of the parameter such that p% of the data is less than or equal to Xp
      • e.g., The median is the 50th percentile
slide21
Expert-Opinion Elicitation
  • Class Example
    • Six experts required
    • Unknown issue given to experts
      • Define assumptions of issue
      • Elicit first values
      • First results
      • Expert Discussion
      • Elicit second values
      • Show final elicitation results
ad