Methodologies for sorting through the chaff presentation to dhs iaip 27 january 2005
Download
1 / 22

Methodologies for Sorting Through the Chaff Presentation to: DHS - PowerPoint PPT Presentation


  • 151 Views
  • Uploaded on

Methodologies for Sorting Through the Chaff Presentation to: DHS/IAIP 27 January 2005. Pherson Associates, LLC • Email: [email protected] Recognizing the Good Stuff: Using What If? Analysis and Outside-In Thinking to generate generic Indicators or Signposts Empirically-derived Checklists

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Methodologies for Sorting Through the Chaff Presentation to: DHS' - noe


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Methodologies for sorting through the chaff presentation to dhs iaip 27 january 2005

Methodologies for Sorting Through the ChaffPresentation to: DHS/IAIP27 January 2005

Pherson Associates, LLC • Email: [email protected]


Five approaches

Recognizing the Good Stuff:

Using What If? Analysis and Outside-In Thinking to generate generic Indicators or Signposts

Empirically-derived Checklists

Eliminating the Bad Stuff:

Deception Detection

Analysis of Competing Hypotheses

Five Approaches


Recognizing the good stuff
Recognizing the Good Stuff


What if analysis

Definition: Taking as a given that an event has occurred and then explaining how it came about.

Example:

Three years ago, terrorists just tried to crash a plane into the Eiffel tower. What if we had asked ourselves then: “Would they do something similar in the United States? How would they pull this off?”

What If? Analysis


What if analysis1

Value Added:

Focuses attention on all the things that must fall into place for a low probability--but high impact--event to actually occur.

Alerts you to potentially useful reporting that you might have ignored or would have regarded as noise.

What If? Analysis


What if analysis2

The Method:

Assume that what might be the case, is the case.

Develop a chain of argumentation based on both evidence and logic explaining how this outcome actually could have come about. This is called “thinking backwards.”

Generate a list of signposts or “observables” that would indicate that this outcome is coming about.

Monitor the traffic for any evidence that relates to the signposts or indicators.

What If? Analysis


Outside in thinking

Definition: A technique for identifying the full range of forces, factors, and trends that would indirectly shape an issue.

Examples:

In brainstorming how al-Qaeda elements are communicating with each other, are there any technological trends or new technologies that we need to consider (eg., use of “unsent” email messages, MP3, or IPods)?

Outside-In Thinking


Inside out versus outside in thinking
Inside-Out versus Outside-In Thinking


Competing approaches

Question: How do we assess a terrorist threat?

Inside-Out Approach:

Monitor reporting for tipoffs/lead information.

Extrapolate patterns from reporting trends.

Outside-In Approach:

Identify relevant global trends.

Assess how they might affect when, where, and how a terrorist might launch an attack.

Competing Approaches


Outside in thinking1

The Method:

Generate a generic description of the problem at hand.

List all the factors (social, technological, economic) that could have an impact (the subject usually has little influence over these factors but can exploit them).

Next list the factors over which the subject can exert some influence (choice of partners, methods of communication, capability to acquire feedback, etc.).

Assess how each of these factors could have an impact.

Look for data that suggests they actually have an impact.

Outside-In Thinking


Empirically derived checklists

The Method:

Establish categories of data (walk-ins, detainee reports, émigré reporting, human sources, etc.)

Review the reporting within each category and establish criteria for what turned out to be useful or not.

Develop a rough scale. For example, reporting that turned out to be useful usually met these criteria; bad reporting often fell into these boxes, etc.

Use these lists to rate the utility of incoming reporting.

Rate the new reporting based on these lists and revise/refine the lists over time.

Empirically-derived Checklists


Eliminating the bad stuff
Eliminating the Bad Stuff


Detecting deception

Look for deception when:

Accepting new information would require you to change your mind, alter a key assumption, or divert significant resources (protect all apartment buildings or shopping centers).

Your analysis hinges on a single or key piece of data.

The terrorists have a great deal to gain, or lose, if you take a specific action (discount a key source).

You know they have an effective feedback channel. (they are likely to learn of your reaction in the press).

Detecting Deception


Tactical indicators of deception

Is the source reliable?

Does the source have access?

Is the source vulnerable to control or manipulation by the terrorists?

Have the terrorists tried to deceive us in this way in the past?

Tactical Indicators of Deception


Tactical indicators of deception1

How accurate is the source’s reporting?

Examine the whole chain of evidence, including translations!

Does the critical evidence check out?

The subsource can be more critical than the source.

Does evidence from one source (HUMINT) conflict with another source (OSINT)?

Do other sources of information provide corroborating evidence?

Tactical Indicators of Deception


How to avoid deception

Be suspicious if forced to rely on sources who have not been seen or directly interviewed.

Try not to rely exclusively on non-material evidence (verbal intelligence).

Check all instances in which a source’s reports that initially appeared correct later turned out to be wrong-- and yet the source always seemed to offer a good explanation for the discrepancy.

Heed the opinions of those closest to the reporting.

Know the enemy’s limitations as well as his capabilities.

How to Avoid Deception


Analysis of competing hypotheses

Definition: not been seen or directly interviewed.

The identification of a complete set of alternative hypotheses, the systematic evaluation of data that is consistent and inconsistent with each hypothesis, and the rejection of hypotheses that contain too much inconsistent data.

Analysis of Competing Hypotheses


The value of ach

ACH helps you overcome three fundamental analytic traps: not been seen or directly interviewed.

Selective perception (or coming to closure too quickly) that usually results from focusing on a single hypothesis.

A failure to generate—at the outset—a complete set of alternative hypotheses.

Focusing on the evidence that tends to confirm rather than to disconfirm the hypothesis.

The Value of ACH


Analysis of competing hypotheses1

Advantages: not been seen or directly interviewed.

Ensures that all the information and argumentation is evaluated.

Helps avoid premature closure.

Highlights the evidence that is most “discriminating” in making the case.

Removes the relatively unimportant data from the equation.

Analysis of Competing Hypotheses


Ach the eight step process

1) Identify the possible hypotheses to be not been seen or directly interviewed.

considered. (use brainstorming techniques)

2) List significant evidence and arguments

for and against each hypothesis.

(include the absence of evidence)

3) Prepare a matrix to analyze the “diagnosticity”

of the evidence.

ACH: The Eight Step Process


Ach the eight step process1

4) Delete evidence and arguments that have not been seen or directly interviewed.

no diagnostic value.(that support all hypotheses)

5) Assess the relative likelihood of each

hypothesis. (try to refute each hypothesis rather

than confirm it)

6)Determine how sensitive the conclusion is

to just a few critical pieces of evidence.

(would the judgment still stand if the evidence

were wrong?)

ACH: The Eight Step Process


Ach the eight step process2

7) Report conclusions; establish the not been seen or directly interviewed.

relative likelihood of all hypotheses.

8) Identify milestones for further observation.

(to validate that the most likely hypothesis is

correct or to show that events are taking a

different direction than anticipated)

ACH: The Eight Step Process


ad