Slide1 l.jpg
Sponsored Links
This presentation is the property of its rightful owner.
1 / 36

“To Ignore or Not to Ignore?” Follow-up to Statistically Significant Signals" PowerPoint PPT Presentation


  • 205 Views
  • Updated On :
  • Presentation posted in: General

“To Ignore or Not to Ignore?” Follow-up to Statistically Significant Signals". Reflections from San Diego County . Biosurveillance Information Exchange Working Group. 2/23/06. Jeffrey Johnson, MPH San Diego County Health & Human Services Agency. SAN DIEGO COUNTY.

Download Presentation

“To Ignore or Not to Ignore?” Follow-up to Statistically Significant Signals"

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


“To Ignore or Not to Ignore?”

Follow-up to Statistically Significant Signals"

Reflections from San Diego County

Biosurveillance Information Exchange Working Group

2/23/06

Jeffrey Johnson, MPH

San Diego County

Health & Human Services Agency


SAN DIEGO COUNTY

  • Nearly 3 million population

  • International border

  • Large military presence

  • Biotechnology Hub

  • 21 Emergency Departments


Early Event Detection in San Diego

  • Evolving effort since pre - 9/11

  • Data sources: ER Visits, Paramedic transports, 911 calls, school surveillance, OTC sales

  • Systems:Local SAS/Minitab system, ESSENCE, and BioSense

  • Statistical

  • Methods:Descriptive, time series, CUSUM, EWMA, process control methods (P&U Charts)

  • Multiple syndromes

  • Visualization and alerting

  • Incident Characterization

  • Follow-up to signals

County of San Diego

Health & Human Services Agency


If We Ignore A Signal……

  • We take no action or follow-up

  • Save staff resources

  • Avoid bothering hospital staff yet again

  • Another data source may signal

  • “The Feds may pick it up”

  • Might lose an earlier start to a response

  • We might be dead wrong to ignore


If We Do Not Ignore a Signal……

  • Will it be another “false alarm”

  • May detect an event earlier

  • Earlier response

  • Continued interaction with the medical community

  • Gain experience with follow-up

  • Increased situational awareness


Characterization of Detections

  • Detection Method

  • Syndrome group

  • % Admitted

  • Deaths?

  • Geographic cluster?

  • Prior day’s level?

  • Recent level?

  • Age groups?

  • Severe syndrome?

  • Detections in other data sources?

  • Other epidemiological intelligence?

  • Other diagnostic information

Follow-up?

Action

or

No Action

or

Watch


Detection Follow-up

with Medical Community

What is the final diagnosis of Patients A, B, C?

Is there a common pattern among admitted patients?

Did any have lab test results that might suggest a larger event?

Among patients with a common zip code, was there a

shared living setting or common exposure?

Can we send someone out to review medical charts?

What is your facility’s assessment of the situation?

County of San Diego

Health & Human Services Agency


ABERRATION DETECTION RESPONSE GUIDELINES

Routine Surveillance Activities

Ignore?

IDENTIFY

Aberration

detected

NO

YES

Rule out system error

Potential

false positive

VERIFY

Preliminary evaluation

Ignore?

Describe initial results

Ignore?

“False Positive”

YES

NO

True Positive

NOTIFY

Inform key departmental staff

Inform key divisional staff

Intensive

monitoring & surveillance

Evaluate other

data sources

Ignore?

County of San Diego

Health & Human Services Agency

Cluster check


GI Syndrome Over Time (10/31/04 – 8/24/05)

ED

911

Paramedic Runs


The Significant Aspects

of Syndromic Surveillance

  • StatisticalSignificance

  • Public Health Significance

  • Significant Event

  • Significant Public Awareness

  • SignificantBiological Agent Detection


Statistical significance vs. public health significance

HAZMAT FLAG – 12/04/2004

County of San Diego

Health & Human Services Agency


Statistical significance vs. public health significance

County of San Diego

Health & Human Services Agency


County of San Diego

Health & Human Services Agency


Significant event with statistically significance outcomes

Syndromic Surveillance for Natural Disasters

San Diego Wild Fires, 2003

San Diego County


Significant event with statistically significance outcomes

Syndromic surveillance for natural disasters


Significant Public Awareness

“The Clinton Effect”

September 4, 2004

While spikes in both datasets are apparent, normalized counts show a relatively larger increase in ED visits on Sept. 6, 2004.


Significant Public Awareness

7/7/05

London Bombings

San Diego County

Paramedic Transports

for

“Chest Pain”


Significant BT Agent Detection

Biowatch

  • BioWatch Detection

    • Tells us agent, sensor site and date

    • Plume plot may help us narrow surveillance on a geographic area

Application of Syndromic Surveillance

Agent:

Syndrome categories

Specific word search in CC or DX fields

Sensor site:

Zip codes, population (schools)

Date:

Temporal based surveillance

New pre-detection baselines


Anatomy of a Detection

(a case example)


Daily Email Report

Feb 5, 2006

911 Call Data

Attached Table


911 Call Center - GI Syndrome Signal


Line listing for review

Non-specific call complaints


911 Call Center - GI Syndrome Signal

The count for the signals include a consistent range

Various statistical signals

21 Signals since 07/01/03


What did we do?

  • Magnitude of cases

  • Which method(s) signaled?

  • Check the other call centers

  • Check the other data sources

    • (ED data, EMS transports)

  • Review the line listing

  • Our conclusion…..

…... 14 vs mean of 7.8

…... CUSUM (2), P-Chart, U-Chart

…… No signals

…… No Signals

……. No apparent pattern

>>>>>

  • Super Bowl Sunday

  • Fewer trauma calls

  • Smaller denominator (P-Chart)

  • Traditional increase in GI on this day

  • Watch next day’s results


Case Example #2

Hospital 9 ED Data

Respiratory Syndrome


Hospital 9 - Daily Results Table


Hospital 9 Respiratory Syndrome

01/01/04 - 02/03/06

Many signals….

So what’s the context?

Do we ever ignore the signals?


Hospital 9 Respiratory Syndrome

  • 24 signals over a 37 day period

  • Count range: 11 – 34

  • Over time an increasing mean


Greater Syndrome Specificity……

Hospital 9 Influenza-like-illness (ILI) Syndrome

  • “ILI syndrome” has greater syndrome specificity than “Respiratory” syndrome”

  • 16 signals over a 37 day period


What We Have Learned

  • S$gn&ls Happen!

  • Make sure you see flames before yelling “Fire”

  • CUSUM 2 & 3 STD may be too sensitive

  • We lose precision with non-specific syndromes

  • Everyone wants to know what’s going on all the time

  • Increasing focus on situational awareness

  • Further evaluation and testing required


Hype Cycle of Emerging

“Syndromic Surveillance” Technologies

Adapted from the Gartner Hype Cycle

Dual use, situational awareness, appropriate signals

Too many signals?, IT Costs, poor syndrome specificity, evaluation results

The “magic bullet”

9/11, Anthrax attacks

Prioritized data sets, protocols in place,

Event

or

Technology Trigger

Peak

of

Inflated Expectations

Trough

of Disillusionment

Slope

of

Enlightenment

Plateau

of

Productivity


Considerations

  • More work in all areas of syndromic surveillance is needed

  • Knowledge requires responsibility

  • The enemy is studying our efforts

  • Current/future funding levels require reliability, efficiency and sustainability of systems and approaches

  • The Future:

  • Neural networks and Artificial Intelligence (AI)?

  • Are we ready?


Contact Information

Jeffrey Johnson

619-531-4945

[email protected]

Thank You


  • Login