Slide1 l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

“To Ignore or Not to Ignore?” Follow-up to Statistically Significant Signals" PowerPoint PPT Presentation


  • 187 Views
  • Updated On :
  • Presentation posted in: General

“To Ignore or Not to Ignore?” Follow-up to Statistically Significant Signals". Reflections from San Diego County . Biosurveillance Information Exchange Working Group. 2/23/06. Jeffrey Johnson, MPH San Diego County Health & Human Services Agency. SAN DIEGO COUNTY.

Download Presentation

“To Ignore or Not to Ignore?” Follow-up to Statistically Significant Signals"

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

“To Ignore or Not to Ignore?”

Follow-up to Statistically Significant Signals"

Reflections from San Diego County

Biosurveillance Information Exchange Working Group

2/23/06

Jeffrey Johnson, MPH

San Diego County

Health & Human Services Agency


Slide2 l.jpg

SAN DIEGO COUNTY

  • Nearly 3 million population

  • International border

  • Large military presence

  • Biotechnology Hub

  • 21 Emergency Departments


Slide3 l.jpg

Early Event Detection in San Diego

  • Evolving effort since pre - 9/11

  • Data sources: ER Visits, Paramedic transports, 911 calls, school surveillance, OTC sales

  • Systems:Local SAS/Minitab system, ESSENCE, and BioSense

  • Statistical

  • Methods:Descriptive, time series, CUSUM, EWMA, process control methods (P&U Charts)

  • Multiple syndromes

  • Visualization and alerting

  • Incident Characterization

  • Follow-up to signals

County of San Diego

Health & Human Services Agency


Slide4 l.jpg

If We Ignore A Signal……

  • We take no action or follow-up

  • Save staff resources

  • Avoid bothering hospital staff yet again

  • Another data source may signal

  • “The Feds may pick it up”

  • Might lose an earlier start to a response

  • We might be dead wrong to ignore


Slide5 l.jpg

If We Do Not Ignore a Signal……

  • Will it be another “false alarm”

  • May detect an event earlier

  • Earlier response

  • Continued interaction with the medical community

  • Gain experience with follow-up

  • Increased situational awareness


Slide6 l.jpg

Characterization of Detections

  • Detection Method

  • Syndrome group

  • % Admitted

  • Deaths?

  • Geographic cluster?

  • Prior day’s level?

  • Recent level?

  • Age groups?

  • Severe syndrome?

  • Detections in other data sources?

  • Other epidemiological intelligence?

  • Other diagnostic information

Follow-up?

Action

or

No Action

or

Watch


Slide7 l.jpg

Detection Follow-up

with Medical Community

What is the final diagnosis of Patients A, B, C?

Is there a common pattern among admitted patients?

Did any have lab test results that might suggest a larger event?

Among patients with a common zip code, was there a

shared living setting or common exposure?

Can we send someone out to review medical charts?

What is your facility’s assessment of the situation?

County of San Diego

Health & Human Services Agency


Slide8 l.jpg

ABERRATION DETECTION RESPONSE GUIDELINES

Routine Surveillance Activities

Ignore?

IDENTIFY

Aberration

detected

NO

YES

Rule out system error

Potential

false positive

VERIFY

Preliminary evaluation

Ignore?

Describe initial results

Ignore?

“False Positive”

YES

NO

True Positive

NOTIFY

Inform key departmental staff

Inform key divisional staff

Intensive

monitoring & surveillance

Evaluate other

data sources

Ignore?

County of San Diego

Health & Human Services Agency

Cluster check


Slide9 l.jpg

GI Syndrome Over Time (10/31/04 – 8/24/05)

ED

911

Paramedic Runs


Slide10 l.jpg

The Significant Aspects

of Syndromic Surveillance

  • StatisticalSignificance

  • Public Health Significance

  • Significant Event

  • Significant Public Awareness

  • SignificantBiological Agent Detection


Slide11 l.jpg

Statistical significance vs. public health significance

HAZMAT FLAG – 12/04/2004

County of San Diego

Health & Human Services Agency


Slide12 l.jpg

Statistical significance vs. public health significance

County of San Diego

Health & Human Services Agency


Slide13 l.jpg

County of San Diego

Health & Human Services Agency


Slide14 l.jpg

Significant event with statistically significance outcomes

Syndromic Surveillance for Natural Disasters

San Diego Wild Fires, 2003

San Diego County


Slide15 l.jpg

Significant event with statistically significance outcomes

Syndromic surveillance for natural disasters


Slide16 l.jpg

Significant Public Awareness

“The Clinton Effect”

September 4, 2004

While spikes in both datasets are apparent, normalized counts show a relatively larger increase in ED visits on Sept. 6, 2004.


Slide17 l.jpg

Significant Public Awareness

7/7/05

London Bombings

San Diego County

Paramedic Transports

for

“Chest Pain”


Slide18 l.jpg

Significant BT Agent Detection

Biowatch

  • BioWatch Detection

    • Tells us agent, sensor site and date

    • Plume plot may help us narrow surveillance on a geographic area

Application of Syndromic Surveillance

Agent:

Syndrome categories

Specific word search in CC or DX fields

Sensor site:

Zip codes, population (schools)

Date:

Temporal based surveillance

New pre-detection baselines


Slide19 l.jpg

Anatomy of a Detection

(a case example)


Slide20 l.jpg

Daily Email Report

Feb 5, 2006

911 Call Data

Attached Table


Slide21 l.jpg

911 Call Center - GI Syndrome Signal


Slide23 l.jpg

Line listing for review

Non-specific call complaints


Slide24 l.jpg

911 Call Center - GI Syndrome Signal

The count for the signals include a consistent range

Various statistical signals

21 Signals since 07/01/03


Slide25 l.jpg

What did we do?

  • Magnitude of cases

  • Which method(s) signaled?

  • Check the other call centers

  • Check the other data sources

    • (ED data, EMS transports)

  • Review the line listing

  • Our conclusion…..

…... 14 vs mean of 7.8

…... CUSUM (2), P-Chart, U-Chart

…… No signals

…… No Signals

……. No apparent pattern

>>>>>

  • Super Bowl Sunday

  • Fewer trauma calls

  • Smaller denominator (P-Chart)

  • Traditional increase in GI on this day

  • Watch next day’s results


Slide26 l.jpg

Case Example #2

Hospital 9 ED Data

Respiratory Syndrome


Slide27 l.jpg

Hospital 9 - Daily Results Table


Slide30 l.jpg

Hospital 9 Respiratory Syndrome

01/01/04 - 02/03/06

Many signals….

So what’s the context?

Do we ever ignore the signals?


Slide31 l.jpg

Hospital 9 Respiratory Syndrome

  • 24 signals over a 37 day period

  • Count range: 11 – 34

  • Over time an increasing mean


Slide32 l.jpg

Greater Syndrome Specificity……

Hospital 9 Influenza-like-illness (ILI) Syndrome

  • “ILI syndrome” has greater syndrome specificity than “Respiratory” syndrome”

  • 16 signals over a 37 day period


Slide33 l.jpg

What We Have Learned

  • S$gn&ls Happen!

  • Make sure you see flames before yelling “Fire”

  • CUSUM 2 & 3 STD may be too sensitive

  • We lose precision with non-specific syndromes

  • Everyone wants to know what’s going on all the time

  • Increasing focus on situational awareness

  • Further evaluation and testing required


Slide34 l.jpg

Hype Cycle of Emerging

“Syndromic Surveillance” Technologies

Adapted from the Gartner Hype Cycle

Dual use, situational awareness, appropriate signals

Too many signals?, IT Costs, poor syndrome specificity, evaluation results

The “magic bullet”

9/11, Anthrax attacks

Prioritized data sets, protocols in place,

Event

or

Technology Trigger

Peak

of

Inflated Expectations

Trough

of Disillusionment

Slope

of

Enlightenment

Plateau

of

Productivity


Slide35 l.jpg

Considerations

  • More work in all areas of syndromic surveillance is needed

  • Knowledge requires responsibility

  • The enemy is studying our efforts

  • Current/future funding levels require reliability, efficiency and sustainability of systems and approaches

  • The Future:

  • Neural networks and Artificial Intelligence (AI)?

  • Are we ready?


Contact information l.jpg

Contact Information

Jeffrey Johnson

619-531-4945

[email protected]

Thank You


  • Login