anticipating and satisfying clinician information needs the infobutton manager project l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project PowerPoint Presentation
Download Presentation
Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project

Loading in 2 Seconds...

play fullscreen
1 / 72

Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project - PowerPoint PPT Presentation


  • 121 Views
  • Uploaded on

Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project. James J. Cimino, M.D. Department of (Bio)Medical Informatics Weekly Seminar March 6, 2003. Studying Clinician Information.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project' - barny


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
anticipating and satisfying clinician information needs the infobutton manager project

Anticipating and Satisfying Clinician Information Needs:The Infobutton Manager Project

James J. Cimino, M.D.

Department of (Bio)Medical Informatics

Weekly Seminar

March 6, 2003

studying clinician information
Studying Clinician Information
  • Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med 1985 Oct;103(4):596-9
  • Other studies: Osheroff, Forsythe, Gorman, Ely…
  • Types of needs
    • Patient-specific
    • General knowledge
    • Institutional
  • Resources
    • Medical record
    • Colleagues
    • Journals
    • Books
do clinicians satisfy their needs
Do Clinicians Satisfy their Needs?
  • Needs are underestimated
  • Information needs may not be recognized
  • Resources are outdated or unavailable
  • Computers not used
  • Resolution may be deferred
  • Unsatisfied information needs are a potential source for errors
needs related to cis use
Needs Related to CIS Use?
  • Very small part of the clinician’s day
  • Very important part of the day
    • New information stimulates needs
    • Decision-making moments
  • Opportunity to resolve needs (using a computer)
  • Context might predict needs
  • Context might help resolve needs
  • Needs in this setting have not been studied
context dependent information needs
Context-Dependent Information Needs

!

?

Age

Sex

Role

Training

Task

Data

Institution

Context

hypothesis
Hypothesis:
  • Information needs are stereotypical
  • Information needs can be predicted based on context
  • Appropriate resource can be identified (automatically or in advance)
  • Resource can be queried automatically
  • Information from context can be used for query
  • “Infobuttons”
a brief history of infobuttons
A Brief History of Infobuttons
  • 1991: “Medline Button” translated ICD9 to MeSH and performed Medline searches
  • 1994: World Wide Web
  • 1994: First CPMC Web-based CIS
  • 1996: WebCIS
  • 1996: Infobuttons link microbiology and medication data to PubMed and Micromedex
  • 1997: Infobuttons for patients (PatCIS)
  • 2002: Handheld, wireless infobuttons (PalmCIS)
infobutton manager project
Infobutton Manager Project
  • Study information needs of CIS users
  • Characterize needs based on context
  • Develop way to anticipate needs
  • Develop ways to resolve needs
  • Deploy anticipation/solution
  • Evaluate
infobutton manager team members
Infobutton Manager Team Members
  • Jim Cimino
  • Jianhua Li
  • Sue Bakken
  • Vimla Patel
  • Mark Graham
  • Leanne Currie
  • Mureen Allen
  • Anne-Marie Ramierez
  • Your name here?
methods
Methods
  • Determine information needs
    • WebCIS log files
    • WebCIS user questionnaires
    • Observational studies
  • Characterize information needs
  • Build Infobutton Manager (IM)
  • Build infobuttons
  • Populate IM database
  • Integrate IM with WebCIS
  • Evaluation
webcis log file methods
WebCIS Log File Methods
  • Health Resources page in WebCIS
  • Infobuttons in WebCIS
  • Problems:
    • Which user selected which resource?
    • What was the user was doing?
  • Solution: look at previous activity at same IP address
webcis log file records
WebCIS Log File Records

Mar 2 16:16:46 webcis3-i syslog: |WebCIS|ciminoj|156.111.145.61|3131313 |lab^2003-03-02-05.19.00.819650|view

Mar 2 16:18:55 webcis3-i syslog: |WebCIS|ciminoj|156.111.145.61|3131313 |healthresource|view

Mar 2 16:18:58 webcis3-i syslog: |Resources|(nobody)|156.111.145.61|| |url^http://salk.cpmc.columbia.edu:70/mdxdocs

webcis log files results
WebCIS Log Files Results
  • Six months of log files
  • 2,607 users (1,034 per month) used a resource
  • 2931 IPs (2008 “156.x.x.x”)
  • User types:

Housestaff: 51%

Attendings: 34%

Nurses: 5%

Other: 10%

webcis log files results15
WebCIS Log Files Results

38,763 uses of resources

14,036 outside WebCIS (2339/month)

24,727 within WebCIS (4,104/month; 4/user)

19,913 uses of 24 resources (3,319/month)

4,814 uses of 3 infobuttons (802/month)

webcis log files results16
WebCIS Log Files Results

Popular resources: Inside Outside

Micromedex: 56.5% 51.7%

Ovid: 7.4% 8.2%

NYP Formulary 7.2% 7.1%

Harrison’s 5.1% 5.1%

PubMed 5.1% 5.1%

Health Journals 5.0% 8.2%

Medline Plus 3.3% 3.6%

ICD9-CM 3.3% 2.9%

Infobuttons

Pharmacy/Micromedex 92.9% -

Microbiology/PubMed 6.1% -

Pharmacy/PubMed 1.0% -

webcis log files results17
WebCIS Log Files Results

Health Resources: 19,913 times in 105 contexts:

Lab results: 52.1%

Radiology results: 9.0%

Clinical notes: 7.4%

Pathology results: 2.8%

Visit list: 2.1%

Pharmacy: 2.0%

Discharge summary: 1.9%

80.4%

Infobuttons: 4,814 times in 2 contexts:

Pharmacy: 93.9%

Lab results (Micro): 6.1%

100%

webcis log files results18
WebCIS Log Files Results

Context: Lab Rad Note Card Path Vis Phar DSum

MDX 53% 9% 3% 3% 3% 2% 2% 2%

Ovid 49% 9% 6% 2% 3% 1% 1% 1%

Formul. 52% 8% 6% 3% 3% 1% 3% 2%

Harrison 49% 10% 5% 3% 4% 1% 1% 3%

PubMed 48% 10% 7% 3% 5% 1% 3% 1%

Journals 43% 8% 7% 6% 5% 2% 2% 1%

Med+ 50% 8% 6% 3% 2% 4% 2% 1%

ICD9 37% 12% 10% 2% 3% 2% 1% 2%

Infobuttons

MDX - - - - - - 100% -

PubMed 86% - - - - - 14% -

webcis log files results19
WebCIS Log Files Results

Context: Lab Rad Note Card Path Vis Phar DSum

MDX 56% 57% 61% 59% 49% 42% 9% 62%

Ovid 7% 7% 8% 5% 8% 4% 1% 6%

Formulary 7% 6% 6% 6% 7% 3% 1% 7%

Harrisons 5% 6% 4% 5% 6% 3% 1% 7%

PubMed 5% 5% 5% 4% 8% 2% 1% 3%

Journals 4% 4% 5% 9% 8% 3% 1% 3%

Med+ 3% 3% 2% 3% 3% 32% 1% 2%

ICD9 1% 4% 4% 2% 3% 2% .2% 3%

Infobuttons

MDX - - - - - - 84% -

PubMed 3% - - - - - 2% -

webcis log files results20
WebCIS Log Files Results

User: Housestaff Attending Nurse Other

MDX 62% 61% 42% 27%

Ovid 6% 7% >1% 1%

Formulary 6% 5% 3% 5%

Harrison 4% 3% 4% 3%

PubMed 4% 5% 1% 1%

Journals 3% 0% 3% 1%

Med+ 2% 2% 3% 2%

ICD9 3% 5% 1% 1%

Statref 1% 1% 1% 3%

PIER 0% 2% 3% 1%

Infobuttons

MDX 5% 6% 42% 53%

PubMed 2% 1% 2% 1%

webcis usage logs results
WebCIS Usage Logs Results
  • Take home messages:
    • Micromedex is a popular resource
    • Ovid edges out PubMed
    • Lab tests are a popular context for resources
    • No correlation between context and resource
    • Nurses and pharmacists like the Micromeex infobutton for microbiology
  • Still needed:
    • Context/resource/user type interactions
    • Concept-oriented contexts
  • Questions:
    • How much is based on knowledge of resources?
    • What are the questions in each context?
webcis user questionnaires
WebCIS User Questionnaires
  • Health resources page outside vs. inside WebCIS
  • Resources used, how often, how useful, anecdotes
  • Infobuttons used, how often, how useful, anecdotes
  • Other resources used
webcis user questionnaire results
WebCIS User Questionnaire Results

1055 users in January

300 randomly selected

250 with valid e-mail addresses surveyed

42 responded

21 attendings

19 housestaff

1 nurse

1 pharmacist

webcis user questionnaire results24
WebCIS User Questionnaire Results

How often do you access:

Never<MonthMonthlyWeeklyDaily

Attend 2 1 3 5 9

House 3 11 5

Nurse 1

Pharm 1

Attend 11 8 2

House 14 3 1 1

Nurse 1

Pharm 1

Resource

Infobut

webcis user questionnaire results25
WebCIS User Questionnaire Results

Why do you access:

Patient QsGenl KnowOther

Attend 20 12 3

House 18 10 1

Nurse 1

Pharm 1 1

Attend 10

House 4

Nurse

Pharm 1

Resource

Infobut

webcis user questionnaire results26
WebCIS User Questionnaire Results

How do you access:

NoneWC-LoginWC-MenuBothOther

Attend 2 4 7 8 12

House 0 8 8 3 17

Nurse 1

Pharm 1 1

Resource

webcis user questionnaire results27
WebCIS User Questionnaire Results

What resources and how useful:

Very Somewhat Not _

AHAHAH

MMDX 17 17 0 2 1 0

Ovid 10 11 1 2 0 0

PubMed 12 7 2 3 0 0

Medline+ 4 4 0 1 1 0

FullText 8 7 2 2 0 0

Harrisons 7 4 3 9 1 0

PIER 2 4 3 1 1 0

Formulary 2 3 2 1 1 0

Health J 6 9 0 1 0 1

Science J 3 4 0 1 1 0

STATref 1 2 6 0 1 1

ICD9 1 2 5 0 0 1

webcis user questionnaire results28
WebCIS User Questionnaire Results

How useful are infobuttons:

Very Somewhat Not _

AHAHAH

Micro Org 3 3 5 1 2 0

Micro Sens 7 3 2 3 0 0

Pharmacy 6 1 0 2 0 0

webcis user questionnaire results29
WebCIS User Questionnaire Results

How does self-reporting correlate with log files?

100

80

60

40

20

0

A

A

A

A

A

H

A

A

A

A

A

H

H

H

A

H

H

H

H

A

A

A

H

H

H

A

H

H

H

H

H

A

H

A

A

H

A

A

H

Never <Monthly Monthly Weekly Daily

Health Resources

N

P

webcis user questionnaire results30
WebCIS User Questionnaire Results

How does self-reporting correlate with log files?

25

20

15

10

5

0

A

H

A

H

A

P

A

H

H

A

H

H

A

A

A

H

H

H

H

H

H

A

H

H

A

A

A

H

A

H

A

N

A

A

A

A

A

H

A

H

A

Never <Monthly Monthly Weekly Daily

Infobuttons

webcis user questionnaire results31
WebCIS User Questionnaire Results
  • Anecdotes:
    • Avoid errors (drug interactions)
    • Identify correct doses
    • Identify pills
    • Prepare publications
    • Teaching
    • Understand abbreviations (sensitivity tests)
  • Other desired resources:
    • Up To Date
    • NEJM
    • MDConsult
    • Books
webcis user questionnaire results32
WebCIS User Questionnaire Results
  • Resources are used and liked
  • Resources are used within WebCIS
  • Resources are used to answer patient-specific questions
  • Use of resources may be overestimated (can’t tell for sure)
  • Infobuttons are used and liked
  • Additional resources requested
  • Questions:
    • How knowledgeable are users about resources?
    • What are the context-specific questions
observational studies
Observational Studies
  • Connect portable usability lab to clinical workstation
  • Ask users to think aloud about information needs
  • Videotape screen and audiotape users
  • Transcribe tapes
  • Code transcripts, using tapes
  • Analysis of coding
slide34

Portable Usability Lab

User’s Workstation

75 foot cable

Video Monitor

Converter Controller

Microphone

Video Converter

Headphones

VCR

Cassette Recorder

observational results
Observational Results
  • Sites:
    • Medical /Surgical
    • CCU
    • Ambulatory Care
  • Number of Days:
    • 7 different days
    • Varying days of the week
    • Varying times of day
  • Total hours of tape: 15 hrs 29 minutes
  • Total Number of events: 154
observational results37
Observational Results

Total Number of events: 154

observational results38
Observational Results

Total Number by User Type

event coding
Event Coding
  • Types of Events:
    • Foreground/Background
    • Implicit/Explicit
    • Patient/Institution/Administrative/Domain
  • Types of Resources
    • Human/Computer/Paper
  • Types of Outcomes
    • Success/Deferred/Failed
coding event type
Coding Event Type
  • Foreground
    • Specific information about patient management
    • Problem, Intervention, Comparison, Outcome
  • Background
    • Root question: who, what, where,when, why, how?
    • Related to health disorder or aspect of a disorder
coding event type42
Coding Event Type
  • Explicit: Subject expresses an information need
  • Implicit: Subject uses a resource other than WebCIS
coding event type44
Coding Event Type
  • Patient
    • A patient related need
  • Institution
    • An information need that is related to the infrastructure of the institution
  • Administrative
    • An institutional information need that is related to a specific patient
  • Domain
    • A domain related information need that is related to a specific patient
coding resources used
Coding Resources Used
  • Human
    • The subject sought information from a human interaction in person or via telephone
  • Computer
    • The subject sought information from a computer interaction either webCIS or other internet resource
  • Paper
    • The subject sought information from a paper resource – patient chart or text book
coding outcomes
Coding Outcomes
  • Success
    • The subject sought and found the desired information
  • Deferred
    • The subject did not seek an answer
  • Failed
    • The subject sought, but did not find desired information
coding events
Coding Events
  • Need to correlate across event types
  • Especially need to look at types of needs that were deferred or that failed
  • Good news:
    • Information needs exist and can be identified
    • Needed resources are computer-based
  • Bad news (good news for us):
    • Many needs failed or deferred
    • Solutions will require integration of patient data with information resources
  • Questions:
    • What are the triggers?
    • What resources can help?
methods classification scheme
Methods: Classification Scheme

General question: What is the normal value for this calcium result?

Generic question: What is the normal value of this laboratory test?

Classification: Navigation: because normal laboratory values are available in WebCIS, this IB link would take the user to another page informing the users on how to access the information needed.

methods classification scheme53
Methods: Classification Scheme

General question: Since the CT scan of the liver is abnormal what are his liver function test results?

Generic question: What laboratory tests are associated with this radiology report?

Classification: Functionality group since it will require designing an IB that would bring information to the user from another area in WebCIS (or some other resource) while he is at a particular site.

slide54

Observational Studies Results

Classification Code

Percent

Count

Navigation

48

31.2

Functionality

27

17.5

Laboratory

19

12.3

Pharmacy

31

20.1

Differential

9

5.8

Diag.

Definition/info.

15

9.7

Miscellany

5

3.2

Total

154

100.0

observational studies results
Observational Studies Results
  • Wide variety of information needs
  • Many needs are stereotypical
  • Many can be answered with simple resources
  • Some complex solutions will be needed
  • Next steps:
    • Finish the characterizations of needs
    • Propose ways to solve each need
    • Build infobuttons (project opportunities!)
the infobutton manager
The Infobutton Manager
  • CGI script using “get” or “post”
  • Context parameters as input
  • Uses context table and infobutton table
  • Returns a list of questions that are links
  • Log files for calls to IM and selecting links
  • Infobutton Manager Manager for maintenance
matching context to need
Matching Context to Need

Infobutton Manager

I

n

f

o

b

u

t

t

o

n

s

Context

Matching

Context Table

Question

Selection

Question Table

Instantiation

C

o

n

t

e

x

t

context parameters
Context Parameters
  • Patient: age, gender
  • User: physician, nurse, student, patient, …
  • Setting: lab reports, medications, …
  • Concept of interest: lab test, drug, …
infobutton parameters
Infobutton Parameters
  • Question type: fixed, fill-in-the-blank
  • Source type: simple query, additional data
  • URL:

http://www.guideline.gov/SEARCH/search_results.asp? sSearch_string=<>&bSort_by_date=&query_id=& bDebug=&results=10&view=search_results

  • Question:

What are guidelines for evaluation of <>?

infobutton manager manager demo
Infobutton Manager Manager Demo

Insert Infobutton Manager Demo Here

Insert Infobutton Manager Manager Demo Here

evaluation plan
Evaluation Plan
  • Baseline observation
    • Creation of Infobutton Manager tables
    • Resources used
    • Unmet information needs
  • Log of access to online resources
  • Introduction of infobuttons
    • Usability studies
  • Log of access to online resources (infobuttons)
  • Post-intervention observation
    • Use of infobuttons and other resources
    • Unmet information needs
discussion
Discussion
  • Information needs occur and are detectable
  • Information needs are unmet
  • Information needs are predictable based on context
  • Solutions are at hand
  • Solutions are institution independent
  • Integration into clinical systems is easy
  • Will infobuttons reduce unmet information needs?
  • Next project: show that this reduces errors!
acknowledgments
Acknowledgments
  • National Library of Medicine grant R01-LM07593
  • Rick Gallagher
  • The team: Jianhua Li, Sue Bakken, Vimla Patel, Mark Graham, Leanne Currie, Mureen Allen, Anne-Marie Ramierez
slide66

<a href="Javascript:get_info('35702')"><img src="info.gif"></a>

function get_info(mc){ var infowin = window.open("","INFO");

infowin.focus(); document.INFO.info_med.value=mc;

document.INFO.submit(); }

<form method="POST" name=“INFO”

action="wc_infomanage.cgi">

<input type="HIDDEN" name="info_med" value="">

<input type="HIDDEN" name="info_context" value="LabDetail">

<input type="HIDDEN" name="info_institute" value="CPMC">

<input type="HIDDEN" name="DOB" value="1951-05-26">

<input type="HIDDEN" name="SEX" value="F">

<input type="HIDDEN" name="USER" value="ciminoj">

</form>

context parameters70
Context Parameters
  • Patient: age, gender
  • User: physician, nurse, student, patient, …
  • Setting: lab reports, medications, …
  • Concept of interest: lab test, drug, …
  • Institution: CPMC, RMRS, LDS, …
slide71

<form method="POST" action="wc_infomanage.cgi">

<input type="HIDDEN" name="info_med" value=“35702">

<input type="HIDDEN" name="info_context" value="LabDetail">

<input type="HIDDEN" name="info_institute" value="CPMC">

<input type="HIDDEN" name="DOB" value="1951-05-26">

<input type="HIDDEN" name="SEX" value="F">

<input type="HIDDEN" name="USER" value="ciminoj">

</form>

slide72

<form method="POST" action="wc_infomanage.cgi">

<input type="HIDDEN" name="info_med" value=“DIGOXIN|..">

<input type="HIDDEN" name="info_context" value="LabDetail">

<input type="HIDDEN" name="info_institute" value=“RMRS">

<input type="HIDDEN" name="DOB" value="1951-05-26">

<input type="HIDDEN" name="SEX" value="F">

<input type="HIDDEN" name="USER" value="ciminoj">

</form>