Measuring technical progress and potential impact in real time
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

Measuring Technical Progress and Potential Impact in Real Time PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on
  • Presentation posted in: General

Measuring Technical Progress and Potential Impact in Real Time. Presented at the American Evaluation Association Conference 2009 by Jerald Hage, Wilbur Hadden Center for Innovation University of Maryland Gretchen Jordan Sandia National Laboratories.

Download Presentation

Measuring Technical Progress and Potential Impact in Real Time

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Measuring technical progress and potential impact in real time

Measuring Technical Progress and Potential Impact in Real Time

Presented at the

American Evaluation Association Conference 2009

by

Jerald Hage, Wilbur Hadden

Center for Innovation

University of Maryland

Gretchen Jordan

Sandia National Laboratories


Measuring technical progress and potential impact in real time1

Measuring Technical Progress and Potential Impact in Real Time

Why be concerned about this issue?

  • Papers and patents occur to late for effective interventions for research management

  • Metrics provide opportunities for expanding the number of social and economic benefits

  • Without better and more timely measures, there will be a considerable backlash against the research stimulus package

  • Need to shift focus from productivity to innovation


Measuring technical progress and potential impact in real time2

Measuring Technical Progress and Potential Impact in Real Time

Current studies :

  • NSF study of 6 morbidities at two Institutes of NIH (200 projects) (focus of this talk)

  • NSF study of 5 research areas in 6 national laboratories N = 72 projects

    Past studies:

  • Scientific and technological progress at Sandia National Laboratories (22 projects)

  • New products at STAR at NOAA (5 projects)


Review of current cost benefit studies of health research

Review of current cost-benefit studies of health research

  • Too much emphasis on mortality rates

  • Too much emphasis on economic returns

  • Too much emphasis on long time periods

  • Little consideration of the deconstruction of the streams of research


Correcting the problems

Correcting the Problems

  • View health care as a process:

    - prevention

    - diagnosis

    - treatment

    - post-treatment

  • Develop various social measures (e.g. reduction in home care, improved quality of life, etc.)

  • Focus on specific research projects as completed

  • Isolate core ideas in streams of research


A framework for understanding the impacts of health research

A Framework for Understanding the Impacts of Health Research

Knowledge Pool

Global Research

Health

Industry

Healthcare

Appropriateness,

Access, etc.

Prevention, Diagnosis, Treatment, and Post Treatment

(Population) Health Status

and Function

Interactions/Feedback

  • Health Research

  • Biomedical

  • Clinical

  • Health services

  • Population and Public health

  • Cross cutting research

Other

Industries

Government

Well Being

And

Economic Prosperity

Research Agenda

Determinants of health (social, environmental, etc.)

Public Information, Groups

Research Capacity

Our focus for measures of potential impact

Modified from CAHS Report on ROI for Health research

available at http://www.cahs-acss.ca/e/publications/

6

J. Hage October 2009


Measuring health outcomes as percent change in

Measuring Health Outcomesas(percent change in)

Prevention

  • effectiveness of prevention intervention (decline in incidence of morbidity)

  • severity of incidence of morbidity

    Intake and Assessments

  • the speed of diagnosis (reduction in the number of tests and their duration)

  • the accuracy of diagnosis (reduction in false positives or negatives)

  • accuracy of prognosis (duration and quality of life, etc.)


Measuring social economic benefits

Measuring Social/Economic Benefits

Prevention

  • Value of illness days saved from decline in morbidity incidence

  • Value of reduction in cost of treatments for less severe morbidity incidence

    Intake and Assessments

  • Reduction in the costs of tests for diagnosis

  • Reduction in costs of false positives or negatives

  • Reduction in the costs of futile interventions


Measuring social economic benefits example one

Measuring Social/Economic Benefits: Example One

In a clinical trial of a new staining technique for earlier diagnosis of metastasis, the treatment group had their sentinel lymph node removed.

The treatment group had a much better survival rate after four years and with fewer new tumors

Using the treatment groups experience, one takes the number of new patients in the next year and computes the percentage with metastasis that would be found.


Measuring social economic benefits example one1

Measuring Social/Economic Benefits: Example One

Example: diagnostic technique in melanoma

Formula for social value:

new patients X % metastasis X% mortality = Life years*

53,120 15.8% 22.5%1,888

* only in fourth year

Formula for economic value:

Life years X Value of one year =Economic value

  • $45,000$85 million

    Does not include the value of reduction in the number of tumors


Measuring social economic benefits example one2

Measuring Social/Economic Benefits: Example One

Assumptions:

  • The percentage of sentinel nodes where metastasis has occurred in the study is the same as in the general population.

  • No confounding differences between the populations of Australia (the majority of the patients), the U.S. or Europe.

    Reference: Morton et al. September 28, 2006 “Sentinel-Note Biopsy or Nodal Observation in Melanoma” New England Journal of Medicine, 355, 13: 1307-1317


Measuring social economic benefits example two

Measuring Social/Economic Benefits: Example Two

In a clinical trial of new patients with Alheizmer’s Disease those with depression were given anti-depressant drug

The treatment group had a slower on set of Alheizmer’s disease by on average about three months

Using the treatment groups’ experience, one takes the number of new patients in the next year and computes the savings in home care


Measuring social economic benefits example two1

Measuring Social/Economic Benefits: Example Two

Metric for Prevention: Alheizmer’s Disease

Formula for social value:

new patients X % depression X Reduction in hours

454,000 28% 520 per case

Or 2,754,237 days of home care

Formula for economic value:

Value per Value of

Hour Total hours

$10$66 million

Does not include the value of reduced stress


Measuring social economic benefits example two2

Measuring Social/Economic Benefits: Example Two

Assumptions:

  • The percentage of depression among new patients is the same as existed in the study, which to us seems to be an underestimation.

  • For new patients only home care is used, which again is a conservative estimate.

  • Only 40 hours per week of home care for each of the three months or 13 weeks because the new patients do not need constant watching.

  • An average wage of only $10.

  • The assumption of an improvement of three months is based on the conservative estimate of the findings from five drugs that have been approved by the FDA for Alzheimer’s which reduced symptoms for six to 12 months for about 50 percent of the population, i.e. 50 % of six months or three months.

    Reference: Lu et al., June 16, 2009 “Donepezil delays progression of AD in MCI subjects with depressive symptoms” Neurology, 72, 24, 2115-21


Measuring technical progress in real time

Measuring Technical Progress in Real Time

Next Steps:

  • Apply all health outcome and social/economic benefit metrics in six morbidity areas

  • Determine what proportion of the funding was completed by NIH

  • Determine the stream of development of ideas that led to the outcome


Measuring technical progress in real time1

Measuring Technical Progress in Real Time

Please contact us if you have questions, suggestions, or opportunities to collaborate.

  • Jerald Hage: Email [email protected], phone 301-405-6437.

  • Wilbur Hadden: Email [email protected], phone 310-405-9746

  • Gretchen Jordan: Email [email protected], phone 505-844-9075


  • Login