themes that cut across disciplines quality control and research methods n.
Download
Skip this Video
Download Presentation
Themes that Cut Across Disciplines: Quality Control and Research Methods

Loading in 2 Seconds...

play fullscreen
1 / 38

Themes that Cut Across Disciplines: Quality Control and Research Methods - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

Themes that Cut Across Disciplines: Quality Control and Research Methods. Faina Linkov, PhD Research Assistant Professor University of Pittsburgh Cancer Institute. The journey of quality control in the Supercourse The journey of a thousand miles starts with a single step. Overview.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Themes that Cut Across Disciplines: Quality Control and Research Methods' - najila


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
themes that cut across disciplines quality control and research methods

Themes that Cut Across Disciplines: Quality Control and Research Methods

Faina Linkov, PhD

Research Assistant Professor

University of Pittsburgh Cancer Institute

The journey of quality control in the Supercourse

The journey of a thousand miles starts with a single step

overview
Overview
  • Overview the history of quality control and peer review
  • Quality control on Amazon.com
  • Quality control in the Supercourse: QC on PPT lectures is at the core of what we do
  • Supercourse studies of quality control
  • Our conclusions and future directions
what do scientists have in common
What do scientists have in common?
  • Quest for discovery
  • Common research methods, such as statistics
  • Journal article format
  • Quality control mechanisms
  • Peer review: is it the same as quality?
history of quality early times
History of Quality: Early times
  • 384-322 BCE-Aristotle and the birth of peer review
  • 854-931 “Ethics of the Physician” by Ishap bin Ali Al Rahwi-first documented description of peer review in Medicine
royal society of london
Royal Society of London

Philosophical Transactions- Originally edited by Henry Oldenburg

First peer reviewed journal

1665

quality control criteria of the past
Quality control criteria of the past

Albert Einstein, 1905

His famous work, Annus Mirabilis, was not peer reviewed!

history of quality continued
History of quality continued

19th century – birth of quality control discipline

1950 - W. Edwards Deming and statistical quality control

1970’s- Total Quality Management (TQM)

what is quality
What is Quality

“ Doing the right thing right, right away”

W. Edwards Deming, 1982

“a measure of goodness that relates to the intended use of a product and the expectations customers have concerning this product”

Barkman, 1989

traditional quality control of scientific materials peer review
Traditional Quality control of Scientific Materials: Peer Review

“A peer-reviewed journal is one that has submitted most of its published articles for review by experts who are not part of the editorial staff”

Vancouver Group of Editors

is peer review the same as quality
Is peer review the same as quality?

South Korean cloning scandal, alongside with other examples, demonstrate that peer review is not perfect

the state of qc for powerpoint on the web
The State of QC for PowerPoint on the Web
  • Over 7,600,000 PowerPoint files on the Internet give us unique new ways to do QC
  • Only 3383 lectures tested for QC-Supercourse
  • We published almost all articles concerning the state QC of PowerPoint on the web
powerpoint lectures
PowerPoint lectures
  • No information on quality control for PowerPoint lectures on-line
  • There is a growing need to establish quality control for Scientific PPT lectures on Line
  • Migration of 7 million lectures to the web by 2008
  • Establishment of “Quality Circles” in the Supercourse: John Last, Vint Cerf, and many others
progress report in the supercourse
Progress Report in the Supercourse

Over 14000 lecture reviews were obtained in the last 6 years

Lecture review forms utilize Likert like scales, regularly used for the evaluation of educational programs and amazon, ebay, consumers reports..

space for written comments
Space for written comments

Some of the URLs have changed since the lecture was developed.

It would be helpful if that was corrected, so you could actually see the information

referred to.

Is there any text(s) that you could recommend for epidemiology, i.e., a basic text, an intermediate text, and an advanced text?

my dissertation 2005
My dissertation, 2005

First evaluation of peer review mechanisms for PPT lectures on-line

results and conclusions of this research
Results and conclusions of this research
  • Lectures were rated highly by experts and non-experts
  • People are primarily rating at the upper tail of the distribution
  • There is a poor agreement among reviewers (very low Kappa statistic)
  • The role of personal background in the quality of the peer review is not clear and needs to be further investigated
  • There is a pressing need for scientific evaluation of PPT lectures on the web
implications for the supercourse of science
Implications for the Supercourse of Science?
  • There is no consistency about the use of peer review on the web
  • Utility of peer review strategies for PPT lectures is not clear
  • New approaches will be developed for the Supercourse of Science.
  • Shall we allow disciplines to decide the QC techniques which are most applicable for their fields?
  • There is a need to build a science of QC
where do we go from here for the development of quality control mechanisms in the supercourse
Where do we go from here for the development of quality control mechanisms in the Supercourse?

Continuous quality improvement (CQI): not a destination, but a journey

supercourse aim
Supercourse aim:

To scientificallydevelop a system of QC on the web by making multiple measures available

Power of the Internet makes the development of the Science of QC possible.

moving beyond peer review
Moving beyond peer review

“… We had great difficulty in finding any real hard evidence of the system’s effectiveness, which is disappointing, as peer review is the cornerstone of editorial policies worldwide”

Tom Jeffersen

suggested quality control mechanisms for the supercourse of science multiple measures
Suggested Quality Control Mechanisms for the Supercourse of Science: Multiple Measures

a) Expert editorial board

b)Screening to identify non-appropriate lectures, as done in nature and BMJ

c) Five star system(or Priority Score) similar to that in Amazon.com

d) Opinion of experts (professors)

e) Personal Characteristics of authors (rank, university, citations)

f) Web statistics for utilization of lectures: hits, links, Page Rankings, etc.

g) Key note speeches

h) Publications and citations from Google Scholar

i) Model similar to NIH style review

editorial board in formation
Editorial Board (in formation)
  • John Last, MD
    • father of modern environmental epidemiology
  • Peter Greenwald, MD, PhD

- leading cancer control specialist, NCI

  • Paul Zimmet, MD

-leading diabetes research in Australia

  • Dr. Steven N. Blair

-leader in exercise science

Plus 8 others

screening to eliminate poor quality lecture
Screening to eliminate poor quality lecture

Key criteria for lecture rejection in the parent Supercourse project

  • Not related to prevention
  • Inaccurate
  • Out of date
  • Incomplete
amazon com like system
Amazon.com like system

42 Reviews

Customer Reviews

<Book title>

mean lecture scores in the supercourse
Mean Lecture Scores in the Supercourse

First 2000 surveys

Mean Score: 4.12

personal characteristics of authors rank university citations
Personal Characteristics of Authors(rank, university, citations)

1288 authors in the Supercourse

  • 30 authors from Harvard
  • 8 authors from Johns Hopkins
  • 100 authors from Pitt
  • 7 authors from Columbia
keynote speeches
Keynote speeches

Someone chosen for keynote address is expected to give a good lecture

nih model and continuous quality monitoring
NIH Model and Continuous Quality Monitoring

Priority Scores

Submission

Triage

Resubmission

Supercourse Model: Continuous Quality Control

google model for qc opportunities for exploration
Google model for QC: Opportunities for exploration?

Do not exclude anything, but give it a rank

Development of search mechanisms where you can find anything you like

why do this research
Why do this research?

“I believe this lecture provides simple, understandable, thorough and consolidated information of the subject. This is actual service to the humanity”

From the evaluator of the

Supercourse lectures

conclusions
Conclusions
  • There is a need to define the science of QC for online lectures
  • Scientists need to be able to choose what they want in terms of QC mechanisms, including peer review and other systems
  • Science needs to replicate success of industry in the area of QC
  • Traditional approaches (such as peer review) may not be bad, but with Internet technologies much more can be done
vision for the supercourse of science
Vision for the Supercourse of Science
  • Development of novel QC systems for the Supercourse of science.
  • Need to systematically and continuously evaluate QC systems
  • Upfront QC systems and information collection
thank you
Thank you!

Please e-mail your ideas to super1@pitt.edu