1 / 15

Unreliable research

Unreliable research. Trouble at the lab. Group Members:. Patrik Ahlberg Magnus Åhberg Syeda Rabab Naqvi Thawatachart Chulapakorn. Outline:. Introduction. Irreproducibility is on rampage Self corrections are not working IRL “There is no cost to getting things wrong,

obelia
Download Presentation

Unreliable research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unreliable research Trouble at the lab

  2. Group Members: Patrik Ahlberg Magnus Åhberg Syeda Rabab Naqvi Thawatachart Chulapakorn

  3. Outline:

  4. Introduction • Irreproducibility is on rampage • Self corrections are not working IRL “There is no cost to getting things wrong, The cost is not getting them published.” Brian Nosek, University of Virginia

  5. Understanding Insignificance • False Positives + False Negatives = Problem • We are not using statistics • Not possible with power of 0.8

  6. Not evenwrong • Research is commonly not thoughtthrough or not wellexecuted • Examples: • The ”pentaquark” saga (experiment not properlyblinded) • Risks with computer model ”tuning” • Longevityassociatedgenetic variations duetoinproper handling of research samples

  7. Blame the ref • Peer reviewersare not detectingerrors • John Bohannon´smade-up paper waspublished in 157 outof 304 journals • Fiona Godleeyieldedsimilarresultswhensending an 8-error paper to the 200 reviewersof BMJ • Errorsareexplained by incompetenceratherthanfraud • The replicationmechanism for scientificself-correction is not functioningwell

  8. Harder to Clone than you wish : To find errors in the publications , we need to replicate the work of others, but process is not according to our wish. Replication is hard to meet our standard Replication Lack of interest by Journals and Academic Researchers

  9. Lack of Interest • Journals thirsty of novelty, show little interest in it. Replication is Hard • Academic researchers mostly spend time on work which is more likely to enhance their careers. • Original methods and data required • Unpublished Research • Clinical trials are very costly to rerun • Software may be different during the replication • Failure of replication due to tacit knowledge • Expermenter’s regress

  10. Making the paymasters care Quality is more important than Quantity Finally, we would say that…

  11. Patriks work • Unethical publishing cancels this problem • No reproducibility possible • Small feasibility tests • The system has given up on the system • Seniors warning juniors • Reliability comes from people not publications

  12. Ownreflections • The possibilityto ”tune”computer modelsuntil the desiredresultsappearrelatesto my own research • It is easytopercievepatterns and get resultsthatareexpected and might not evenexist • Responsibility as a reviewer: • When do I accept a reviewer invitation? • Do I expecttofinderrors? Whatam I looking for as a reviewer?

  13. Own Reflections If some one tries to repeat my experiment, he must have • Access to original methods and data • Awareness of simulation procedure • Similar samples for study

  14. Related to my (future) research • Replication • Same or not? Why? • Scrutiny • Procedure • Measurement • Bias discussion • Raw data • Log book!!!

More Related