1 / 30

Rethinking Research Evaluation in the Open Science Era

Explore the paradigm shift towards Open Science and the need for comprehensive revisions in evaluating researchers. Discover the challenges, alternatives, and the importance of assessing research on multiple criteria for a more inclusive and impactful academic landscape.

kneil
Download Presentation

Rethinking Research Evaluation in the Open Science Era

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Open Science will never prevail without a thorough revisiting of the way evaluations of researchers are conducted Bernard Rentier OAI11 - CERN-UNIGE Workshop on Innovations in Scholarly Communication Geneva, June 20, 2019

  2. Open Access Open Publishing Open Science: a paradigm shift OSC Open Scholarly Communication Open flow: free to publish & free to read

  3. -Gilles Allain « Saisissez une citation ici. »

  4. -Gilles Allain « Saisissez une citation ici. »

  5. -Gilles Allain « Saisissez une citation ici. »

  6. -Gilles Allain « Saisissez une citation ici. »

  7. Accepted for publication after anonymous peer review in International Journal of Advanced Computer Technology (2014)

  8. Open Science: a paradigm shift • Open Scholarly Communication • Open citations • Open Research Data • Open Source Software • Citizen Science • Open Education

  9. Why do we assess ? • Resources are scarce (money, time) • Selection is permanently needed • Researchers are accountable for public funds

  10. How do we assess ? • Today mostly on quick proxies (Journal Impact Factors,…), not enough on content and advancement of knowledge • Impact of research is a good indicator but it should be clear which impact, on what and how to measure it

  11. Why should we change ? • Growing distrust and frustration • Quantitative assessment generates competition, not sharing • Quantitative metrics induce overpublication • Current assessment practices reduce researchers’ diversity

  12. Why should we change ? • Landscape is changing towards Open Science, based on exchange and sharing • Assessment should take the OS principles into account.

  13. Are there alternatives ? • DORA, Leyden Manifesto • Several theoretical paths • Concrete initiatives

  14. Yes indeed, but... • Tells us what we sould’t do any longer • Not what we should do

  15. Assessment must be based on MULTIPLE CRITERIA

  16. Research output • Research activity • Publications • Datasets • Open source • Funding • Research Process • Stakeholder engagement/citizen science • Collaboration & interdisciplinarité • Research integrity • Risk management • Service & Leadership • Leadership • Academic standing • Peer review • Networking • Research Impact • Communication & dissemination • IP (patents, licenses) • Societal impact • Knowledge exchange • Teaching and supervision • Teaching • Mentoring • Supervision • Professional Experience • Continuing professional development • Project management • Personal qualities

  17. « MATRIX, NOT METRICS »

  18. OS-CAM, the Career Assessment Matrix

  19. OS-CAC, the Career Assessment Cube

  20. Sharing experience • In universities • Role for libraries, in synergy with researchers, monitoring impact (macro & micro)

  21. Public authorities, funders, OA publishers, researchers: same combat • Awareness of OS benefits and collateral damage • Consensus on where we are, not yet on how to get where... • Compliance of funders to O.S. principles is increasing but not fully enforced

  22. Take-home message It will be impossible to implement Open Science harmoniously without a large, significant and determined consensus on new ways to evaluate research and researchers.

  23. E-book freely available at: https://academie-editions.be/

  24. Thank you ! https://academie-editions.be/

More Related