Competências Básicas de Investigação Científica e de Publicação

1. CompetênciasBásicas de InvestigaçãoCientífica e de Publicação Lecture 8: Global science and the need for metrics Ganesha Associates 2013

2. Brasil’s economic growth creates a demand for scientists and technologists Ganesha Associates 2013

3. Science drives the economy • “The ability to create, distribute and exploit scientific and technological knowledge has become a major source of competitive advantage, wealth creation and improvements in the quality of life. • Some of the main features of this transformation are; • the growing impact of information and communications technologies (ICT) on the economy and on society • the rapid application of recent scientific advances in new products and processes • a high rate of innovation across economically developed countries • a shift to more knowledge-intensive industries and services • rising skill requirements.” • Source: Organisation for Economic Co-operation and Development (OECD)Policy document 2000 Ganesha Associates 2013

4. Ganesha Associates 2013

5. Global research investment Ganesha Associates

6. Ganesha Associates 2013

7. Ganesha Associates 2013

8. Document output in 2011 Ganesha Associates 2013

9. Brasil: Ganesha Associates 2013

10. BRIC output: documents Ganesha Associates 2013 Source: International Comparative Activity Performance – Elsevier 2011

11. Citation quality is a problem Ganesha Associates 2013

12. International collaboration is low Ganesha Associates 2013

13. Share of articles by co-authorship type Ganesha Associates 2013

14. EPI : English Proficiency Index by age

15. Ganesha Associates 2013

16. Institutional ranking Ganesha Associates 2013

17. Brasil Qualis and the importance of Indexação • O Qualisé um levantamentorealizadopela Capes com o intuito de mensurar a qualidade da produçãocientífica dos programas de pós-graduação. • A classificação de periódicosérealizadapelasáreas de avaliação e passapor um processoanual e atualização. Essesveículossãoenquadradosemestratosindicativos de qualidade; A1, o maiselevado; A2; B1; B2; B3; B4; B5; C, com peso zero. • O seuprograma de pós-graduaçãoseráavaliadosegundoessaescala. Ganesha Associates 2013

18. UK’s Research Excellence Framework 2014 • Outputs: The panels will assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standards. This element will carry a weighting of 65 per cent. • Impact: The sub-panels will assess the ‘reach and significance’ of impacts on the economy, society and/or culture that were underpinned by excellent research conducted in the submitted unit, as well as the submitted unit’s approach to enabling impact from its research. This element will carry a weighting of 20 per cent. • Environment: The sub-panels will assess the research environment in terms of its ‘vitality and sustainability’, including its contribution to the vitality and sustainability of the wider discipline or research base. This element will carry a weighting of 15 per cent.

19. Global metrics summary • Science and technology are too politically important for them to be left un-managed • The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form • Metrics will become more important as better standards are defined

20. Break

21. Portfolio analysis at NIH • What is the state of the science? • Are there gaps in our research portfolio? • How do the gaps compare to research carried out in other agencies? • How much do we spend on mitochondrial research? • What is the average number of scientific papers generated per $100K spent per grant?

22. Define: indexação • A indexação de uma revista é o processo que confere o indicador de qualidade necessário para que esta possa pertencer a um ou mais bancos de dados de renome internacional, em uma área específica do conhecimento, como por exemplo, MEDLINE, ISI, LILACS, etc., segundo critérios preestabelecidos por essas instituições. • É um fator importante para que a produção científica nela apresentada se torne conhecida e reconhecida internacionalmente. • O periódico necessita, para isto, estar de acordo com os padrões básicos de apresentação formal e excelência de conteúdo científico, determinados internacionalmente. Ganesha Associates

23. Ganesha Associates

24. Ganesha Associates

25. As fontesde‘indexação’emordem de ‘qualidade’ • Thomson Web of Science • PubMed/Medline • Scopus • ScieLo • Google • Scirus • LILACS Ganesha Associates

26. Journal selection process - ISI • To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI seeks to cover the best regional journals as well. • However, rather than compare a regional journal with all journals in a particular subject category, the ISI editor considers it in terms of those journals in the category from the same geographic area. • High journal publishing standards, especially timeliness, and English language bibliographic elements remain essential. Ganesha Associates

27. Impact Factor defined Impact Factor 2011 = # citations received in 2011 to articles published in 2010/2009 ______________________________________________________ # number of source items published 2010 and 2009 “The Impact Factor of a journal is probably the only quantitative way of assessing its worth and relevance to the academic community it serves.” Editorial, Journal of Computer-assisted Learning, 2001 Ganesha Associates

28. Journal of Cell Science - IF

29. Nature citations per article

30. Citation statistics – PLoS Biology Ganesha Associates

31. Ganesha Associates

32. Impact Factors over time Ganesha Associates

33. Impact Factor and area Ganesha Associates

34. Impact Factors for Microbiology Ganesha Associates

35. Impact Factors for Neuroscience Ganesha Associates

36. Impact Factors and article type Ganesha Associates

37. Impact Factors and Authors Correlation between article citation rate and journal impact for four authors Ganesha Associates

38. Impact Factors and Journals Citation rates in 1986 or 1987 of articles published in three biochemical journals in 1983 or 1984, respectively. From Seglen, BMJ 1997;314:497 Ganesha Associates

39. Alternatives ? • The Hirsch Index is rapidly gaining favor as a performance measurement tool for scientific authors. • A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np - h) papers have no more than h citations • Thus, the h-index reflects both the number of publications and the number of citations per publication. The index is designed to improve upon simpler measures such as the total number of citations or publications. • Like the IF, the index works properly only for comparing scientists working in the same field. • For more information see Scopus or Wikipedia Ganesha Associates

40. H-index - weaknesses • The h-index is bounded by the total number of publications. • This means that scientists with a short career are at an inherent disadvantage, regardless of the importance of their discoveries. • For example, had Albert Einstein died in early 1906, his h-index would be stuck at 4 or 5, despite his being widely acknowledged as one of the most important physicists, even considering only his publications to that date. Ganesha Associates

41. Declaration onResearchAssessment • Made during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco, CA, on December 16, 2012. • Main recommendation: Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions. • Alternatives?

42. New Usage Metrics Impact Factor Well-established, easily understood and accepted Endorsed by funding agencies and researchers Several competitors: H-index, Eigenfactor, SJR (Google Page Rank) COUNTER Usage Factor Journal/book usage-based alternative perspective Reflects value of journals to all categories of user Easy to understand PIRUS (Publisher and Institutional Repository Usage Statistics) COUNTER-compliant usage reports at the article level Will enable the usage of individual research outputs to be measured ORCID (Open Researcher and Contributor ID) Central registry of identifiers from individual researchers Will enable aggregation of multiple forms of research output

43. FundRef

44. Altmetrics, an alternative? Ganesha Associates

45. Many metrics, many uses… • Development of national-level social contracts for science and technological investment • Does investment lead to the achievement of national/international objectives for increased employment, economic growth, improved healthcare? • At a national/institutional/departmental/individual level how do we compare with the competition? • Management of human resources and tacit intellectual property in a multinational organisation • Do I get tenure/promotion? • How often has my article been downloaded/cited • How well has my paper contributed to building the knowledge network in my field? • Which journal should I publish in? • Which articles should I read? • Has my article been picked-up by the media? • Standards: CrossRef, ORCID, DataCite, FundRef Ganesha Associates

46. Snowball metrics

50. Global metrics summary • Science and technology are too politically important for them to be left un-managed • The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form • Metrics will become more important as better standards are defined