Ecological Data Sharing:

1. Ecological Data Sharing: Current practice and lessons for “scaling up” Ann Zimmerman LTER Ecoinfomatics Workshop October 30, 2003

2. M.A. Library and Information Science, University of Iowa, 1986 Ph.D., Information and Library Studies, University of Michigan, 2003

3. Librarian, 1991-Nov. 2003 U.S. Geological Survey Great Lakes Science Center Ann Arbor, MI Librarian, 1987-1991 U.S. Fish and Wildlife Service Northern Prairie Wildlife Research Center Jamestown, ND

4. Data sharing is necessary in order to address many environmental problems. Airborne pollutants from one country affect the health of another nation’s water supply. The destruction of rainforests influences weather patterns in other parts of the world.

5. We must have your data to save the planet! No one could use my data! They wouldn’t understand them!

6. Why ecological data? • Ecologists work at small spatial and temporal scales • Data sets are small and highly diverse • Standard methods are difficult to achieve • Ecology is a craft science • There is a high level of data ownership

7. Intriguing Questions • Why are some data easier/harder to share than others? • Do standards really facilitate data sharing? If so, when? If so, how? • How do secondary users judge data quality?

8. Answers are relevant to… • Design of data resources • Standards development • Policy • Education

9. Existing Research • The affect of databases on the practice and communication of science • Scientists’ attitudes toward data sharing • Research-related information that scientists share • Expected returns for sharing • Data withholding

10. RQ: What are the experiences of ecologists who use shared data? • How do ecologists locate data and assess their quality? • What are the characteristics of the data they receive? • What information do ecologists depend on to use the data? • What challenges do they face throughout the process?

11. Qualitative Research Methods • Effective when important variables are unclear and empirical information is scarce • Useful for understanding processes as well as outcomes • Interviewing is a useful method to study past events and when participants cannot be observed

12. Qualitative Research Limitations • Imprecise measurement • Vulnerability to bias • Weak generalizability of findings

13. Key Definitions Data • Scientific data Scientific or technical measurements…and observations or facts that can be represented by numbers…and that can be used as a basis for reasoning or further calculation (NRC, 1997).

14. Ecologists? • “I’ve never seen so many khaki pants in my life.” “…and beards, this must be the ESA crowd.” Rahel, Frank J. 1983. The habitus of ecologists: Fear and clothing at Penn State. Bulletin of the Ecological Society of America 64(3): 219-220. ?

15. Ecologists • Members of ESA, or • Self-identification, or • Affiliation or title contains ecolog*

16. Data Sharing • The voluntary provision of information from one individual or institution to another for purposes of legitimate scientific research (Boruch, 1985) • My study is limited to shared data used for ecological research.

17. Secondary Use of Data • The use of data collected for one purpose to study a new problem • Includes data gathered to address a specific research question & data used to describe biological or physical phenomena

18. Data Collection • Method: Semi-structured, in-depth interviews • Primary subjects: 13 ecologists who reused data (selected from 2 key ecological journals) • Secondary subjects: 4 data managers

19. Data Analysis • Primary data: Interview transcripts • Developed a coding scheme and analyzed data following suggestions from Miles & Huberman* *Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook, 2nd ed. Thousand Oaks, CA: Sage Publications. .

20. Reliability • Reliability as consistency in judgment *detailed descriptions of selection of subjects, data collection, & data analysis *reporting of researcher biases, values, & central assumptions

21. Validity • Validity: accuracy of the information and whether it matches reality • Data triangulation: use of diverse sources to study the same phenomenon • Check results with subjects

22. Conceptual Framework Overcoming Distance

23. Overcoming D i s t a n c e Standards Social Exchange Informal Knowledge Potential Distances: Cultural, Epistemological, Methodological, or Terminological Temporal or Spatial Personal

24. Standards as Distance Spanners:Making Local Knowledge Public Measurement as a social technology (Porter) *Quantification as a technology of distance *Standards as a substitute for trust based on personal knowledge Porter, T. M. (1999). Quantification and the accounting ideal in science. In M. Biagioli (Ed.), The science studies reader (pp. 394-406). New York: Routledge. Porter, T. M.(1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton, NJ: Princeton University Press.

25. Standards Reduce & Amplify • Standard measurements involve a loss of information (reduction). • Reduction turns local knowledge into public knowledge (amplification). Latour, B. (1999). Circulating reference: Sampling the soil in the Amazon forest. In Pandora’s hope: Essays on the reality of science studies (pp. 24-79). Cambridge, MA: Harvard University Press.

26. Circulating Reference The ability of standards to bring the world closer, yet also to push it away

27. Key Findings Overcoming Distances in the Secondary Use of Data

28. Gathering One’s Own Data Helps with Reuse Ecologists' experiences as collectors of their own data in the field or laboratory plays the most important role in their secondary use of data.

29. Field-based knowledge Understanding data Judging data quality

30. Data Gathering Provides: • Expertise to understand the critical link between the purpose, the research methods chosen, and the data that result • Ability to recognize data limitations • Ability to visualize potential points of error • A ‘sense’ for data

31. Research purpose Methods Data What frog species live here? How many frogs live here?

32. Charles: “In some ways it is just very simple. Someone saw an animal on such and such a date at such and such a location. That’s basically it. And you can explain that to six-year-old. The only tricky thing…. and, you know, in some ways it is not that hard conceptually, but I see people making the mistake all the time… What does the absence of a record mean? And the absence of a record doesn’t mean the absence of a species. It may just mean a lack of survey effort. And you see biological reports all the time that people consult the state biodiversity database and say, “Oh, we have no endangered species on this piece of property. It’s okay; go ahead and turn it into a shopping mall.” Note: All interviewee names are pseudonyms

33. Visualizing Potential Points of Error Susan: “Well, where the different sources of error can come in-- things like getting water samples, or running the equipment and running the machines that actually analyze water chemistry, and how where you sample within a lake might influence dissolved organic carbon. So, you just get a better idea of all the different things that could influence the final number.”

34. Factors that Influence Research Methods • The scientific question • The environment • The taxa • Practical considerations such as time, money, and skill

35. Gaining a ‘sense’ for data Nancy: “When you're in the field, most of what you learn is not the data points you're collecting -- it's just that sense.” Michael: “The more you actually go out and do those things the more.... You are sort of more critical of the data.”

36. Standards of Scientific Practice Ecologists recognize the informal knowledge they gain in the field, but it is not discussed publicly in the context of “real science” Formal notions about norms of scientific practice guide the gathering of data for reuse and frame ecologists’ experiences

37. Hindrances to Sharing & Reuse • Challenge of locating and integrating data collected for many different purposes and at varying temporal and spatial scales • Ecologists’ idiosyncratic methods of organizing data

38. Re-circulating Reference Ecologists attempt to reconstruct the original collection of the data they seek to reuse.

39. Ellen: “If honestly I could not figure out what they had done, then I just would not use that data point.”

40. Nathan: “One person could have a table that has a column of species and density. Another person could have a table that says Species I Density, Species II Density, and Species III Density. Those sorts of schema differences when you scale them up to 10, 20, 30 data sets -- and we would like to get to 100, 200, 1000 data sets -- become extremely limiting in your ability to integrate the data and to utilize them in a particular framework.”

41. Factors Influencing Data Reuse • Scientific questions • Data collection methods • Data characteristics • Data ownership • Presence of standards • Reuse potential • Intermediaries • Economic or political value of data • Computational and statistical capacity • Existence of formal data sharing systems

42. Applicability of findings • Data documentation: Develop tools and policies to gather information that data collectors are best suited to provide • “Scaling up”: Recognize the importance of intermediaries • Education: Teach data management, data documentation skills, and ethics and guidelines for secondary data use

43. Christine: “In a field like molecular ecology, you grind up a sample, extract the DNA, and sequence it. It's the same thing over and over regardless of the material, and so it's relatively easy to standardize that. Of course, the more I work in molecular ecology, the more I realize that there are many sources of error, many points of decision making, etc. that can and do make standardization difficult.”

44. Charles: “The economics data is often much more organized and processed. In economics, typically people are working with a shared data set. There are hundreds of people that work with the current population survey, for example, and you can go and find out, "Well, what are the problems with this data set?" Everyone can tell you, "Oh yeah, ’79 was a really bad year, and there’s a glitch, and you are going to have to reprocess this field if you want to use it. … But ecology data is not like that. Typically it never gets re-analyzed. And so you are on your own and kind of starting from scratch working with, untested and unverified, unvalidated, and unchecked out data most of the time.”

45. Most scientific data are not simple “measurements” Taken from a seminary by Paul Avery (University of Florida) Data Grids for 21st Century Data Intensive Science SOC seminar – May 8, 2003 Available at: http://www.scienceofcollaboratories.org/NewsEvents/index.php

46. “…the analysis of proteomics data is currently informal and relies heavily on expert opinion. Databases and software tools developed for the analysis of molecular sequences and microarrays are helpful, but are limited owing to the unique attributes of proteomics data and differing research goals.” Boguski, Mark S. and Martin W. McIntosh. 2003. Biomedical informatics and proteomics. Nature 422: 233-521.

47. Ecological Circuitry Collaboratory “At an individual level, we would like all students in this program to be better able to build, use, and understand models while at the same time have firm grounding in the practices of field- and lab-based empirical science.” http://www.ecostudies.org/cc/index.html

48. www.scienceofcollaboratories.org

49. Project Goals • Perform a comparative analysis of collaboratory projects • through invitational workshops that bring together collaboratory researchers from around the world, • Develop a Collaboratory Knowledge Base • technical and social data and detailed findings from existing collaboratory projects, • Develop general principles and design methods • with broad community participation, • Test these principles on existing collaboratories

50. Work Practices & Information Needsof Ecologists Research goals: • to provide input to the design of better information & collaboration tools for ecologists • to understand ecologists’ innovative ways of approaching complexity as a cultural response to the difficulties created by the massive scale of ecological phenomena Bonnie A. Nardi School of Information and Computer Science University of California Irvine nardi@ics.uci.edu darrouzet-nardi.net/bonnie