A Proper Research Record: The Basic Mechanics

1. A Proper Research Record:The Basic Mechanics Russ Lea, VPR Fall 2010

2. Acknowledgements • Columbia University. (2003–2004). Responsible conduct of research: Courses portal. Course 6: Data acquisition and management. http://www.ccnmtl.columbia.edu/projects/rcr/rcr_data/foundation/index.html. • Magnus, P. D., & Kalichman, M. (2002, September). Data management. Retrieved from RCR Education Resources, Online Resource for RCR Instructors: http://rcrec.org/r/index.php?module=ContentExpress&func=display&meid=29&ceid=2. • Making the Right Moves, A Practical Guide to Scientific Management for Postdocs and New Faculty. Howard Hughes Medical Institute. Chapter 8, Data Management and Laboratory Notebooks. http://www.hhmi.org/resources/labmanagement/downloads/moves2_ch8.pdf • Guidelines for Responsible Data Management in Scientific Research. Developed by Clinical Tools, Inc. Funded by: Office of Research Integrity, US Department of Health and Human Services • Keeping a Lab Notebook, W. Wilson, Department of Engineering & Physics, University of Central Oklahoma. http://www.physics.uco.edu/wwilson/bridge09/Lab%20Notebooks.ppt • University of Wisconsin at Madison RCR Guidelines http://www.grad.wisc.edu/research/policyrp/rcr/data.html • University of California, San Francisco, Data Management: Research Records http://www.research.ucsf.edu/QG/orQgDm.asp

3. Presentation Outline • Why is data management both an issue of scientific rigor and ethics? • What constitutes data? • What are best practices in managing data? • Who is responsible for the accuracy of data collection, recording, and storage? • How long should data be kept; who may access it; and in what contexts may, or must, data be withheld?

4. Research Relies on Trust • Principal Investigators are rarely supervised • Even data collectors are rarely line-of-sight supervised • Research relies on a researchers to: • Develop and employ unbiased research methods • Honestly and accurately report a study’s methods, data handling, and analyses

5. "A laboratory notebook is one of a scientist's most valuable tools. It contains the permanent written record of the researcher's mental and physical activities for experiment and observation, to the ultimate understanding of physical phenomena. The act of writing in the notebook causes the scientist to stop and think about what is being done in the laboratory. It is, in this way, an essential part of doing good science." from Writing the Laboratory Notebook, by Howard M. Kanare, American Chemical Society, 1985

6. What is data? Defined by NIH Grants Policy Statement Typically includes… • Laboratory research • Animal research • Clinical trials • Funding • Correspondence with granting agencies, institutions, and collaborators

7. Researchers Share Responsibility for Data • Data Collector • Collects data • Records data • Stores data while • study is in progress Principal Investigator • Creates system that collects, records, and stores data • Trains data collectors (benefits of good records and problems with poor records) • Supervises data collection, recording, and storage

8. Responsible Data Collection It is unethical to: • Knowingly collect data in a manner that is biased • Falsify or fabricate data • Tailor or change a protocol to alter findings • Change or remove non-conforming data to bend findings

9. Responsible Data Collection cont’d. Rigorous standards have evolved from industry and human subjects research • Universities shifting towards industrial standards for data collection • Description of reasons for experiments • Experimental protocols • Observations, measurements, other results • Printouts, photographs, machine generated data • Mathematical calculations performed on raw data • Brief interpretations of the results

10. Data Collection Detailed records: • Establish good work practices • Are the diaries of the researcher’s work and thoughts • Teach support staff in the researcher’s lab • Allow replication of procedures and studies by the researcher and others • Validate research results • Meet contractual requirements (grants/contracts/patent applications) • Provide evidence of the integrity of research • Can be used to defend patents • Include documentation for publications

11. Data Collection cont’d. There is no one way to keep data; however, data should: • Be written as an individual effort, not as a group project • Explain why the research was done • Explain how it was done • Inform where the primary data is maintained • Illustrate what happened and didn’t happen (data) • Indicate special materials/instruments used • Include an interpretation of the data (and of others if necessary) • Be in compliance with departmental, institutional, and federal regulatory requirements with special care given to human and animal research

12. Data Collection cont’d. The following style conventions are widely recognized: • Permanent binding • Consecutively numbered pages • Table of contents • Explanation of abbreviations • Use of acid free paper for longevity • Dated entries • Date/initial all changes • Keep legible and clear records in permanent ink • Periodic review and signing of notebook by an individual who is not directly involved in the research

13. When is a Witness Warranted? • Entries that merit a witness • When an invention/idea is conceived that may have intellectual property value • When the idea is put into actual practice (reduction to practice) • What constitutes an appropriate witness • Witness should understand the science • Should not be a co-inventor, or have a vested interest in verifying the claim • Someone who is not directly involved in the work, but understands and can explain the idea

14. Reduction to Practice Actual ―experimentally verified to work - OR – Constructive ―filing a patent application that describes the invention in such detail that an individual of ordinary skill, in the relevant field, could understand and practice the invention • Regardless of the type of reduction to practice utilized, patent validity can be challenged if the disclosure fails to enable a person, of ordinary skill in the art, from practicing the invention without undue experimentation

15. What goes in the notebook? • Plans and deviations from the plan • Observations • Sketches, diagrams, and photographs • Ideas (legal document to prove patents) • Links to the notebooks of others in the group • Links to instrument logbooks and data disks • Emails from collaborators (taped/pasted) • Plot-as-you-go graphs • Summaries of papers you have read

16. Research Methods • Reliable data is dependent on reliable methods.  Methods can be compromised by bias, choosing a particular method or set of experimental conditions in order to reach a specific result or conclusion.  Responsible research is research conducted using appropriate, reliable methods, and adequate controls. • Clarity in methods should be used to gather and analyze data. • Work may be judged by others for validity and accuracy of methods used to derive data. • Researchers must be forthcoming about procedures used to obtain each new result. • A clear and comprehensive accounting of a project, its purpose, and direction make it much easier for research to be disseminated, understood, and evaluated by other members of the scientific community. Researchers should analyze current system of note taking―review an old notebook. If the material is difficult to read, a new approach may be warranted

17. Best Practices: Lab Notebook • Recording should be done as soon as possible after data collection • Entries should be recorded chronologically. • Material should be organized using sections/headings. • A second loose-leaf notebook should be kept for items that may not fit into the primary lab notebook, e.g., photographs, machine printouts, chart recordings. • Entries should be written in the first person, and be specific as to who did the work. • Supervisors should review/sign off on notebooks to certify completeness and accuracy.

18. Best Practices: Lab Notebook cont’d. • Material sources should be identified and special attention given to any hazardous use. • Data should be recorded directly into notebooks, not put on scraps of paper or relying on memory. • Data should include periodic factual―not speculative, summaries of status/findings. • Errors should be identified by crossing out mistakes without obscuring the initial data. • Copies of original notebooks should be kept elsewhere for safekeeping.

19. LABNOTEBOOKSGREATEST HITSDiscovery of first Computer Bug. What else would you do but glue it into your notebook?HarvardSept. 9, 1945

20. Merry Christmas, Ma Bell! First TransistorAT&T Bell Labs Note prestigious witness list (some signed), dates, schematic.

21. Library of Congress – Alexander Graham Bell http://memory.loc.gov/ammem/bellhtml/bellhome.html

22. The Telephone Gambit: Chasing Alexander Graham Bell's Secret (2008)by Seth Shulman

23. Best Practices: Electronic Notebook • Develop a data management plan • Assign a designated individual to ensure that the data management plan is executed • Decide what to data store, where to store it, specify a method, and assign an individual to record/log the information • Reference the electronic data in a signed, dated, and witnessed handwritten notebook

24. Best Practices: Electronic Notebook Cont’d. • If electronic records are the only records, a time-stamping program that will date each entry into the system should be utilized • Electronic records should be carefully monitored • Data should be backed up on a disk with a hard copy; relevant software must be retained to ensure future access • Security of data must be maintained to prevent unauthorized access to the system

25. Don’ts: Lab Notebooks • Do not use pencils or something that can be erased easily. • Do not tear out pages. • Do not erase, black out, or paste over data. • Do not modify the original records at a later date. Log modifications on the actual date that they were made and reference the data (i.e., data, page of notebook) that they apply. • Do not put a stray mark on the page without annotating its purpose. (i.e., do not check or circle a number without recording the reason. Otherwise, questions could arise regarding its accuracy.)

26. Correcting Notebook Errors • Draw a single line through the error. • Initial and date each correction when entering the correct information near the original entry. • Provide a brief explanation of each correction.

27. Data Sharing The means by which researchers accurately represent work to the scientific community and the public • As part of the scientific process, it is expected that data would be shared and reported for several purposes, to: • Acknowledge implications of a study • Contribute to a field of study • Stimulate new ideas • Information should be shared cautiously while the project is in progress as full implications of the data may not be fully known. • After publication, allow reasonable access to the raw data, i.e., honor requests that are in the interest of scientific inquiry and that can be accomplished without inordinate expense or delay. • Some sponsor institutions/funding agencies have requirements as to when, and to what degree, a research project may be shared.

28. Data Management Integrity of scholarly work depends on system of data management: • Are records accurate and complete? • Are there reasonable plans for retention and storage of data? • Is the data managed so that it may be shared if required by funding agencies? • Would an audit of records support your claims in publications? • Could co-investigators validate the accuracy of the manuscript from the lab notebook? • Data management is essential for authentication of data, authorship, and intellectual property.

29. Responsible Data Storage • Lab notebooks “in progress” may be kept at the bench. • Consider storing data in a manner that protects it from loss, theft, or damage. • Store historical records of decision making, draft work, and other documents detailing processes with the same care as outcome data. • Keep completed notebooks in a central repository. • Original notebooks typically remain at the institution where the data was generated. • Consider a catalog system; and when someone takes a notebook, it should be checked out and then returned.