Enhancing Research Integrity: Best Practices and Guidelines

1. The Integrity of Data ManagementResponsible Conduct of Research Workshop Howard University November 2, 2010 Peggy Carr, Ph.D. Associate Commissioner National Center for Education Statistics 1

2. Overview of Statistical and Research Standards 2

3. OMB Guidelines for Federal Statistical Agencies • 16 Federal Statistical Agencies adhere to Office of Management and Budget Guidelines to maintain high standards of performance • Development of concepts and methods • Planning and design of surveys and other means of collecting data • Collection of data • Processing and editing of data 3 OMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, February 22, 2002, (67 FR 8752-8460)

4. Best Practices for the Conduct of Research 4

5. Best Practices for the Conduct of Research • Conceptualize Research Problem • Operationalize Study Design • Engage in Peer Review and Networking Activities • Publish and Publish Early/Often • Be Ethical 5

6. Oprah Video Clip

7. 1. Conceptualize Research Problem Literature Review Statement of Problem Definition of Terms, Concepts, and Constructs Conceptual Operational Hypotheses Research Question 7

8. 2a. Operationalize Study Design The Concept of Study Design Explained Variance Hypothesis 1 Unexplained Variance Sampling Error Measurement Error Validity Reliability

9. Validation of Constructs Face – Content validation across judges Criterion – Predictive and concurrent Construct – Methodological approaches to validation 2b. Operationalize Study Design Jealousy Envy 9

10. 2c. Operationalize Study Design • Reliability of Measures • Test and Retest – stability over time • Equivalency – consistency across forms • Internal – internal consistency UNRELIABLE RELIABLE 10

11. 2c. Operationalize Study Design • Research Design • Internal Validity – refuting rival plausible hypotheses • External Validity – generalizability of findings • Choice of Designs – experimental, quasi, survey, correlational Y X 11

12. 2c. Operationalize Study Design • Quality Assurances of Data • Data Checks (e.g., out of range variance, criterion reference checks, replication) • Thoughtful and Appropriate Data Analysis • The nature of your data • Limitations of statistical technique • Discuss Findings and Conclusions with Colleagues • Relate back to literature review • Implications for future research • Flaws and limitations NEXT 12

13. 2c. Operationalize Study Design Trend in fourth-grade NAEP mathematics average scores 13

14. 2c. Operationalize Study Design 14

15. 3. Engage in Peer Review and Networking Activities • Find a specialty and stick with it • Attend professional meetings and conferences – share your research • Keep an open mind and learn from others 15

16. 4. Publish and Publish Early/Often • Start your publication record early in peer reviewed journals • As first, second, or third author • Publish and/or present often • Regardless of professional focus • Annually 16

17. 5. Be Ethical • Be committed and follow through • Adhere to • Subjects’ Rights • Institutional Review Board • Confidentiality and Privacy Rights 17

18. Questions? 18