Writing an Empirical Paper

1. Writing an Empirical Paper PM 515 February 11, 2011

2. Why write an empirical paper • To pass your quals • Publish or perish • To establish the need for your next paper or grant

3. Overall structure • Title page • Abstract • Introduction • Methods • Results • Conclusions • References • Tables • Figures • All of these are important!

4. Title page • Title • Should say what the paper is about • Should contain the keywords that you want to show up in a search • If you can’t get them all in the title, make sure they’re in the abstract.

5. Title page • Authors • You • The PI • Anyone who contributed substantially to • Design and analysis OR • Interpretation • And read/approved the final draft

6. Order of authorship • Talk about authorship at the beginning! • Different PIs have different policies • First draft-first author • PI is always first author • Some projects have a core group of people who are always authors • Most prestigious positions are 1st, last, and 2nd.

7. Abstract • Follow the journal’s format • Some want one big paragraph • Some want Purpose, Methods, Results, and Conclusions • Generally 150-300 words

8. Abstract should contain • Statement of the problem • Research question/hypothesis • Sample characteristics (N, some demographics, how recruited) • Type of analysis • Concise summary of results • Implications

9. Introduction • Define the health issue • How prevalent is it? • Why is it bad? • Why would it be useful to have more information about it?

10. Introduction should answer these questions • What was I studying? • Why was it an important question? • What did we know about it before I did this study? • How will this study advance our knowledge?

11. Introduction – lit review • Summarize previous studies • Draw general conclusions: “Several studies have supported the hypothesis that X is associated with Y (cites). However, other studies (cites) have not found support for this hypothesis.”

12. Introduction – lit review • Don’t just list a bunch of studies: “Pentz et al found X. Richardson et al also found X. Sussman et al found X and Y. Valente et al found X too. Leventhal et al found X in a different kind of sample. Spruijt-Metz et al found X and Z but not Y. Sun et al didn’t measure X but recommended that X should be studied….”

13. Introduction – lit review • Lit review should leave the reader knowing what has been done before. • Now tell the reader what has not been done. • Emphasize what you’re going to do!

14. Introduction • Leave the reader feeling that • This is an important problem. • Some related work has been done, but there are important gaps in the literature. • This study will fill those gaps.

15. Theoretical model • Useful for giving the paper structure • Helps you form the hypotheses • Helps you decide which variables to include

16. Is Theory A supported? • With a different health behavior? • Example: The Transtheoretical Model was developed for smoking cessation. Can it be applied to exercise adherence? • In a different population? • Kids vs. adults • Different ethnic groups • In different cultures • Can also contrast assess fit across groups in the same paper (maybe the theory works among men but not among women)

17. Competing theoretical models • A good strategy for introducing some suspense into your paper • Theory A would predict …. • But Theory B would predict…. • Which theory will win?

18. End of intro • By now, you have convinced the reader that the study needs to be done. • Then add a brief statement about what you did • The present study evaluated the associations between X and Y in a sample of Z…. • Can also state why this study is better than previous studies • Larger sample • More diverse sample • Longitudinal • Better measures • Etc.

19. Visualize your reader…. • Who reads this journal? • What do they know? • What do they not know? • What do they want to learn by reading your paper?

20. Methods • First, ask the PI if they have a standard methods section. • If not, look for previous papers or tech reports. • It’s not plagiarizing if a research group has an accepted standard methods section and you use it. • (But whoever wrote it should be a co-author)

21. Methods • Study design • Give a general overview • Cross-sectional or longitudinal? • Was there an intervention? • Was there random assignment? Or pretest-posttest?

22. Subjects / participants / sample • What was the population (e.g., all students in a school)? • How were potential participants selected? (everyone invited, randomly selected) • How were they contacted and invited to participate? By whom? Where? • How did they give informed consent? Was the procedure IRB-approved?

23. Procedure • How were the data collected? • Where? • When? • By whom? • In what format? (paper-and-pencil, interview, web, etc.)

24. Measures • Describe each variable and scale • How was it measured? • Give a cite • Has it been validated in a similar sample? • State the Cronbach’s alpha, test-retest reliability, or correlation with some objective standard, if known • State how it was coded (5-point scale, dichotomous, etc.) • If you changed it for this study, explain why.

25. Data analysis • Describe the analysis used to test each hypothesis • For simple analyses, just say what statistic you used. • For complex analyses, give some explanation of the method. • Normally one paragraph (or sentence) for each hypothesis or table • Don’t need to describe data entry and cleaning in detail—focus on the technical aspects of the analyses. • The reader assumes you entered the data accurately.

26. Results • Participation rate • How many people were invited? • How many consented? • How many completed the survey? • Attrition (for longitudinal studies) • What percent were retained at follow-up? • Did those lost to attrition differ on pretest variables to those followed successfully? • Was there differential attrition across intervention groups? (e.g., was there more dropout in the control group?)

27. Results • Describe the sample • Demographic characteristics • Prevalence of the outcome variable • Distributions of some of the important IV and covariates • Usually this is Table 1.

28. Results • Describe each table. • Generally one paragraph per table • Just summarize the table’s results in the text! Don’t restate all the numbers in the table. • Direct the reader’s attention to the take-home message of each table (usually which variables were significant)

29. Example • Table 1. Demographic characteristics • Table 2. Demographic variation in the IVs and DVs • Table 3. Zero-order correlations • Table 4. Regression of IVs predicting DVs, controlling for covariates • Table 5. Repeat regressions, stratified by a moderator

30. Tables vs. figures • Figures take up a lot of journal space, so you shouldn’t have too many. • Don’t use a figure for information that can be conveyed in a table (e.g., comparisons of means between groups, a single regression line, etc.) • Figures are useful for graphing interactions or showing complex patterns of results. • Figures are also good for diagramming theoretical models and path models.

31. Example of a path model figure

32. Example of when you don’t need a figure

33. Results • Describe each finding, but don’t interpret! • Save the interpretations and implications for the discussion. • Just say what happened.

34. Discussion • BRIEFLY summarize results. • Don’t repeat the whole Results section! • Link your findings back to previous research. • Were your findings similar or different? If different, why? • How do your results add to what was already known?

35. Link your results back to theory • Do your results support a theory? • Do they refute some other theory?

36. Discuss any unusual or unexpected findings • If you found something totally contrary to the literature, why do you think this happened? • It’s not just that you’re right and everyone else was wrong. What was different?

37. Limitations • Sample size • Small sample – limited power • Big sample – too much power may cause you to interpret unimportant findings as significant • Causality • Can’t prove unless you had random assignment • Even longitudinal studies can only establish temporal precedence

38. Generalizability • Limited to people who weren’t lost to attrition • Limited to people who were successfully recruited • Limited to a particular type of person at a particular place and time

39. Measurement issues • Self-report bias • Invalid measures • Measures with poor psychometric characteristics

40. More research is (always) needed • What questions remained unanswered? • Don’t forget to include the question that you’re going to address in your next grant!

41. Implications • For researchers • For practitioners • For policymakers • For society in general

42. Conclusions • Add with a punchy take-home message

43. References • Include the major players in the field. • Try to cite the most recent studies and seminal older studies. • OK to cite yourself and your friends/mentors

44. The review process • Roles: • Editor • Associate editor • Reviewer • Editorial assistant • Publisher’s editor / typesetter

45. How are reviewers selected? • How I choose them • My friends and colleagues who have done similar work • People in Pubmed who have done similar work • People who have submitted similar manuscripts to the same journal • Not too senior—they never agree • A junior reviewer is OK if there’s also a more senior reviewer, but junior reviewers can be really picky! • People the authors cite • Good mix of substantive and statistical people

46. Review process • Editor may triage (sometimes with agreement of associate editors) • Editor delegates to associate editors • Associate editors assign reviewers • Reviewers review • Associate editor makes recommendation • Editor approves and communicates with authors

47. Decision letters • Accept outright – very rare! • Accept with minor revisions • Revise-resubmit • Reject

48. What to do when you get a revise-resubmit • Resubmit it soon! • Include a point-by-point response to the reviewers’ critiques • Be respectful to the reviewers—they will read this! • You can respectfully disagree and back it up, or you can say that something would have been nice but unfortunately it was impossible. • Don’t just say, “We decided not to do that.” • Don’t be overly hostile or overly solicitous. • It’s a colleague-to-colleague relationship, even though you’re a student.

49. What to do when you get a rejection • Don’t take it personally. • Don’t throw away the paper. • Don’t throw away the reviews. • Don’t quit the Ph.D. program. • Goal—send it out to another journal ASAP. • Make a few changes if the reviewers had some good points, but don’t spend months responding to all their criticisms.

50. After acceptance • It’s “In press” • Wait for the proofs • Return the proofs quickly • Keep the co-authors informed • Put it on your CV