R esearch project planning Overview

1. Research project planningOverview Catherine R. Messina PhD Research Associate Professor Department of Family, Population & Preventive Medicine December 20, 2017

3. The Goal of Research… To discover and disseminate new knowledge… • that can be applied to other contexts (generalizeable) • and that can be repeated (replicable) with similar results (reliable and valid)

4. Research is problem solving • Identify that a problem exists • Define the problem • What are the causes / what contributes to this problem? • Is it modifiable? Preventable? • If so, then how can we modify / prevent it? • Respond to the problem systematically • systematic inquiry: research follows a sequential process

5. The research process No State research question Is there a need for your study? Review literature Yes Determine study design, sample size Select target population State hypothesis Develop protocol, select measures, analysis plan IRB approval Interpret and report results Collect data Data analysis

6. The research question is ….. • A general statement of purpose - identifies the focus of study • Based on current interest in a topic and feasibility of carrying out this research • Should be well grounded in the current knowledge base (i.e., the literature) • Should have the potential to add to the current knowledge base • Should not be too broad or to narrow • What was the rate of flu vaccination of children in Suffolk County for 2014? • How can the rate of flu vaccination of children be increased? • Can an individualized telephone counseling intervention for parents improve the rate of flu vaccination of children aged 6 months and older?

7. Systematically review the relevant literature • Provides background about what is currently known about a problem or what has already been done in a particular area • Helps identify areas needing further inquiry • Answers question of whether proposed study is necessary and if so, why … • The literature review should support that your research question needs to be asked… • Provides information about methodology and what might be expected • What’s feasible • What worked well • What barriers encountered • Rely on primary rather than secondary sources

8. What is a hypothesis? • It is an “educated” statement that specifies the relationship between 2 or more sets of observations • This statement can be tested - confirmed or denied by research. • A hypothesis is a prediction of consequences: we state……. • NO RELATIONSHIP exists between 2 or more observations (null hypothesis), OR • a RELATIONSHIP DOES exist (alternative hypothesis) • A hypothesis is not an ethical or moral question • A hypothesis is valuable even if not supported by data

9. Hypotheses are always stated before data are collected

10. Formulate study hypothesis • Research hypothesis - statement of an expected or predicted relationship between two or more variables • Based on prior findings that parents who believe that the flu vaccine will harm their child, we expect that among children seen at SB Children’s outpatient clinic – children whose parents believe the flu vaccine is harmful will be less likely to be up to date with age-appropriate vaccinations when compared to children whose parents believe the flu vaccine is not harmful. • Statistical hypothesis • H0: No difference in rates of children up to date with age-appropriate vaccinations among children whose parents believe the flu vaccine is harmful compared to children whose parents believe the flu vaccine is not harmful • H1: Fewer children whose parents believe the flu vaccine is harmful will be up to date with vaccinations compared to children whose parents believe the flu vaccine is not harmful

11. Formulate study hypothesis • Hypotheses should be stated in operational – that is - measurable terms • Operational definitions – define critical terms • E.g., We expect that fewer current smokers will identify smoking as a behavior that increases children’s risk for upper respiratory disease compared to adults who never smoked or who are former smokers • Operationalize smoking: 3 categories of smoking status based on self-reported response to questions: “Do you smoke now” if “yes”, current smoker; if “no”, “Have you ever smoked” – if yes, former smoker / if no, never smoked • Operationalize “identify smoking as increasing their children’s risk for upper respiratory disease”: agree or disagree with statement; define “upper respiratory disease” • Operational definitions should be: feasible – resources, access to data; valid, reliable

12. Research Questions vs. Hypotheses • Descriptive studies characterize the extent and distribution of a health factor; who is at risk; when/when it occurs… • Descriptive studies do not test hypotheses but rather respond to general descriptive research questions: To what extent is Type 2 diabetes a health concern among children residing in Suffolk County? • Descriptive studies generally have a set of objectives: • To quantify the proportion of children in Suffolk County NY have Type 2 diabetes • To describe the sociodemographic and other health characteristics of children in Suffolk County NY with Type 2 diabetes … etc • HOWEVER: The results of descriptive studies contribute to the development of hypotheses – “educated expectations” based on prior knowledge

13. Research Questions vs. Hypotheses • Hypotheses are appropriate for relationship and difference questions • Difference questions: compare between or within groups • E.g., Can a school-based education program improve elimination of standing water sources in communities at high risk for Zika, compared to communities without a school-based program. • Ho: no difference in elimination of standing water sources when school-based education program vs. no school-based education program • H1: Elimination of standing water sources more frequent when school-based education program vs. when no school-based education program

14. Research Questions vs. Hypotheses • Hypotheses are appropriate for relationship and difference questions • Relationship questions: examine the degree to which two or more variables / factors are associated • E.g., examine the association between rates of infant mortality and geographic region • Ho: no difference in rates of infant mortality by region • H1: rates of infant mortality lower in region A, compared to region B

15. Identify the target population and sample • Why do we sample? – to obtain information about the sample in order to make inferences about the population • Target or study population – ALL of the members of a group you are interested in – should be clearly defined • Sampling population (or sampling frame) – population from which the sample is drawn (e.g., patients from SBU Peds clinics) • Sampling – the process of selecting a specific number of individuals in such a way (i.e., unbiased) that they represent the larger group from which they were selected • Sample – a set of individuals whose characteristics represent the larger group from which they were selected (population)

16. Identify the target population and sample • Sampling population influenced by: • External validity (generalizability) – how well does the sampling population reflect the overall target population of interest • Feasibility – who do you have access to???

17. Sample selectionInclusion and exclusion criteria • Inclusion criteria • Based on the research question and research plan • Improves feasibility of conducting study • Exclusion criteria • Exclude characteristics that may confound results • Exclude based on ethical considerations Keep in mind that overly stringent exclusion criteria can impact generalizability (i.e., external validity) of study findings to other persons or situations that are similar to study participants and setting

18. How do we select a representative sample? • Selecting a sampling method • Depends on your study goals and available resources • Two main types used in health research: • incidental sampling… • random sampling…

19. Sample selection • Random (probability) sample • Every individual in the targeted population HAS an equal and specified chance of being selected • Advantage - Reduces risk of selection bias • Incidental (non-probability) sample (aka convenience sample) • Used when probability sampling is not possible • Every individual in the targeted population DOES NOT have an equal specified chance of being selected • Advantages – economical and convenient • Disadvantages • Increases risk of selection bias • Can not ensure generalizability because probability of being selected is unknown

20. Sample selection • Examples: Random sampling • Simple random sample • All individuals have an equal and independent chance of being selected • Selection is through the use of random numbers or similar method (e.g., statistical programs can do random selection) • Stratified random sample • Participants are grouped according to strata such as age group, gender, diagnosis (e.g., cases vs. controls) • Used if important grouping characteristics are known ahead of time • Equal numbers of participants are randomly selected from each strata

21. Sample selection • Examples: Incidental (non – random) sampling • Convenience sample • Participants who meet inclusion criteria selected based on availability • Can evaluate after the fact, if characteristics are similar to larger population – but cannot really generalize from sample to population • Consecutive sample • Version of convenience sampling where every available individual or event within an accessible population is chosen as they become available • Best choice for non-random sampling

22. Study design Study design decisions based on: • Goal / Purpose – type of research question asked • Feasibility • Time frame • Available resources • Money • People • Equipment

23. Goal of your research: • Exploratory • To develop an initial understanding of a phenomena • Descriptive • To precisely measure and report the characteristics of a population or phenomenon and • to answer questions about what, where, when and how • Explanatory • Discover and report relationships among different phenomena • To establish cause and effect.

24. Types of study designs for exploratory and descriptive goals • Observational studies – cross-sectional studies, case-control studies, cohort studies • Observational / Descriptive: Study is designed to describe to precisely measure and report the characteristics of a population or phenomenon • Relational: Study is designed to examine relationships among two or more variables • Correlation relationships – associations only - not cause/effect

25. Types of study designs for explanatory goals • Experimental studies – • Randomized controlled trials: true experimental designs – participants randomized to intervention or control group • Randomization “equalizes” known and unknown confounders so that results can be attributed to treatment with reasonable confidence • Conditions highly controlled – reduces confounding • Can infer cause/effect • Non-randomized or quasi-experimental designs • Both address difference questions between groups

26. Study measures • Anticipate and collect ALL data required to address research question / hypotheses • Outcome or dependent variables • Independent variables • Demographics to adequately characterize sample • Covariates / confounders – based on you clinical experience or prior lit • Keep in mind that study measures should be limited to those relevant to your study • IRB will question anything they think is not relevant • Where are data coming from? • Medical record?; patient self reports?; other? • Missing data!!! – how much can you expect and how might it affect your results????

27. Values Data can take on • Qualitative data: values are intrinsically non-numerical • Categorical • Male / female; cause of death; eye color, etc. • Quantitative data: values are intrinsically numerical • Discrete: counts or occurrences - can only take on certain values (integers); gaps between values: number of children / family • Continuous: no gaps between values – can take on all values within an intervals (fractional values): height; weight; survival time after treatment • Use flexible formats • Continuous rather than categorical formats, if possible

28. Scales of Measurement • The degree of precision with which a characteristic is measured • Has implications for the way information is summarized • Determines statistical methods used to analyze data • Nominal or Categorical • Ordinal • Interval • Ratio

29. Nominal Scales • Simplest (i.e. lowest) level of measurement • Data values fit into mutually exclusive categoriesthat are not more or less, but are different from one another in some way • Attributes are only named – arbitrarily assign numbers, codes, or labels • E.g., 1 = female 2 = male; risk factor: 1 = yes 0 = no; blood type; treatment outcome • Two categories = dichotomous, binary, binomial • > two categories = multinomial

30. Ordinal Scales • Inherent (implied) order among categories (hierarchy) and ordering is important • Staging of cancer: 0 – IV; Dukes A,B,C,D • The difference between two adjacent categories is not equal throughout the scale (apparent equal distances between values do not reflect equal intervals) • E.g., Apgar scores: difference between 8 and 10 not of the same magnitude as 0 and 2 • Meaning of different levels may not be the same for different individuals • Likert scales: 1 = Strongly disagree ;2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly agree

31. Interval-Scale Measurement Inherent order and constant, equal, and meaningful intervals (distances) between values Absence of a true or meaningful zero point (i.e., zero can be arbitrary and does not indicate the complete absence of the quantity being measured) E.g., Intelligence test score; temperature, pain level (if measured on scale 1 – 10).

32. Ratio-Scale Measurement Highest level of measurement Continuum of values Equal distances between values reflect equal intervals True and meaningful zero Numerical relationships between values are meaningful E.g., age, weight, pulse rate

33. Estimating the sample size • Crucial!!! • The size of the sample may influence the representativeness of the study sample and the conclusions you can draw from your findings • Key issues: • “Representativeness” – how well does the sample represent the population • NOTE: having enough people in your sample does not necessarily guarantee representativeness if sample selection was biased in some way • Statistical power – the ability to detect significant differences between groups, if they actually exist.

34. Estimating the sample size Considerations! • The sample size you need vs. what is at hand (or your timeline) • Planning ahead for subgroup analyses

35. Describe protocol • Describe methods or work-plan • List ALL required supplies • List ALL activities related to your study • This includes data management • Identify who is responsible for each activity • Maybe it will only be you • Set approximate date for accomplishment of each activity • This includes IRB approval • Take home message: Research ALWAYS takes longer than you think!!!!

36. Evaluate the feasibility of testing the hypothesis • Are adequate resources for testing hypothesis available? • Access to necessary sample and measures? • Have enough time to carry out? • If getting help, will assistants be able to carry out your protocol exactly as planned? • May need to revise

37. Statistical analysis plan • Never too early to start thinking about this • INFORMED BY YOUR RESEARCH QUESTION!!! • Are you describing a set of characteristics? • Are you evaluating degree of correlation between 2 measures? • Are you comparing measure(s) between 2 or more groups? • Goes hand in hand with operational definitions and choice of study measures • E.g., if plan to compare means or compare proportions – need to obtain appropriate data • E.g., cross sectional study or repeated measures design – each requires a different statistical approach • Early consideration of analysis plan can reveal need to collect additional data at the planning stage when you can actually do something about it!

38. Irb!! • Well thought out protocol will make this process go more smoothly!

39. Implement study • Keep track of any deviations from protocol

40. Results, conclusions, and next steps • Hypotheses supported or not supported? • How do your results relate to what is already known? • What have you added? (a null finding is still a finding!) • How generalizable are your results? • Limitations of your study • Next steps

41. Research is to see what everybody else has seen, and to think what nobody else has thought.Albert Szent-Gyorgyi, Hungarian Biochemist, 1937 Nobel Prize for Medicine, 1893-1986

42. Contact information • Catherine R. Messina PhD • Department of Family, Population & Preventive Medicine • HSC-L3, Rm 086 • 4-8266 • catherine.messina@stonybrookmedicine.edu