Finishing up: Statistics & Developmental designs

1 / 24

# Finishing up: Statistics & Developmental designs - PowerPoint PPT Presentation

Finishing up: Statistics & Developmental designs. Psych 231: Research Methods in Psychology. Remember to turn in the second group project rating sheet in labs this week. Announcements. About populations. Real world ( ‘ truth ’ ). H 0 is correct. H 0 is wrong. Type I error. Reject H 0.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## Finishing up: Statistics & Developmental designs

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Finishing up:Statistics & Developmental designs

Psych 231: Research Methods in Psychology

Real world (‘truth’)

H0 is correct

H0 is wrong

Type I error

Reject H0

Experimenter’s conclusions

Fail to Reject H0

Type II error

76%

80%

XB

XA

• Example Experiment:
• Group A - gets treatment to improve memory
• Group B - gets no treatment (control)
• After treatment period test both groups for memory
• Results:
• Group A’s average memory score is 80%
• Group B’s is 76%

H0: μA = μB

H0: there is no difference between Grp A and Grp B

• Is the 4% difference a “real” difference (statistically

significant) or is it just sampling error?

Two sample

distributions

set α-level

Statistics Summary

Observed difference

Computed

test statistic

=

Difference from chance

Make a decision: reject H0or fail to reject H0

The Design of the study determines what statistical tests are appropriate
• 1 factor with two groups
• T-tests
• Between groups: 2-independent samples
• Within groups: Repeated measures samples (matched, related)
• 1 factor with more than two groups
• Analysis of Variance (ANOVA) (either between groups or repeated measures)
• Multi-factorial
• Factorial ANOVA
Some inferential statistical tests

Observed difference

X1 - X2

T =

Diff by chance

Based on sample error

• Design
• 2 separate experimental conditions
• Degrees of freedom
• Based on the size of the sample and the kind of t-test
• Formula:

XB

XA

Computation differs for

between and within t-tests

T-test
• The observed difference between conditions
• Kind of t-test
• Computed T-statistic
• Degrees of freedom for the test
• The “p-value” of the test
• “The mean of the treatment group was 12 points higher than the control group. An independent samples t-test yielded a significant difference, t(24) = 5.67, p < 0.05.”
• “The mean score of the post-test was 12 points higher than the pre-test. A repeated measures t-test demonstrated that this difference was significant significant, t(12) = 5.67, p < 0.05.”
T-test

Observed variance

F-ratio =

XA

XC

XB

Variance from chance

• Designs
• More than two groups
• 1 Factor ANOVA, Factorial ANOVA
• Both Within and Between Groups Factors
• Test statistic is an F-ratio
• Degrees of freedom
• Several to keep track of
• The number of them depends on the design

Can’t just compute a simple difference score since there are more than one difference

A - B, B - C, & A - C

Analysis of Variance

The ANOVA tests this one!!

Do further tests to pick between these

XA = XB = XC

XA ≠ XB ≠ XC

XA ≠ XB = XC

XA = XB ≠ XC

XA = XC ≠ XB

XA

XC

XB

Null hypothesis:

H0: all the groups are equal

Alternative hypotheses

• HA: not all the groups are equal
1 factor ANOVA

XA ≠ XB ≠ XC

XA ≠ XB = XC

XA = XB ≠ XC

XA = XC ≠ XB

• Planned contrasts and post-hoc tests:
• - Further tests used to rule out the different Alternative hypotheses

Test 1: A ≠ B

Test 2: A ≠ C

Test 3: B = C

1 factor ANOVA
• The observed differences
• Kind of test
• Computed F-ratio
• Degrees of freedom for the test
• The “p-value” of the test
• Any post-hoc or planned comparison results
• “The mean score of Group A was 12, Group B was 25, and Group C was 27. A 1-way ANOVA was conducted and the results yielded a significant difference, F(2,25) = 5.67, p < 0.05. Post hoc tests revealed that the differences between groups A and B and A and C were statistically reliable (respectively t(8) = 5.67, p < 0.05 & t(9) = 6.02, p <0.05). Groups B and C did not differ significantly from one another”
1 factor ANOVA
We covered much of this in our experimental design lecture
• More than one factor
• Factors may be within or between
• Overall design may be entirely within, entirely between, or mixed
• Many F-ratios may be computed
• An F-ratio is computed to test the main effect of each factor
• An F-ratio is computed to test each of the potential interactions between the factors
Factorial ANOVAs
• The observed differences
• Because there may be a lot of these, may present them in a table instead of directly in the text
• Kind of design
• e.g. “2 x 2 completely between factorial design”
• Computed F-ratios
• May see separate paragraphs for each factor, and for interactions
• Degrees of freedom for the test
• Each F-ratio will have its own set of df’s
• The “p-value” of the test
• May want to just say “all tests were tested with an alpha level of 0.05”
• Any post-hoc or planned comparison results
• Typically only the theoretically interesting comparisons are presented
Factorial ANOVAs
• Because of the issue of interest
• Limited resources (not enough subjects, observations are too costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
Non-Experimental designs
• Age typically serves as a quasi-independent variable
• Three major types
• Cross-sectional
• Longitudinal
• Cohort-sequential
Developmental designs
Cross-sectional design
• Groups are pre-defined on the basis of a pre-existing variable
• Study groups of individuals of different ages at the same time
• Use age to assign participants to group
• Age is subject variable treated as a between-subjects variable

Age 4

Age 7

Age 11

Developmental designs
• Can gather data about different groups (i.e., ages) at the same time
• Participants are not required to commit for an extended period of time
• Cross-sectional design
Developmental designs
Disavantages:
• Individuals are not followed over time
• Cohort (or generation) effect: individuals of different ages may be inherently different due to factors in the environmental context
• Are 5 year old different from 15 year olds just because of age, or can factors present in their environment contribute to the differences?
• Imagine a 15yr old saying “back when I was 5 I didn’t have a Wii, my own cell phone, or a netbook”
• Does not reveal development of any particular individuals
• Cannot infer causality due to lack of control
• Cross-sectional design
Developmental designs
Follow the same individual or group over time
• Age is treated as a within-subjects variable
• Rather than comparing groups, the same individuals are compared to themselves at different times
• Changes in dependent variable likely to reflect changes due to aging process
• Changes in performance are compared on an individual basis and overall
• Longitudinal design

time

Age 11

Age 15

Age 20

Developmental designs
Example
• Wisconsin Longitudinal Study(WLS)
• Began in 1957 and is still on-going (50+ years)
• 10,317 men and women who graduated from Wisconsin high schools in 1957
• Originally studied plans for college after graduation
• Now it can be used as a test of aging and maturation
Longitudinal Designs
• Can see developmental changes clearly
• Can measure differences within individuals
• Avoid some cohort effects (participants are all from same generation, so changes are more likely to be due to aging)
• Longitudinal design
Developmental designs
• Can be very time-consuming
• Can have cross-generational effects:
• Conclusions based on members of one generation may not apply to other generations
• Numerous threats to internal validity:
• Attrition/mortality
• History
• Practice effects
• Improved performance over multiple tests may be due to practice taking the test
• Cannot determine causality
• Longitudinal design
Developmental designs
Measure groups of participants as they age
• Example: measure a group of 5 year olds, then the same group 10 years later, as well as another group of 5 year olds
• Age is both between and within subjects variable
• Combines elements of cross-sectional and longitudinal designs
• Addresses some of the concerns raised by other designs
• For example, allows to evaluate the contribution of cohort effects
• Cohort-sequential design
Developmental designs

Cohort-sequential design

Time of measurement

1975

1985

1995

Cohort A

1970s

Age 5

Age 5

Age 5

Cross-sectional component

Cohort B

1980s

Age 15

Age 15

Cohort C

1990s

Age 25

Longitudinal component

Developmental designs