Techniques exp rimentelles 2
1 / 20

Techniques expérimentelles 2 - PowerPoint PPT Presentation

  • Uploaded on

Techniques expérimentelles 2. Barbara Hemforth Most of this is stolen from a lecture by Chuck Clifton

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Techniques expérimentelles 2' - mason

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Techniques exp rimentelles 2

Techniques expérimentelles 2

Barbara Hemforth

Most of thisisstolenfrom a lecture by Chuck Clifton

Ii how to do experiments part 1 general design principles
II: How to do experiments. Part 1, General design principles

  • Dictum 1: Formulate your question clearly

  • Dictum 2: Keep everything constant that you don’t want to vary

  • Dictum 3: Know how to deal with unavoidable extraneous variability

  • Dictum 4: Have enough power in your experiment

  • Dictum 5: Pay attention to your data, not just your statistical tests

C. Clifton Jr

Dictum 1 formulate your question clearly
Dictum 1: Formulate your question clearly

  • Independent variable: variation controlled be experimenter, not by what subject does

  • Dependent variable: variation observed in subject’s behavior, perhaps dependent on IV

  • Operationalization of variables

C. Clifton Jr

Dictum 2 try to keep everything constant except what you want to vary
Dictum 2: Try to keep everything constant except what you want to vary

  • Try to hold extraneous variables constant through norms, pretests, corpora…

  • When you can’t hold them constant, make sure they are not associated (confounded) with your IV

What happens when there is unavoidable variation
What happens when there is unavoidable variation? want to vary

  • Subdictum B: When in doubt, randomize

    • Random assignment of subjects to conditions

    • Questionnaire: order of presentation of items?

      • Single randomization: problems

      • Different randomization for each subject

      • Constrained randomizations

  • Equate confounds by balancing and counterbalancing

    • Alternative to random assignment of subject to conditions: match squads of subjects

Counterbalancing of materials
Counterbalancing of materials want to vary

  • Counterbalancing

    • Ensure that each item is tested equally often in each condition.

    • Ensure that each subject receives an equal number of items in each condition.

  • Why is it necessary?

    • Since items and subjects may differ in ways that affect your DV, you can’t have some items (or subjects) contribute more to one level of your IV than another level.

Sometimes you don t have to counterbalance
Sometimes you don want to vary’t have to counterbalance

  • If you can test each subject on each item in each condition, life is sweet

  • E.g., Ganong effect (identification of consonant in context)

    • Vary VOT in 8 5-ms steps

      • /dais/ - /tais/

      • /daip/ - /taip/

    • Classify initial segment as /d/ or /t/

      • Present each of the 80 items to each subject 10 times

      • Ganong effect: biased toward /t/ in “type,” /d/ in “dice”

Connine, C. M., & Clifton, C., Jr. (1987). Interactive use of information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 13, 291-299.

If you have to counterbalance
If you have to counterbalance… want to vary

  • Simple example

    • Questionnaire, 2 conditions, N items

    • Need 2 versions, each with N items, N/2 in condition 1, remaining half in condition 2

      • Versions 1 and 2, opposite assignment of items to conditions

  • More general version

    • M conditions, need some multiple of M items, and need M different versions

      • Embarrassing if you have 15 items, 4 conditions…

      • That means that some subjects contributed more to some conditions than others did; bad, if there are true differences among subjects

Counterbalancing things besides items
Counterbalancing things besides items want to vary

  • Order of testing

    • Don’t test all Ss in one condition, then the next condition…

    • At least, cycle through all combinations of conditions (all lists) before testing a second subject with the same list

    • Fancier, latin square

      • Avoid minor confound if always test cond 1 before cond 2 etc.

      • N x n square, sequence x squad, containing condition numbers, such that each condition occurs once in each column, each order

  • Location of testing

    • E.g., 2 experiment stations

Latin squares euler 1773
Latin Squares (Euler, 1773) want to vary

Latin square of order 2 Latin square of order 3

ab x   y   z

b   a z   x   y

y   z   x

  • A latin square of order n is an n by n array of n symbols in which every symbol occurs exactly once in each row and column of the array.

Variance in an experiment
Variance in an experiment want to vary

  • Systematic variance: variability due to manipulation of IV and other variables you can identify

  • Random variance: variability whose origin you’re ignorant of

  • Point of inferential statistics: is there really variability associated with IV, on top of other variability?

    • Is there a signal in the noise?

Best way to deal with extraneous variability minimize it
Best way to deal with extraneous variability: Minimize it! want to vary

  • Keep everything constant

    • Reduce experimental noise

      • See the signal easier

    • Keep environment, instructions, distractions, experimenter, response manipulanda, etc. constant

    • Pretest subjects and select homogeneous ones, if that suits your purposes

One way to minimize extraneous variance within subject designs
One way to minimize extraneous variance: Within-subject designs

  • Subjects differ

    • …a lot, in some measures, eg. Reading speed, reaction time

  • Present all levels of your IV to each subject

    • Assume the subject effect is a constant across all the levels.

    • Differences among conditions thus abstracted from subject differences

  • Counterbalancing necessary

    • Test each item in each condition for an equal number of subjects.

  • Worry about experience changing what your subject did

    • E.g., will reading an unreduced relative clause (The horse that was raced past the barn fell) affect reading of a reduced relative clause sentence?

Dictum 4 have enough power to overcome extraneous variability
Dictum 4: Have enough power to overcome extraneous variability

  • Add more data!

    • Minimizes noise component of differences among condition means

  • Law of large numbers

    • The larger the sample size, the more probable it is that the sample mean comes arbitrarily close to the population mean

    • If you’re (almost) looking at population means, any differences have to be real – not sampling error

Dictum 5 pay attention to your data not just your statistical tests
Dictum 5: Pay attention to your data, not just your statistical tests

  • Look at your data, graph them, try to make sense out of them

    • Don’t just look for p < .05!

  • Examine confidence intervals

  • Look at your data distributions

    • Stem and leaf graphs

    • By subjects…

Magnitude estimation steven s power law
Magnitude estimation: statistical testsSteven’s Power Law

Steven s power law
Steven’s Power Law statistical tests

Steven s power law1
Steven’s Power Law statistical tests

Magnitude estimation an example
Magnitude estimation: an example statistical tests

Which man did you wonder when to meet?

Assign an arbitrary number to that item, greater than zero.

Now, for each of the following items, assign a number. If the item is better than the first one, use a larger number; if it’s worse, smaller. Make the item proportional to how much better or worse the item is than the original – if twice as good, make the number 2x the start; if 1/3 as good, make the number 1/3 as big as the start.

Magnitude estimation an example1
Magnitude estimation : an example statistical tests

  • Which man did you wonder when to meet?

    • Assign an arbitrary number, greater than 0, to this first item.

    • Now, for each successive item, assign a number – bigger if the item is better, smaller if worse, and proportional – if the item is 2x as good, make the number 2x the original; if ¼ as good, make the number ¼ as big as the original.

  • Which book would you recommend reading?

  • When do you know the man whom Mary invited?

  • This is a paper that we need someone who understands.

  • With which pen do you wonder when to write.

  • Who did Bill buy the car to please?

Bard, E. G., Robertson, D., & Sorace, A. (1996). Magnitude estimation of linguistic acceptability. Language, 72.