Techniques expérimentelles 2 - PowerPoint PPT Presentation

Techniques exp rimentelles 2
1 / 20

  • Uploaded on
  • Presentation posted in: General

Techniques expérimentelles 2. Barbara Hemforth Most of this is stolen from a lecture by Chuck Clifton

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Techniques expérimentelles 2

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Techniques exp rimentelles 2

Techniques expérimentelles 2

Barbara Hemforth

Most of thisisstolenfrom a lecture by Chuck Clifton

Ii how to do experiments part 1 general design principles

II: How to do experiments. Part 1, General design principles

  • Dictum 1: Formulate your question clearly

  • Dictum 2: Keep everything constant that you don’t want to vary

  • Dictum 3: Know how to deal with unavoidable extraneous variability

  • Dictum 4: Have enough power in your experiment

  • Dictum 5: Pay attention to your data, not just your statistical tests

C. Clifton Jr

Dictum 1 formulate your question clearly

Dictum 1: Formulate your question clearly

  • Independent variable: variation controlled be experimenter, not by what subject does

  • Dependent variable: variation observed in subject’s behavior, perhaps dependent on IV

  • Operationalization of variables

C. Clifton Jr

Dictum 2 try to keep everything constant except what you want to vary

Dictum 2: Try to keep everything constant except what you want to vary

  • Try to hold extraneous variables constant through norms, pretests, corpora…

  • When you can’t hold them constant, make sure they are not associated (confounded) with your IV

What happens when there is unavoidable variation

What happens when there is unavoidable variation?

  • Subdictum B: When in doubt, randomize

    • Random assignment of subjects to conditions

    • Questionnaire: order of presentation of items?

      • Single randomization: problems

      • Different randomization for each subject

      • Constrained randomizations

  • Equate confounds by balancing and counterbalancing

    • Alternative to random assignment of subject to conditions: match squads of subjects

Counterbalancing of materials

Counterbalancing of materials

  • Counterbalancing

    • Ensure that each item is tested equally often in each condition.

    • Ensure that each subject receives an equal number of items in each condition.

  • Why is it necessary?

    • Since items and subjects may differ in ways that affect your DV, you can’t have some items (or subjects) contribute more to one level of your IV than another level.

Sometimes you don t have to counterbalance

Sometimes you don’t have to counterbalance

  • If you can test each subject on each item in each condition, life is sweet

  • E.g., Ganong effect (identification of consonant in context)

    • Vary VOT in 8 5-ms steps

      • /dais/ - /tais/

      • /daip/ - /taip/

    • Classify initial segment as /d/ or /t/

      • Present each of the 80 items to each subject 10 times

      • Ganong effect: biased toward /t/ in “type,” /d/ in “dice”

Connine, C. M., & Clifton, C., Jr. (1987). Interactive use of information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 13, 291-299.

If you have to counterbalance

If you have to counterbalance…

  • Simple example

    • Questionnaire, 2 conditions, N items

    • Need 2 versions, each with N items, N/2 in condition 1, remaining half in condition 2

      • Versions 1 and 2, opposite assignment of items to conditions

  • More general version

    • M conditions, need some multiple of M items, and need M different versions

      • Embarrassing if you have 15 items, 4 conditions…

      • That means that some subjects contributed more to some conditions than others did; bad, if there are true differences among subjects

Counterbalancing things besides items

Counterbalancing things besides items

  • Order of testing

    • Don’t test all Ss in one condition, then the next condition…

    • At least, cycle through all combinations of conditions (all lists) before testing a second subject with the same list

    • Fancier, latin square

      • Avoid minor confound if always test cond 1 before cond 2 etc.

      • N x n square, sequence x squad, containing condition numbers, such that each condition occurs once in each column, each order

  • Location of testing

    • E.g., 2 experiment stations

Latin squares euler 1773

Latin Squares (Euler, 1773)

Latin square of order 2Latin square of order 3

abx   y   z

b   az   x   y

y   z   x

  • A latin square of order n is an n by n array of n symbols in which every symbol occurs exactly once in each row and column of the array.

Variance in an experiment

Variance in an experiment

  • Systematic variance: variability due to manipulation of IV and other variables you can identify

  • Random variance: variability whose origin you’re ignorant of

  • Point of inferential statistics: is there really variability associated with IV, on top of other variability?

    • Is there a signal in the noise?

Best way to deal with extraneous variability minimize it

Best way to deal with extraneous variability: Minimize it!

  • Keep everything constant

    • Reduce experimental noise

      • See the signal easier

    • Keep environment, instructions, distractions, experimenter, response manipulanda, etc. constant

    • Pretest subjects and select homogeneous ones, if that suits your purposes

One way to minimize extraneous variance within subject designs

One way to minimize extraneous variance: Within-subject designs

  • Subjects differ

    • …a lot, in some measures, eg. Reading speed, reaction time

  • Present all levels of your IV to each subject

    • Assume the subject effect is a constant across all the levels.

    • Differences among conditions thus abstracted from subject differences

  • Counterbalancing necessary

    • Test each item in each condition for an equal number of subjects.

  • Worry about experience changing what your subject did

    • E.g., will reading an unreduced relative clause (The horse that was raced past the barn fell) affect reading of a reduced relative clause sentence?

Dictum 4 have enough power to overcome extraneous variability

Dictum 4: Have enough power to overcome extraneous variability

  • Add more data!

    • Minimizes noise component of differences among condition means

  • Law of large numbers

    • The larger the sample size, the more probable it is that the sample mean comes arbitrarily close to the population mean

    • If you’re (almost) looking at population means, any differences have to be real – not sampling error

Dictum 5 pay attention to your data not just your statistical tests

Dictum 5: Pay attention to your data, not just your statistical tests

  • Look at your data, graph them, try to make sense out of them

    • Don’t just look for p < .05!

  • Examine confidence intervals

  • Look at your data distributions

    • Stem and leaf graphs

    • By subjects…

Magnitude estimation steven s power law

Magnitude estimation:Steven’s Power Law

Steven s power law

Steven’s Power Law

Steven s power law1

Steven’s Power Law

Magnitude estimation an example

Magnitude estimation: an example

Which man did you wonder when to meet?

Assign an arbitrary number to that item, greater than zero.

Now, for each of the following items, assign a number. If the item is better than the first one, use a larger number; if it’s worse, smaller. Make the item proportional to how much better or worse the item is than the original – if twice as good, make the number 2x the start; if 1/3 as good, make the number 1/3 as big as the start.

Magnitude estimation an example1

Magnitude estimation : an example

  • Which man did you wonder when to meet?

    • Assign an arbitrary number, greater than 0, to this first item.

    • Now, for each successive item, assign a number – bigger if the item is better, smaller if worse, and proportional – if the item is 2x as good, make the number 2x the original; if ¼ as good, make the number ¼ as big as the original.

  • Which book would you recommend reading?

  • When do you know the man whom Mary invited?

  • This is a paper that we need someone who understands.

  • With which pen do you wonder when to write.

  • Who did Bill buy the car to please?

Bard, E. G., Robertson, D., & Sorace, A. (1996). Magnitude estimation of linguistic acceptability. Language, 72.

  • Login