This presentation is the property of its rightful owner.
1 / 20

# Techniques expérimentelles 2 PowerPoint PPT Presentation

Techniques expérimentelles 2. Barbara Hemforth Most of this is stolen from a lecture by Chuck Clifton

Techniques expérimentelles 2

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Techniques expérimentelles 2

Barbara Hemforth

Most of thisisstolenfrom a lecture by Chuck Clifton

### II: How to do experiments. Part 1, General design principles

• Dictum 1: Formulate your question clearly

• Dictum 2: Keep everything constant that you don’t want to vary

• Dictum 3: Know how to deal with unavoidable extraneous variability

• Dictum 4: Have enough power in your experiment

• Dictum 5: Pay attention to your data, not just your statistical tests

C. Clifton Jr

### Dictum 1: Formulate your question clearly

• Independent variable: variation controlled be experimenter, not by what subject does

• Dependent variable: variation observed in subject’s behavior, perhaps dependent on IV

• Operationalization of variables

C. Clifton Jr

### Dictum 2: Try to keep everything constant except what you want to vary

• Try to hold extraneous variables constant through norms, pretests, corpora…

• When you can’t hold them constant, make sure they are not associated (confounded) with your IV

### What happens when there is unavoidable variation?

• Subdictum B: When in doubt, randomize

• Random assignment of subjects to conditions

• Questionnaire: order of presentation of items?

• Single randomization: problems

• Different randomization for each subject

• Constrained randomizations

• Equate confounds by balancing and counterbalancing

• Alternative to random assignment of subject to conditions: match squads of subjects

### Counterbalancing of materials

• Counterbalancing

• Ensure that each item is tested equally often in each condition.

• Ensure that each subject receives an equal number of items in each condition.

• Why is it necessary?

• Since items and subjects may differ in ways that affect your DV, you can’t have some items (or subjects) contribute more to one level of your IV than another level.

### Sometimes you don’t have to counterbalance

• If you can test each subject on each item in each condition, life is sweet

• E.g., Ganong effect (identification of consonant in context)

• Vary VOT in 8 5-ms steps

• /dais/ - /tais/

• /daip/ - /taip/

• Classify initial segment as /d/ or /t/

• Present each of the 80 items to each subject 10 times

• Ganong effect: biased toward /t/ in “type,” /d/ in “dice”

Connine, C. M., & Clifton, C., Jr. (1987). Interactive use of information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 13, 291-299.

### If you have to counterbalance…

• Simple example

• Questionnaire, 2 conditions, N items

• Need 2 versions, each with N items, N/2 in condition 1, remaining half in condition 2

• Versions 1 and 2, opposite assignment of items to conditions

• More general version

• M conditions, need some multiple of M items, and need M different versions

• Embarrassing if you have 15 items, 4 conditions…

• That means that some subjects contributed more to some conditions than others did; bad, if there are true differences among subjects

### Counterbalancing things besides items

• Order of testing

• Don’t test all Ss in one condition, then the next condition…

• At least, cycle through all combinations of conditions (all lists) before testing a second subject with the same list

• Fancier, latin square

• Avoid minor confound if always test cond 1 before cond 2 etc.

• N x n square, sequence x squad, containing condition numbers, such that each condition occurs once in each column, each order

• Location of testing

• E.g., 2 experiment stations

### Latin Squares (Euler, 1773)

Latin square of order 2Latin square of order 3

abx   y   z

b   az   x   y

y   z   x

• A latin square of order n is an n by n array of n symbols in which every symbol occurs exactly once in each row and column of the array.

### Variance in an experiment

• Systematic variance: variability due to manipulation of IV and other variables you can identify

• Random variance: variability whose origin you’re ignorant of

• Point of inferential statistics: is there really variability associated with IV, on top of other variability?

• Is there a signal in the noise?

### Best way to deal with extraneous variability: Minimize it!

• Keep everything constant

• Reduce experimental noise

• See the signal easier

• Keep environment, instructions, distractions, experimenter, response manipulanda, etc. constant

• Pretest subjects and select homogeneous ones, if that suits your purposes

### One way to minimize extraneous variance: Within-subject designs

• Subjects differ

• …a lot, in some measures, eg. Reading speed, reaction time

• Present all levels of your IV to each subject

• Assume the subject effect is a constant across all the levels.

• Differences among conditions thus abstracted from subject differences

• Counterbalancing necessary

• Test each item in each condition for an equal number of subjects.

• E.g., will reading an unreduced relative clause (The horse that was raced past the barn fell) affect reading of a reduced relative clause sentence?

### Dictum 4: Have enough power to overcome extraneous variability

• Minimizes noise component of differences among condition means

• Law of large numbers

• The larger the sample size, the more probable it is that the sample mean comes arbitrarily close to the population mean

• If you’re (almost) looking at population means, any differences have to be real – not sampling error

### Dictum 5: Pay attention to your data, not just your statistical tests

• Look at your data, graph them, try to make sense out of them

• Don’t just look for p < .05!

• Examine confidence intervals

• Look at your data distributions

• Stem and leaf graphs

• By subjects…

### Magnitude estimation:Steven’s Power Law

http://www.cis.rit.edu/people/faculty/montag/vandplite/pages/chap_6/ch6p10.html

### Steven’s Power Law

http://www.cis.rit.edu/people/faculty/montag/vandplite/pages/chap_6/ch6p10.html

### Steven’s Power Law

http://www.cis.rit.edu/people/faculty/montag/vandplite/pages/chap_6/ch6p10.html

### Magnitude estimation: an example

Which man did you wonder when to meet?

Assign an arbitrary number to that item, greater than zero.

Now, for each of the following items, assign a number. If the item is better than the first one, use a larger number; if it’s worse, smaller. Make the item proportional to how much better or worse the item is than the original – if twice as good, make the number 2x the start; if 1/3 as good, make the number 1/3 as big as the start.

### Magnitude estimation : an example

• Which man did you wonder when to meet?

• Assign an arbitrary number, greater than 0, to this first item.

• Now, for each successive item, assign a number – bigger if the item is better, smaller if worse, and proportional – if the item is 2x as good, make the number 2x the original; if ¼ as good, make the number ¼ as big as the original.

• Which book would you recommend reading?

• When do you know the man whom Mary invited?

• This is a paper that we need someone who understands.

• With which pen do you wonder when to write.