210 likes | 213 Views
Exploring Individual Variability Using ACT-R. Christian Schunn George Mason University. Understanding Variability in Performance. Within and between subjects variability are important sources of information (beyond average performance) Differentiate models
E N D
Exploring Individual Variability Using ACT-R Christian Schunn George Mason University
Understanding Variability in Performance • Within and between subjects variability are important sources of information (beyond average performance) • Differentiate models • Indicate whether average is representative of any individual's behavior (both within and between subjects)
Sources of Variability: Naïve View • Random sampling with fixed probabilities • e.g., most mathematical models of memory • e.g., most mathematical models of choice
ACT-R view • “Noise” (variations on Naïve) • Expected gain noise • Activation noise (encoding and transient) • Perceptual noise • Parametric variation • Global architecture differences • Ability (W, d) & motivational differences (G) • Experiential differences (expertise & luck) • e.g., q&r, a&b, activation, strength, aij, etc • Knowledge variation (expertise & luck) • Productions & Chunks
An example: Are there individual differences in adaptivity? • On average, people select choices according to base-rates of success • e.g., Probability matching • On average, people adapt (or change) strategies when base-rates change • Reder 82, Siegler 87, Lovett & Anderson 96 • Do people systematically differ in how much or how fast they adapt? • Also, is average meaningful?
Experiment Details Block = 10 trials
A mathematical model of aggregate performance } r2=.93 (zero parameters)
Variation in Adaptivity • Suggests more than just noise?
ACT-R BST model • Adapted from Lovett (1998) • Same productions & parameter values • Force-over, force-under, then finish task, retry with other strategy if can't solve • Can ACT-R provide better fit and more insights than the Naïve Monte Carlo simulation?
Fit to Data (default) r2=.77, RMS=.06 • Luck/experiential differences plus noise • Less variability than humans
Sensitivity Analyses • What influences mean and variance in adaptivity? • EGS settings • Motivational levels • Learning • Prior experiences • Each model uses default settings and tweaks one feature
Noise settings (EGS) .84, .08 .77, .06 .72, .09 • Even at better settings, variability is low
Motivation settings (G) .84, .11 .77, .06 .67, .13 • At high G settings, variability goes down
Learning decay (d) .75,.13 .77,.06 • Decay makes model too sensitive • And variability still too low
Prior experiences .78, .25 .77, .06 • Apparently subjects have been playing BST previously? • Can get greater variability, but fit to mean becomes worse.
Relationship to awareness data • Suggests more than just noise • ACT-R fits unaware best?
Conclusions/Questions • ACT-R gives more persuasive exploration of chance variability • Variability and mean affected differently • EGS, G affect means but not variablity levels • For adaptivity only! (both affect block variability) • Amount of prior experiences affects both • Watch out for individual differences: • Evidence for parameter learning decay just a mixture of aware and unaware?