1 / 21

Educational data mining overview & Introduction to Exploratory Data Analysis

Educational data mining overview & Introduction to Exploratory Data Analysis. Ken Koedinger CMU Director of PSLC Professor of Human-Computer Interaction & Psychology Carnegie Mellon University. Plan. Because it is technical, will start with learning curve formulas …

saunders
Download Presentation

Educational data mining overview & Introduction to Exploratory Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Educational data mining overview & Introduction to Exploratory Data Analysis Ken Koedinger CMU Director of PSLC Professor of Human-Computer Interaction & Psychology Carnegie Mellon University

  2. Plan • Because it is technical, will start with learning curve formulas … • Then go to exploratory data analysis • Return to in next session to use of formulas in Item Response Theory, Learning Factors Analysis • (Provide some “spaced” practice for you)

  3. Overview • Questions on yesterday’s intro? • Another example of learning curves • Quantitative models of learning curves • Power law, logistic regression • Exercise: • Goals: 1) Get familiar with data, 2) Learn/practice Excel skills • Tasks: 1) create a “step table”, 2) graph learning curves using a) error rate & b) assistance score

  4. Student Performance As They Practice with the LISP Tutor

  5. Production Rule Analysis Evidence for Production Rule as an appropriate unit of knowledge acquisition

  6. Using learning curves to evaluate a cognitive model • Lisp Tutor Model • Learning curves used to validate cognitive model • Fit better when organized by knowledge components (productions) rather than surface forms (programming language terms) • But, curves not smooth for some production rules • “Blips” in leaning curves indicate the knowledge representation may not be right • Corbett, Anderson, O’Brien (1995) • Let me illustrate …

  7. What’s happening on the 6th & 10th opportunities? Curve for “Declare Parameter” production rule • How are steps with blips different from others? • What’s the unique feature or factor explaining these blips?

  8. Can modify cognitive model using unique factor present at “blips” • Blips occur when to-be-written program has 2 parameters • Split Declare-Parameter by parameter-number factor: • Declare-first-parameter • Declare-second-parameter

  9. Overview • Questions on yesterday’s intro? • Another example of learning curves • Quantitative models of learning curves • Power law, logistic regression • Exercise: • Goals: 1) Get familiar with data, 2) Learn/practice Excel skills • Tasks: 1) create a “step table”, 2) graph learning curves using a) error rate & b) assistance score

  10. Learning curve analysis • The Power Law of Learning (Newell & Rosenbloom, 1993) Y = a Xb Y – error rate X – opportunities to practice a skill a – error rate on 1st opportunity b – learning rate After the log transformation “a” is the“intercept” or starting point of the learning curve “b” is the “slope” or steepness of the learning curve

  11. More sophisticated learning curve model • Generalized Power Law to fit learning curves • Logistic regression (Draney, Wilson, Pirolli, 1995) • Assumptions • Different students may initially know more or less => use an intercept parameter for each student • Students learn at the same rate => no slope parameters for each student • Some productions may be more known than others => use an intercept parameter for each production • Some productions are easier to learn than others => use a slope parameter for each production • These assumptions are reflected in detailed math model …

  12. More sophisticated learning curve model p  Probability of getting a step correct (p) is proportional to: • if student i performed this step = Xi, add overall “smarts” of that student = i • if skill j is needed for this step = Yj, add easiness of that skill = jadd product of number of opportunities to learn = Tj & amount gained for each opportunity = j Use logistic regression because response is discrete (correct or not) Probability (p) is transformed by “log odds” “stretched out” with “s curve” to not bump up against 0 or 1 (Related to “Item Response Theory”, behind standardized tests …)

  13. Overview • Questions on yesterday’s intro? • Another example of learning curves • Quantitative models of learning curves • Power law, logistic regression • Exercise: • Goals: 1) Get familiar with data, 2) Learn/practice Excel skills • Tasks: 1) create a “step table”, 2) graph learning curves using a) error rate & b) assistance score

  14. TWO_CIRCLES_IN_SQUARE problem: Initial screen

  15. TWO_CIRCLES_IN_SQUARE problem: An error a few steps later

  16. TWO_CIRCLES_IN_SQUARE problem: Student follows hint & completes prob

  17. Exported File Loaded into Excel

  18. DataShop Export & Using Excel • Get files from • Go to Learnlab.org • Click on “Enabling Technologies” • Click on “Meetings” • Click on “Documents” • Don’t do yet … • Demo …

  19. Demo: Export Step Roll Up from Data Shop …

  20. Now try it yourself … • Follow instructions in download from two slides ago …

  21. END

More Related