Carnegie Learning
Download
1 / 34

Marcos Almeida Sue Young Chung Matthew Easterday Andrea Knight Scott Robertson - PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on

Carnegie Learning Assistments Interface. Marcos Almeida Sue Young Chung Matthew Easterday Andrea Knight Scott Robertson. Agenda. Project overview Translating from Standards to Skills Understanding skills Remedy Data driven design Demo Evaluation and Next Steps. Assistments Project.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Marcos Almeida Sue Young Chung Matthew Easterday Andrea Knight Scott Robertson' - xander-rivers


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Carnegie Learning

Assistments Interface

Marcos Almeida

Sue Young Chung

Matthew Easterday

Andrea Knight

Scott Robertson


Agenda

  • Project overview

  • Translating from Standards to Skills

  • Understanding skills

  • Remedy

  • Data driven design

  • Demo

  • Evaluation and Next Steps


Assistments Project

  • Web-based computer tutor “Assistment” system

  • that will help students prepare

  • for standardized mathematics tests.

  • Why important

  • No Child Left Behind Act (accountability, consequences)

  • Effective use of classroom time

  • Goals

  • predict a student’s score on a standardized test

  • provide feedback to teachers about how they can adapt lessons to address gaps in students' knowledge


Our project
Our Project

Design teacher interface so they see where their students need help.


The Challenge

A Standard

8.N.10 Estimate and compute with fractions (including simplification of fractions), integers, decimals, and percents (including those greater than 100 and less than 1).


The Challenge

A Standard

8.N.10 Estimate and compute with fractions (including simplification of fractions), integers, decimals, and percents (including those greater than 100 and less than 1).

A Skill

Translation (Symbolic to Visualization)


A skill
A Skill

Skills are the steps taken to solve a problem.



Reporting

Category

State

Standard

Questions

Skills


Project goals
Project Goals

  • Communicate student and class progress

  • Provide guidance for remedy


Our solution
Our Solution

  • Make the relationship between standards, skills, and questions explicit.

  • Provide different views of the data to support teachers various uses.


Three Focus Questions

  • How do teachers understand skills? What can we do to help them understand skills?

  • 2. How to translate from the standards (that teachers understand) to skills (operational knowledge that the students have and that CL has)

  • 3. How do teachers diagnose/evaluate and then what do they to remedy?


  • Interviews in their classrooms/offices/schools [5]

  • Think Alouds at CL-sponsored conference [10]

  • Card-sorting task on CL algebra and geometry skills

  • Prototype walkthrough on our 5 designs

User Studies

We talked to 18 algebra/geometry teachers who use CL’s Cognitive Tutor and/or prepare their students for the PSSA


Understanding skills
Understanding Skills

  • Interviews and Card Sort

  • What we learned:

    • Teachers can identify what skills mean by labels

    • Consensus on grouping of skills (what the correlation was)

    • Non-consensus in labeling of group

    • Teachers naturally break down questions into steps


Understanding skills1
Understanding Skills

  • Important for us to make mapping from standards to skills very explicit

  • Skill to question relationship is crucial

  • Skills should be illustrated as a step within a question

  • Definitions of skills should include examples


Translating from standards to skills
Translating from Standards to Skills

  • Interviews and Prototype Think Alouds

  • What we learned:

    • Standards are more concrete

    • Evaluated against standards

    • Practical needs to see progress

    • Teachers get a feel of how their students are doing

    • Data loses meaning without the personal context


Translating from standards to skills1
Translating from Standards to Skills

  • Need to explain skills in terms of standards

  • Important for us to provide context

  • Help teachers link Assistments to other sources of information / remedy


Remedy
Remedy

  • Interviews, Think Alouds, Background Questionnaires

  • What we learned:

    • Teachers jump to remedy from performance reports

    • Inexperienced teachers (under 3-5 years experience) were unable to generate remedy methods

    • Teachers value class interaction to gauge if remedy is needed

    • Re-use questions for quizzes, exams, and practice

    • Most commonly assign more practice problems


Remedy1
Remedy

  • Dependant upon first two focus’ being a success

    • Trust Assistments analysis

  • Provide guidance for remedy

  • Provide support for class or student interaction

  • Allow questions to be accessed for quizzes, exams, and practice problems


Design activity
Design Activity

Aggregate

Prototypes

Skills

Standards

&

Relationship

Remedy



Feedback on prototype
Feedback on Prototype

Assistments data is not currently something teachers would know how to use.

"I would go back and teach the part again“ (U14, U22)

"I would assign more worksheets" (U20)

Practical Needs

“What should I address next” (U14, U15)

Teachers’ sense of standards is more concrete than skills

“Have I covered all the standards”

“Where in the standards are they stuck” (U15, U20)


Feedback on prototype1
Feedback on Prototype

Individual Questions give teachers concrete idea on where the students struggle

“Like most missed question displayed” (U14, U20)

Good visual presentation important for good testing results

“this has stuff all over the place” (U15)

“What does performance mean? Does higher number mean good or bad?” (U14, U15, U20)


Design requirement doc
Design Requirement Doc

  • Evaluation of designs

  • Formation of designs

  • Support future design for client

    (support our designs)


Explain skill in context of a question

Immediately view problem areas


A new design
A New Design

  • Most importantly:

    How many of my students will pass?

  • Different tabs show further details:

    • Performance on state standards

    • Prioritized list of skills needing improvement

    • Progress

    • Questions the class missed the most

  • Could be extended with:

    • Comparison between classes

    • Skill-Standard relation

    • Remedies




Demo

  • Student progress

  • Communicating skills and standards

  • Predict a student’s score on a standardized test

  • Provide feedback to teachers about gaps in students' knowledge



ad