Loading in 2 Seconds...

The Design Phase: Using Evidence-Centered Assessment Design

Loading in 2 Seconds...

- 181 Views
- Uploaded on

Download Presentation
## The Design Phase: Using Evidence-Centered Assessment Design

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### The Design Phase:Using Evidence-Centered Assessment Design

Monty Python argument

Workshop Flow

- The construct of MKT
- Gain familiarity with the construct of MKT
- Examine available MKT instruments in the field
- Assessment Design
- Gain familiarity with the Evidence-Centered Design approach
- Begin to design a framework for your own assessment
- Assessment Development
- Begin to create your own assessment items in line with your framework
- Assessment Validation
- Learn basic tools for how to refine and validate an assessment
- Plan next steps for using assessments

What is Evidence-Centered Design?

- Approach developed by ETS
- Uses “evidentiary reasoning” to design the underlying principles of an assessment
- Answers these questions:
- What complex of KSAs should be assessed?
- What types of evidence would we need to show a test-taker has these KSAs?
- What kinds of assessment items would allow us to gather this evidence?
- Provides a set of tools for structuring a systematic approach to doing this

Phases of ECD

- Domain Analysis

Big-picture narrative of the knowledge domain

- Domain Modeling

Organization of the information in the domain in terms of:

- The aspect of proficiency of the test-taker in the domain
- The kinds of things the test-taker might do to provide evidence of their proficiency
- The kinds of situations that might make it possible to provide such evidence
- Conceptual Assessment Framework

Blueprint for the actual assessment that takes all of this into account

- The Assessment

Phases of ECD for Assessing MKT

- Domain Analysis of MKT

Big-picture narrative of the knowledge domain

Rate of change

MKT

- Interpreting unconventional forms or representations
- Choosing problems and examples that can illustrate key curricular ideas
- Differentiating between colloquial and mathematical uses of language
- Linking precise aspects of representations
- Understanding implications of models and representations
- Evaluating mathematical statements

y=ax+b

Proportionality

Phases of ECD for Assessing MKT

- Domain Modeling

Organization of the information in the domain in terms of:

- The aspect of proficiency of the test-taker in the domain
- The kinds of things the test-taker might do to provide evidence of their proficiency
- The kinds of situations that might make it possible to provide such evidence

Create a “Design Pattern”

Phases of ECD for Assessing MKT

- Conceptual Assessment Framework

Blueprint for the actual assessment that takes all of this into account

Types of Relationships to Curriculum/PD

Assessment

Development

Curriculum/PD

Development

Assessment

Development

Curriculum/PD

Development

Assessment

Development

Curriculum/PD

Development

Adapting Off-the-Shelf Instruments

- Start with this process for your own assessment needs
- Evaluate the fit of the instrument for your needs
- Important: Do not simply mix and match!

Examples of Design Patterns

- MKT for SimCalc
- Co-construct as a group
- Do your own

Foundationsof proportionality

Find the missing

number

Consider the function

Proportionality includes: linearity, rate, function, slope in graphs, interpreting tables with an underlying rate

NCTM Curriculum Focal Points

Students graph proportional relationships and identify

the unit rate as the slope of the related line.

They distinguish proportional relationships

(y/x = k, or y = kx) from other relationships,

including inverse proportionality (xy = k, or y = k/x).

Grade 7: Developing an understanding of and applying proportionality, including similarity.

Students extend their work with ratios to develop an understanding of proportionality that they apply to solve single and multistep problems in numerous contexts. They use ratio and proportionality to solve a wide variety of percent problems, including problems involving discounts, interest, taxes, tips, and percent increase or decrease. They also solve problems about similar objects (including figures) by using scale factors that relate corresponding lengths of the objects or by using the fact that relationships of lengths within an object are preserved in similar objects. Students graph proportional relationships and identify the unit rate as the slope of the related line. They distinguish proportional relationships (y/x = k, or y = kx) from other relationships, including inverse proportionality (xy = k, or y = k/x).

Grade 8: Analyzing and representing linear functions and solving linear equations and systems of linear equations.

Students use linear functions, linear equations, and systems of linear equations to represent, analyze, and solve a variety of problems. They recognize a proportion (y/x = k, or y = kx) as a special case of a linear equation of the form y = mx + b, understanding that the constant of proportionality (k) is the slope and the resulting graph is a line through the origin. Students understand that the slope (m) of a line is a constant rate of change, so if the input, or x-coordinate, changes by a specific amount, a, the output, or y-coordinate, changes by the amount ma. Students translate among verbal, tabular, graphical, and algebraic representations of functions (recognizing that tabular and graphical representations are usually only partial representations), and they describe how such aspects of a function as slope and y-intercept appear in different representations. Students solve systems of two linear equations in two variables and relate the systems to pairs of lines that intersect, are parallel, or are the same line, in the plane. Students use linear equations, systems of linear equations, linear functions, and their understanding of the slope of a line to analyze situations and solve problems.

They recognize a proportion as a special case of a

linear equation of the form y = mx + b, understanding

that the constant of proportionality (k) is the slope

and the resulting graph is a line through the origin.

Students translate among verbal, tabular, graphical,

and algebraic representations of functions.

KSAs

- Common Content Knowledge
- Proportionality concepts
- Specialized Content Knowledge
- Interpreting unconventional forms or representations
- Choosing problems and examples that can illustrate key curricular ideas
- Differentiating between colloquial and mathematical uses of language
- Linking precise aspects of representations
- Understanding implications of models and representations
- Evaluating mathematical statements

Workshop Flow

- The construct of MKT
- Gain familiarity with the construct of MKT
- Examine available MKT instruments in the field
- Assessment Design
- Gain familiarity with the Evidence-Centered Design approach
- Begin to design a framework for your own assessment
- Assessment Development
- Begin to create your own assessment items in line with your framework
- Assessment Validation
- Learn basic tools for how to refine and validate an assessment
- Plan next steps for using assessments

Activity #2Create a Design Pattern

- Find Activity #2 in your binder
- Pick a design pattern topic
- Work on your own, with a partner, or with a small group
- Complete the Design Pattern form
- Feedback / Review Process
- Discussion to follow
- Show-and-tell of 2 or 3 Design Patterns
- Your insights, questions, challenges

Some Useful References(all available on the web)

- Baxter, G. P., & Mislevy, R. J. (2005). The Case for an Integrated Design Framework for Assessing Science Inquiry. PADI Technical Report 5. Menlo Park, CA: SRI International.
- Embretson, S. E. (Ed) (1985). Test Design: Developments in psychology and psychometrics. New York: Academic Press, Inc.
- Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to Evidence-Centered Design. CRESST Technical Paper Series. Los Angeles, CA: CRESST.
- Mislevy, R. J, Hamel, L. et al. (2003). Design Patterns for Assessing Science Inquiry. PADI Technical Report 1. Menlo Park, CA: SRI International.
- PADI Website: http://padi.sri.com

Feedback

- What worked?
- What didn’t work?
- What do you still want to learn?

Q1. Definition of MKT

“By mathematical knowledge for teaching, we mean the mathematical knowledge used to carry out the work of teaching mathematics. Examples of this “work of teaching” include explaining terms and concepts to students, interpreting students’ statements and solutions, judging and correcting textbook treatments of particular topics, using representations accurately in the classroom, and effects of teachers’ mathematical knowledge on student achievement providing students with examples of mathematical concepts, algorithms, or proofs” (Hill, Rowan, & Ball, 2005).

“We look across mathematical knowledge needed for or used in teaching, including “pure” content knowledge as taught in secondary, undergraduate, or graduate mathematics courses; pedagogical content knowledge and curricular knowledge (also described by Shulman) both possibly taught in mathematics methods courses; and what is more elusive, knowledge that, while also mathematical, is not typically taught in undergraduate mathematics courses and is not be entirely pedagogical. Mathematical knowledge for teaching, keeping the emphasis on mathematics and acknowledging that teachers may know and use mathematics that is different from what is required for other professions” (Ferrini-Mundy, et. al., 2008)

Question 2

Teacher’s attitudes about being evaluated

Q3. Adapting Off-the-Shelf Instruments

- Start with this process for your own assessment needs
- Evaluate the fit of the instrument for your needs
- Important: Do not simply mix and match!

Download Presentation

Connecting to Server..