The future of Assessments
This presentation is the property of its rightful owner.
Sponsored Links
1 / 37

The future of Assessments Lessons learned internationally PowerPoint PPT Presentation


  • 83 Views
  • Uploaded on
  • Presentation posted in: General

The future of Assessments Lessons learned internationally. Washington, 9 March 2010 Andreas Schleicher Head, Indicators and Analysis Division OECD Directorate for Education. The future of assessments. Or the Alchemists’ Stone?. The Holy Grail?. Know w hy you are looking

Download Presentation

The future of Assessments Lessons learned internationally

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The future of assessments lessons learned internationally

The future of AssessmentsLessons learned internationally

Washington, 9 March 2010

Andreas SchleicherHead, Indicators and Analysis DivisionOECD Directorate for Education


The future of assessments

The future of assessments

Or the Alchemists’ Stone?

The Holy Grail?

  • Know why you are looking

    • You cannot improve what you cannot measure

    • The yardstick for success is no longer just improvement by national standards but the best performing education systems globally

  • Know what you are looking for

    • A new assessment culture

    • Responsive to changing skill requirements

    • Capitalising on methodological advances

    • Not sacrificing validity gains for efficiency gains

  • Know howyou will recognise it when you find it

    • Gauging predictive validity

    • Impact on improving learning and teaching

  • Implications and lessons learned .


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Cost per student

Graduate supply

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

United States

Cost per student

Finland

Graduate supply

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Australia

Finland

United Kingdom

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

Tertiary-type A graduation rate


The future of assessments lessons learned internationally

A world of change – higher education

Expenditure per student at tertiary level (USD)

United States

Australia

Finland

Tertiary-type A graduation rate


Latin america then

Latin America then…

Hanushek 2009


Latin america then and now

Latin America then and now…

Hanushek 2009


Latin america then and now1

Latin America then and now…

Why quality is the key

Hanushek 2009


Know what you are looking for

Know what you are looking for

The Holy Grail was a well-described object, and there was only one true grail…


The future of assessments lessons learned internationally

Schooling in the medieval age:

The school of the church


The future of assessments lessons learned internationally

Schooling in the industrial age:

Uniform learning


The future of assessments lessons learned internationally

Schooling in the industrial age:

Uniform learning

The challenges today:

Universal quality

Motivated and self-reliant citizens

Risk-taking entrepreneurs, converging and continuously emerging professions tied to globalising contexts and technological advance


The future of assessments lessons learned internationally

How the demand for skills has changedEconomy-wide measures of routine and non-routine task input (US)

Mean task input as percentiles of the 1960 task distribution

The dilemma of assessments:

The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource

(Levy and Murnane)


The future of assessments lessons learned internationally

  • Education today needs to prepare students…

    …to deal with more rapid change than ever before…

    …for jobs that have not yet been created…

    …using technologies that have not yet been invented…

    …to solve problems that we don’t yet know will arise

  • It’s about new…

    • Ways of thinking

      • involving creativity, critical thinking, problem-solving and decision-making

    • Ways of working

      • including communication and collaboration

    • Tools for working

      • including the capacity to recognise and exploit the potential of new technologies

    • The capacity to live in a multi-faceted world as active and responsible citizens.


Mathematics in pisa

Mathematics in PISA

The real world

The mathematical World

Making the problem amenable to mathematical treatment

A mathematical model

A model of reality

Understanding, structuring and simplifying the situation

Using relevant mathematical tools to solve the problem

A real situation

Validating the results

Mathematical results

Real results

Interpreting the mathematical results


The future of assessments lessons learned internationally

  • By whom?

  • Evaluators

  • Users of feedback

  • Evaluation agencies

National assessment systems differ…

Student assessment

Classroom

  • How? Methods and procedures, mix of criteria and instruments

  • Mapping of feedback to different units

  • For what? E.g.

  • Accountability

  • Improvement

Teacher appraisal

Who is assessed

School

School evaluation

System

System assessment

  • What?

  • Inputs

  • Processes

  • Outcomes

  • With whom?

  • Agents involved


The future of assessments lessons learned internationally

Assessment culturesPutting the pressure on top of the education system is the easy part, building capacity is harder

Participative/internal

Interactive, reflective, critical friend

Survey

Summative

Formative

Standardised assessment

Inspectorate

Administrative external


But there are global trends

…but there are global trends

  • Understanding what the assessment reveals about students’ thinking to shape better opportunities for student learning

  • Responding to assessments can enhance student learning if tasks are well crafted to incorporate principles of learning

  • Capitalise on improved data handling tools and technology connectivity to combine formative and summative assessment interpretations for a more complete picture of student learning

  • Multi-layered, coherent assessment systems from classrooms to schools to regional to national to international levels that…

    • Support improvement of learning at all levels of the education system

    • Are largely performance-based

    • Make students’ thinking visible and allow for divergent thinking

    • Are adaptable and responsive to new developments

    • Add value for teaching and learning by providing information that can be acted on by students, teachers, and administrators

    • Are part of a comprehensive and well-aligned continuum, communicate what is expected and hold relevant stakeholders accountable .

  • Integrate, synthesize and creatively apply content knowledge in novel situations

  • Activate students as owners of their own learning and activate students as learning resources for one another


Know how you will recognise it when you find it

Know how you will recognise it when you find it

The Alchemists’ stone was to be recognised by transforming ordinary metal into gold…


The future of assessments lessons learned internationally

Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada)after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1)

Odds ratioCollege entry

School marks at age 15

PISA performance at age 15


The future of assessments lessons learned internationally

Relationship between test performance and economic outcomesAnnual improved GDP from raising performance by 25 PISA points

Percent addition to GDP


Implications and lessons learned

Implications and lessons learned

The medieval Alchemists’ followed the dictates of a well-established science but that was built on wrong foundations

The search for the Holy Grail was overburdened by false clues and cryptic symbols


From assessment inhibited practice towards outcome driven reform

From assessment-inhibited practice towards outcome driven reform

Strong focus on processes

Integrated quality management

Good willand trust

Weak outcome-based management

Strong outcome-based management

External control, uninformed prescription

Deprivation

Weak focus on processes


Some criteria used in the world

Some criteria used in the world

  • Coherence

    • Built on a well-structured conceptual base—an expected learning progression—as the foundation both for large scale and classroom assessments

    • Consistency and complementarity across administrative levels of the system and across grades

  • Comprehensiveness

    • Using a range of assessment methods to ensure adequate measurement of intended constructs and measures of different grain size to serve different decision-making needs

    • Provide productive feedback, at appropriate levels of detail, to fuel accountability and improvement decisions at multiple levels

  • Continuity

    • A continuous stream of evidence that tracks the progress of both individual students .


Understanding learning progressions

Understanding learning progressions

  • Learning targets

    • Defining what mastery means for a given skill level

  • Progress variables

    • Delineate a pathway that characterise the steps that learners typically follow as they become more proficient

    • Evaluation of students reasoning in terms of the correctness of their solutions as well as in terms of their complexity, validity and precision

  • Levels of achievement

    • Describing the breadth and depth of the learner’s understanding of the domain at a particular level of advancement

  • Learning performances

    • The operational definitions of what student’s understanding would look like at each of the stages of progress .

Wilson, ATC21S


The future of assessments lessons learned internationally

OECD Level 2

OECD Level 6

  • Identifying

  • Recognising issues that can be investigated scientifically

  • Identifying keywords in a scientific investigation

  • Recognising the key features of a scientific investigation

  • Explaining

  • Applying knowledge of science in a situation

  • Describing or interpreting phenomena scientifically or predicting change

  • Using evidence

  • Interpreting scientific evidence and drawing conclusions

  • Identifying the assumptions, evidence and reasoning behind conclusions

Students can determine if

scientific measurement can be applied to a given variable in an investigation. Students can appreciate the relationship between a simple model and the phenomenon it is modelling.

Students can demonstrate ability to understand and articulate the complex modelling inherent in the design of an investigation.

  • Context

    • - Personal

    • Social/public

    • Global

  • Competencies

    • Identify scientific issues

    • Explain phenomena scientifically

    • Use scientific evidence

Students can recall an

appropriate, tangible, scientific fact applicable in a simple and straightforward context and can use it to explain or predict an outcome.

Students can draw on

a range of abstract scientific knowledge and concepts and

the relationships between these in developing explanations of

processes

  • Knowledge

  • Knowledge of science

  • Knowledge about science

  • Attitudes

  • -Interest in science

  • -Support for scientific enquiry

  • -Responsibility

Students demonstrate

ability to compare and differentiate among competing explanations by

examining supporting evidence. They can formulate arguments by synthesising evidence from multiple

sources.

Students can point to an obvious feature in a simple table in support of a given statement. They are able to recognise if a set of given characteristics apply to the function of everyday

artifacts.


Some methodological challenges

Some methodological challenges

  • Can we sufficiently distinguish the role of context from that of the underlying cognitive construct ?

  • Do new types of items that are enabled by computers and networks change the constructs that are being measured ?

  • Can we drink from the firehose of increasing data streams that arise from new assessment modes ?

  • Can we utilise new technologies and new ways of thinking of assessments to gain more information from the classroom without overwhelming the classroom with more assessments ?

  • What is the right mix of crowd wisdom and traditional validity information ?

  • How can we create assessments that are activators of students’ own learning ?

Wilson, ATC21S


The future of assessments lessons learned internationally

High policy value

A real-time assessment environment that bridges the gap between formative and summative assessment .

Quick wins

Must haves

Examine individual, institutional and systemic factors associated with performance

Extending the range of competencies through which quality is assessed

Monitor educational progress

Measuring growth in learning

Low feasibility

High feasibility

Establish the relative standing of students and schools

Assuming that every new skill domain is orthogonal to all others

Money pits

Low-hanging fruits

Low policy value


The future of assessments lessons learned internationally

Getting the sequencing right

Phases of development

Adequate  Good

Poor  Adequate

Good  Great

Main focus of assessment

  • Tackling underperformance

  • Transparency .

  • Spreading best practice

  • World class performance.

  • Continuous learning and innovation .

Role of government

  • Regulating .

  • Capacity-building

  • Prescribing .

  • Justifying

  • Enabling

  • Incentivising .

Role of professions

  • Implementing

  • Accepting evidence

  • Adopting minimum standards

  • Accommodating

  • Evidence-based

  • Adopting best . practice

  • Leading

  • Evidence-driven

  • Achieving high reliability and innovation .

  • Principled

  • Strategic partnership

  • Negotiated

  • Pragmatic .

  • Top-down

  • Antagonistic .

Nature of relationship between government and professions

  • Steady improvement

  • Growing public satisfaction .

  • Consistent quality

  • Public engagement and co-production .

Main outcomes

  • Improvement in outcomes

  • Reduction of public anxiety.


Thank you

Thank you !

www.oecd.org; www.pisa.oecd.org

All national and international publications

The complete micro-level database

email: [email protected]

Twitter: @SchleicherEDU

…and remember:

Without data, you are just another person with an opinion


  • Login