1 / 25

USES AND INTERPRETATIONS OF INTERNATIONAL ASSESSMENT DATA

USES AND INTERPRETATIONS OF INTERNATIONAL ASSESSMENT DATA. Perspectives from the international Academy of Education Invited session AERA, APRIL 6,2014. Participants. The uses of PISA in Mexico Maria de Ibarrola and Sylvia Schmelkes, Mexico

kaida
Download Presentation

USES AND INTERPRETATIONS OF INTERNATIONAL ASSESSMENT DATA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. USES AND INTERPRETATIONS OF INTERNATIONAL ASSESSMENT DATA Perspectives from the international Academy of Education Invited session AERA, APRIL 6,2014

  2. Participants • The uses of PISA in Mexico • Maria de Ibarrola and Sylvia Schmelkes, Mexico • Validity of Inferences from International Assessment: Cautions about two typical uses • Kadriye Ercican, Canada • Using International Assessment to Investigate Cognitive Performance Differentials by Socioeconomic Status • Servaas Van de Berg, South Africa • Measuring College Value Added: a delicate instrument • Richard J. Shavelson,USA • Collaborative Problem Solving in Different National Settings • Patrick Griffin, Australia

  3. The uses of PISA* in Mexico. 2013 radical reforms for improving quality of education Maria de Ibarrola Department of Educational Research Center for Research and Advanced Studies, Mexico Sylvia Schmelkes President, National Institute of Educational Evaluation, Mexico AERA Invited Session for the International Academy of Education Uses and Interpretations of International Assessment Data: Perspectives from the International Academy of Education (IAE) Philadelphia, PA, USA April 6 2014 *Program for International Student Assessment

  4. Purposes of thistalk • Reportthedramaticresults of PISA in Mexico • Describe some uses of PISA in educationalpolicydecisionmakingforimprovingquality (or at leastforimproving PISA results) • Enumeratethe radical educationalreformproposed in 2013 (and itsemphasisonteachers) • Giveanaccount of theseriouslack of consensusonthemeasurestaken • (Thispresentationwillnotdealwith PISA psychometric, statisticalorinternalvalidity)

  5. Mexicoamong OECD countries • Mexicobecamean OECD country member in 1994 • Part of Nationalpoliciesforthe “modernization” of the country: ( signing NAFTA alsoforinstance) . • A hugereform in 1989 forthemodernization of education • (At aboutthesame dates, the Zapatista Movement of Liberationexploded in the South of the country, remindingus of another, and manyothers, Mexicos) • As an OECD contrymember, Mexico’sfirstparticipation in PISA was in 2000: (Reading) • Regular participationafterthateverythreeyears: • 2003: Mathematics • 2006 ( Sciences) • 2009 ( Reading) • 2012 ( Mathematics )

  6. Among OECD countries, Mexico… • Has thelowestincome per capita: 6400 US doll / vs a mean of 23 100 • Isthesecond country withincreasinginequalityamongsitspopulation: 10 % of therichest sector of thepopulationis 29 times richerthanthelowest 10% • Spendsonly a third ( 7. 4% of GNP) in social expenses as against21.9% as a mean forall OCDE countries • Spends half( USD5 000 or less) per child per year from primary through tertiary education in 2010 against OECD countries USD 9 300 Schoolingforchildren of Migrantdailylaborers: poorestamongpoors

  7. A synthesis of results, PISA 2012

  8. The PISA shock! • After 9 years of schooling, 52 % of 15 yearoldstudentsdo nothavetheminimumcompetencesfortheKnowledgeSociety (northeConstitutionaleducationalobjectivesmandated ). • Onlyabout 4% havethedesiredcompetences • National Educational Research has tried to make public awareness of the poor educational resources and the poor educational outcomes since the 1960’s; highly critical of educational policies • New awareness, nowbased in a “solid, scientific, internationalevidenceof students’ achievements” that has interestedmanyother social sectors in theeducationalproblems of the country

  9. Badstudents and boggeddown!

  10. Lastamong OECD countries 8/10 studentsunderachievers 0.1% excellent in Sciences 77 years to achieve OECD mean, a 149 years to get the first place!

  11. Thesearchfor causes and explanations • Mexico’sdevelopment status: “notyet a developednation” • Poorsocioeconomicconditions of moststudents, and deep cultural differences • Small proportion of GNP destinedtoeducation, and unequallydistributedamongmodalities and levels • BadTeacherworkingconditions, poorinitial and continuous training, largeclasssize, etc • 27% of schoolswithoutwater; 42% withoutsewage; 20% schoolswithoutdesk and chairsforteachers • Poor eduational governance and administration at school, local, regional, State, and National level

  12. USES of PISA in Mexico, as proposedby OCDE exploration Progressive acceptance of PISA as a valid and reliable indicator of the quality of education, in spite of the explanations, (because they are taken into account in the construction of the instrument) • Changes in educational policies and practices in light of PISA results YES • Use of PISA policy findings in national/federal education debate and policy-making YES • The role of PISA in national/federal assessment and evaluation policies and practices YES • Setting and revising curriculum standards YES • Setting and monitoring performance targets and indicators YES • Breakspear, S. (2012), “The Policy Impact of PISA: AnExploration of the Normative Effects of InternationalBenchmarking in School System Performance”, OECDEducation Working Papers, No. 71, OECD Publishing. http://dx.doi.org/10.1787/5k9fdfqffr28-en

  13. Setting and monitoring performance targets and indicators • Objective 1 of the Education Sector Program 2007-2012 establishes as a goal for 2012 to raise performance on PISA to 435 points as an average for both mathematics and reading, taking the average score of 392 points, attained in PISA 2006, as the base. • The objective was not reached (415 points), • some propose a new target, • others warn against the use of this kind of targets as substitute to an integral education

  14. Setting and revising curriculum standards • In 2011 a hugecurriculumreformwaslaunchedintegrating K-12. • “The Agreement for the Articulation of Basic Education asserts that the whole curriculum should set a vision for 2021 that includes generalizing the competencies described at PISA Level 3; eliminating the gap between the students who perform below Level 2 and those who perform at orabovethatlevel”. • As of 2013 thereformiscritcized as a huge and monumental disasterfor Basic education, a new basiccurriculum debate has beencalled at a Nationallevel . • Whatistoday’s Basic Education: knowledge, abilities and values • Whatistheprofile of a basiceducationgraduate • Whatshouldbethe curricular structure of basiceducation

  15. Led to the formation, or increased the scope, of a national/federal assessment system

  16. Unreliability of ENLACE(Studentswithbestresults) ENLACE waslinkedtoteacher incentives since 2006 Alwayscriticized as foritsvalidity And was suspended in 2014 Weakness of the instrument? Corruption? Fraud?

  17. PrivatesectorsDefend ENLACE

  18. Light improvements in PISA results • Between 2003 and 2012, Mexico’sschoolingforyoungpeopleincreasedfrom 55% tocover 75% of theagegroup, butthe country isstillfarfromreachingthedesiredcoveragerate of a 12 yearcompulsoryeducation. • Between 2003 and 2012, Mexicanstudentsincreasedthepointsachieved in PISA results; from 385 to 413 • And Finlandlost a highernumber • WHY?? regressiontothe mean? Weakness of theinstrument?

  19. 15 yearoldcoverage and PISA evaluationcycles 69.6 66.2 62.9 58.1 51.6

  20. Meaningfuldifferences2003-2012 in the mean performance in mathematicsby country

  21. Lookingforthekeyfactorstoimprovequality • Socioeconomicconditions of the country? • Inequality has increased! • EducationalBudget? • Expenditure per studentisthelowest and has notimproved • Hugedifferences in expenditurebyregion and schoolmodality: provisional measuresbecame regular pooreducationalsystemsforpoorsectors • ImprovingInfrastructure? • Severalprogramstoimproveschoolinfrastructure • Severaleffortsto introduce ICTs in schools and USE themforpedagogicalmeans • Butrecentcensus ( 2014) revealsseriousproblems • Schoolautonomy ? • Severalefforts and normsfordecentralizingschooloperation up toschoollevel • Butteacher labor conditions do notfostercollegiatework

  22. ItistheTeachers…!!!. • “What determines schoolprogressisthequality of theteachingbody! • “Theteachermakesthedifference. And ifhis/hers performance takes place within a context of clear and ambitiousobjectives, ( highstandards) accompaniedbymeasuretools, training and informatintoadjustthesystem, thedifferenceisstillbetter”. • (PISA 2012, Mexico. Mtra. Gabriela Ramos, Directora de Gabinete y Sherpa, Dec. 2013) • Thesecrets of Finlandsuccess: expertteachers: • Highlyvalued; rigourouslyselected; verywelltrained; widedefinition of functions/adequateworking time; complete pedagogialfreedom; associatedtoUniversities; continuoustrained; regularlyevaluated…

  23. The 2013 Reform: evaluationisthename of thegame • Constitutionalamendments and a New General Law of Education: a fundamental righttoreceive QUALITY EDUCATION • Full AutonomytotheNationalInstitute of EducationalEvaluation (INEE) • The role of INEE: formativeevaluation! • The INEE isthe MAIN authority in evaluationmatters: 19 General Directoratestofulfillalltheresponsibilitiesassignedbylaw • Law of Teaching as a professionalservice • Evaluation of teachersforentry, promotion and permanence, as indicatedby INEE • Radical change in theactors and thecriteriathatsustainthosedecisions: theUnionisout • A system of reliableinformationonteachers: Nationalcensus (March 2014) • After a century of building a Nationaleducationsystemwegettoknowhowmanyteachers are there in thepayroll. ( and 13% are notaccountedfor)

  24. Teachers’ evaluation. A passionate debate • Educationalauthoritiesambiguousgame: • A callforNational and Regional proposalson a “new educationalmodel” • But ...Drawing up laws at the highest level and Normative policy instruments at high speed • TheVerystrongNationalUnion of Teachers: evaluationisacceptedbut“Labor rightshavetoberespected” • Radical Union’ssections in pooreststatesviolentlyopposethelaws: stoppages, strikes, highway and streetblockades, • Strongprivate sector NGOs pressing forteacherevaluation and highstakeresults: Teachers can befired !and some (many?) of themshouldbe. • Regular Teachersoffendedbyanevaluationthatdoesnotconsidertheirdailyteachingconditions and do notrecognizetheirefforts and contributionsforimprovement • Scholars and Researchers: tryingtomake a sound and objectiveevaluation of the 2013 reforms and itseffects: isit a labor reformoraneducationalproposal?

  25. Evaluationistheinformation, notthesolution. • Deep and seriouslack of consensusonwhatisquality of education and howtoachieveit • withinanextremelydiverse and unequalsociety • Uncompletereforms, continuouschanges, ambiguousmeasures, broken links amongthemanylayers of implementation • Serious gaps and contradictionsbetweendiscourses, laws, budget and dailyconditions • Allsorts of active and passiveoppositioncomingfromdifferentsectors • Educational Research (National and International) has a hugebibliographyonhowcomplexEducationalchange and improvements are. • Sorry, Wehavenotachievedtheproper formula

More Related