1 / 25

Study Overview

Study Overview. National Research Coordinators Meeting Amsterdam, October 16-19. IEA studies in the field. 1971: First IEA Civic Education Study in 10 countries (part of six-subject study) 1999/2000: IEA CIVED Assessment of 14-year-olds in 28 countries

raanan
Download Presentation

Study Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Study Overview National Research Coordinators Meeting Amsterdam, October 16-19

  2. IEA studies in the field • 1971: First IEA Civic Education Study in 10 countries (part of six-subject study) • 1999/2000: IEA CIVED • Assessment of 14-year-olds in 28 countries • Assessment of upper secondary students in 16 countries • 2008/2009: IEA ICCS (more than 40 expressions of interest so far)

  3. International Study Centre and partner institutions - 1 • Australian Council for Educational Research (ACER), Melbourne/Australia • International Study Centre • Overall Project management • Test Development • Student Background Questionnaire • Scaling and Data analysis • John Ainley (Project Coordinator) • Wolfram Schulz (Research Director)

  4. International Study Centre and partner institutions - 2 • National Foundation for Educational Research (NFER), Slough/UK • National Contexts Survey • Student Perceptions Questionnaire • European Regional Module • David Kerr (Associate Research Director)

  5. International Study Centre and partner institutions - 3 • Laboratorio di Pedagogia Sperimentale (LPS), Rome/Italy • School Questionnaire • Teacher Questionnaire • Bruno Losito (Associate Research Director)

  6. International Study Centre and partner institutions - 4 • IEA Data Processing Center (DPC), Hamburg/Germany • Sampling and Weighting • Data Processing • Scaling and Analysis • Falk Brese (Coordinator) • Heiko Sibberns (TEG and PAC representative)

  7. International Study Centre and partner institutions - 5 • IEA Secretariat, Amsterdam/Netherlands • Translation Verification • Quality Monitoring • Barbara Malak (Manager Membership Relations) • Hans Wagemaker (Executive Director)

  8. Project Management • Joint Management Committee (JMC) • Meets frequently (≥ monthly) with consortium staff • Meets occasionally as a JMC • Project Advisory Committee (PAC) • 1st meeting July 2006 • 2nd meeting October 2006 • IEA Secretariat • IEA Technical Expert Group (TEG)

  9. Purposes of study • International comparison of outcomes of civic and citizenship education at start of 21st century • Provide trend data for comparison with CIVED 1999 • Response to challenge in civic and citizenship education in changed contexts of democracy and participation

  10. ICCS and CIVED • Opportunity to review changes • Extended framework and broadening of scope • Different approach to teacher survey • Regional modules

  11. Research questions directed at exploration of • Variations in student achievement • Changes since 1999 • Extent of interest and dispositions towards citizenship • Student perceptions of recent threats and future developments • Relationship of learning outcomes with curriculum, teaching and school organisation • Relationship of learning outcomes with student background

  12. Assessment Framework • Outlines what will be assessed in ICCS • Consists of two parts: • Civics and Citizenship framework • Contextual framework • Current version to be discussed at this meeting

  13. Instrumentation • Student instruments comprising: • International (core) cognitive test (60 minutes) • International (core) student background questions (15 minutes) • International (core) student perceptions on civic-related issues (30 minutes) • Regional cognitive test module(s) (20-30 minutes) • Regional student questionnaire on civic-related issues (10-20 minutes) • Teacher questionnaire (30 minutes) • School questionnaire (30 minutes) • National Contexts Survey (on-line)

  14. Instrument development • Establishing assessment framework • Piloting of items in subset of participating countries (Feb-Mar 07) • Review with experts and NRCs • Field Trial of preliminary set of instruments (Oct-Nov 07) • Review with experts and NRCs • Final selection (Feb-Apr 08)

  15. General survey design • Population: Grade 8 (minimum average age of 13.5) • Two-stage cluster samples in participating countries • PPS sampling of (at least 150) schools • Random selection of one classroom within each sampled school • Random selection of teachers within sampled schools (no student-teacher link) teaching grade 8

  16. Instrument administration • Test administrators responsible for test and questionnaire sessions • Student instrument administration: • International cognitive test • International background questionnaire • International perceptions questionnaire • Regional Cognitive test (optional) • Regional perceptions questionnaire (optional)

  17. Translation and verification • Instruments translated from (English) source version into national languages • National adaptations reviewed in cooperation with international study centre • Translations verified by language experts • Final layout review

  18. Field Operations • Manuals will be developed to guide operations undertaken by national centres (coordination at school, test administration, coding, data entry) • School and teacher questionnaires administered through school coordinators • International control monitors visiting sub-samples of schools

  19. Data Management and Scaling • DPC creates international database (cleaning procedures) • DPC computes sampling weights • DPC provides item statistics • DPC and ACER undertake scaling of data • IRT scaling model still to be decided (1-PL vs. 3-PL) • Plausible values for cognitive measures • IRT weighted likelihood scores for some questionnaire-based indices

  20. Analysis and Reporting • (Probably) Jackknife Repeated Replication (JRR) to provide correct standard errors • Described scales (cognitive and some affective-behavioural) • Descriptive tables with uni- and bivariate analyses • Single and multi-level analyses

  21. Reports • International Report (June 2010) • Regional Report(s) (June 2010) • Technical Report (Nov 2010) • Instrument development • Field Operations • Sampling • Scaling • Reporting

  22. International Database • Student, teacher and school databases including: • ID variables • Item responses • Cognitive Scale(s) – Plausible Values • Affective-Behavioural Scales • Weights • JRR Indicators

  23. Assistance to NRCs • Item development workshop (this meeting) • Data Management and Marker Training seminars (FT and MS) • International Database Seminar (Nov 2010)

  24. Timeline

  25. Questions?

More Related