Qatar Comprehensive Educational Assessment QCEA 2008: Discussion Session For QCEA Support

1. Qatar Comprehensive Educational Assessment (QCEA) 2008: Discussion Session For QCEA Support

2. 2 Overview of the QCEA program QCEA program: Measures student performance with respect to the Qatar Curriculum Standards Is an indicator for the success of the reform Assesses Arabic, English, Mathematics, and Science learning in Grades 4 to11 Assesses the success of the implementation of the Qatari curriculum standards Starting from 2007 QCEA has been applied only in Independent Schools Administered to approximately 18,000 students, attending Qatar independent schools, in the Spring of 2007 According to 2008 enumeration, it will be administered to approximately 24,000 students.

3. 3 How is QCEA Prepared? QCEA questions are written by contracting companies ETS and CTB under the supervision and control of the Students Assessment Office (SAO, Evaluation Institute) QCEA assesses the highlighted standards set by the Curriculum Standards Office ( Education Institute) Question are reviewed by the four content area experts of the SAO along with a group of highly experienced Qatari teachers from Independent Schools, trained in workshops organized by SAO. Only questions of high quality standards should be retained. Cultural sensitivity and language correctness are also taken into account. In the context of localizing the whole operation, item writing workshops have been held in the SEC in order to train local teachers to write QCEA questions aligned to the Qatari Curriculum Standards. Three workshops have been organized since January 2007 where 100 teachers have been trained

4. 4 How is scoring carried out? The scoring operation has been 100% localized since 2006. The SAO is in charge of the scoring in cooperation with ETS and CTB. Leadership positions are assigned to highly qualified nationals who are trained on Online Scoring Network operations ( OSN) and electronic hand scoring (EHS). Multiple choice questions are scored by the computer. Constructed questions are scored using OSN and EHS systems by trained teachers. Daily calibrations are carried out to avoid discrepancies.

5. 5 Better Understanding of the QCEA 2007 Results Performance levels and scale scores: Each student receives a scale score in each content area Scale scores are also converted into performance levels thus indicating whether a student has met the standards Five performance levels are considered, with three of those embedded within the “Below Standards” description for the sake of better discrimination among the poorer performers: “Meets Standards” “Approaches Standards” “Below Standards” (levels 1–3), level3, being the highest About performance levels Students who receive the qualification “Meets Standards” (level 5) have demonstrated that they are proficient in understanding and applying the skills and concepts included in the curriculum standards for that grade level. Students who receive the qualification “Approaches Standards” (level 4) are just beginning to approach proficiency. They need to continue to work on understanding and applying the skills and concepts contained in the curriculum standards. Students who receive the qualification “Below Standards” (levels 1–3) have not yet acquired sufficient knowledge of the skills and concepts contained in the curriculum standards for that grade level. They need help in understanding the concepts, and they need to practice applying their knowledge to solve problems and answer questions based on the curriculum standards. For the purposes of tracking student performance more closely, the “below standards” category has been divided into three levels, as described below: · Level 3: he/she may reach the “Approaches Standards” level with some additional effort · Level 2: he/she may reach the “Approaches Standards” level with considerable additional effort · Level 1: he/she may reach the “Approaches Standards” level with extensive additional effort About performance levels Students who receive the qualification “Meets Standards” (level 5) have demonstrated that they are proficient in understanding and applying the skills and concepts included in the curriculum standards for that grade level. Students who receive the qualification “Approaches Standards” (level 4) are just beginning to approach proficiency. They need to continue to work on understanding and applying the skills and concepts contained in the curriculum standards. Students who receive the qualification “Below Standards” (levels 1–3) have not yet acquired sufficient knowledge of the skills and concepts contained in the curriculum standards for that grade level. They need help in understanding the concepts, and they need to practice applying their knowledge to solve problems and answer questions based on the curriculum standards. For the purposes of tracking student performance more closely, the “below standards” category has been divided into three levels, as described below: · Level 3: he/she may reach the “Approaches Standards” level with some additional effort · Level 2: he/she may reach the “Approaches Standards” level with considerable additional effort · Level 1: he/she may reach the “Approaches Standards” level with extensive additional effort

6. 6 Summary of Results: QCEA 2007Performance level results across subject-matter: overall and gender Subject-matter Although scale score comparisons among subject- matters are not valid, performance level proportions can be contrasted. In this respect students tended to perform better on the Arabic and English tests than on the mathematics and science instruments A larger proportion of students scored in the “Meets Standards” performance level in Arabic and English, as compared to mathematics and science. A smaller proportion of students scored in the “Below Standards: Level 1” in Arabic and English, as compared to mathematics and science. Gender Girls tended to perform better than boys in all subjects.

7. 7 How Were the Results in General The results did not meet expectations based on the efforts deployed by the Evaluation Institute, the Education Institute and the schools. They did not reflect the actual level of the students. One of the most obvious reasons is the absentee rate. Students either : Did not attend at all Did not make a serious effort Did not answer ( blank papers)

8. 8 Absentee Rate 2007

9. 9 Important Consequences If results do not accurately reflect students’ real level, it impacts: Assumptions, analysis, decisions, orientations for improvement and solutions. Efforts and investments. National and international results/ranking.

10. 10 In order to make the QCEA a Success 1) On the Student Assessment Level: Further awareness campaigns and more interactions with schools Further coordination with related offices in the Education Institute and more focus on common goals 2) On the School Level: Better understanding of QCEA objectives and no more considering it as overburdening schools’ schedule Making use of the morning assemblies a week at least prior to QCEA assessments to motivate students Further interactions with parents for better involvement and commitment.

11. 11 In order to make the QCEA a Success (2) The Test is not designed to be a fail/pass tool, so it cannot be used for such a purpose in order to motivate students. Therefore, In the absence of the self assessment culture and competitive spirit, a compensation strategy needs to be used ( familiar to students): Awards from the school Top achieving students’ names posted on the honor board in all Independent Schools Acknowledgement on the SEC Day of Excellence Ceremony.

12. 12 Further suggestions