1 / 61

Building a Comprehensive Assessment System Edward Roeber

Building a Comprehensive Assessment System Edward Roeber Office of Educational Assessment and Accountability. This is the time of assessment…. . This is the era of standards-based reform, based on student assessment information Federal laws require much assessment at the state level

bosticf
Download Presentation

Building a Comprehensive Assessment System Edward Roeber

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Building a Comprehensive Assessment System Edward Roeber Office of Educational Assessment and Accountability 1

  2. This is the time of assessment…. • This is the era of standards-based reform, based on student assessment information • Federal laws require much assessment at the state level • District response … more assessment! • Using assessment information is now “high stakes” - school survival now depends on it • Yet, educators are not always prepared to develop assessments nor use the information provided by them 2

  3. Added Assessments • New assessments will add even more external pressures • Michigan Merit Examination • End-of-course examinations • With added state level assessment, as well as district-level assessments, our assessment systems may be out of balance - too much external testing, too little classroom-based assessment 3

  4. What’s Needed • We need a balanced assessment system - one that honors the work in classroom assessment and is aligned to the state and district assessments • A variety of approaches to assessment are needed - perhaps more assessment, but of different kinds • All assessments needs to work in the same direction - a coordinated system of assessment 4

  5. What is a Balanced Assessment System? • Formative • Supports on-going learning in the classroom • Interim • Provides periodic snapshots of student achievement at the district level • Summative • Assesses student learning at the conclusion of a phase of learning • All parts work together and are equally important 5

  6. Balanced Assessment Systems • Are all elements of the assessment system coordinated? Are all three types present? • Formative • Interim • Summative • Are the content and skills measured compatible? • Are standards, curriculum, instruction, learning, assessment and professional development aligned 6

  7. Balanced Assessment Systems • Is assessment continuous - do the assessments measure student progress over time (days, weeks, months, school years, and school careers)? • Are a variety of formal and less formal assessment methods used on an on-going basis? • Is assessment integrated - does the assessment system fit in the larger educational system? • Is assessment seamlessly integrated into instruction? 7

  8. Balanced Assessment Systems • Are a variety of high quality assessments being used? • Do all assessments meet professional standards (reliability, validity, and so forth)? • Are the assessment purposes clearly spelled out and are assessments used that can achieve these purposes? • Do all students participate in assessments appropriate for them? • Do assessments have positive consequences - for students, educators, parents, and others? 8

  9. What are Summative Assessments? • State Level • Michigan Educational Assessment Program • Michigan Merit Examination • MI-Access • English Language Proficiency Assessment • Secondary Credit Assessments? • District Level 9

  10. Uses of Summative Assessments • District • Prioritize schools for improvement assistance efforts • Provide data for the local school board to understand student and school achievement • Report academic progress of the district to parents and other taxpayers, including key community decision-makers • Increase community support for schools and for school improvement efforts • Key Question: Did students learn what they should have? 10

  11. Uses of Summative Assessment • State • Determine priority schools for the interventions required by Federal (and state) law • Guide state-level assistance to “high priority” schools - those continuing not to make adequate yearly progress • Fund schools - could be general state aid and/or special programs to impact struggling schools • Allocate human resources - such as providing staff or consultants to provide supportive management advice and strategies to schools 11

  12. What are Interim Assessments? • Periodic assessments given school- or district- wide at fixed times during the school year • Measure the school’s or district’s curricula in important content areas • Provide evidence that all students have been taught key skills • Serve to reinforce a common curriculum and sets of learning experiences 12

  13. Interim Assessments • District-constructed assessments, or • Commercially-available assessments • Replicas of the MEAP assessments • Pacing assessments that follow the curriculum • Assessments that do not follow MEAP nor the district curriculum • Key Question: Are students in each school on track for proficiency? 13

  14. Interim Assessments • Examples • Quarterly tests based on the instructional sequence(s) used to measure the pacing guides used instructionally • Advantages • May determine if students are learning the enabling skills • Might catch students who are experiencing difficulties in learning before they fall far behind • Challenges • Instructional sequences are not always universal - are multiple forms of the assessments needed? • Are the assessments of high quality and/or match instruction? 14

  15. Uses of Interim Assessment • Increase instructional alignment - horizontal and vertical • Inform parents of all students about the level of student achievement and improvement efforts under way • Assure consistency of instruction across schools and the district • Program evaluation - evaluate the effectiveness of new initiatives 15

  16. What is Formative Assessment? • Classroom-based assessments used on an on-going basis in every classroom • Carefully-thought-out strategies to engage students in learning in and outside of the classroom • May encourage different ways of learning - moving from passive to active student learning • May encourage student self-assessment/self-monitoring 16

  17. Formative Assessment • Teachers are engaged actively in the assessment process by: • Sharing instructional targets with students • Questioning students • Observing students • Examining and evaluating student work • Providing feedback to students • Determining instructional implications of the assessments • Conferencing with students, other teachers and parents 17

  18. Formative Assessments • Students are actively engaged in the assessment process by: • Learning what is expected of them • Taking responsibility for their own learning • Actively participating in the learning process • Participating in how students document what they have learned and how they learned it • Demonstrating their learning to peers, educators and their parents 18

  19. Uses of Formative Assessment • Guide student learning on a daily basis by providing information about what critical skills were and were not learned • Provide extra learning opportunities to students who are struggling academically • Provide additional learning opportunities for students who are doing well academically • Report student progress to students, parents, and other educators • Key Question: Has each student learned? 19

  20. Characteristics of Both Large-Scale and Classroom Assessments • Shared model of student learning • Should include learning progressions over time • Shared concept of disciplinary knowledge and competence • Focus assessment on the most valued knowledge and skills • Focus on understanding and reasoning, not recall • Assess enabling skills and procedural knowledge • Signal to teachers and students what is most important • Base assessment on clearly written standards 20

  21. Characteristics of Both Large-Scale and Classroom Assessments • Measure a manageable body of knowledge and a limited number of the most important skills • Target general forms of cognition • problem solving and inductive reasoning • skills that are more domain-specific • Move from single-answer assessments towards greater use of tasks • content based • measure rich and well-structured knowledge • are open to multiple approaches 21

  22. Characteristics of Both Large-Scale and Classroom Assessments • Designed to be valid and useful to support decisions (large-scale and classroom) • Are technically sound and timely • Are designed in accordance with the purpose for which results will be used 22

  23. Characteristics of Both Large-Scale and Classroom Assessments • Measure the knowledge and skills they purport to measure • Are designed to be reliable, valid, and fair for the inferences that will be drawn from results • Report results in enough detail to reveal needed instructional changes and highlight deficiencies in the system or in instruction • Focus on student learning in school, rather than what students have learned outside of school 23

  24. Characteristics of Both Large-Scale and Classroom Assessments • Provide opportunities for students of different backgrounds to connect their knowledge to relevant school expectations. • A range of measurement approaches are used to provide a variety and range of evidence of student achievement • Provide multiple opportunities for students to demonstrate competence • Look at differences in how students perform in different assessments as rich data about students, not “error” 24

  25. Unique Characteristics of Large-Scale Assessment • Provide comparative data (normative or standards-based) • judge the adequacy of performance • determine specific curricular/instructional areas for improvement • Provide quality feedback to educators about patterns of errors that could be target for instruction in the future 25

  26. Unique Characteristics of Large-Scale Assessment • Must be cost-effective and feasible • instructional uses must be worth the time taken from instruction • Results must be reported to stakeholders 26

  27. Unique Characteristics of Classroom Assessments • Must be on-going and integrated seamlessly into instruction so teachers and students receive frequent feedback on their progress • Assess some desired proficiencies that cannot be effectively assessed on large-scale assessments • Provide quality ongoing feedback to teachers about patterns of errors that could indicate the need for modification of future instructional strategies 27

  28. Unique Characteristics of Classroom Assessments • Help teachers to identify and reconstruct students’ misconceptions • Provide quality feedback to students about their performance and specific guidance about how to improve • Help students learn how to assess their current levels of understanding in relation to learning goals; learn to recognize and produce quality work 28

  29. What are ways we can assess students? • Selected-response tests • Forced-choice - multiple-choice, true-false, matching • Written response - short-answer, extended response • Observation • Structured • Unstructured • Samples of Student Work 29

  30. What are ways we can assess students? • Performance Events • Individual interviews • Hands-on performance assessments • Performance Tasks • Individual assessments • Group performance assessments • Projects • Datafolios • Portfolios 30

  31. Forced-Choice Items • Examples • multiple-choice, true-false, matching • Advantages • Can cover much content in little time • Inexpensive • Easy to score - “Objective” • Challenges • Difficult to write high-quality items • May stress memorization over understanding and application • Could encourage instruction that emphasizes content coverage over deep understanding of concepts 31

  32. Written Response Items • Examples • essays, short-answer, solve math problems, fill in the blank • Advantages • Tap student understanding • Can measure application of knowledge • Can be administered in group settings • Challenges • More time-consuming, expensive to score • Scoring may not be objective • Students may not respond 32

  33. Structured Observation • Examples • A topic for group discussion question by small groups of students • Teamwork exercise with observation protocol • Science laboratory exercise • Advantages • Assess application of skills • Can observe students using skills in real contexts • Challenges • Difficult to observe multiple groups (or individuals) at the same time • Time consuming; labor intensive 33

  34. Unstructured Observation • Examples • Student classroom participation • Quality of student responses to teacher’s questions • Advantages • Naturalistic observation of students • Collect data (systematically) not usually collected or used • Challenges • Lack of opportunity for all students to be observed • Anecdotal in nature - how generalizable? 34

  35. Samples of Student Work • Examples • Daily classroom work • Term papers • Advantages • Captures work already being done in the classroom • Could reinforce the on-going creation of products by students • Challenges • The information from each student may be different • May be difficult to “standardize” across multiple students 35

  36. Performance Events • Examples - • Musical performance sing, play a musical instrument • Science experiment • Solve mathematical problem • Advantages • Often, these assess the most important skills • Challenges • Time consuming to assess all student • Expensive to develop, use, and score 36

  37. Performance Tasks • Examples • Paper on an English topic • Compose a song • Plan for a school or community improvement project • Advantages • Sustain and extend classroom learning • Encourage students to work deeply in the content area • Challenges • Time consuming - for students and teachers • Students need support and assistance 37

  38. Projects • Examples • Senior project on a topic of the student’s interest • Implementation of a plan to improve the school or community • Advantages • Students can engage actual work that interests them • Challenges • Very time consuming for both students and teachers, to carry out, to present, and to score 38

  39. Datafolios • Examples • Collection of data from one or more sources, such as observation, ratings of student work, test scores in one folio • Advantages • Captures and organizes information about a student in one place • Challenges • Time consuming for students and/or teachers to select and organize the information • Data itself may not tell much of a story; narrative commentary may also be needed 39

  40. Portfolios • Examples • Showcase of best arts products • Collection of essays written in English class • Advantages • Show student progress • Capture best work of the student • Encourage students to produce work • Challenges • Very time consuming to develop, collect, organize, and score • May be idiosyncratic to the individual students 40

  41. Questions to Consider First • Do we know what students need to learn? • Do we understand what students need to learn? • How do students best learn? • What instructional strategies are most effective? • How do we know that students have learned? • Can state, district, and classroom assessments work together to promote quality student learning? 41

  42. Questions to Consider First • What work on assessment is needed to improve how assessment contributes to student learning? • What professional learning needs to occur for this work to happen? • How can the state encourage work on all three parts of the balanced assessment system? 42

  43. Choosing Assessment Methods • Each educator and school should use a variety of assessment methods (multiple methods) • The methods selected locally (district, school, and classroom) should complement and build on the assessments used at the state and national level • Coordinated assessments are most useful, since they can present a more complete picture of student achievement and other accomplishments 43

  44. Coordinated Assessment Systems • Use both large-scale and classroom-based assessments • Measure the same skills or related skills using different measures of related skills • One possible model: • State or district: end-skills • Classroom: learning progressions leading to the accomplishment of end skills 44

  45. Coordinated Assessment Systems • Developed to provide “comprehensive” information on learning and achievement • Different assessment methods may or may not yield consistent information • If the results are consistent, still use multiple methods • If not consistent, look for reasons - may yield insights into student behavior and/or the quality of the measures used. • Avoid using multiple assessments as a multiple-choice item - selecting one source and ignoring the others 45

  46. Developing the Needed System • Examine the academic content standards valued at the state level • Format - end-skills or learning progressions? • Breadth and depth - how thorough? • Completeness - few or many in number? • Determine how the state will assess these • What types of measures will be used? • What types of measures are missing? • What other types of data do classroom teachers need? 46

  47. Developing the Needed System • If the skills being measured are end-skills, then create the desired learning progressions in grade-level expectations • Work should involve educators from all levels to assure horizontal and vertical alignment • Determine the assessments methods to be used to assess the learning progressions • Determine assessment methods needed to gauge student learning • Develop the needed assessments 47

  48. Developing the Needed System • Create and implement instructional plans to provide the instruction on learning progressions • Make sure all teachers are “on the same page” • Horizontal alignment • Vertical alignment • Use assessments to determine student learning • Embed assessments in on-going instruction • Review and use assessment results to improve instruction • Examine assessment results to determine individual student and group instructional needs 48

  49. Michigan’s Assessment System • In Michigan, the assessment system needs to be more fully balanced • Summative assessments play a major role • Interim assessments are needed • Formative assessments are also needed • Teachers need to be given time to actively discuss instructional strategies • Assessments need to be embedded in good instruction • Teachers need to be formally prepared to work on the assessments to be used 49

  50. Secondary Credit Assessment Program • State and districts need to determine whether students have learned enough to receive credit in a variety of high school credit areas • This could be done in conventional end-of-course exams or through a different manner • We propose to do both - to develop a balanced assessment system in Michigan • We invite interested districts, schools, and educators to become partners in this effort 50

More Related