Montana Comprehensive Assessment System - PowerPoint PPT Presentation

Montana comprehensive assessment system
1 / 65

  • Uploaded on
  • Presentation posted in: General

Montana Comprehensive Assessment System. MontCAS. Montana Office of Public Instruction Linda McCulloch, Superintendent May 2006. Welcome Panelists Sharon Houle, Measured Progress Carol Taylor-Cann, Riverside Publishing Bob Runkel, OPI—GAP Students

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Montana Comprehensive Assessment System

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Montana comprehensive assessment system

Montana Comprehensive Assessment System


Montana Office of Public Instruction

Linda McCulloch, Superintendent

May 2006

Montana comprehensive assessment system



  • Sharon Houle, Measured Progress

  • Carol Taylor-Cann, Riverside Publishing

  • Bob Runkel, OPI—GAP Students

  • Lynn Hinch, OPI—English Language Proficiency

  • Donna O’Neill, OPI--NAEP Coordinator

  • Linda Peterson, OPI—Science Standards

  • Moderator, Judy Snow, OPI--Assessment



  • MontCAS Overview

    • Judy Snow

  • MontCAS—Updates on Other Assessments

    • Panelists

  • MontCAS Issues and Details

    • Judy Snow

Montcas overview

MontCAS Overview

  • Assessment 101

  • MontCAS Phases 1 and 2

Assessment 101

Standardized Test

Norm-referenced test

Criterion-referenced test

Performance levels

Alternate Assessment



No Child Left Behind

Adequate Yearly Progress (AYP)

Administrative Rules of Montana


Board of Public Education (BPE)

Assessment 101

Montana comprehensive assessment system

Phase 1

Norm-referenced test (NRT)—the Iowas—Riverside Publishing Company

Aligned to national standards

Grades 4, 8, 11

Reading, language arts, math, social studies, science

Multiple choice

Alternate assessment scales

Funded by the state

Phase 2

Criterion-referenced test (CRT)—Measured Progress

Aligned to Montana Content Standards

Grades 3-8, and 10 (science in spring 2008)

Reading and math

Multiple choice, math short answer and constructed response

Evidence based alternate

Funded by NCLB


Montana comprehensive assessment system

Other AssessmentsPanelists

Bob runkel modified crt

Bob RunkelModified CRT

Gap students

GAP Students

  • In between CRT-Alternate and CRT

  • Modified performance standards

  • Feasibility Study

    • Grade 5 students

    • Identified as special education students for the grade 4 CRT in spring 2006

    • Pilot math CRT based on modified performance standards

  • Tentative Window

    • September 11-19, 2006

Lynn hinch english language proficiency test

Lynn Hinch English Language Proficiency Test

Montana comprehensive assessment system


  • English Language Proficiency

    • Required by NCLB

    • Grades K-12

    • Test Development through Mountain West Consortium (MWAC)

    • Date TBA

No child left behind requirements


  • Both Titles I and III require an annual assessment of English language proficiency of all K - 12 limited English proficient students

  • Assessment must be aligned to state English language proficiency standards

  • Assessment must report a separate score for listening, speaking, reading, and writing

Title iii requirements


  • States must establish Annual Measurable Achievement Objectives (AMAOs) that determine how LEP students are making progress in learning English, and whether students are attaining proficiency in English.

Status of english language proficiency assessment

Status of English Language Proficiency Assessment

  • Mountain West Assessment Consortium

  • Test Development and Delivery

  • RFP Process

  • March '06 Request to the Department of Education for timeline extension

Donna o neill naep

Donna O’NeillNAEP

Montana comprehensive assessment system


  • National Assessment of Educational Progress

    • NAEP--Nation’s Report Card

    • Odd-numbered Years

    • Grades 4 and 8, Reading and Math

    • Required Sample

All about naep

All About NAEP

  • What is NAEP?

  • Why is it important to Montana?

  • Who takes the NAEP Assessment?

  • What are the requirements for NAEP?

  • New developments in Science & Writing

  • What is the timeline for the NAEP Assessment?

Naep national assessment

NAEPNational Assessment

  • The Nation’s Report Card

  • Measures academic achievement

  • Reports to the public what students know and can do:

    • average scale scores

    • proficiency levels

  • Formats vary by grade:

    • multiple choice

    • constructed-response

  • Links performance to educational variables

    • Teacher background information

    • School characteristics

  • NAEP is confidential



The nation s report card

The Nation’s Report Card



Montana comprehensive assessment system


  • Random sample

    • 2,000 students per

      • Subject

      • Grade

  • Grades 4, 8 & 12

  • Students with disabilities

  • Limited-English proficient

Naep requirements

NAEP Requirements

  • Linked to Title I funding

    • Schools

    • Districts

  • NAEP is designed

    • Minimal disruption of instruction

    • 90 to 120 minutes of each student’s time

    • Teachers are encouraged to observe

    • Field staff are many Montana teachers

New developments

New Developments

  • Science Release

    • May

  • Writing (8, 12)

  • Pilot and Special Studies

    • National Indian Education Study (NIES)

      • Grades 4, 8

      • NAEP data will be linked to survey data

      • Student, Teacher and Principal Questionnaires

Naep timeline

NAEP Timeline

Key dates

Key Dates

  • Secure cooperation of districts and schools

    • Notify districts in early May

    • Notify schools of selection during May

    • Notify schools of assessment dates in June

    • Maintain communication with schools throughout the process



  • Packet of Information

    • Facts About Montana Education

    • Historical information on results

    • NAEP Questions Tool

    • NAEP Data Explorer (researchers)


Sharon houle measured progress

Sharon HouleMeasured Progress

Crt science test

CRT Science Test

  • Grades 4, 8, and 10

  • Spring 2008

  • Aligned to Revised Science Standards

Criterion referenced test where are we going

Criterion-Referenced Test Where are we going?

Science--grades 4, 8 and 10

  • May 31, 2006

    • National science content and bias reviews

  • July 25

    • Montana science content and bias reviews

  • Spring 2007

    • Field test

  • Spring 2008

    • First Administration—CRT and CRT-Alternate

Linda peterson

Linda Peterson



Montana comprehensive assessment system

5YCEP Continuous Improvement Process









Montana Surveys of the Enacted Curriculum (MSEC) – Alignment Assessment Process

5YCEP Continuous Improvement Process

5YCEP Continuous Improvement Process





5YCEP Continuous Improvement Process

The Five-Year Comprehensive Education Plan (5YCEP)

Building a Continuous Improvement Process for Montana Schools

Montana board of public education

Montana Board of Public Education



10.54.2503 STANDARDS REVIEW SCHEDULE (1) Montana's content and performance standards shall be reviewed and revised on a five-year cycle beginning July 1, 2005.

Montana board of public education1

Montana Board of Public Education

Standards Revision 2005-2010


  • Assure Montana citizens that its public schools are providing all children of our great state with challenging academic expectations

  • Revised standards clearly and consistently identify what students should know, understand and be able to do

  • Revised standards provide a framework to help guide local curriculum and instruction

Montana board of public education criteria to guide standards revision

Montana Board of Public EducationCriteria to Guide Standards Revision

Standards will be:

  • academic in nature and content specific.

  • challenging and rigorous.

  • clear, understandable and free of jargon.

  • measurable.

  • address diversity, specifically fulfilling the commitment to implementing 20-1-501, Indian Education for All.

K 12 science standards

K-12 Science Standards

Montana Standards Framework

  • Content Standards

  • Benchmarks - Grades 4 and 8 and upon graduation

  • Performance Descriptors

K 12 science standards1

K-12 Science Standards

Performance Descriptors

  • Advanced – superior performance

  • Proficient – solid academic competency

  • Nearing Proficiency – partial mastery of prerequisite knowledge and skills

  • Novice – beginning to attain prerequisite knowledge and skills

K 12 science standards2

Content Standard 1

Students design, conduct, evaluate, and communicate processes and results of scientific investigations, and demonstrate thinking skills associated with this procedural knowledge


Grade 4 (3) - describe and communicate the results of scientific investigations

Grade 8 (3) - review, communicate and defend results of investigations

Upon Graduation – review evidence, communicate and defend results, and recognize that the results of a scientific investigation are always open to revision by further investigation

K-12 Science Standards

Performance descriptors content standard 1

Performance DescriptorsContent Standard 1


  • Grade 4 – with direction, completes a simple investigation with identified variables, using appropriate tools and communicates results

  • Grade 8 – identifies and communicates testable questions, plans and conducts experimental investigations and communicates results

  • Upon Graduation – generates testable questions, constructs a plan for a controlled investigation, makes logical inferences based on observations, accurately interprets data by identifying the strengths and weaknesses in an investigation design, and communicates results.

K 12 science standards3

Content Standard 3

Students demonstrate knowledge of characteristics, structures, and function of living things, the process and diversity of life, and how lining organisms interact with each other and their environment, and demonstrate thinking skills associated with this knowledge.


Grade 4 – create and use a classification system to group a variety of plants and animals according to their similarities and differences, (e.g., American Indian medicinal plants, American Indian dwellings)

Grade 8 – create and use a basic classification scheme to identify plants and animals, (e.g., classification scheme based on American Indian use of indigenous plants for medicinal, food and other uses)

Upon Graduation – recognize, generate and apply biological classification schemes to infer and discuss the degree of divergence using ecosystems, (e.g., American Indian food collections systems)

K-12 Science Standards

Performance descriptors content standard 3

Performance DescriptorsContent Standard 3


  • Grade 4 – identifies attributes of biotic (living) things and abiotic (non-living) objects, including classification based on similarities and differences, basic structure and function, processes of each system

  • Grade 8 – identifies and classifies biotic things and abiotic objects through application of common classification schemes; identifies the interdependence of life and the environment and explains how characteristics of living things change because of the environment

  • Upon Graduation – organizes, classifies, and describes interactions of the biotic and abiotic parts of the biosphere as well as the natural history of interactions of life on Earth and uses these skills to solve related novel (to the student) problems

K 12 science standards proposed time line

K-12 Science Standards Proposed Time Line

  • March – MayGather Public Comments

  • May 11Present Update to BPE

  • May 15Presentation and Discussion with MACIE

  • May – JuneWriting Team Work Sessions

  • JunePrepare draft standards for notice of hearing

K 12 science standards proposed time line1

K-12 Science Standards Proposed Time Line

  • JulyPresent to BPE draft Notice of Hearing and proposed time line

  • AugustNotice of Hearing posted

  • OctoberPublic Hearing

  • NovemberFinal BPE Action Anticipated

  • 2007Implementation, Outreach, and Dissemination

K 12 science standards revisions

K-12 Science Standards Revisions

Linda Vrooman Peterson

Accreditation Division Administrator

Office of Public Instruction

(406) 444-5726

Draft K-12 Science Standards link:

Assessment issues and details

Assessment Issues and Details

Standardized Administration

Accurate Data

Technical Adequacy

Impact on Instruction

Standard administration accurate reliable data

Standard Administration = Accurate + Reliable Data

  • Participation

  • Training

  • Test Security

  • Special Populations

    • Accommodations

    • Alternate Assessments

Participation all means all

Phase 1 Options

No accommodations

If in IEP, 504, or LEP plan

Standard or non standard accommodations

Alternate Assessment Scale


Phase 2 Options

No accommodations

Standard accommodations

If in IEP or 504 plan

Non standard accommodations


Participation All Means All!

Training so that

Training so that . . .

  • everyone understands the procedures for administration.

  • the test is administered in a comparable way in all locations across the state.

  • quality control procedures are utilized when returning test materials.

Test security

Test Security

All test items and responses to those items in MontCAS are secure materials and may not be copied or duplicated in any way; CRT materials may not be retained in the school after testing is completed.

It is very important to review test security information with test administrators during test administration training.

Opi test security

OPI Test Security

  • Any concern about breaches in test security or noncompliance with test administration procedures must be reported immediately to the principal and system test coordinator and to the State Assessment Director.   

  • OPI Guidelines and Procedures for Test Security

    • This OPI publication outlines procedures for reporting testing irregularities and should be made available to system superintendents, principals, and test administrators.



Standard accommodations do not change what we expect students to know and be able to do.

Test accommodations are based on individual student needs and classroom assessment practice; they are not for the use by an entire classroom or category of students.

They should not give students unfair advantages; rather they are meant to remove barriers that may exist due to a student’s learning style or disability.

Standard and non standard accommodations


Do not change the intent/content of the assessment

Phase 1: Only by IEP, 504, or LEP plan

Phase 2: By IEP, 504, LEP, or regular routine in the classroom

Non Standard

Change the content/intent of the assessment

Phase 1: Only by IEP,504, or LEP plan

Phase 2: Only by IEP or 504 plan

Standard and Non StandardAccommodations

Accurate data

Accurate Data

  • the information collected is of high quality

    • demographic information

    • enrollment information

    • disability information

    • decisions related to the use of accommodations and the alternate assessment (CRT-Alternate)

2006 data check points

2006 Data Check Points

  • OPI CRT Testing Cycle Enrollment Count

    • March 6, 2006Window Opens

    • April 13, 2006 Data due to County Superintendent

    • April 20, 2006 Data due to OPI

  • Measured Progress CRT Online Demographic Data Verification

    • April 24 – May 19, 2006

  • OPI Comparability Study—June 2006

  • OPI Iowa Tests (NRT) Disaggregated Data Review

    • July 24 - August 4 , 2006

Technical quality and expertise

Technical Quality and Expertise

  • Test Development

  • Technical Advisory Committee (TAC)

  • Technical Manuals

  • CRT and CRT-Alternate Standard Setting

Montana performance levels

Montana Performance Levels

  • Advanced

  • Proficient

  • Nearing Proficiency

  • Novice

Standard setting

Standard Setting

Standard Setting

  • Standard setting is the process of determining cut points for performance levels for the CRT and the CRT-Alternate.

  • Panels of Montana educators

  • Facilitated by Measured Progress

2006 standard setting details

2006 Standard Setting Details

  • For more information:

  • To apply:


  • Travel, meals (those not provided during the sessions), and lodging reimbursements

  • Renewal units and honoraria

2006 standard setting where and when

2006 Standard SettingWhere and When

Red Lion Colonial Inn in Helena

  • June 21-22

    • CRT, Reading and Math, grades 4, 8, and 10

  • June 20-22

    • CRT-Alternate, Reading and Math, grades 3-8 and 10

  • July 26-27

    • CRT, Reading and Math, grades, 3, 5, 6, and 7

Impact on instruction

Impact on Instruction

  • Reporting

  • Data Interpretation

  • Tools

Reporting and impact on instruction

Reporting and Impact on Instruction

  • Phase 1—NRT

    • NPR

    • NCE

    • Stanine

    • Performance Level

  • Phase 2—CRT

    • Scaled Score

      • Performance Level for each content area

    • Raw Score

      • Number of points for each standard

Iowa tests interpretive guide

Iowa Tests Interpretive Guide


  • Relationship of stanines, percentile ranks, and normal curve equivalents (p. 5)

  • Comparison of NRT Scores (p. 6)

  • Performance Profile (p. 12)

  • Individual Customized Skills Report (p. 16)

Crt results interpretation

CRT Results Interpretation

  • 2005 Interpretive Guide

    • Roster and Item-Level Report (p. 6)

  • Released Items

    • Items that scored are released.


  • iAnalyze

    • A secure website where data gathered from the CRT assessments are reported

Montana comprehensive assessment system

Judy Snow

State Assessment Director

Office of Public Instruction


Assessment link:

JUMP link:

  • Login