Hawaii statewide assessment program maintaining test security and reporting testing incidents
This presentation is the property of its rightful owner.
Sponsored Links
1 / 42

Hawaii Statewide Assessment Program: Maintaining Test Security and Reporting Testing Incidents PowerPoint PPT Presentation


  • 53 Views
  • Uploaded on
  • Presentation posted in: General

Hawaii Statewide Assessment Program: Maintaining Test Security and Reporting Testing Incidents. Fall 2008 Hawaii Department of Education Systems Accountability Office (SAO).

Download Presentation

Hawaii Statewide Assessment Program: Maintaining Test Security and Reporting Testing Incidents

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Hawaii statewide assessment program maintaining test security and reporting testing incidents

Hawaii Statewide Assessment Program:Maintaining Test Security andReporting Testing Incidents

Fall 2008

Hawaii Department of Education

Systems Accountability Office (SAO)


Hawaii statewide assessment program maintaining test security and reporting testing incidents

This presentation focuses on testing incidents and the procedures for reporting testing incidents and inquiries into those reports.


Board of education policy no 2520 statewide assessment program policy

Board of Education Policy No. 2520STATEWIDE ASSESSMENT PROGRAMPOLICY

“Systematic school, curricular, and program improvement efforts directed toward student attainment of the Hawaii Content and Performance Standards (HCPS) requires the collection of relevant and accurate student assessment information.”


Maintaining integrity of the hawaii state assessment hsa

Maintaining Integrity of theHawaii State Assessment (HSA)

Criterion Referenced Test (CRT) and Norm Referenced Test (NRT) data

Curricular, instructional, personnel, and financial decisions

Improved student achievement

Strict adherence to test administration and security procedures is required to ensure that the student assessment results are accurate.


Hsa procedures

HSA Procedures

All school personnel involved in the administration of the HSA are accountable for knowledge of and adherence to all test security procedures as specified for their roles.


Hawaii statewide assessment program maintaining test security and reporting testing incidents

HSA guidelines and procedures can be found in the Test Administrator’s Handbook (TAH), the Test Coordinator’s Handbook (TCH) and the Student Participation and Accommodations Handbook (SPAH).

Additional mandatory resources:

  • HSA District Meeting sessions or school level trainings

  • DVD: Hawaii Statewide Assessment Program

    Watching the DVD and attending the mandatory training are not substitutes for reading the HSA handbooks.


Hawaii state alternate assessment hsaa hawaiian aligned portfolio assessment hapa

Hawaii State Alternate Assessment (HSAA) &Hawaiian Aligned Portfolio Assessment (HAPA)

HSAA procedures can be found in the:

  • HSAA Rater/Coordinator Manual

  • HSAA Test Coordinator’s Handbook

    HAPA procedures can be found in the:

  • School Administrator - Test Administrator - Test Coordinator Handbook


Testing incidents

Testing Incidents

Any event or procedure that could potentially impact the integrity of the tests, the data, and the test results before, during, and after the test administration is a testing incident.

All testing incidents must be reported immediately.


Types of testing incidents

Types of Testing Incidents

  • Testing irregularity

  • Standardized administration irregularity

  • Test security violation


Testing irregularity

Testing Irregularity

Is an incident that affects the student’s performance on the test, but may, or may not, compromise the test result.

  • Must be reported

  • May or may not trigger an inquiry

  • May require the administration of the replacement test form

  • May require invalidation of the student’s score


Testing irregularity examples

Testing Irregularity Examples

Include, but are not limited to:

  • Electrical outage during testing

  • Fire alarm during testing

  • Many students taken ill due to vog

  • Hurricane evacuation during testing

  • Medical emergencies involving key personnel

  • Defective materials

  • Student bubbled the wrong portion of the response booklet


Standardized administration irregularity

Standardized Administration Irregularity

Is a change in the established test administration procedure and protocols that may, or may not, compromise the test result.

  • Must be reported

  • May or may not trigger an inquiry

  • May affect the student’s performance on the test

  • May result in the administration of the replacement test form

  • May require invalidation of the student’s score


Standardized administration irregularity examples

Standardized Administration Irregularity Examples

Include, but are not limited to:

  • Disallowed classroom displays containing content information visible during test sessions (see handbooks for details)

  • Test related content instruction, review or practice tests are provided between testing sessions (if it implies prior knowledge of test items, this could be a test security violation)

  • Test administrator fails to read the directions or conduct the examples as directed

  • Providing inappropriate accommodations

  • Selecting specific students to “not test”

  • Restricting or extending time to complete the testing sessions


Test security violation

Test Security Violation

A test security violation is a serious incident that compromises test security, data security, or both, and the integrity of the student score results.

  • Must be reported

  • Will trigger an inquiry into the incident

  • May affect the student’s performance on the test

  • May result in the administration of the replacement test form

  • May require invalidation of the student’s score

  • Superintendent will be notified


Test security violation examples

Test Security Violation Examples

Include, but are not limited to:

  • Use of mobile phones or cameras during testing sessions to record test items

  • Test items or student responses are copied, discussed, reviewed or analyzed before, during or after the assessment is administered

  • Test administrator or proctor leaves the classroom unattended or with an unqualified adult during a session (see handbooks for qualifications)

  • Test booklets are assigned to test administrator or proctors for their reference and review

  • Test responses are altered

  • Prompting (verbal or non-verbal) or giving answers to students during testing to review specific items

  • Students sharing answers, creating cheat sheets or developing group answers


Procedure sequence and expectations for reporting a testing incident

Procedure, Sequence and Expectations forReporting a Testing Incident


Who reports

Who Reports?

All Hawaii Department of Education employees are required to report testing incidents to the school Administrator.

Test Coordinators and school Administrators must report any knowledge of testing incidents to:

  • Student Assessment Section (SAS) Administrator

  • Complex Area Superintendent


Students parents and community members

Students, Parents, and Community Members

Non-DOE personnel may report testing incidents to school personnel.

School personnel receiving such information are responsible for reporting the information to the school Administrator.


Test coordinators and school administrators

Test Coordinators and School Administrators

If you have witnessed, been informed of, or suspect the possibility of a testing incident that could potentially impact the integrity of the tests, the data, and the test results, immediately contact:

  • Student Assessment Section (SAS) Administrator at 808-394-1300

    In no answer, call…

  • System Planning and Improvement (SPIS) Administrator at 808-586-3285


If no answer or after hours

If No Answer or After Hours

Send a LN email to Kent Hinton

(cc: Christina Tydeman and Cara Tanimura)

Provide:

  • Your name,

  • Your school,

  • Phone numbers with hours for contact, and

  • The nature of your call.

    You will be contacted by one of the individuals within your specified hours of contact.

    The SAO Director will inform the Superintendent of all reported testing incidents.


Besides school personnel reporting to sas

Besides School Personnel Reporting to SAS…

State level data processing can reveal potential testing incidents or school level data quality concerns that may trigger an inquiry.


Red flag examples

“Red Flag” Examples

Include, but are not limited to:

  • Reviews of student demographic data

    • Subgroup identifiers (i.e., ethnicity, SPED, ELL, disadvantaged)

    • Grade level/birth date

  • Analyses of testing data

    • Student performance trends (e.g., school level, grade level, subgroups)

    • Irregular scoring patterns

    • Inappropriate accommodations provided

    • Out of grade level testing

  • Missing testing materials

  • Altered documents

  • Test scores for students who have been exempted from taking the test by parent letter


Testing irregularity form

Testing Irregularity Form

All schools reporting any testing incidents must complete and submit a “Testing Irregularity Form” for each incident to the Student Assessment Section.


Information requested when reporting to sas

Information Requested When Reporting to SAS

  • Role/position of individual reporting

    • Name is optional

  • School name

  • Date(s) of incident(s)

  • Description of incident(s)

    • Type (e.g., cheating, missing materials, copying items)

    • Scope (e.g., individual, class, grade level, school-wide)

    • By whom (e.g., student, school personnel)


Step 1 did a testing incident occur

Step 1: Did a Testing Incident Occur?

  • What happened?

  • Who was involved?

  • How widespread?

  • Who initiated the incident?


Step 1 outcomes

Step 1: Outcomes

If no testing incident is found, no further action is needed.

If a test/administration irregularity is found, SAS shall direct school personnel with appropriate actions.

If a test security violation is suspected, the school shall be directed to take immediate action (e.g., stop all testing) and a inquiry team shall be sent to the school.


Step 2 if an inquiry is necessary

Step 2: If an Inquiry is Necessary

School Administrator and the respective Complex Area Superintendent or Charter School Executive Director will be notified.

Upon notification, School Administrator must immediately:

  • Cease testing

  • Provide full cooperation and assistance to the inquiry team

    • Private room for interviews by the inquiry team

    • Access to all individuals involved in the incident

    • Copies of documents upon request of SAO


Documents to have handy

Documents to Have Handy

  • FERPAforms – with parent/guardian signature for students involved in the incident

  • Test Administration Training Session Attendance Verification Form

  • Hawaii State Assessment Security and Administration Procedures Acknowledgment Form


Step 2 outcomes

Step 2: Outcomes

Inquiry team is established.

  • Systems Accountability Office (SAO) personnel

  • Dispatched to the school upon notification of alleged test security violation


Step 3 the site visit

Step 3: The Site Visit

Upon arrival at the school, the team will:

  • Meet with Test Coordinator/School Administrator

    • Summarize and clarify the testing incident

    • Share the scope of the inquiry

    • Clarify the roles of school personnel within the inquiry

    • Discuss possible outcomes

    • Review copies of all FERPA forms

  • Conduct interviews

    • Staff may have union representation


Scope of the inquiry

Scope of the Inquiry

  • Who will be interviewed

  • How the findings will be determined

  • When and if testing will be resumed

  • Possible consequences regarding test security and the validity of test scores

  • When the report will be completed


At the end of the site visit

At the End of the Site Visit

The inquiry team will brief the Administrator and Complex Area Superintendent or Charter Schools Executive Director on the findings.


Components of the inquiry report

Components of the Inquiry Report

  • School context

  • Summary of the inquiry methodology

  • Descriptions of the incident, findings, and conclusions

  • Recommended actions and timeline

    • Test security and administration procedures

    • Validity of scores

      No recommendations for disciplinary actions shall be included.


Suppressed test scores

Suppressed Test Scores

In the event the inquiry team determines a student’s scores to be invalid, the scores and the related test materials will be suppressed.

  • In other words, it will be as if the student(s) did not take the test session(s).

    Schools may not view any testing materials or scores that have been suppressed as a result of a testing incident inquiry.


Effect of suppressed scores

Effect of Suppressed Scores

Scores from individual testing sessions may be deemed invalid resulting in partial scores in the subject area for the affected students.

If all of a student’s scores for a subject area are deemed invalid, the student will have no score and will be considered a non-participant.


Step 3 outcomes

Step 3: Outcomes

SAO makes the final determinations of score validity.

Inquiry report is presented by SAO Director:

  • Complex Area Superintendent or Charter Schools Executive Director

  • State Superintendent.

    Superintendent approves the final recommended actions for improving test security and administration procedures.


Possible determinations

Possible Determinations

  • No testing incident occurred

  • Test/administration irregularity occurred = valid scores

    • Did not compromise test security

    • Did not compromise test results (i.e., scores)

  • Test/administration irregularity occurred = invalid scores

    • Did not compromise test security

    • Compromised test results (i.e., scores)

  • Test security violation occurred = invalid scores

    • Compromised test security

    • Compromised test results (i.e., scores)


At the superintendent s discretion

At the Superintendent’s Discretion…

The Superintendent may share the testing incident inquiry report with the Board of Education or the Charter Schools Review Panel.

The school may be required to cover the costs of any replacement test forms used and other related costs (e.g., re-printing of score reports and testing materials, hiring raters).


Step 4 public disclosure

Step 4: Public Disclosure

Approved recommendations are implemented.

The school Administrator must notify the affected parents in writing of the test security violation and its impact on their children’s scores.

The school, in consultation with the DOE’s Communications Office, must respond to Freedom of Information requests regarding the incident.


Overall outcomes

Overall Outcomes

Improving test security contributes to

  • Improved validity and accuracy of assessment scores.

  • Transparency of the test security inquiry process for schools.

  • Public accountability through full compliance with No Child Left Behind.

  • Increased credibility and overall confidence in the student performance results.


For more information regarding testing incidents and inquiries please contact

For more information regarding Testing Incidents and Inquiries, please contact:

Systems Accountability Office at 808-586-3283


  • Login