The view october 4 2012
This presentation is the property of its rightful owner.
Sponsored Links
1 / 87

The View October 4, 2012 PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on
  • Presentation posted in: General

The View October 4, 2012. Dr. Valorie Foy [email protected] Dr. Valorie Foy, Director Jeremy Heneger, Assistant Director John Moon, NeSA Project Manager Dr. Ed Foy, Writing Coordinator Dr. Ted Larson, NAEP Coordinator Jackie Naber, Administrative Assistant

Download Presentation

The View October 4, 2012

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The view october 4 2012

The ViewOctober 4, 2012

Dr. Valorie Foy

[email protected]


The assessment team

Dr. Valorie Foy, Director

Jeremy Heneger, Assistant Director

John Moon, NeSA Project Manager

Dr. Ed Foy, Writing Coordinator

Dr. Ted Larson, NAEP Coordinator

Jackie Naber, Administrative Assistant

Carol Bom, Administrative Assistant

The Assessment Team


Today s agenda

NSSRS

C4L Update

Writing

NeSA

Nebraska Performance Accountability System [NePAS ]

Today’s Agenda


Nssrs

NSSRS


The view october 4 2012

NeSA results ARE an important data source!

When combined with other information, these data can support curricular, instructional, and learning support decision making.

For NeSA...

every student

will be tested!


The view october 4 2012

Commit to data analysis as a continuous process, not an event. (Reeves, 2009)


The view october 4 2012

Nebraska schools should use NeSA data to . . .

  • Provide feedback to students, parents and the community

  • Inform instructional decisions.

  • Inform curriculum development and revision.

  • Measure program success and effectiveness.

  • Promote accountability to meet state and

  • federal requirements.


The view october 4 2012

  • Ensuring Quality Data

  • Online Assessments

    • Make sure students use their own Test Session Ticket

    • Test sessions can be taken in any order

    • Have students exit NeSA completely after testing

    • Install NeSA software early and verify it works using Practice Tests

    • Paper/Pencil Assessments

    • Make sure the student ID is correct and complete (page 2)

    • Affix labels on front of every answer document

    • Separate answer sheets and test booklets

    • Leave all test booklets and answer sheets intact

    • Inventory test materials early and order more if needed

    • Bubble answers on correct part of answer sheet


The view october 4 2012

  • Ensuring Quality Data

  • Code Not Tested (NT) Reasons

    • Use Management Tools to enter NT codes for the online assessments

    • Bubble NT codes on answer sheets for pencil/paper assessments

    • Tested Codes include:

      • No Longer Enrolled (NLE)

      • Parent Refusal (PAR)

      • Emergency Medical Waiver (EMW)

      • Recently Arrived LEP (RAL)

      • Student Absent Entire Window (SAE)

      • Invalid (INV)

      • Other (OTH)


The view october 4 2012

  • Ensuring Quality Data

    • Score Corrections (ZEROS and NT codes)

    • June Review by NDE Staff

      • New Student

      • No Attempted Documents Returned

      • Duplicate State ID

      • Multiple Documents Returned

      • Online Demographics Mismatch

      • Alt Assessment Flag Changed

    • Late June districts are contacted for NT Reasons for DRC reports

    • August corrections DRC reports revised in NSSRS for SOSR

    • October 2012 SOSR District Preview for Final Corrections (Two weeks from release to make corrections)


The view october 4 2012

C4L


Check 4 learning

  • Timeline for System Updates

    • October 1st System Management Update

      • Improved method for uploading students

    • October 29th Test Creation Update

      • Improved process for building reading tests

    • New Items Added Throughout Fall

Check 4 Learning


Check 4 learning1

  • Currently Districts Can

    • Upload students to the system

    • Place students into classes

    • Use old tests

    • Create and use new math and science tests

    • Please note:

      • System will be down: Sept. 28 & Oct. 26

Check 4 Learning


Check 4 learning2

  • WebEx for System Update

    • October 30

      • 2:00 p.m.

    • October 31

      • 10:00 a.m.

    • Recorded and posted on NDE website

Check 4 Learning


Check 4 learning3

  • Ideas for Use:

    • Point of Instruction Assessment

    • Curriculum Unit Assessment

    • Standard Review

    • District/Building Level Assessment

    • New Student Skill Assessment

    • L to J Model

Check 4 Learning

http://www.education.ne.gov/Assessment/C4Learning.htm


Check 4 learning4

  • Moving Forward

    • Currently

      • Review and Approval of Items

      • Seeking Feedback

      • Determine Needs

    • Minimize Burden

      • Earlier MOU

      • Maintain System that is Useful

Check 4 Learning


Check 4 learning5

  • Statistics

    • Over 4000 items in the system

    • Over 1500 ready to be approved

    • Last year from Feb. until the end of the year-135,000 assessments were administered online

    • Approximately 180 Districts take part in C4L

Check 4 Learning


Nesa writing

NeSA Writing


Nebraska educators continue to be involved in these aspects of nesa w

Prompt Creation

Field Testing

Prompt Selection

Rangefinding

Score Verification (formally Writing Appeals)

Continued training and communication

Nebraska Educators continue to be involved in these aspects of NeSA-W:


The rangefinding process

  • Nebraska Director of Statewide Writing

  • Includes 7-10 Nebraska educators per grade level from a variety of educational backgrounds.

  • Two DRC reps facilitate the work.

The Rangefinding Process:


The rangefinding process1

  • Analytically score, discuss, and come to consensus on 90-100 papers from the field testing of the test prompt.

  • DRC uses the papers selected and scored by this group to establish the anchors and train the raters prior to the scoring of the assessment.

The Rangefinding Process:


On line writing times

On-line Writing Times


Grade 8 and 11 testing time

When time on task was plotted with the scale score the following general conclusions seem plausible:

1. Student scores start to level off after a 75-90 minute period so students should be encouraged to spend a minimum of 75 minutes.

Grade 8 and 11 Testing Time


Grade 8 and 11 testing time1

2. With SPED students, the trend is somewhat upward for the entire time tracked, so SPED students could be encouraged to spend as much time as they need as long as they are productive.

3. However, keep in mind students cannot be controlled for time during testing. Training and instruction must come in the classroom before the test.

Grade 8 and 11 Testing Time


Scoring shift grades 8 and 11

In April NDE conducted a standard setting for

grades 8 and 11. The results of the process were

presented to the State Board, resulting in new cut

scores for these grades.

The quality of writing did not decline but rather

the board--after discussion over a two day period-

raised the requirements for Meets the Standards.

Scoring Shift – Grades 8 and 11


The view october 4 2012

  • Under the old holistic grading system, a student at grade 8 and grade 11 would reach Met the Standard with a 2 and a 2+ (4.33).

  • Descriptions in the 2 column included:

    • Digressions

    • Limited unrelated details

    • Occasionally forced or contrived

    • Errors may district


The view october 4 2012

  • With new analytic system, a student must have a score that falls in the 3 range of the rubric (if equivocated to the old rubric the student needs a 3 and a 3 -).

  • 3 Column: Generally

    • Adequate

    • Logical

    • Functional

    • Paragraphing successful

    • Conventions usually correct and do not distract


Scoring shift grades 8 and 111

When looking at the rubrics (old and new), the 3

column is the acceptable standard. The Nebraska

PLD’s (Performance Learning Descriptor) also

clearly reflect the 3’s column on each rubric as the

state writing goal.

The new cut scores will allow some 2’s, but most

likely in the domains of word choice and

conventions.

Scoring Shift – Grades 8 and 11


Field testing at 4 8 11

September 13 – New prompts were written by Nebraska teachers at grades 4, 8, 11. The new prompts need to be field tested.

Field Testing at Grade 4 will remain the same as in the past. Testing will be in the spring from the end of February to middle of May using paper/penciltests.

Field Testing at 4, 8, 11


New process field testing grades 8 and 11

Grades 8 and 11 will be on-line through the NeSA testing site. Prompts are assigned randomly to students.

November 12 - tickets are available and new download if needed. New button on NeSA icon will say NeSA-W field testing.

The field testing window will be November 14 to December 21. (5 1/2 weeks)

Field Testing WebExes November 6-8

New Process - Field Testing Grades 8 and 11


Field testing

This is an opportunity for students to practice taking the test under the same conditions as the regular NeSA-W test.

As a field test of future prompts for NeSA–W, the test results cannot be used in any other way. (For example: no local scoring)

NDE appreciates schools supporting this project as we want to keep prompt writing in the hands of Nebraska educators.

Field Testing


Analytic scoring training dates grade 4

Designed for teachers.

November 8 - Scottsbluff

November 14 - Kearney

November 16 – Lincoln

Times: 8:30 – 11:30

Materials from the training will be posted to our website at:

http://www.education.ne.gov/Assement Writing.htm

Analytic Scoring Training Dates Grade 4


Statewide writing assessment grade 4 2012 2013

Will remain in the narrative mode.

Will continue to be two timed 40-minutes sessions

Will still be tested in paper/pencil format. (No online administration planned)

Will allow access to commercial dictionary and thesaurus.

Statewide Writing AssessmentGrade 4 – 2012-2013


Statewide writing assessment grade 4 2012 20131

Will be scored analytically in 2013

Will have a new cut score set in April 2013 and approved by the State Board of Education in May.

Will have test results released publicly in the spring of 2013 along with grades 8 and 11.

Statewide Writing AssessmentGrade 4 – 2012-2013


Statewide writing assessment grade 4 2012 20132

Will include a new self-assessment tool for analytic scoring.

Information about the modes of writing and student writing samples are found on the assessment website in the writing section:

http://www.educatin.ne.gov/Assessment/NeSAWriting.htm

Statewide Writing AssessmentGrade 4 – 2012-2013


New at grade 4 on the test self assessment tool

ON

New at grade 4 ON the Test – Self-Assessment Tool


Analytic scoring grade 4 starting 2013

Analytic scoring provides five scores for each student: a score for content, a score for organization, a score for word choice and voice, a score for sentence fluency and conventions (punctuation, grammar, spelling), and a total overall composite score.

After the standard setting process, the composite score will be converted to a scale score of 0-70. Students in grade 4 will receive analytic scoring information in 2013.

Analytic Scoring – Grade 4Starting 2013


Analytic scoring grade 4 starting 20131

  • The four domains of writing to be used in analytic scoring:

    Scoring domains were weighted as follows:

  • Content and ideas35%

  • Organization25%

  • Word choice and Voice20%

  • Conventions (grammar, punctuation,

    spelling)20%

Analytic Scoring – Grade 4 Starting 2013


Grade 4 cut score

In April of 2013, a cut score process will be conducted for the writing assessments in grade 4 so that the following classifications of proficiency determinations can be made:

Exceeds the standards

Meets the standards

Below the standards

Grade 4 - Cut Score


Grade 4 cut score1

In December of 2011, NDE conducted a session to create the Performance Level Descriptors for the expectations of Nebraska students based on the analytic rubric. The PLD’s are available on the Assessment web page:

http://www.education.ne.gov/Assessment/NeSA_Writing.htm

Grade 4 - Cut Score


Grade 4 cut scores

The State Board of Education will approve the scores at their May meeting. Following the cut score process and approval in May, the conversion to the scale score (0-70) will occur, and the public release will follow.

Grade 4 - Cut Scores


Grade 4 cut score2

Based on what happened at Grades 8/11, we can anticipate a significant shift at Grade 4 in the requirements for Meets the Standard.

Grade 4 – Cut Score


Lessons learned from 2012

For any paper/pencil tests, students must write in

#2 PENCIL. Papers written in ink will not be

scored and will result in zero scores.

In 2012 the papers/pencil booklet submitted in

ink, therefore, not scorable, were for the most

part administered in make-up sessions by

someone other than the regular classroom

teacher.

Lessons Learned from 2012


Lessons learned from 20121

In paper/pencil testing, the student response to the writing prompt must be written in the actual test booklet. If a student with an IEP is word processing a paper or using a scribe, the student response must be transcribed into the test booklet in pencil.

Originals of transcribed papers will be sent to DRC in an envelope provided with the shipping materials.

Lessons Learned from 2012


The view october 4 2012

This score verification process is appropriate only for very specific, not general, scoring issues. Only the total score will be submitted. The score verification process is designed for student scores that fall below the state cut score.

Statewide Writing Appeals Process in the future under analytic score will be called Score Verification


Appeals score verification cont

Districts will need to complete the score verification form, on the website or from the SAA update. Districts must provide a written explanation indicating the reason for the score verification change and a copy of each paper in question. The writer of the score verification should focus on the particular domain or domains in question. Score verifications must reach the Assessment Office by the deadline.

Score Verification is completed by Nebraska educators who have been involved in the writing grading for years.

Appeals – Score Verification (cont)


The view october 4 2012

Questions contact:

[email protected]

Phone: 402-471-2947


The view october 4 2012

NeSA


Nesa news 2013 nesa testing window

NeSA News2013 – NeSA Testing Window

January 21 – February 8

February 11 – March 22

March 25 – May 3

NeSA Writing

ELDA

NeSA – Reading

Mathematics

Science

NeSA-AA Alternate Tests


Online testing 2013

All paper/pencil tests must be ordered October 8-19 through eDirect – writing, reading, mathematics, and science. Enrollment form

Online Testing 2013


The view october 4 2012

Students whose IEP plans or 504 plans specify an accommodation that would require a paper/pencil.

ELL students who need an accommodation that would require a paper/pencil.

Students responding in a language other than English or Spanish (NeSA-W).

Students contracted to institutions where online access is not allowed.

Students who take the alternate assessments-R/M/S

2013 – Online TestingRequirement in reading, mathematics, science, writing (Grades 8 and 11) for all students except:


Research for the october paper pencil order

Review the revised Accommodations document-On Assessment Page-NDE

District-Accommodations that are best for the student

A student who takes one content area paper/pencil does not necessarily need paper/pencil in all

Students who qualify for speech may not need any paper/pencil

Inform the student

Research for the October Paper/Pencil order


Enrollment webexes

October 2-4, 2012 – “Invitation” is posted

Enrollment WebExes


Nepas

NePAS


Nebraska performance accountability system nepas

  • Federal Accountability – AYP/PLAS

  • State Accountability – NePAS

    • Tied to LB # 870

Nebraska Performance Accountability System - NePAS


Section i scores with rankings

1. Three NePAS Grade-Level Configurations

2. School District

Section I-Scores with Rankings


Section i 4 charts for each school district

Section I: 4 charts for each school district

  • Elementary Grade-Level Configuration

    Grades 3-5

  • Middle Grade-Level Configuration

    Grades 6-8

  • Secondary Grade-Level Configuration

    Grades 9-12

  • District

    Grades 3-12


Components

Status—Scale Score

Improvement—Scale Score

Growth—Scale Score

Participation--Met/Not Met or Percent

Graduation Rate--Percent

Components


Scale scores and rankings status average scale score

ReadingE M S D

MathE M S D

Science E M S D

Writing E M S D

Scale Scores and RankingsStatus=Average Scale Score


The view october 4 2012

Reading E M S D

Math E M S D

Science*E M S D

Writing*E M S D

Scale Scores and RankingsImprovement= Difference of Average Scale ScoresCompares scores of groups of students


The view october 4 2012

Reading E M D

Math E M D

Scale Scores and RankingsGrowth= Average of Differences in Scale ScoresGrowth based on matched student data in the same district-consecutive years


Graduation rate percent and ranking

Will be accompanied by Enrollment- Grades 9-12

4 Year 2012 (Secondary & District)

6 Year* Available 2013 (Secondary & District)

Graduation RatePercent and Ranking


Participation 95 met not met no rankings

Reading E M S D

Math E M S D

Science E M S D

Writing E M S D

Participation (95%)Met/Not Met-No Rankings


Cut scores charts are on nepas screens

Reading, Math, Science

8th and 11th Grade Writing

Cut Scores Charts are on NePAS screens


The view october 4 2012

  • Section 1--Anywhere District

    Nebraska Performance Accountability (NePAS)Elementary Grade-Level Configuration (Grades 3-5)

DRAFT


Rankings

Ties will receive the same ranking

The next school will receive the next lower number.

Example:

If 20 districts are ranked # 1 in a component,the next district will receive a ranking of 21.

Rankings


Nepas business rules still being determined

Appropriate masking (under 10)

Any groups 10 and above to be included

In any criterion where 100% of the students place in a single rating, the scores must be masked—by law. Example 100% graduation rate

NePAS business rules- still being determined


With inclusion of ten or more number of districts possible in reading and math

With inclusion of ten or more-Number of districts possible in reading and math


Nepas part ii

By Grade Level-Building (Actual physical building)

By Grade Level-District

No Rankings

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

NePAS Part II


For each building in a district for each grade scale score

Example

Grade 4-Reading

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

For eachBuilding in a district for each Grade-Scale Score


For each district for each grade scale score

Example

Grade 4-Reading

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

For eachDistrict-for each Grade- Scale Score


The view october 4 2012

Section 2.1

School Building Accountability By Grade

Choose Grade: (Drop Down Menu)

Grade 05


State of the schools report

State of the Schools Report

October

State of the Schools Report--available to districts

  • Disaggregated data

  • AYP/PLAS

  • NePAS

  • 15 days—then rankings added

    November

    State of the Schools Report Public Release

  • Disaggregated data

  • AYP/PLAS

  • NePAS


Your story

The month of October gives districts the opportunity to prepare a message

Others-Focus on the district chart

Your Story


The view october 4 2012

Anywhere School District

School District (Grades 3-12)


Your story1

Many rankings—focus on strengths

If there are scores that are weak, provide information that informs the scores

Use the counterbalance of information—the story behind the rankings

Our community wanted to know what we were doing to improve

Your Story


Test security

Test Security


Two security events

  • Caveon Security Report

  • United State Department of Education:

    Office of the Inspector General

Two Security Events


Security

Equity

Security


Examples of score invalidation

Allowing word walls, personal dictionaries, graphic organizers on writing test

Editing student first drafts on writing test

Reading the reading passages to students

Allowing students to carry tests from one teacher to another unmonitored

Providing accommodations not allowed by an IEP

Allowing dictionaries on reading test.

Examples of Score Invalidation


Examples of score invalidation1

Examples of Score Invalidation

  • Student accessed help on cell phone

  • Student used inappropriate math document from notebook


Reasons for breach in security that called for score invalidations

New Teacher

New Principal

New District Assessment Contact

Substitute Teacher

Paraprofessional

Make-up Test

Reasons for breach in security- that called for score invalidations


Security breach form

District information

Date of incident

Description of incident

Action taken

District response (if known)

Actions taken to assure situation will not be repeated

Security Breach Form


Missing materials

List of missing materials

Action district took regarding the breach (missing materials)

District Outcome/Resolution

Missing Materials


Webexes galore

  • Enrollment for Paper/Pencil Orders

    • October 2 and 3

    • “Invitations” posted on Assessment Page

  • C4L Update

    • October 30 and 31

    • “Invitations” posted on Assessment Page

  • Technology Update/Writing Field Test 8 & 11

    • November 6-8

      • Dates Posted

WebExes Galore


The view october 4 2012

Questions?

http://www.education.ne.gov/Assessment/Index.html


  • Login