the view october 4 2012
Skip this Video
Download Presentation
The View October 4, 2012

Loading in 2 Seconds...

play fullscreen
1 / 87

The View October 4, 2012 - PowerPoint PPT Presentation

  • Uploaded on

The View October 4, 2012. Dr. Valorie Foy [email protected] Dr. Valorie Foy, Director Jeremy Heneger, Assistant Director John Moon, NeSA Project Manager Dr. Ed Foy, Writing Coordinator Dr. Ted Larson, NAEP Coordinator Jackie Naber, Administrative Assistant

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' The View October 4, 2012' - jess

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the assessment team

Dr. Valorie Foy, Director

Jeremy Heneger, Assistant Director

John Moon, NeSA Project Manager

Dr. Ed Foy, Writing Coordinator

Dr. Ted Larson, NAEP Coordinator

Jackie Naber, Administrative Assistant

Carol Bom, Administrative Assistant

The Assessment Team
today s agenda


C4L Update



Nebraska Performance Accountability System [NePAS ]

Today’s Agenda

NeSA results ARE an important data source!

When combined with other information, these data can support curricular, instructional, and learning support decision making.

For NeSA...

every student

will be tested!


Nebraska schools should use NeSA data to . . .

  • Provide feedback to students, parents and the community
  • Inform instructional decisions.
  • Inform curriculum development and revision.
  • Measure program success and effectiveness.
  • Promote accountability to meet state and
  • federal requirements.

Ensuring Quality Data

  • Online Assessments
    • Make sure students use their own Test Session Ticket
    • Test sessions can be taken in any order
    • Have students exit NeSA completely after testing
    • Install NeSA software early and verify it works using Practice Tests
    • Paper/Pencil Assessments
    • Make sure the student ID is correct and complete (page 2)
    • Affix labels on front of every answer document
    • Separate answer sheets and test booklets
    • Leave all test booklets and answer sheets intact
    • Inventory test materials early and order more if needed
    • Bubble answers on correct part of answer sheet

Ensuring Quality Data

  • Code Not Tested (NT) Reasons
    • Use Management Tools to enter NT codes for the online assessments
    • Bubble NT codes on answer sheets for pencil/paper assessments
    • Tested Codes include:
      • No Longer Enrolled (NLE)
      • Parent Refusal (PAR)
      • Emergency Medical Waiver (EMW)
      • Recently Arrived LEP (RAL)
      • Student Absent Entire Window (SAE)
      • Invalid (INV)
      • Other (OTH)

Ensuring Quality Data

    • Score Corrections (ZEROS and NT codes)
    • June Review by NDE Staff
      • New Student
      • No Attempted Documents Returned
      • Duplicate State ID
      • Multiple Documents Returned
      • Online Demographics Mismatch
      • Alt Assessment Flag Changed
    • Late June districts are contacted for NT Reasons for DRC reports
    • August corrections DRC reports revised in NSSRS for SOSR
    • October 2012 SOSR District Preview for Final Corrections (Two weeks from release to make corrections)
check 4 learning

Timeline for System Updates

    • October 1st System Management Update
      • Improved method for uploading students
    • October 29th Test Creation Update
      • Improved process for building reading tests
    • New Items Added Throughout Fall
Check 4 Learning
check 4 learning1

Currently Districts Can

    • Upload students to the system
    • Place students into classes
    • Use old tests
    • Create and use new math and science tests
    • Please note:
      • System will be down: Sept. 28 & Oct. 26
Check 4 Learning
check 4 learning2

WebEx for System Update

    • October 30
      • 2:00 p.m.
    • October 31
      • 10:00 a.m.
    • Recorded and posted on NDE website
Check 4 Learning
check 4 learning3

Ideas for Use:

    • Point of Instruction Assessment
    • Curriculum Unit Assessment
    • Standard Review
    • District/Building Level Assessment
    • New Student Skill Assessment
    • L to J Model
Check 4 Learning

check 4 learning4

Moving Forward

    • Currently
      • Review and Approval of Items
      • Seeking Feedback
      • Determine Needs
    • Minimize Burden
      • Earlier MOU
      • Maintain System that is Useful
Check 4 Learning
check 4 learning5


    • Over 4000 items in the system
    • Over 1500 ready to be approved
    • Last year from Feb. until the end of the year-135,000 assessments were administered online
    • Approximately 180 Districts take part in C4L
Check 4 Learning
nebraska educators continue to be involved in these aspects of nesa w

Prompt Creation

Field Testing

Prompt Selection


Score Verification (formally Writing Appeals)

Continued training and communication

Nebraska Educators continue to be involved in these aspects of NeSA-W:
the rangefinding process

Nebraska Director of Statewide Writing

  • Includes 7-10 Nebraska educators per grade level from a variety of educational backgrounds.
  • Two DRC reps facilitate the work.
The Rangefinding Process:
the rangefinding process1

Analytically score, discuss, and come to consensus on 90-100 papers from the field testing of the test prompt.

  • DRC uses the papers selected and scored by this group to establish the anchors and train the raters prior to the scoring of the assessment.
The Rangefinding Process:
grade 8 and 11 testing time

When time on task was plotted with the scale score the following general conclusions seem plausible:

1. Student scores start to level off after a 75-90 minute period so students should be encouraged to spend a minimum of 75 minutes.

Grade 8 and 11 Testing Time
grade 8 and 11 testing time1

2. With SPED students, the trend is somewhat upward for the entire time tracked, so SPED students could be encouraged to spend as much time as they need as long as they are productive.

3. However, keep in mind students cannot be controlled for time during testing. Training and instruction must come in the classroom before the test.

Grade 8 and 11 Testing Time
scoring shift grades 8 and 11

In April NDE conducted a standard setting for

grades 8 and 11. The results of the process were

presented to the State Board, resulting in new cut

scores for these grades.

The quality of writing did not decline but rather

the board--after discussion over a two day period-

raised the requirements for Meets the Standards.

Scoring Shift – Grades 8 and 11

Under the old holistic grading system, a student at grade 8 and grade 11 would reach Met the Standard with a 2 and a 2+ (4.33).

  • Descriptions in the 2 column included:
        • Digressions
        • Limited unrelated details
        • Occasionally forced or contrived
        • Errors may district

With new analytic system, a student must have a score that falls in the 3 range of the rubric (if equivocated to the old rubric the student needs a 3 and a 3 -).

  • 3 Column: Generally
      • Adequate
      • Logical
      • Functional
      • Paragraphing successful
      • Conventions usually correct and do not distract
scoring shift grades 8 and 111

When looking at the rubrics (old and new), the 3

column is the acceptable standard. The Nebraska

PLD’s (Performance Learning Descriptor) also

clearly reflect the 3’s column on each rubric as the

state writing goal.

The new cut scores will allow some 2’s, but most

likely in the domains of word choice and


Scoring Shift – Grades 8 and 11
field testing at 4 8 11

September 13 – New prompts were written by Nebraska teachers at grades 4, 8, 11. The new prompts need to be field tested.

Field Testing at Grade 4 will remain the same as in the past. Testing will be in the spring from the end of February to middle of May using paper/penciltests.

Field Testing at 4, 8, 11
new process field testing grades 8 and 11

Grades 8 and 11 will be on-line through the NeSA testing site. Prompts are assigned randomly to students.

November 12 - tickets are available and new download if needed. New button on NeSA icon will say NeSA-W field testing.

The field testing window will be November 14 to December 21. (5 1/2 weeks)

Field Testing WebExes November 6-8

New Process - Field Testing Grades 8 and 11
field testing

This is an opportunity for students to practice taking the test under the same conditions as the regular NeSA-W test.

As a field test of future prompts for NeSA–W, the test results cannot be used in any other way. (For example: no local scoring)

NDE appreciates schools supporting this project as we want to keep prompt writing in the hands of Nebraska educators.

Field Testing
analytic scoring training dates grade 4

Designed for teachers.

November 8 - Scottsbluff

November 14 - Kearney

November 16 – Lincoln

Times: 8:30 – 11:30

Materials from the training will be posted to our website at: Writing.htm

Analytic Scoring Training Dates Grade 4
statewide writing assessment grade 4 2012 2013

Will remain in the narrative mode.

Will continue to be two timed 40-minutes sessions

Will still be tested in paper/pencil format. (No online administration planned)

Will allow access to commercial dictionary and thesaurus.

Statewide Writing AssessmentGrade 4 – 2012-2013
statewide writing assessment grade 4 2012 20131

Will be scored analytically in 2013

Will have a new cut score set in April 2013 and approved by the State Board of Education in May.

Will have test results released publicly in the spring of 2013 along with grades 8 and 11.

Statewide Writing AssessmentGrade 4 – 2012-2013
statewide writing assessment grade 4 2012 20132

Will include a new self-assessment tool for analytic scoring.

Information about the modes of writing and student writing samples are found on the assessment website in the writing section:

Statewide Writing AssessmentGrade 4 – 2012-2013
analytic scoring grade 4 starting 2013

Analytic scoring provides five scores for each student: a score for content, a score for organization, a score for word choice and voice, a score for sentence fluency and conventions (punctuation, grammar, spelling), and a total overall composite score.

After the standard setting process, the composite score will be converted to a scale score of 0-70. Students in grade 4 will receive analytic scoring information in 2013.

Analytic Scoring – Grade 4Starting 2013
analytic scoring grade 4 starting 20131

The four domains of writing to be used in analytic scoring:

Scoring domains were weighted as follows:

  • Content and ideas 35%
  • Organization 25%
  • Word choice and Voice 20%
  • Conventions (grammar, punctuation,

spelling) 20%

Analytic Scoring – Grade 4 Starting 2013
grade 4 cut score

In April of 2013, a cut score process will be conducted for the writing assessments in grade 4 so that the following classifications of proficiency determinations can be made:

Exceeds the standards

Meets the standards

Below the standards

Grade 4 - Cut Score
grade 4 cut score1

In December of 2011, NDE conducted a session to create the Performance Level Descriptors for the expectations of Nebraska students based on the analytic rubric. The PLD’s are available on the Assessment web page:

Grade 4 - Cut Score
grade 4 cut scores

The State Board of Education will approve the scores at their May meeting. Following the cut score process and approval in May, the conversion to the scale score (0-70) will occur, and the public release will follow.

Grade 4 - Cut Scores
grade 4 cut score2

Based on what happened at Grades 8/11, we can anticipate a significant shift at Grade 4 in the requirements for Meets the Standard.

Grade 4 – Cut Score
lessons learned from 2012

For any paper/pencil tests, students must write in

#2 PENCIL. Papers written in ink will not be

scored and will result in zero scores.

In 2012 the papers/pencil booklet submitted in

ink, therefore, not scorable, were for the most

part administered in make-up sessions by

someone other than the regular classroom


Lessons Learned from 2012
lessons learned from 20121

In paper/pencil testing, the student response to the writing prompt must be written in the actual test booklet. If a student with an IEP is word processing a paper or using a scribe, the student response must be transcribed into the test booklet in pencil.

Originals of transcribed papers will be sent to DRC in an envelope provided with the shipping materials.

Lessons Learned from 2012

This score verification process is appropriate only for very specific, not general, scoring issues. Only the total score will be submitted. The score verification process is designed for student scores that fall below the state cut score.

Statewide Writing Appeals Process in the future under analytic score will be called Score Verification
appeals score verification cont

Districts will need to complete the score verification form, on the website or from the SAA update. Districts must provide a written explanation indicating the reason for the score verification change and a copy of each paper in question. The writer of the score verification should focus on the particular domain or domains in question. Score verifications must reach the Assessment Office by the deadline.

Score Verification is completed by Nebraska educators who have been involved in the writing grading for years.

Appeals – Score Verification (cont)
nesa news 2013 nesa testing window
NeSA News2013 – NeSA Testing Window

January 21 – February 8

February 11 – March 22

March 25 – May 3

NeSA Writing


NeSA – Reading



NeSA-AA Alternate Tests

online testing 2013

All paper/pencil tests must be ordered October 8-19 through eDirect – writing, reading, mathematics, and science. Enrollment form

Online Testing 2013

Students whose IEP plans or 504 plans specify an accommodation that would require a paper/pencil.

ELL students who need an accommodation that would require a paper/pencil.

Students responding in a language other than English or Spanish (NeSA-W).

Students contracted to institutions where online access is not allowed.

Students who take the alternate assessments-R/M/S

2013 – Online TestingRequirement in reading, mathematics, science, writing (Grades 8 and 11) for all students except:
research for the october paper pencil order

Review the revised Accommodations document-On Assessment Page-NDE

District-Accommodations that are best for the student

A student who takes one content area paper/pencil does not necessarily need paper/pencil in all

Students who qualify for speech may not need any paper/pencil

Inform the student

Research for the October Paper/Pencil order
section i 4 charts for each school district
Section I: 4 charts for each school district
  • Elementary Grade-Level Configuration

Grades 3-5

  • Middle Grade-Level Configuration

Grades 6-8

  • Secondary Grade-Level Configuration

Grades 9-12

  • District

Grades 3-12


Status—Scale Score

Improvement—Scale Score

Growth—Scale Score

Participation--Met/Not Met or Percent

Graduation Rate--Percent


Reading E M S D

Math E M S D

Science* E M S D

Writing* E M S D

Scale Scores and RankingsImprovement= Difference of Average Scale ScoresCompares scores of groups of students

Reading E M D

Math E M D

Scale Scores and RankingsGrowth= Average of Differences in Scale ScoresGrowth based on matched student data in the same district-consecutive years

graduation rate percent and ranking

Will be accompanied by Enrollment- Grades 9-12

4 Year 2012 (Secondary & District)

6 Year* Available 2013 (Secondary & District)

Graduation RatePercent and Ranking
participation 95 met not met no rankings

Reading E M S D

Math E M S D

Science E M S D

Writing E M S D

Participation (95%)Met/Not Met-No Rankings

Section 1--Anywhere District

Nebraska Performance Accountability (NePAS)Elementary Grade-Level Configuration (Grades 3-5)



Ties will receive the same ranking

The next school will receive the next lower number.


If 20 districts are ranked # 1 in a component,the next district will receive a ranking of 21.

nepas business rules still being determined

Appropriate masking (under 10)

Any groups 10 and above to be included

In any criterion where 100% of the students place in a single rating, the scores must be masked—by law. Example 100% graduation rate

NePAS business rules- still being determined
nepas part ii

By Grade Level-Building (Actual physical building)

By Grade Level-District

No Rankings

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

for each building in a district for each grade scale score


Grade 4-Reading

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

For eachBuilding in a district for each Grade-Scale Score
for each district for each grade scale score


Grade 4-Reading

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

For eachDistrict-for each Grade- Scale Score

Section 2.1

School Building Accountability By Grade

Choose Grade: (Drop Down Menu)

Grade 05

state of the schools report
State of the Schools Report


State of the Schools Report--available to districts

  • Disaggregated data
  • NePAS
  • 15 days—then rankings added


State of the Schools Report Public Release

  • Disaggregated data
  • NePAS
your story

The month of October gives districts the opportunity to prepare a message

Others-Focus on the district chart

Your Story

Anywhere School District

School District (Grades 3-12)

your story1

Many rankings—focus on strengths

If there are scores that are weak, provide information that informs the scores

Use the counterbalance of information—the story behind the rankings

Our community wanted to know what we were doing to improve

Your Story
two security events

Caveon Security Report

  • United State Department of Education:

Office of the Inspector General

Two Security Events
examples of score invalidation

Allowing word walls, personal dictionaries, graphic organizers on writing test

Editing student first drafts on writing test

Reading the reading passages to students

Allowing students to carry tests from one teacher to another unmonitored

Providing accommodations not allowed by an IEP

Allowing dictionaries on reading test.

Examples of Score Invalidation
examples of score invalidation1
Examples of Score Invalidation
  • Student accessed help on cell phone
  • Student used inappropriate math document from notebook
reasons for breach in security that called for score invalidations

New Teacher

New Principal

New District Assessment Contact

Substitute Teacher


Make-up Test

Reasons for breach in security- that called for score invalidations
security breach form

District information

Date of incident

Description of incident

Action taken

District response (if known)

Actions taken to assure situation will not be repeated

Security Breach Form
missing materials

List of missing materials

Action district took regarding the breach (missing materials)

District Outcome/Resolution

Missing Materials
webexes galore

Enrollment for Paper/Pencil Orders

    • October 2 and 3
    • “Invitations” posted on Assessment Page
  • C4L Update
    • October 30 and 31
    • “Invitations” posted on Assessment Page
  • Technology Update/Writing Field Test 8 & 11
    • November 6-8
      • Dates Posted
WebExes Galore