The View October 4, 2012. Dr. Valorie Foy [email protected] Dr. Valorie Foy, Director Jeremy Heneger, Assistant Director John Moon, NeSA Project Manager Dr. Ed Foy, Writing Coordinator Dr. Ted Larson, NAEP Coordinator Jackie Naber, Administrative Assistant
Jeremy Heneger, Assistant Director
John Moon, NeSA Project Manager
Dr. Ed Foy, Writing Coordinator
Dr. Ted Larson, NAEP Coordinator
Jackie Naber, Administrative Assistant
Carol Bom, Administrative AssistantThe Assessment Team
Nebraska Performance Accountability System [NePAS ]Today’s Agenda
When combined with other information, these data can support curricular, instructional, and learning support decision making.
will be tested!
Commit to data analysis as a continuous process, not an event. (Reeves, 2009)
Nebraska schools should use event. (Reeves, 2009)NeSA data to . . .
Prompt Creation event. (Reeves, 2009)
Score Verification (formally Writing Appeals)
Continued training and communicationNebraska Educators continue to be involved in these aspects of NeSA-W:
When time on task was plotted with the scale score the following general conclusions seem plausible:
1. Student scores start to level off after a 75-90 minute period so students should be encouraged to spend a minimum of 75 minutes.Grade 8 and 11 Testing Time
2 following general conclusions seem plausible:. With SPED students, the trend is somewhat upward for the entire time tracked, so SPED students could be encouraged to spend as much time as they need as long as they are productive.
3. However, keep in mind students cannot be controlled for time during testing. Training and instruction must come in the classroom before the test.Grade 8 and 11 Testing Time
In April NDE conducted a standard setting for following general conclusions seem plausible:
grades 8 and 11. The results of the process were
presented to the State Board, resulting in new cut
scores for these grades.
The quality of writing did not decline but rather
the board--after discussion over a two day period-
raised the requirements for Meets the Standards.Scoring Shift – Grades 8 and 11
When looking at the rubrics (old and new), the 3 following general conclusions seem plausible:
column is the acceptable standard. The Nebraska
PLD’s (Performance Learning Descriptor) also
clearly reflect the 3’s column on each rubric as the
state writing goal.
The new cut scores will allow some 2’s, but most
likely in the domains of word choice and
conventions.Scoring Shift – Grades 8 and 11
September following general conclusions seem plausible: 13 – New prompts were written by Nebraska teachers at grades 4, 8, 11. The new prompts need to be field tested.
Field Testing at Grade 4 will remain the same as in the past. Testing will be in the spring from the end of February to middle of May using paper/penciltests.Field Testing at 4, 8, 11
Grades 8 and 11 following general conclusions seem plausible:will be on-line through the NeSA testing site. Prompts are assigned randomly to students.
November 12 - tickets are available and new download if needed. New button on NeSA icon will say NeSA-W field testing.
The field testing window will be November 14 to December 21. (5 1/2 weeks)
Field Testing WebExes November 6-8New Process - Field Testing Grades 8 and 11
This is an following general conclusions seem plausible:opportunity for students to practice taking the test under the same conditions as the regular NeSA-W test.
As a field test of future prompts for NeSA–W, the test results cannot be used in any other way. (For example: no local scoring)
NDE appreciates schools supporting this project as we want to keep prompt writing in the hands of Nebraska educators.Field Testing
Designed for teachers. following general conclusions seem plausible:
November 8 - Scottsbluff
November 14 - Kearney
November 16 – Lincoln
Times: 8:30 – 11:30
Materials from the training will be posted to our website at:
http://www.education.ne.gov/Assement Writing.htmAnalytic Scoring Training Dates Grade 4
Will remain in the narrative mode. following general conclusions seem plausible:
Will continue to be two timed 40-minutes sessions
Will still be tested in paper/pencil format. (No online administration planned)
Will allow access to commercial dictionary and thesaurus.Statewide Writing AssessmentGrade 4 – 2012-2013
Will be scored analytically in 2013 following general conclusions seem plausible:
Will have a new cut score set in April 2013 and approved by the State Board of Education in May.
Will have test results released publicly in the spring of 2013 along with grades 8 and 11.Statewide Writing AssessmentGrade 4 – 2012-2013
Will include a new self-assessment tool for analytic scoring following general conclusions seem plausible:.
Information about the modes of writing and student writing samples are found on the assessment website in the writing section:
http://www.educatin.ne.gov/Assessment/NeSAWriting.htmStatewide Writing AssessmentGrade 4 – 2012-2013
ON following general conclusions seem plausible:New at grade 4 ON the Test – Self-Assessment Tool
Analytic scoring provides five scores for each student: a score for content, a score for organization, a score for word choice and voice, a score for sentence fluency and conventions (punctuation, grammar, spelling), and a total overall composite score.
After the standard setting process, the composite score will be converted to a scale score of 0-70. Students in grade 4 will receive analytic scoring information in 2013.Analytic Scoring – Grade 4Starting 2013
Scoring domains were weighted as follows:
In April of 2013, a cut score process will be conducted for the writing assessments in grade 4 so that the following classifications of proficiency determinations can be made:
Exceeds the standards
Meets the standards
Below the standardsGrade 4 - Cut Score
In December of 2011, NDE conducted a session to create the Performance Level Descriptors for the expectations of Nebraska students based on the analytic rubric. The PLD’s are available on the Assessment web page:
http://www.education.ne.gov/Assessment/NeSA_Writing.htmGrade 4 - Cut Score
The State Board of Education will approve the scores at their May meeting. Following the cut score process and approval in May, the conversion to the scale score (0-70) will occur, and the public release will follow.Grade 4 - Cut Scores
Based on what happened at Grades 8/11, we can anticipate a significant shift at Grade 4 in the requirements for Meets the Standard.Grade 4 – Cut Score
For any paper/pencil tests, students must write in significant shift at Grade 4 in the requirements for Meets the Standard.
#2 PENCIL. Papers written in ink will not be
scored and will result in zero scores.
In 2012 the papers/pencil booklet submitted in
ink, therefore, not scorable, were for the most
part administered in make-up sessions by
someone other than the regular classroom
teacher.Lessons Learned from 2012
In paper/pencil testing, the student response to the writing prompt must be written in the actual test booklet. If a student with an IEP is word processing a paper or using a scribe, the student response must be transcribed into the test booklet in pencil.
Originals of transcribed papers will be sent to DRC in an envelope provided with the shipping materials.Lessons Learned from 2012
This score verification process is appropriate only for very specific, not general, scoring issues. Only the total score will be submitted. The score verification process is designed for student scores that fall below the state cut score.Statewide Writing Appeals Process in the future under analytic score will be called Score Verification
Districts will need to complete the specific, not general, scoring issues. Only the total score will be submitted. The score verification form, on the website or from the SAA update. Districts must provide a written explanation indicating the reason for the score verification change and a copy of each paper in question. The writer of the score verification should focus on the particular domain or domains in question. Score verifications must reach the Assessment Office by the deadline.
Score Verification is completed by Nebraska educators who have been involved in the writing grading for years.Appeals – Score Verification (cont)
January 21 – February 8
February 11 – March 22
March 25 – May 3
NeSA – Reading
NeSA-AA Alternate Tests
All paper/pencil tests must be ordered specific, not general, scoring issues. Only the total score will be submitted. The October 8-19 through eDirect – writing, reading, mathematics, and science. Enrollment formOnline Testing 2013
Students whose specific, not general, scoring issues. Only the total score will be submitted. The IEP plans or 504 plans specify an accommodation that would require a paper/pencil.
ELL students who need an accommodation that would require a paper/pencil.
Students responding in a language other than English or Spanish (NeSA-W).
Students contracted to institutions where online access is not allowed.
Students who take the alternate assessments-R/M/S2013 – Online TestingRequirement in reading, mathematics, science, writing (Grades 8 and 11) for all students except:
District-Accommodations that are best for the student
A student who takes one content area paper/pencil does not necessarily need paper/pencil in all
Students who qualify for speech may not need any paper/pencil
Inform the studentResearch for the October Paper/Pencil order
October 2-4, 2012 Page-NDE– “Invitation” is postedEnrollment WebExes
1. Three NePAS Grade-Level Page-NDEConfigurations
2. School DistrictSection I-Scores with Rankings
Status—Scale Score Page-NDE
Participation--Met/Not Met or Percent
Reading E M S D Page-NDE
Math E M S D
Science E M S D
Writing E M S DScale Scores and RankingsStatus=Average Scale Score
Reading E M S D Page-NDE
Math E M S D
Science* E M S D
Writing* E M S DScale Scores and RankingsImprovement= Difference of Average Scale ScoresCompares scores of groups of students
Reading E M D Page-NDE
Math E M D
Scale Scores and RankingsGrowth= Average of Differences in Scale ScoresGrowth based on matched student data in the same district-consecutive years
4 Year 2012 (Secondary & District)
6 Year* Available 2013 (Secondary & District)Graduation RatePercent and Ranking
Reading E M S D Page-NDE
Math E M S D
Science E M S D
Writing E M S DParticipation (95%)Met/Not Met-No Rankings
Reading, Math, Science Page-NDE
8th and 11th Grade WritingCut Scores Charts are on NePAS screens
Nebraska Performance Accountability (NePAS)Elementary Grade-Level Configuration (Grades 3-5)
Ties will receive the same ranking Page-NDE
The next school will receive the next lower number.
If 20 districts are ranked # 1 in a component,the next district will receive a ranking of 21.Rankings
Appropriate masking (under 10) Page-NDE
Any groups 10 and above to be included
In any criterion where 100% of the students place in a single rating, the scores must be masked—by law. Example 100% graduation rateNePAS business rules- still being determined
By Grade Level-District
Participation (%)NePAS Part II
Participation (%)For eachBuilding in a district for each Grade-Scale Score
Participation (%)For eachDistrict-for each Grade- Scale Score
School Building Accountability By Grade
Choose Grade: (Drop Down Menu)
State of the Schools Report--available to districts
State of the Schools Report Public Release
The month of October gives districts the opportunity to prepare a message
Others-Focus on the district chartYour Story
Anywhere School District prepare a message
School District (Grades 3-12)
Many prepare a messagerankings—focus on strengths
If there are scores that are weak, provide information that informs the scores
Use the counterbalance of information—the story behind the rankings
Our community wanted to know what we were doing to improveYour Story
Office of the Inspector General
Equity prepare a messageSecurity
Allowing word walls, personal dictionaries, graphic organizers on writing test
Editing student first drafts on writing test
Reading the reading passages to students
Allowing students to carry tests from one teacher to another unmonitored
Providing accommodations not allowed by an IEP
Allowing dictionaries on reading test.Examples of Score Invalidation
New Teacher organizers on writing test
New District Assessment Contact
Make-up TestReasons for breach in security- that called for score invalidations
District information organizers on writing test
Date of incident
Description of incident
District response (if known)
Actions taken to assure situation will not be repeatedSecurity Breach Form
List of missing materials organizers on writing test
Action district took regarding the breach (missing materials)
District Outcome/ResolutionMissing Materials
Questions? organizers on writing test