The view october 4 2012
Download
1 / 87

The View October 4, 2012 - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

The View October 4, 2012. Dr. Valorie Foy valorie.foy@nebraska.gov. Dr. Valorie Foy, Director Jeremy Heneger, Assistant Director John Moon, NeSA Project Manager Dr. Ed Foy, Writing Coordinator Dr. Ted Larson, NAEP Coordinator Jackie Naber, Administrative Assistant

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The View October 4, 2012' - jess


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
The view october 4 2012

The ViewOctober 4, 2012

Dr. Valorie Foy

valorie.foy@nebraska.gov


The assessment team

Dr. Valorie Foy, Director

Jeremy Heneger, Assistant Director

John Moon, NeSA Project Manager

Dr. Ed Foy, Writing Coordinator

Dr. Ted Larson, NAEP Coordinator

Jackie Naber, Administrative Assistant

Carol Bom, Administrative Assistant

The Assessment Team


Today s agenda

NSSRS

C4L Update

Writing

NeSA

Nebraska Performance Accountability System [NePAS ]

Today’s Agenda



The view october 4 2012

NeSA results ARE an important data source!

When combined with other information, these data can support curricular, instructional, and learning support decision making.

For NeSA...

every student

will be tested!



The view october 4 2012

Nebraska schools should use event. (Reeves, 2009)NeSA data to . . .

  • Provide feedback to students, parents and the community

  • Inform instructional decisions.

  • Inform curriculum development and revision.

  • Measure program success and effectiveness.

  • Promote accountability to meet state and

  • federal requirements.


The view october 4 2012

  • E event. (Reeves, 2009)nsuring Quality Data

  • Online Assessments

    • Make sure students use their own Test Session Ticket

    • Test sessions can be taken in any order

    • Have students exit NeSA completely after testing

    • Install NeSA software early and verify it works using Practice Tests

    • Paper/Pencil Assessments

    • Make sure the student ID is correct and complete (page 2)

    • Affix labels on front of every answer document

    • Separate answer sheets and test booklets

    • Leave all test booklets and answer sheets intact

    • Inventory test materials early and order more if needed

    • Bubble answers on correct part of answer sheet


The view october 4 2012

  • E event. (Reeves, 2009)nsuring Quality Data

  • Code Not Tested (NT) Reasons

    • Use Management Tools to enter NT codes for the online assessments

    • Bubble NT codes on answer sheets for pencil/paper assessments

    • Tested Codes include:

      • No Longer Enrolled (NLE)

      • Parent Refusal (PAR)

      • Emergency Medical Waiver (EMW)

      • Recently Arrived LEP (RAL)

      • Student Absent Entire Window (SAE)

      • Invalid (INV)

      • Other (OTH)


The view october 4 2012

  • E event. (Reeves, 2009)nsuring Quality Data

    • Score Corrections (ZEROS and NT codes)

    • June Review by NDE Staff

      • New Student

      • No Attempted Documents Returned

      • Duplicate State ID

      • Multiple Documents Returned

      • Online Demographics Mismatch

      • Alt Assessment Flag Changed

    • Late June districts are contacted for NT Reasons for DRC reports

    • August corrections DRC reports revised in NSSRS for SOSR

    • October 2012 SOSR District Preview for Final Corrections (Two weeks from release to make corrections)


The view october 4 2012

C4L event. (Reeves, 2009)


Check 4 learning

  • Timeline for System Updates event. (Reeves, 2009)

    • October 1st System Management Update

      • Improved method for uploading students

    • October 29th Test Creation Update

      • Improved process for building reading tests

    • New Items Added Throughout Fall

Check 4 Learning


Check 4 learning1

  • Currently Districts Can event. (Reeves, 2009)

    • Upload students to the system

    • Place students into classes

    • Use old tests

    • Create and use new math and science tests

    • Please note:

      • System will be down: Sept. 28 & Oct. 26

Check 4 Learning


Check 4 learning2

  • WebEx for System Update event. (Reeves, 2009)

    • October 30

      • 2:00 p.m.

    • October 31

      • 10:00 a.m.

    • Recorded and posted on NDE website

Check 4 Learning


Check 4 learning3

  • Ideas for Use: event. (Reeves, 2009)

    • Point of Instruction Assessment

    • Curriculum Unit Assessment

    • Standard Review

    • District/Building Level Assessment

    • New Student Skill Assessment

    • L to J Model

Check 4 Learning

http://www.education.ne.gov/Assessment/C4Learning.htm


Check 4 learning4

  • Moving Forward event. (Reeves, 2009)

    • Currently

      • Review and Approval of Items

      • Seeking Feedback

      • Determine Needs

    • Minimize Burden

      • Earlier MOU

      • Maintain System that is Useful

Check 4 Learning


Check 4 learning5

  • Statistics event. (Reeves, 2009)

    • Over 4000 items in the system

    • Over 1500 ready to be approved

    • Last year from Feb. until the end of the year-135,000 assessments were administered online

    • Approximately 180 Districts take part in C4L

Check 4 Learning


Nesa writing

NeSA Writing event. (Reeves, 2009)


Nebraska educators continue to be involved in these aspects of nesa w

Prompt Creation event. (Reeves, 2009)

Field Testing

Prompt Selection

Rangefinding

Score Verification (formally Writing Appeals)

Continued training and communication

Nebraska Educators continue to be involved in these aspects of NeSA-W:


The rangefinding process

The Rangefinding Process:


The rangefinding process1

The Rangefinding Process:


On line writing times
On-line Writing Times papers from the field testing of the test prompt.


Grade 8 and 11 testing time

When time on task was plotted with the scale score the following general conclusions seem plausible:

1. Student scores start to level off after a 75-90 minute period so students should be encouraged to spend a minimum of 75 minutes.

Grade 8 and 11 Testing Time


Grade 8 and 11 testing time1

2 following general conclusions seem plausible:. With SPED students, the trend is somewhat upward for the entire time tracked, so SPED students could be encouraged to spend as much time as they need as long as they are productive.

3. However, keep in mind students cannot be controlled for time during testing. Training and instruction must come in the classroom before the test.

Grade 8 and 11 Testing Time


Scoring shift grades 8 and 11

In April NDE conducted a standard setting for following general conclusions seem plausible:

grades 8 and 11. The results of the process were

presented to the State Board, resulting in new cut

scores for these grades.

The quality of writing did not decline but rather

the board--after discussion over a two day period-

raised the requirements for Meets the Standards.

Scoring Shift – Grades 8 and 11


The view october 4 2012

  • Under the old holistic grading system, a student following general conclusions seem plausible:at grade 8 and grade 11 would reach Met the Standard with a 2 and a 2+ (4.33).

  • Descriptions in the 2 column included:

    • Digressions

    • Limited unrelated details

    • Occasionally forced or contrived

    • Errors may district


The view october 4 2012

  • With new analytic system, a student must have following general conclusions seem plausible:a score that falls in the 3 range of the rubric (if equivocated to the old rubric the student needs a 3 and a 3 -).

  • 3 Column: Generally

    • Adequate

    • Logical

    • Functional

    • Paragraphing successful

    • Conventions usually correct and do not distract


Scoring shift grades 8 and 111

When looking at the rubrics (old and new), the 3 following general conclusions seem plausible:

column is the acceptable standard. The Nebraska

PLD’s (Performance Learning Descriptor) also

clearly reflect the 3’s column on each rubric as the

state writing goal.

The new cut scores will allow some 2’s, but most

likely in the domains of word choice and

conventions.

Scoring Shift – Grades 8 and 11


Field testing at 4 8 11

September following general conclusions seem plausible: 13 – New prompts were written by Nebraska teachers at grades 4, 8, 11. The new prompts need to be field tested.

Field Testing at Grade 4 will remain the same as in the past. Testing will be in the spring from the end of February to middle of May using paper/penciltests.

Field Testing at 4, 8, 11


New process field testing grades 8 and 11

Grades 8 and 11 following general conclusions seem plausible:will be on-line through the NeSA testing site. Prompts are assigned randomly to students.

November 12 - tickets are available and new download if needed. New button on NeSA icon will say NeSA-W field testing.

The field testing window will be November 14 to December 21. (5 1/2 weeks)

Field Testing WebExes November 6-8

New Process - Field Testing Grades 8 and 11


Field testing

This is an following general conclusions seem plausible:opportunity for students to practice taking the test under the same conditions as the regular NeSA-W test.

As a field test of future prompts for NeSA–W, the test results cannot be used in any other way. (For example: no local scoring)

NDE appreciates schools supporting this project as we want to keep prompt writing in the hands of Nebraska educators.

Field Testing


Analytic scoring training dates grade 4

Designed for teachers. following general conclusions seem plausible:

November 8 - Scottsbluff

November 14 - Kearney

November 16 – Lincoln

Times: 8:30 – 11:30

Materials from the training will be posted to our website at:

http://www.education.ne.gov/Assement Writing.htm

Analytic Scoring Training Dates Grade 4


Statewide writing assessment grade 4 2012 2013

Will remain in the narrative mode. following general conclusions seem plausible:

Will continue to be two timed 40-minutes sessions

Will still be tested in paper/pencil format. (No online administration planned)

Will allow access to commercial dictionary and thesaurus.

Statewide Writing AssessmentGrade 4 – 2012-2013


Statewide writing assessment grade 4 2012 20131

Will be scored analytically in 2013 following general conclusions seem plausible:

Will have a new cut score set in April 2013 and approved by the State Board of Education in May.

Will have test results released publicly in the spring of 2013 along with grades 8 and 11.

Statewide Writing AssessmentGrade 4 – 2012-2013


Statewide writing assessment grade 4 2012 20132

Will include a new self-assessment tool for analytic scoring following general conclusions seem plausible:.

Information about the modes of writing and student writing samples are found on the assessment website in the writing section:

http://www.educatin.ne.gov/Assessment/NeSAWriting.htm

Statewide Writing AssessmentGrade 4 – 2012-2013


New at grade 4 on the test self assessment tool

ON following general conclusions seem plausible:

New at grade 4 ON the Test – Self-Assessment Tool


Analytic scoring grade 4 starting 2013

Analytic scoring provides five scores for each student: a score for content, a score for organization, a score for word choice and voice, a score for sentence fluency and conventions (punctuation, grammar, spelling), and a total overall composite score.

After the standard setting process, the composite score will be converted to a scale score of 0-70. Students in grade 4 will receive analytic scoring information in 2013.

Analytic Scoring – Grade 4Starting 2013


Analytic scoring grade 4 starting 20131

  • The four domains of writing to be used in analytic scoring: score for content, a score for organization, a score for word choice and voice, a score for sentence fluency and conventions (punctuation, grammar, spelling), and a total overall composite score.

    Scoring domains were weighted as follows:

  • Content and ideas 35%

  • Organization 25%

  • Word choice and Voice 20%

  • Conventions (grammar, punctuation,

    spelling) 20%

Analytic Scoring – Grade 4 Starting 2013


Grade 4 cut score

In April of 2013, a cut score process will be conducted for the writing assessments in grade 4 so that the following classifications of proficiency determinations can be made:

Exceeds the standards

Meets the standards

Below the standards

Grade 4 - Cut Score


Grade 4 cut score1

In December of 2011, NDE conducted a session to create the Performance Level Descriptors for the expectations of Nebraska students based on the analytic rubric. The PLD’s are available on the Assessment web page:

http://www.education.ne.gov/Assessment/NeSA_Writing.htm

Grade 4 - Cut Score


Grade 4 cut scores

The State Board of Education will approve the scores at their May meeting. Following the cut score process and approval in May, the conversion to the scale score (0-70) will occur, and the public release will follow.

Grade 4 - Cut Scores


Grade 4 cut score2

Based on what happened at Grades 8/11, we can anticipate a significant shift at Grade 4 in the requirements for Meets the Standard.

Grade 4 – Cut Score


Lessons learned from 2012

For any paper/pencil tests, students must write in significant shift at Grade 4 in the requirements for Meets the Standard.

#2 PENCIL. Papers written in ink will not be

scored and will result in zero scores.

In 2012 the papers/pencil booklet submitted in

ink, therefore, not scorable, were for the most

part administered in make-up sessions by

someone other than the regular classroom

teacher.

Lessons Learned from 2012


Lessons learned from 20121

In paper/pencil testing, the student response to the writing prompt must be written in the actual test booklet. If a student with an IEP is word processing a paper or using a scribe, the student response must be transcribed into the test booklet in pencil.

Originals of transcribed papers will be sent to DRC in an envelope provided with the shipping materials.

Lessons Learned from 2012


The view october 4 2012

This score verification process is appropriate only for very specific, not general, scoring issues. Only the total score will be submitted. The score verification process is designed for student scores that fall below the state cut score.

Statewide Writing Appeals Process in the future under analytic score will be called Score Verification


Appeals score verification cont

Districts will need to complete the specific, not general, scoring issues. Only the total score will be submitted. The score verification form, on the website or from the SAA update. Districts must provide a written explanation indicating the reason for the score verification change and a copy of each paper in question. The writer of the score verification should focus on the particular domain or domains in question. Score verifications must reach the Assessment Office by the deadline.

Score Verification is completed by Nebraska educators who have been involved in the writing grading for years.

Appeals – Score Verification (cont)


The view october 4 2012

Questions contact: specific, not general, scoring issues. Only the total score will be submitted. The

edward.foy@nebraska.gov

Phone: 402-471-2947


The view october 4 2012

NeSA specific, not general, scoring issues. Only the total score will be submitted. The


Nesa news 2013 nesa testing window
NeSA News specific, not general, scoring issues. Only the total score will be submitted. The 2013 – NeSA Testing Window

January 21 – February 8

February 11 – March 22

March 25 – May 3

NeSA Writing

ELDA

NeSA – Reading

Mathematics

Science

NeSA-AA Alternate Tests


Online testing 2013

All paper/pencil tests must be ordered specific, not general, scoring issues. Only the total score will be submitted. The October 8-19 through eDirect – writing, reading, mathematics, and science. Enrollment form

Online Testing 2013


The view october 4 2012

Students whose specific, not general, scoring issues. Only the total score will be submitted. The IEP plans or 504 plans specify an accommodation that would require a paper/pencil.

ELL students who need an accommodation that would require a paper/pencil.

Students responding in a language other than English or Spanish (NeSA-W).

Students contracted to institutions where online access is not allowed.

Students who take the alternate assessments-R/M/S

2013 – Online TestingRequirement in reading, mathematics, science, writing (Grades 8 and 11) for all students except:


Research for the october paper pencil order

Review the revised Accommodations document-On Assessment Page-NDE

District-Accommodations that are best for the student

A student who takes one content area paper/pencil does not necessarily need paper/pencil in all

Students who qualify for speech may not need any paper/pencil

Inform the student

Research for the October Paper/Pencil order


Enrollment webexes

October 2-4, 2012 Page-NDE– “Invitation” is posted

Enrollment WebExes


Nepas

NePAS Page-NDE


Nebraska performance accountability system nepas

Nebraska Performance Accountability System - NePAS


Section i scores with rankings

1. Three NePAS Grade-Level Page-NDEConfigurations

2. School District

Section I-Scores with Rankings


Section i 4 charts for each school district
Section I: 4 charts Page-NDEfor each school district

  • Elementary Grade-Level Configuration

    Grades 3-5

  • Middle Grade-Level Configuration

    Grades 6-8

  • Secondary Grade-Level Configuration

    Grades 9-12

  • District

    Grades 3-12


Components

Status—Scale Score Page-NDE

Improvement—Scale Score

Growth—Scale Score

Participation--Met/Not Met or Percent

Graduation Rate--Percent

Components


Scale scores and rankings status average scale score

Reading E M S D Page-NDE

Math E M S D

Science E M S D

Writing E M S D

Scale Scores and RankingsStatus=Average Scale Score


The view october 4 2012

Reading E M S D Page-NDE

Math E M S D

Science* E M S D

Writing* E M S D

Scale Scores and RankingsImprovement= Difference of Average Scale ScoresCompares scores of groups of students


The view october 4 2012

Reading E M D Page-NDE

Math E M D

Scale Scores and RankingsGrowth= Average of Differences in Scale ScoresGrowth based on matched student data in the same district-consecutive years


Graduation rate percent and ranking

Will be accompanied by Enrollment- Grades 9-12 Page-NDE

4 Year 2012 (Secondary & District)

6 Year* Available 2013 (Secondary & District)

Graduation RatePercent and Ranking


Participation 95 met not met no rankings

Reading E M S D Page-NDE

Math E M S D

Science E M S D

Writing E M S D

Participation (95%)Met/Not Met-No Rankings


Cut scores charts are on nepas screens

Reading, Math, Science Page-NDE

8th and 11th Grade Writing

Cut Scores Charts are on NePAS screens


The view october 4 2012

  • Section 1 Page-NDE--Anywhere District

    Nebraska Performance Accountability (NePAS)Elementary Grade-Level Configuration (Grades 3-5)

DRAFT


Rankings

Ties will receive the same ranking Page-NDE

The next school will receive the next lower number.

Example:

If 20 districts are ranked # 1 in a component,the next district will receive a ranking of 21.

Rankings


Nepas business rules still being determined

Appropriate masking (under 10) Page-NDE

Any groups 10 and above to be included

In any criterion where 100% of the students place in a single rating, the scores must be masked—by law. Example 100% graduation rate

NePAS business rules- still being determined


With inclusion of ten or more number of districts possible in reading and math
With inclusion of ten or more- Page-NDENumber of districts possible in reading and math


Nepas part ii

By Grade Level-Building (Actual physical building) Page-NDE

By Grade Level-District

No Rankings

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

NePAS Part II


For each building in a district for each grade scale score

Example Page-NDE

Grade 4-Reading

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

For eachBuilding in a district for each Grade-Scale Score


For each district for each grade scale score

Example Page-NDE

Grade 4-Reading

Status (SS)

Improvement (SS)

Growth (SS)

Participation (%)

For eachDistrict-for each Grade- Scale Score


The view october 4 2012

Section Page-NDE2.1

School Building Accountability By Grade

Choose Grade: (Drop Down Menu)

Grade 05


State of the schools report
State of the Schools Report Page-NDE

October

State of the Schools Report--available to districts

  • Disaggregated data

  • AYP/PLAS

  • NePAS

  • 15 days—then rankings added

    November

    State of the Schools Report Public Release

  • Disaggregated data

  • AYP/PLAS

  • NePAS


Your story

The month of October gives districts the opportunity to prepare a message

Others-Focus on the district chart

Your Story


The view october 4 2012

Anywhere School District prepare a message

School District (Grades 3-12)


Your story1

Many prepare a messagerankings—focus on strengths

If there are scores that are weak, provide information that informs the scores

Use the counterbalance of information—the story behind the rankings

Our community wanted to know what we were doing to improve

Your Story


Test security

Test Security prepare a message


Two security events

  • Caveon Security Report prepare a message

  • United State Department of Education:

    Office of the Inspector General

Two Security Events


Security

Equity prepare a message

Security


Examples of score invalidation

Allowing word walls, personal dictionaries, graphic organizers on writing test

Editing student first drafts on writing test

Reading the reading passages to students

Allowing students to carry tests from one teacher to another unmonitored

Providing accommodations not allowed by an IEP

Allowing dictionaries on reading test.

Examples of Score Invalidation


Examples of score invalidation1
Examples of Score Invalidation organizers on writing test

  • Student accessed help on cell phone

  • Student used inappropriate math document from notebook


Reasons for breach in security that called for score invalidations

New Teacher organizers on writing test

New Principal

New District Assessment Contact

Substitute Teacher

Paraprofessional

Make-up Test

Reasons for breach in security- that called for score invalidations


Security breach form

District information organizers on writing test

Date of incident

Description of incident

Action taken

District response (if known)

Actions taken to assure situation will not be repeated

Security Breach Form


Missing materials

List of missing materials organizers on writing test

Action district took regarding the breach (missing materials)

District Outcome/Resolution

Missing Materials


Webexes galore

  • Enrollment for Paper/Pencil Orders organizers on writing test

    • October 2 and 3

    • “Invitations” posted on Assessment Page

  • C4L Update

    • October 30 and 31

    • “Invitations” posted on Assessment Page

  • Technology Update/Writing Field Test 8 & 11

    • November 6-8

      • Dates Posted

WebExes Galore


The view october 4 2012

Questions? organizers on writing test

http://www.education.ne.gov/Assessment/Index.html