1 / 48

Assessment Update Webinar

This webinar provides important updates on assessment operations, TIDE, reporting documents, parent communication, data, reporting timelines, system launches, TIDE updates, content development, teacher involvement, digital library storefront, ELA scoring, NAEP, ELPA21, and technology.

sallyj
Download Presentation

Assessment Update Webinar

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Update Webinar SY 16-17 Update #10 JUNE 21, 2017 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  2. Topics • Operations • TIDE • Reporting Documents • Parent Communication • Data • Reporting Timelines • System Launches • TIDE Updates • Content Development • Teacher Involvement/Development Opportunities • Digital Library Storefront • ELA Scoring • NAEP • Participation/Field Notification • ELPA21 • Screener and Reports • Technology OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  3. Operations OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  4. TIDE Appeals • Online Testers: Appeals to Invalidate and Reset student online test results may be flagged in TIDE now through June 25. • Paper Testers: Appeals to Invalidate and Reset student paper test results may be flagged in TIDE according to the schedule outlined below. Only state-level users will be able to approve invalidation appeals.

  5. Participation Codes in TIDE Participation Window in TIDE closes July 3 • Within this application, districts can establish students attempt code (reason not tested). It also allows for districts to identify exemptions (NNEP, Home-based, Private school, F1 Visa). • The state does not utilize the Reason not Tested and Exemptions data from TIDE. The Exemption calculations are collected from CEDARS. • L2 Basic must be applied in TIDE, under Participation Codes, prior to the end of the window on July 3. The state utilizes this data from TIDE for reporting to ORS, ISRs, and the Report Card web site. Data Required for State Use Optional; District Use Only

  6. Reporting Documents All reporting documents, as outlined in the PIRG, are to be reported to the state. • Test Incident Report:All test incidents must be reported by Friday, June 23 • Note whether students were able to complete their testing opportunity • District Administration and Security Report: (Required) – Completed reports for each administration must be signed by the district Superintendent and submitted to the state within one week of the conclusion of the district’s test administration. • Test Material Variance Form: Discrepancies in the quantity of materials received or returned to vendor, or damaged materials shall be reported immediately. • Missing or damaged materials must also be notated in your District Administration and Security Report.   Do notsend secure or confidential student information through email

  7. Parent Communication Spring 2017: Students took one or more of the Smarter Balanced, MSP, Off-Grade Level, and/or EOC assessments. • Individual Score Reports (ISRs): Hard copy reports arrive in district August 17—18 • Shipment includes two copies of each ISR. One copy for parents and one copy for student folder. • Support Documents: Materials in support of parent communication available the first of August (English versions) • State Testing Parent Letter Templates • Your Child’s Progress handout • Graduation in Washington Toolkit Translated versions available the first of September

  8. Testing Options SY 2017-18 • School Year 2017-18 State Testing Schedule Calendar Posted: http://www.k12.wa.us/assessment/StateTesting/timelines-calendars.aspx • School Year 2016-17 calendar available for reference • Considerations for School Testing Schedules • Paper-pencil tests for students with IEP designating paper as the testing platform • Early release and late starts • Holidays and Winter and Spring breaks • PT 15 day expiration and CAT 45 day expiration

  9. Data OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  10. Data – CEDARS/DDSM activities • Directory updates due by July 3 • Opening Schools, reopening schools • Grade spans • Category/type changes • LEA changes • Work with DDSM for more information • End of year and start of year rollover activites

  11. Data Reporting Timelines • Final enrollment data extract from CEDARS: July 10 • Complete scorefiles arriving at OSPI: • First file to OSPI from AIR (MSP Science) July 17 • Last file to OSPI from AIR (EOC math and bio) July 24 • Final data all approved July 25 • Then, we merge with data from CEDARS, QC the data, and load the data to our applications: • EDS applications (CAA/CIA, WAMS, Report Card Preview, WA Query): August 8 (target) • Report Card Live (current schedule) August 15 • SGPs: tentatively late August or early September (tbd)

  12. Systems Launches: 2017-18 • TIDE: August 15 • TS GOLD: August 15 • eDirect: tbd (early September) • WAMS Registrations open during week of Labor Day • Spring WA-AIM • Hard-copy accommodated materials • Winter and Spring EOC • ELPA21 • NOTE: specific registration deadlines vary by task

  13. TIDE Update for 2017-18 • Observations from several years of automated TIDE processes • Routine: “Student enrolled here, but the old district hasn’t exited” • Routine: “Student isn’t enrolled but will be testing with us” • Common: “Student enrolled with us all year is somewhere else now” • Common: “Student has two records in TIDE, one with the correct SSID…” • Rare: “Student took part of test under SSID 1 and another under SSID 2…” • Rare: “Student showed up and MUST test before our next CEDARS submission” • Rarest: “Student showed up today and MUST test today!” • Conclusion 1: 5 minute files from issuance are creating as many problems as they solve. • Conclusion 2: If a student could be in 2 schools at once in TIDE, we wouldn’t need so many workarounds OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  14. TIDE Update for 2017-18 (part 2) • TIDE able to support visibility of student records in multiple locations • Current proposal for data flow redesign: • 1) Nightly files continue based on full CEDARS enrollment records • BUT during the lag between a student exiting and a student entering, a record is sent at new site • All test settings follow student to new site • Demographics/identifiers update based on new site • Both sites can see record until old site reports ‘exit’ • 2) No more ‘5 minute files’ directly from issuance • Caused HUGE volume of SSID merges and deletes needed in TIDE • Created a 3-points-of-contact solution to fully clean up records in TIDE OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  15. TIDE Update for 2017-18 (part 3) • TIDE able to support visibility of student records in multiple locations • 3) Add WAMS ‘Send Student Record’ screen • Only connects to TIDE (not AIM which requires SpEd flag) • Leverages same SSID Issuance table as 5 minute files process • Only sends records intentionally (no more ‘accidentally pulled all the skills center kids…’) • 4) OSPI Data Analysts retain add/upload student function • “Last Resort” for students needing to test RIGHT NOW!! • Will only add/upload based on real SSIDs – no temp records without SSIDs (except ELPA21 screener) OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  16. NAEP OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  17. 2018 NAEP and International Assessments Update For 2018, Washington is participating in NAEP, TIMSS, and ICILS. Assessed grades and subjects have been posted to the NAEP website at http://www.k12.wa.us/assessment/NAEP/default.aspx . • On June 16, 2017 OSPI notified superintendents and assessment coordinators of districts selected to participate in 2018 NAEP, ICILS, or TIMSS. • Selected schools will be notified between June 23-June 26, 2017. School notifications will include the estimated sample size, the school’s assessment date, and space and time requirements. District assessment coordinators will be included in these notifications. • Schools should notify OSPI by July 10, 2017 to request a new assessment date if there is a conflict. • The NAEP administration will occur during Winter 2018, TIMSS and ICILS administrations will occur during Spring/Summer 2018.

  18. Content Development OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  19. Science Assessment Development OpportunitiesFall 2017—Summer 2018 Ongoing Educator Workgroup Assessment Development Timeline • September 2017: Content Review with Data Review student performance data for every field tested item and make recommendations for the operational item bank. • October 2017: Content Review Review the products of Cluster Development to assure every stimulus, item, and rubric is scientifically accurate and gathers evidence about student mastery of the Washington State 2013 K-12 Science Learning Standards (NGSS). 

  20. Science Assessment Development OpportunitiesFall 2017—Summer 2018 (part 2) Ongoing Educator Workgroup Assessment Development Timeline (continued) • Spring 2018: Item Cluster Work Groups Write items aligned to the Washington State 2013 K-12 Science Learning Standards (NGSS) for science stimuli. • Summer 2018: Pilot Rangefinding and Rubric Validation of field-test items Pilot Range Finding: Review rubrics and student responses for hand-scored constructed-response field test items; finalize decisions about how every student answer on these items will be scored Rubric Validation: Review rubrics and student responses for machine-scored technology-enhanced field test items; finalize decisions about how every student answer on these items will be machine-scored.

  21. Science Assessment Opportunitiesfor new NGSS test, only in 2017—2018 • Fall/Winter 2017: Achievement Level Descriptors (ALDs) Produce draft ALDs, descriptions of the knowledge and skills students demonstrate in each performance level. • Winter/Spring 2018: Alignment Study Evaluate the alignment of the science tests to the Washington State 2013 K-12 Science Learning Standards (NGSS). • Winter/Spring 2018: Contrasting Groups Study Use the Achievement Level Descriptors (ALDs) to predict how students will perform on the science assessment. Online ALD training is provided prior to this study. • Late Summer 2018: Achievement Level Setting Develop threshold score recommendations for student performance levels (Level 1, Level 2, Level 3, Level 4) on the state science tests.

  22. Science Assessment Listserv Interested in receiving email/text updates from Science Assessment? • Go to the Subscribe page for GovDelivery. • Choose a subscription type. • Enter your email address/wireless number. • On the Subscriptions page, select Content Areas > Science, then select the grade band(s) for which you would like to receive information. • If you are interested in receiving updates from other OSPI departments, you can select those areas as well. • Follow the directions until you have completed registration.

  23. Teacher Involvement Update:Smarter Balanced • In March, we had over 200 educators apply for a wide range of Smarter Balanced activities through the end of the year. • Recently, two more opportunities came open for work in October. • About 100 educators applied; application closed June 15. • OSPI is notifying all applicants this week of status • Smarter Balanced will make the final selection based on representation of grade levels, content, and demographic data across states • Educators will be contacted directly by Smarter Balanced or vendors managing the projects • Interested in receiving notices about activities like this? Subscribe for Math, ELA, and/or Digital Library email/text alerts. OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  24. Teacher Involvement Update:WA Specific In addition to Smarter Balanced opportunities… • ELA and Mathematics content teams will be doing “Member-managed” item development • Washington educators will work with OSPI staff to write and review assessment items that will be used on Smarter Balanced assessments • Originally scheduled for April • Rescheduled for July • Interested in receiving notices about activities like this? Subscribe for Math and/or ELA email/text alerts. OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  25. Smarter Balanced Digital Library:New Storefront • Smarter Balanced has released a public-facing Digital Library Storefront web page: SBDigitalLibrary.org • Gives examples of the ideas, lessons, and activities you’ll find in the Digital Library • Provides videos showing what teachers have to say about the Digital Library • Contains videos showing some of the new features of the Digital Library • Includes links to the Digital Library log-in page and a link for educators in Washington to register for their free account. • Please share this page with educators, parents, community partners, etc. OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  26. ELA Performance Task Full Write Scoring • Questions: • How can students earn 0 points on the rubric and still be at/near or above standard in their writing claim? • Has the scoring of the full write items changed from previous years? • What kinds of responses earn 0 points? • Are more students earning 0 points this year compared to last year? • Where in ORS can districts find data about writing mode and scores? • How can teachers help students prepare for full write items?

  27. Full Write Scoring and Writing Claim • How can students earn 0 points on the rubric and still be at/near or above standard in their writing claim? • There are 11 items that contribute to the writing claim. The Full Write item is only 1 of those items. • The other 10 items are included in the CAT. • Examples of the other items can be found in the “LangVocab,” “Edit,” “Revise,” and “Brief Write” IABs. See the WCAP Portal for more information about: • Claim scores in the Understanding Smarter Balanced Assessment Scores Audio presentation in the Test Coordinators > Modulesfolder. • Writing targets in the ELA Summative Blueprint in the Test Administrators > General Information folder.

  28. Full Write Scoring Process • Has the scoring of full write items changed from previous years? • No, the processes and procedures for training the professional scorers have not changed from previous years. • In addition, the processes and procedures for scoring the full write items have notchanged. • All the rubrics, training materials, and training processes have been approved by Smarter Balanced. • We continuously track the statistics used to ensure the quality of scoring. • Washington uses human readers to professionally score all constructed response items on state exams.

  29. Causes of 0 points on Full Write • What kinds of responses earn 0 points? • Blank • Written in a language other than English • Off Topic • Off Purpose (Mode) • Insufficient

  30. Causes of 0 points (continued) • Off Purpose (Mode): The response does not follow the mode of writing given in the prompt. • Example: When told to write a narrative based on the information in the sources, the student writes an informational text. • Insufficient: The response is too brief to score. • The response is not meaningful. For example, “IDK” and “I don’t care.” • The student has not provided enough of their own, original writing to determine whether or not they are on purpose (mode). The response may include an abundance of text copied verbatim from the prompt or from the sources.

  31. Statewide Data • Are more students earning 0 points this year compared to last year? • No, the percent of non-scorable responses (full write items earning zeros) decreased from Spring 2016 to Spring 2017. • What trends can be seen in the data? • Insufficient responses are the cause of most zeros. • Off-purpose (mode) responses are more common in the Narrative mode than in the other two modes.

  32. Statewide Data (percent non-scorable) *2017 stats as of May 12, 2017

  33. Statewide Data (trends in grades 3-5) *2017 stats as of May 12, 2017

  34. Statewide Data (trends in grades 6-8) *2017 stats as of May 12, 2017

  35. Statewide Data (trends in high school) *2017 stats as of May 12, 2017

  36. District data in Online Reporting System • Where in ORS can districts find data about writing mode and scores? • After logging in, select Retrieve Student Results • In the Step 1 box: • Report Type: select “Student Data” • Tested Grade: you can leave as “All Grades” but we suggest selecting a specific level • Download Format: leave as “Excel” • In the Step 2 box: • School: leave as “All” or select a specific school • Then select “Download”

  37. Columns AE through AM • 3 columns per mode (Informational/Explanatory, Opinion/Argumentative, Narrative) • Organization/purpose = 4 points possible • Evidence/elaboration = 4 points possible • Conventions = 2 points possible • 1 row per student • Digits are the scores which show in the online ISRs. Non-scorable responses earn 0’s. • N/A = the student’s PT was not in this mode

  38. Guidance for teachers • How can teachers help students prepare for full writes? • Clarify all the modes of writing and have students write for different purposes using resource texts • Grades 3-5: narrative, opinion, informational • Grades 6-8: narrative, argumentative, explanatory • Grades HS: argumentative, explanatory • Show students how to use informational texts to inspire narratives • Teach students to develop their own thoughts first, then add quotations from the sources to support as evidence, and follow up with explanations of the text evidence to make connections.

  39. ELPA21 OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  40. ELPA21 Update: Screener Transition Timeline Changes • OSPI has made a partial revision to the transition timeline for the ELPA21 Screener. This revision affects only in-coming kindergarten students. The change was made in response to communication that districts were unable to plan for the onset of a new school year with unknown elements around proctoring the screener (test duration, proctor-student ratios, etc.) • In-coming kindergarten students can take the WELPA through September 30, 2017. • Once the ELPA21 screener is launched (August 15) for statewide access, districts will have the discretion to continue with use of the WELPA for kindergartener screening or may opt to use the ELPA21 screener.

  41. ELPA21 Update: Screener • Scoring Turnaround • OSPI has heard concerns around the 7-day turnaround needed for third party scoring. We are exploring ways to lessen the time it takes for scoring. For the coming school year, no changes will be made to the scoring process. Our vendor is committed to meeting the 7-day turnaround and is working to identify ways to streamline scoring to minimize the time it takes. • What you can do to ensure scores are timely • 1. Screen potential ELs as soon as possible after enrollment. • 2. Make sure screeners are submitted, not paused, at the end of testing. • 3. Check ORS for scores regularly after submission of screeners. OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  42. ELPA21 Update: Score Reports • Sending Score Reports to Families • OSPI has become aware of an issue with K-1 test booklets that led to students receiving “not attempted” in the writing domain. This problem is limited to Kindergarten and first grade ELs. • Districts have been contacted depending on their status as “affected” or “not affected” by this scoring error. • All districts can disseminate score reports for all second grade and older ELs. • “Not affected” districts can disseminate all score reports (including K-1 reports). • “Affected” districts should pull score reports with “not attempted” reported for the writing domains of K-1 students and hold until given further direction. • Students with actual impacted results will receive updated score reports. These are scheduled to come in August prior to the start of school. OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  43. Technology OFFICE OF SUPERINTENDENT OF PUBLIC INSTRUCTION

  44. Technology Update: 17-18 Secure Browser • Updated Secure Browsers for use in 2017-18 are expected to be released by AIR on July 18, and will be available for download from WCAP portal • Updated Windows Secure Browser (v.10) will no longer support Vista • Updated MacintoshSecure Browsers (v.10 for OS X 10.9 and higher; v.9.5 for OS X 10.7 and 10.8) will no longer support OS X 10.6 • Updated Chrome Secure Browser will block ChromeVox • If updates to iOS client is needed, it will be available at the same time (note iOS 8.x will no longer be supported)

  45. Contacts for Technology, System, or Program Issues • Test Administrators, School Test Coordinators, and District Test Coordinators contact the WCAP Help Desk (TAs and School Test Coordinators must also alert the District Test and Technology Coordinators of issues). The WCAP Help Desk is open Monday–Friday from 6:00 a.m. to 6:00 p.m. PT (except holidays or as otherwise indicated on the Washington Comprehensive Assessment Program Portal) and can be reached by: • Toll-Free Phone Support: 1-844-560-7366 • Email Support: wahelpdesk@air.org

  46. USER SUPPORT • Questions regarding information in the Test Administration Manuals • Test Administrators and School Test Coordinators, contact your District Test Coordinator. • District Test Coordinators contact the OSPI Assessment Operations Team at: Assessment@k12.wa.us Policy or Test Administration Questions • Test Administrators and School Test Coordinators, refer to the Test Administration Manual. If additional support is needed, contact your District Test Coordinator. • District Test Coordinators contact the OSPI Assessment Operations Team at: Assessment@k12.wa.us

  47. Contacts AIR Help Desk for Technical, Network, and password issues • AIR Help Desk: wahelpdesk@air.org 1-844-560-7366 OSPI Assessment Analysts for student data issues • OSPI Assessment Analysts: assessmentanalysts@k12.wa.us 360-725-6109 OSPI Assessment Operations for assessment policy and test materials • Assessment Operations Assessment@k12.wa.us(360) 725-6348 or (800) 725-4311, press option 3 Smarter Balanced has updated its website. Check out the revamped and more user-friendly site at smarterbalanced.org

  48. Thank you!

More Related