Deborah Swensen, Ph. D. Assessment Director

1. Assessment Directors November 13, 2008 Deborah Swensen, Ph. D. Assessment Director Assessment Directors Meeting

2. UALPA • Constructed Reponses will be scored by contractor • Districts must meet the deadlines • Problems from last year Assessment Directors Meeting

3. AMAOs • Accountability • 2006-2007 AMAO II and III • 2007-2008 AMAO I, II, and III • AMAO I • New calculations • AMAO II • AMAO III Assessment Directors Meeting

4. AMAO I • Numerator = All ELL students who moved • Denominator = All ELL students with 2 score points on UALPA • 7 ways to make progress • Between proficiency levels • With E and I (low and high) • Maintaining A • 2008 – 29% • 2009 – 32.5% • Every year after increase 2.5% Assessment Directors Meeting

5. AMAO II and III • AMAO II • Increase for 2007-2008 school year • 2006-07  20.1% (K-6); 26.2% (7-12) • 2007-08  24.9% (K-6); 30.7% (7-12) • AMAO III • 2008-2009 another increase in AMOs • Possible change in AMO target Assessment Directors Meeting

6. USTAR • Under the direction of Curriculum • Curriculum is gathering additional information • Configurations of schools/programs • Windows of instruction • Number of students • Courses being taught Assessment Directors Meeting

7. Trainings • Utilize data available • Cognos • Access data for CRT tests and enrollment numbers • Drill down to school, grade, and demographic levels • Export the information to PDF or Excel • Look-up tables for UALPA and UBSCT • Available to schools and districts Assessment Directors Meeting

8. Nolan FawcettUBSCT & Secondary Math Specialist Assessment Directors Meeting

9. Printing Problem From October Test Assessment Directors Meeting

10. What Happened!!!!! • After USOE approval Manuals are sent to contractor for publication. • Contractor put them through a “pre-flight” process that looks for problems with font or graphics issues. • Manual was suppose to be checked by contractor or changes introduced by “pre-flight” • This review was missed in the process. Assessment Directors Meeting

11. Letter from Measured Progress Assessment Directors Meeting

12. Dates for Results • November 21st – AD Preliminary Results (Remember this is Rough Data Results) • November 24th – Preliminary Results to Schools • December 15th – Slice File and Fix-it File • January 6th – Paper Reports • Late November 2008 – Release Oct Test Assessment Directors Meeting

13. Preliminary Results Website • Website is a secure system with usernames and passwords assigned at the district and school level. • Passwords, a user guide, and Helpdesk support information was distributed to District Assessment Directors at the end of November. • Measured Progress was responsible for supporting and maintaining the website. http://iservices.measuredprogress.org Assessment Directors Meeting

14. Getting Your Results • Using Internet Explorer, navigate to:  http://iservices.measuredprogress.org • Once at the web page, select “Utah Basic Skills Competency Test” from the drop-down box and click Enter. • Click on the View UBSCT Preliminary Results link on the left of the page. Assessment Directors Meeting

15. Results Continued • On the “Log In” screen: • Enter your district’s user name in the User Name box (your district’s user name is your 2-digit district code – for example, Alpine district’s user name is 01). • Then, enter your secure password in the Password box.  • If you have any questions or need assistance, please contact the UBSCT Helpdesk, 1.866.502.2742. Assessment Directors Meeting

16. Preliminary Results Website Screen Shots Login Screen Assessment Directors Meeting

17. You will be directed to a screen labeled, “Home.”  After reading the text on this page, click on Reporting on the menu at the top of the page. Welcome Screen Assessment Directors Meeting

18. In the section labeled, “Basic Filtering Options,” select the school, grade, and subject for which you would like to view student information. (Please note that only results for reading and mathematics are available.) Reporting Screen Filter Options Report File Download Options (PDF, Excel, CSV) Assessment Directors Meeting

19. After you have made a filter selection, select your preferred download file type. To do this, click on the icon for Adobe PDF, Excel, or CSV. A file will be created in the format you have selected. Student records in the file will appear in alphabetical order by school. • Repeat the three steps above to view student data records for other schools, grades, and subtests within your district by changing your selections in the “Basic Filtering Options” section. Assessment Directors Meeting

20. For more information about the data provided on the roster reports, click on Documentation on the menu at the top of the page. You will find a PDF file for reading and mathematics. Assessment Directors Meeting

21. Preliminary Results EMAIL PROBLEMS: We had several locations block access to Measured Progress’s emails. Please make sure that you fix the problem so that the following emails can come through. • Measured Progress • Andrea Chan - (800) 431-8901 ext 2298 E-Mail chan.andrea@measuredprogress.org • Tinamarie Skillings (800) 431-8901 ext 2146 E-mail skillings.tinamarie@measuredprogress.org Assessment Directors Meeting

22. Calculator Reminder • Algebra 2 students will need to have their own graphing calculator for the CBT. Assessment Directors Meeting

23. UALPA 2008-09 Rita Brock ELL Specialist Assessment Directors Meeting

24. UALPA 2008-2009 Administering Utah’s Academic Language Proficiency Assessment Assessment Directors Meeting

25. Assessment Directors Meeting

26. Assessment Directors Meeting

27. Overview: What, Why, Who, When • District Assessment Coordinators’ and Test Administrators’ Roles & Responsibilities • III. Structure and Format of the Assessment • Test Administration Procedures • After Testing Assessment Directors Meeting

28. I. UALPA: What, Why • Statewide assessment of all identified ELL/LEP students • Mandated by the No Child Left Behind Act of 2001 Assessment Directors Meeting

29. Who Students to be tested include: • All newly enrolled LEP students • All LEP students whose proficiency level was Advanced or below, as measured by the 2007-2008 UALPA Assessment Directors Meeting

30. Who • Kindergarten Testing with UALPA • Only test those students identified as English Language Learner students with UALPA Assessment Directors Meeting

31. When Testing window: January 12, through May 8, 2009 (for both traditional and year-round schools) ELLs should be tested annually until they are determined to be at the Fluentproficiency level. Assessment Directors Meeting

32. District Testing Schedules • January 12 through March 2nd 2009 • Reports will be returned in April • March 2nd through May 8th 2009 • Reports will be returned in June • New students enrolled AFTER April 24, 2009 DO NOT need to be tested on the UALPA Assessment Directors Meeting

33. II. Roles and Responsibilities of the District Assessment Coordinator & the Test Administrator Assessment Directors Meeting

34. District Assessment Coordinator • Before testing: • Creates master list of LEP students to be assessed. • Orders test materials; inventories and confirms materials received. • Selects and trains Test Administrators using “Testing Ethics” policy material required by the Utah Board of Education. Information and forms can be found at http://www.schools.utah.gov/arc/AAEthicsPolicy.pdf • Trains Test Administrators on the administration of the UALPA • Orders answer documents and pre-printed barcode labels, as needed. Assessment Directors Meeting

35. Pre-printed Student Barcode Labels • District submits a pre-print file to USOE Computer Services at least 10 working days prior to date the student barcode labels and answer documents will be needed. • Before this is done, ELL students will need to have been identified as receiving Level 1 –Beginner or Level 2 -Intermediate. • Test Administrators will apply the labels to the answer documents of those students they are assessing. Assessment Directors Meeting

36. New students without labels For new students without pre-printed labels: • Student information will need to be hand bubbled • An SSID number must be assigned before UALPA administration Assessment Directors Meeting

37. District Assessment Coordinator • During testing: • Be available to answer Test Administrator & Site Test Coordinator questions. • Distribute additional materials to schools if needed. Assessment Directors Meeting

38. Test Administrator • During testing: • Administer the one-on-one Speaking tests first. • Administer the other 3 tests in a group setting. • Continually monitor students. • Maintain and oversee the security of all test materials. • Arrange make-up testing sessions for absentees. Assessment Directors Meeting

39. IV. Test Administration Procedures A. General Assessment Directors Meeting

40. Test Site • Individual Testing • Quiet one-on-one environment • Seating • Group Testing • Quiet room • Do Not Disturb sign on door • Desks must be cleared Assessment Directors Meeting

41. Test Security • Responsibility of both the District Assessment Coordinator and Test Administrator • All test materials must be accounted for and returned to the District Assessment Coordinator. Assessment Directors Meeting

42. Testing Ethics • No questions from the UALPA may be taught to or reviewed with students. • Copying any test material is prohibited. • All Test Administrators must be trained under the direction of the District Assessment Director. Assessment Directors Meeting

43. Grouping Students for Testing • As long as the group is not too large, all students within a grade span, taking the same form may be tested together in a group setting. • Group size: • Optimal group size: 10 • Maximum group size: 20 • Take into account the maturity of the students, the number of available proctors, and the acoustics of the room. Assessment Directors Meeting

44. Prompting • In general, prompting is not allowed. • Exceptions: • To clarify a student’s response • If student responded in another language • Test Administrator may repeat a question if: • There was a distraction or interruption • Student did not yet begin to respond and asks for question to be repeated Assessment Directors Meeting

45. Translating Directions • No item directions or item content may be translated. The script must be read in English exactly as printed in the Test Administration Manual. Assessment Directors Meeting

46.  Timing • The UALPA is an untimed test. • During individual testing, Test Administrators should wait approximately 15 seconds for a student to begin a response. • During group testing, Test Administrators should use their best judgment in allowing sufficient time for students to finish multiple-choice and extended responses. Assessment Directors Meeting

47. Frustration Level • Frustration level is defined as when a student has made 5 consecutive scores of “0” or is obviously unable to respond to test prompts. • If frustration level is reached on a Form 1 Speaking test, administer all other tests individually. • If frustration level is reached on a Form 2 Speaking test, then switch to a Form 1 test. Assessment Directors Meeting

48. Frustration Level • If frustration level is reached on any test, mark the “FL” bubble in the Office Use Only box on the answer document. • Determination of frustration level cannot be made during group testing. Assessment Directors Meeting

49. Accommodations • Decisions regarding accommodations must be made ahead of time by the IEP, ELL or Section 504 team and documented in the student’s file. • Test Administrators and/or teachers should not make accommodation decisions at the time of testing. • Braille and Enlarged Print versions of the test are available, but must be ordered 2 months in advance. Assessment Directors Meeting

50.  Non-allowable Accommodations • Certain accommodations are not allowed during the UALPA. These include: • Reading directions or questions aloud to a student (except as indicated by the Test Administrator script). • Translating directions or questions for a student. • Providing bilingual word lists to a student. Assessment Directors Meeting