deborah swensen ph d assessment director n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Deborah Swensen, Ph. D. Assessment Director PowerPoint Presentation
Download Presentation
Deborah Swensen, Ph. D. Assessment Director

Loading in 2 Seconds...

play fullscreen
1 / 113

Deborah Swensen, Ph. D. Assessment Director - PowerPoint PPT Presentation


  • 322 Views
  • Uploaded on

Assessment Directors November 13, 2008. Deborah Swensen, Ph. D. Assessment Director. UALPA. Constructed Reponses will be scored by contractor Districts must meet the deadlines Problems from last year. AMAOs . Accountability 2006-2007 AMAO II and III 2007-2008 AMAO I, II, and III

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Deborah Swensen, Ph. D. Assessment Director' - benjamin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
deborah swensen ph d assessment director
Assessment Directors

November 13, 2008

Deborah Swensen, Ph. D. Assessment Director

Assessment Directors Meeting

ualpa
UALPA
  • Constructed Reponses will be scored by contractor
  • Districts must meet the deadlines
  • Problems from last year

Assessment Directors Meeting

amaos
AMAOs
  • Accountability
    • 2006-2007 AMAO II and III
    • 2007-2008 AMAO I, II, and III
  • AMAO I
    • New calculations
  • AMAO II
  • AMAO III

Assessment Directors Meeting

amao i
AMAO I
  • Numerator = All ELL students who moved
  • Denominator = All ELL students with 2 score points on UALPA
  • 7 ways to make progress
    • Between proficiency levels
    • With E and I (low and high)
    • Maintaining A
  • 2008 – 29%
  • 2009 – 32.5%
  • Every year after increase 2.5%

Assessment Directors Meeting

amao ii and iii
AMAO II and III
  • AMAO II
    • Increase for 2007-2008 school year
      • 2006-07  20.1% (K-6); 26.2% (7-12)
      • 2007-08  24.9% (K-6); 30.7% (7-12)
  • AMAO III
    • 2008-2009 another increase in AMOs
    • Possible change in AMO target

Assessment Directors Meeting

ustar
USTAR
  • Under the direction of Curriculum
  • Curriculum is gathering additional information
    • Configurations of schools/programs
    • Windows of instruction
    • Number of students
    • Courses being taught

Assessment Directors Meeting

trainings
Trainings
  • Utilize data available
  • Cognos
    • Access data for CRT tests and enrollment numbers
    • Drill down to school, grade, and demographic levels
    • Export the information to PDF or Excel
    • Look-up tables for UALPA and UBSCT
  • Available to schools and districts

Assessment Directors Meeting

printing problem from october test
Printing Problem From October Test

Assessment Directors Meeting

what happened
What Happened!!!!!
  • After USOE approval Manuals are sent to contractor for publication.
  • Contractor put them through a “pre-flight” process that looks for problems with font or graphics issues.
  • Manual was suppose to be checked by contractor or changes introduced by “pre-flight”
  • This review was missed in the process.

Assessment Directors Meeting

letter from measured progress
Letter from Measured Progress

Assessment Directors Meeting

dates for results
Dates for Results
  • November 21st – AD Preliminary Results

(Remember this is Rough Data Results)

  • November 24th – Preliminary Results to Schools
  • December 15th – Slice File and Fix-it File
  • January 6th – Paper Reports
  • Late November 2008 – Release Oct Test

Assessment Directors Meeting

preliminary results website
Preliminary Results Website
  • Website is a secure system with usernames and passwords assigned at the district and school level.
  • Passwords, a user guide, and Helpdesk support information was distributed to District Assessment Directors at the end of November.
  • Measured Progress was responsible for supporting and maintaining the website. http://iservices.measuredprogress.org

Assessment Directors Meeting

getting your results
Getting Your Results
  • Using Internet Explorer, navigate to:  http://iservices.measuredprogress.org
  • Once at the web page, select “Utah Basic Skills Competency Test” from the drop-down box and click Enter.
  • Click on the View UBSCT Preliminary Results link on the left of the page.

Assessment Directors Meeting

results continued
Results Continued
  • On the “Log In” screen:
    • Enter your district’s user name in the User Name box (your district’s user name is your 2-digit district code – for example, Alpine district’s user name is 01).
    • Then, enter your secure password in the Password box. 
  • If you have any questions or need assistance, please contact the UBSCT Helpdesk, 1.866.502.2742.

Assessment Directors Meeting

preliminary results website screen shots
Preliminary Results Website Screen Shots

Login Screen

Assessment Directors Meeting

slide17

You will be directed to a screen labeled, “Home.”  After reading the text on this page, click on Reporting on the menu at the top of the page.

Welcome Screen

Assessment Directors Meeting

slide18

In the section labeled, “Basic Filtering Options,” select the school, grade, and subject for which you would like to view student information. (Please note that only results for reading and mathematics are available.)

Reporting

Screen

Filter

Options

Report File

Download Options (PDF, Excel, CSV)

Assessment Directors Meeting

slide19

After you have made a filter selection, select your preferred download file type. To do this, click on the icon for Adobe PDF, Excel, or CSV. A file will be created in the format you have selected. Student records in the file will appear in alphabetical order by school.

  • Repeat the three steps above to view student data records for other schools, grades, and subtests within your district by changing your selections in the “Basic Filtering Options” section.

Assessment Directors Meeting

slide20

For more information about the data provided on the roster reports, click on Documentation on the menu at the top of the page. You will find a PDF file for reading and mathematics.

Assessment Directors Meeting

preliminary results
Preliminary Results

EMAIL PROBLEMS:

We had several locations block access to Measured Progress’s emails.

Please make sure that you fix the problem so that the following emails can come through.

  • Measured Progress
    • Andrea Chan - (800) 431-8901 ext 2298 E-Mail chan.andrea@measuredprogress.org
    • Tinamarie Skillings (800) 431-8901 ext 2146 E-mail skillings.tinamarie@measuredprogress.org

Assessment Directors Meeting

calculator reminder
Calculator Reminder
  • Algebra 2 students will need to have their own graphing calculator for the CBT.

Assessment Directors Meeting

rita brock ell specialist
UALPA 2008-09 Rita Brock ELL Specialist

Assessment Directors Meeting

slide24

UALPA 2008-2009

Administering Utah’s

Academic Language Proficiency Assessment

Assessment Directors Meeting

slide27

Overview: What, Why, Who, When

  • District Assessment Coordinators’ and Test Administrators’ Roles & Responsibilities
  • III. Structure and Format of the Assessment
  • Test Administration Procedures
  • After Testing

Assessment Directors Meeting

slide28

I. UALPA: What, Why

  • Statewide assessment of all identified ELL/LEP students
  • Mandated by the No Child Left Behind Act of 2001

Assessment Directors Meeting

slide29

Who

Students to be tested include:

  • All newly enrolled LEP students
  • All LEP students whose proficiency level was Advanced or below, as measured by the 2007-2008 UALPA

Assessment Directors Meeting

slide30
Who
  • Kindergarten Testing with UALPA
    • Only test those students identified as English Language Learner students with UALPA

Assessment Directors Meeting

slide31

When

Testing window:

January 12, through May 8, 2009

(for both traditional and year-round schools)

ELLs should be tested annually until they are determined to be at the Fluentproficiency level.

Assessment Directors Meeting

district testing schedules
District Testing Schedules
  • January 12 through March 2nd 2009
    • Reports will be returned in April
  • March 2nd through May 8th 2009
    • Reports will be returned in June
  • New students enrolled AFTER April 24, 2009 DO NOT need to be tested on the UALPA

Assessment Directors Meeting

slide33

II. Roles and Responsibilities

of the

District Assessment Coordinator

& the Test Administrator

Assessment Directors Meeting

slide34

District Assessment Coordinator

  • Before testing:
  • Creates master list of LEP students to be assessed.
  • Orders test materials; inventories and confirms materials received.
  • Selects and trains Test Administrators using “Testing Ethics” policy material required by the Utah Board of Education. Information and forms can be found at http://www.schools.utah.gov/arc/AAEthicsPolicy.pdf
  • Trains Test Administrators on the administration of the UALPA
  • Orders answer documents and pre-printed barcode labels, as needed.

Assessment Directors Meeting

slide35

Pre-printed Student Barcode Labels

  • District submits a pre-print file to USOE Computer Services at least 10 working days prior to date the student barcode labels and answer documents will be needed.
  • Before this is done, ELL students will need to have been identified as receiving Level 1 –Beginner or Level 2 -Intermediate.
  • Test Administrators will apply the labels to the answer documents of those students they are assessing.

Assessment Directors Meeting

slide36

New students without labels

For new students without pre-printed labels:

  • Student information will need to be hand bubbled
  • An SSID number must be assigned before UALPA administration

Assessment Directors Meeting

slide37

District Assessment Coordinator

  • During testing:
  • Be available to answer Test Administrator & Site Test Coordinator questions.
  • Distribute additional materials to schools if needed.

Assessment Directors Meeting

slide38

Test Administrator

  • During testing:
  • Administer the one-on-one Speaking tests first.
  • Administer the other 3 tests in a group setting.
  • Continually monitor students.
  • Maintain and oversee the security of all test materials.
  • Arrange make-up testing sessions for absentees.

Assessment Directors Meeting

slide39

IV. Test Administration Procedures

A. General

Assessment Directors Meeting

slide40

Test Site

  • Individual Testing
    • Quiet one-on-one environment
    • Seating
  • Group Testing
    • Quiet room
    • Do Not Disturb sign on door
    • Desks must be cleared

Assessment Directors Meeting

slide41

Test Security

  • Responsibility of both the District Assessment Coordinator and Test Administrator
  • All test materials must be accounted for and returned to the District Assessment Coordinator.

Assessment Directors Meeting

slide42

Testing Ethics

  • No questions from the UALPA may be taught to or reviewed with students.
  • Copying any test material is prohibited.
  • All Test Administrators must be trained under the direction of the District Assessment Director.

Assessment Directors Meeting

slide43

Grouping Students for Testing

  • As long as the group is not too large, all students within a grade span, taking the same form may be tested together in a group setting.
  • Group size:
    • Optimal group size: 10
    • Maximum group size: 20
  • Take into account the maturity of the students, the number of available proctors, and the acoustics of the room.

Assessment Directors Meeting

slide44

Prompting

  • In general, prompting is not allowed.
  • Exceptions:
    • To clarify a student’s response
    • If student responded in another language
  • Test Administrator may repeat a question if:
    • There was a distraction or interruption
    • Student did not yet begin to respond and asks for question to be repeated

Assessment Directors Meeting

slide45

Translating Directions

  • No item directions or item content may be translated. The script must be read in English exactly as printed in the Test Administration Manual.

Assessment Directors Meeting

slide46

Timing

  • The UALPA is an untimed test.
  • During individual testing, Test Administrators should wait approximately 15 seconds for a student to begin a response.
  • During group testing, Test Administrators should use their best judgment in allowing sufficient time for students to finish multiple-choice and extended responses.

Assessment Directors Meeting

slide47

Frustration Level

  • Frustration level is defined as when a student has made 5 consecutive scores of “0” or is obviously unable to respond to test prompts.
  • If frustration level is reached on a Form 1 Speaking test, administer all other tests individually.
  • If frustration level is reached on a Form 2 Speaking test, then switch to a Form 1 test.

Assessment Directors Meeting

frustration level
Frustration Level
  • If frustration level is reached on any test, mark the “FL” bubble in the Office Use Only box on the answer document.
  • Determination of frustration level cannot be made during group testing.

Assessment Directors Meeting

slide49

Accommodations

  • Decisions regarding accommodations must be made ahead of time by the IEP, ELL or Section 504 team and documented in the student’s file.
  • Test Administrators and/or teachers should not make accommodation decisions at the time of testing.
  • Braille and Enlarged Print versions of the test are available, but must be ordered 2 months in advance.

Assessment Directors Meeting

slide50

Non-allowable Accommodations

  • Certain accommodations are not allowed during the UALPA. These include:
    • Reading directions or questions aloud to a student (except as indicated by the Test Administrator script).
    • Translating directions or questions for a student.
    • Providing bilingual word lists to a student.

Assessment Directors Meeting

slide51

Scoring Guides

  • Oral responses on the Speaking test are scored by Test Administrators at the time of testing.
  • Responses are rated using the Scoring Guides in the flipchart.
  • The Test Administrator must study the Scoring Guides before giving the test for the first time.

Assessment Directors Meeting

slide52

Pre-printed Student Barcode Labels

  • District submits a pre-print file to USOE Computer Services at least 10 working days prior to date the student barcode labels and answer documents will be needed.
  • Before this is done, ELL students will need to have been identified as receiving Form 1 or Form 2.
  • Test Administrators will apply the labels to the answer documents of those students they are assessing.

Assessment Directors Meeting

slide53

New students without labels

For new students without pre-printed labels:

  • Student information will need to be hand bubbled
  • An SSID number must be assigned before UALPA administration

Assessment Directors Meeting

slide54

Testing Absentees

  • All LEP students should be administered all four tests.
  • If a student is absent for a particular testing session, schedule a make-up test for that student as soon as possible.

Assessment Directors Meeting

slide55

Kindergarten Test Administration

  • All four tests are administered individually
  • All responses are recorded by the Test Administrator on the student’s Form A answer document

Assessment Directors Meeting

slide56

Grade-Span 1-2 Test Administration

  • The Speaking test is administered individually using the Speaking test flipchart, and is always administered first.
  • Speaking responses are recorded by the Test Administrator on in the consumable booklet.
  • The other three tests are administered to small groups ideally of not more than 10 students, with students marking their answers in the consumable booklet.

Assessment Directors Meeting

slide57

Grades 3-12 Test Administration

  • Speaking Test is administered individually using the flipchart, and must be administered first. Speaking responses are recorded by the Test Administrator on the student’s answer document
  • The other 3 tests are group administered (20 student maximum) using the script in the Test Administration Manual.
  • These responses are marked or written on the student answer document by the student.

Assessment Directors Meeting

slide58

V. After Testing

Assessment Directors Meeting

slide59

Test Administrator

  • After testing:
  • Checks answer documents for completeness & proper bubbling.
  • Checks to see if any applicable testing accommodations are marked.
  • Organizes test materials.
  • Returns test materials to the District Assessment Coordinator.

Assessment Directors Meeting

slide60

District Assessment Coordinator

  • After testing:
  • Organizes scoring of constructed responses.
  • Returns completed answer documents to USOE.
  • Collects and securely destroys all 2008-2009 test materials at end of school year.

Assessment Directors Meeting

slide61

Returning completed answer documents to USOE

  • Verify that all answer documents have had the constructed responses scored.
  • Separate each grade within boxes by teacher/school header sheets.
  • Use packing material to avoid movement of materials inside box. Do NOT use foam as this affects scan ability of documents.
  • Tape box securely with heavy-duty tape (not cellophane or masking tape).
  • Ship to USOE via UPS, FedEx, or other carrier that tracks shipments. Do not send by U.S. Mail.

Assessment Directors Meeting

key dates for ualpa 2008 2009
Key Dates for UALPA 2008-2009
  • January 12, 2009
    • Test materials shipped to districts
  • January 12– May 8 2009
    • Testing Window
  • February 15th 2009, May 1st 2009
    • Deadline for Districts to send Constructed Responses to USOE

Assessment Directors Meeting

key dates for ualpa 2008 20091
Key Dates for UALPA 2008-2009

District Deadlines for return of answer documents to USOE

  • March 2, 2009 (50%) 1st Scoring
  • May 8, 2009 (50%) 2nd Scoring

Assessment Directors Meeting

reports returned for answer documents submitted for 1 st and 2 nd scoring window
Reports returned for answer documents submitted for 1st and 2nd scoring window
  • April 13 (approximately) –for 1st scoring window.
  • June 22 (approximately) –for 2nd scoring window

Assessment Directors Meeting

slide65

If you have questions...

Assessment Directors Meeting

pre print ualpa uploads the how who when
Pre-Print UALPA Uploads The How- Who- When

Assessment Directors Meeting

the how
The How

Assessment Directors Meeting

obtaining a user id and password
Obtaining a User ID and Password
  • LEAs must submit a New User Agreement Form to USOE in order to request a user ID. The New User Agreement form must be filled out and faxed to Sharon Marsh at
  • (801) 538-7938. After checking the contents and forms for accuracy, the administrator will create a user ID for the requestor

Assessment Directors Meeting

user id and password format
User ID And Password Format
  • Your User ID is made up using your LEA number, first initial, last name.
  • The password for the first time that you log on to the testing web-site is Password1
  • Passwords must meet the following criteria:
  • Be at least 8 characters in length
  • Contain at least 1 number (0-9),
  • Contain at least 1 uppercase letter (A-Z),
  • Contain at least 1 lowercase letter (a-z)

Assessment Directors Meeting

logging into the assessment website
Logging into the Assessment Website
  • https://cs.schools.utah.gov/assessment
  • First screen is the Login screen
  • Example: User ID 99smarsh
  • Example: Password Password1

Assessment Directors Meeting

logging into the assessment website1
Logging into the Assessment Website
  • https://cs.schools.utah.gov/assessment
  • When you look at the URL and see the //cs.schools.utah.gov/assessment
  • You can tell that you are logged onto the testing website and not the UTOS site for ordering test materials.
  • Because we store student level data we don’t identify our login page further.

Assessment Directors Meeting

login page
Login Page

99smarsh

Password1

Assessment Directors Meeting

home page
Home Page

Links to various parts of website and contains a Calendar of Events

Assessment Directors Meeting

support page
Support Page

Assessment Directors Meeting

who do i call for help
Who Do I call For Help?

Call Sharon 538-7915

76

6/9/2014

Assessment Directors Meeting

Assessment Directors Meeting

slide77

If Locked Out Of Testing Website Don’t Call Help Desk Call Sharon Marsh 538-7915 or Your Data Tech

Assessment Directors Meeting

before uploading
Before Uploading
  • Before uploading a file to the Assessment system, the file content should be named as outlined the Preprint File Name Specification. Files that fail validation or are named improperly will NOT be accepted by the website. The Preprint File Validation Application and the Preprint File specification is on the Support page.

Assessment Directors Meeting

steps for uploading files
Steps for Uploading Files
  • Create an LEA Preprint file for each test
  • Use the provided Preprint File Validation Application tool to validate the file’s contents.
  • Login to the Assessment website using a valid user ID and password.
  • Click on the Upload File link on the Home page under File Maintenance section. The upload File screen will appear.

Assessment Directors Meeting

upload file screen
Upload File Screen

This is in your handout

Assessment Directors Meeting

selecting your file
Selecting Your File
  • Click on the Browse button to open the choose file window then navigate to where you have your Preprint file.
  • After locating the correct Preprint file, select it and click on Open
  • The Preprint File that was selected along with its path, should now appear in the File Name box.

Assessment Directors Meeting

locating correct preprint file
Locating Correct Preprint File

This is in your hand-outs

Assessment Directors Meeting

review your files after they are sent
Review Your Files After they Are Sent

Assessment Directors Meeting

processing files with errors
Processing Files With Errors
  • If errors were found during validation, a status of Failed will appear next to the file name. The following options are available for files with errors.
  • Click on the filename with status of failed
  • View errors on the file by clicking on the File Summary button. Some of the file errors will be displayed. The user may print these out.

Assessment Directors Meeting

processing files with errors1
Processing Files With Errors

Assessment Directors Meeting

processing files with errors2
Processing Files With Errors
  • Cancel the Failed file to put it at the bottom of the file list.
  • Files with errors must be fixed by the LEA and re-uploaded for processing.
  • Most errors are SSID miss-matches.

Assessment Directors Meeting

the who
The Who

Assessment Directors Meeting

which students test for ualpa
Which Students Test For UALPA?
  • All students that are A, P, E, or I
  • They will be flagged in your SIS systems so that when you create you files to upload you should be able to upload only those students.

Assessment Directors Meeting

which students test for ualpa1
Which Students Test For UALPA?

A= Advanced P=Pre-Emergent E=Emergent I=Intermediate

Assessment Directors Meeting

testing levels
Testing Levels
  • Kindergarten
  • Grade 1-2
  • Grade 3-6
  • Grade 7-8
  • Grade 9-12

Assessment Directors Meeting

the when
The When

Assessment Directors Meeting

when do i send my files
When Do I Send My Files?
  • If the test window for UALPA is in January you will want to start building your files and sending them in 10 days before your want to test.
  • Because all of the documents for UALPA are multiple pages you will receive bar code labels instead of having the students information printed on the answer document.
  • Please make sure that your teachers place the labels on the answer documents within the brackets

Assessment Directors Meeting

will i receive blank documents
Will I Receive Blank Documents?
  • Because we know that you might have new students that move in after you have uploaded your files we will provide you with a few blank forms to use.
  • When filling in the student information use the local LEA number and not the SSID number.

Assessment Directors Meeting

more information
More Information
  • In our December training I will have more information on new answer documents.
  • Last minute information on pre-print uploads
  • Changes if any

Assessment Directors Meeting

questions
QUESTIONS:

Assessment Directors Meeting

slide97
DWA
  • Scoring rubric and student response examples: www.schools.utah.gov/assessment/info_dwa.aspx
  • One prompt for 6th, one prompt for 9th
  • Ensure students have as much time as needed during the one administration session (untimed assessment).
  • Assessment window is determined by school closing date.

Assessment Directors Meeting

slide98
DWA

Closing date:DWA window:

May 20-22, 2009 February 9-27, 2009

May 26-29, 2009 Feb. 16-March 6, 2009

June 2-5, 2009 Feb. 23-March 13, 2009

June 9-12, 2009 March 2-20, 2009

If your school’s closing date is not listed above, please contact Margaret Young at margaret.young@schools.utah.gov.

Assessment Directors Meeting

slide99
DWA
  • #2 lead pencil, no pens
  • Districts are responsible for transcribing onto the answer document all Braille and other accommodated responses.
  • For ordering information, please contact Sarah Moore at sarah.moore@schools.utah.gov.

Assessment Directors Meeting

julie quinn cbt specialist
CBT Update Julie Quinn CBT Specialist

Assessment Directors Meeting

cbt participation projections
CBT Participation Projections

Final CBT participation numbers are due December 12, 2008.

2009 Projections as of November 10

117,000 more tests than 2008

(approx. 62% of eligible tests)

and rising.

Assessment Directors Meeting

december trainings
December Trainings

Please RSVP as soon as possible,

to ensure availability.

Available slots will be released December 4,

for additional individuals from your LEA.

Many LEAs have not yet registered.

Assessment Directors Meeting

load testing
Load Testing

Change in statewide load testing plan:

  • January 19 through January 30
  • Simulate the heaviest two weeks of testing
  • Please have every student possible take training tests during this window
  • Email will be sent asking for when you will plan to participate, to balance loads

Assessment Directors Meeting

load testing1
Load Testing

Local load testing is critical.

  • Mid-December through early January
  • Focus on T1 schools and less robust wireless networks
  • Any problems that impact the CBT projections must be relayed immediately (paper testing order impact)

Assessment Directors Meeting

questions concerns
Questions? Concerns?

Assessment Directors Meeting

utah test ordering system
Utah Test Ordering System
  • UBSCT February
  • DWA
  • UALPA
  • CRT

Assessment Directors Meeting

trimester semester crt
Trimester / Semester CRT
  • No Algebra 2 available for trimester/semester
  • assessments
  • Ordering closes November 30th

Assessment Directors Meeting

ubsct february
UBSCT February
  • Ordering closes DECEMBER 20th
  • 10th graders, 11th and 12th grader
  • retakes

Assessment Directors Meeting

direct writing assessment
Direct Writing Assessment
  • Ordering Closes January 15th
  • 6th and 9th grades
  • Materials to Order:
    • Student Directions
    • TAMS

Assessment Directors Meeting

utah test ordering system1
Utah Test Ordering System
  • CRT / CBT Spring
  • CBT currently closed
  • PBT projections scheduled to pre-populate within UTOS December 1st

Assessment Directors Meeting

pre populate
Pre-Populate?
  • Based off last year’s test totals in each grade/subject
  • Minus current CBT projections
  • Review December 1st

Assessment Directors Meeting

slide113
Call With Questions Regarding UTOS

pre-population!

We can reopen for you to alter counts if needed.

Assessment Directors Meeting