2011 STAR Interpreting and Using Results

1 / 81

# 2011 star interpreting and using results - PowerPoint PPT Presentation

2011 STAR Interpreting and Using Results. August 10, 2011 Webcast Webcast starts at 9:00 a.m. Objectives. Workshop participants will be able to: Describe the purposes of STAR reports Interpret STAR results Explain key statistics Compare and contrast types of reports

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about '2011 star interpreting and using results' - Ava

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### 2011 STARInterpreting and Using Results

August 10, 2011

Webcast

Webcast starts at 9:00 a.m.

Objectives

Workshop participants will be able to:

• Describe the purposes of STAR reports
• Interpret STAR results
• Explain key statistics
• Compare and contrast types of reports
• Identify proper uses of reports

Post-Test Workshop

Agenda
• What’s New?
• Results and Statistical Analysis
• Using Results
• Summary and Internet Reports
• Data CDs
• Individual Student Reports
• Early Assessment Program

Post-Test Workshop

California Modified Assessment (CMA) reports

Scale score, performance levels for

Algebra I

Percent correct for

Geometry

Writing component of grade 4 CST and CMA ELA restored

What’s New in 2011

Manual (M) 2

Post-Test Workshop

Quiz Question 1

Which of these tests had scale scores reported for the first time in 2011?

• CST for World History
• CAPA for Science
• CMA for Algebra I
• CMA for Geometry

Post-Test Workshop

Quiz Question 1

Which of these tests had scale scores reported for the first time in 2011?

C. CMA for Algebra I

Post-Test Workshop

Results: Purposes of STAR Reports

M 3

• Report progress toward proficiency on the state’s academic content standards
• Notify where improvement needed
• To help students’ achievement
• To improve educational programs
• Provide data for state and federal accountability programs

Post-Test Workshop

Results: Performance Levels

M 8−9

• State goal: All students score at proficient or higher
• 350 or higher scale score
• All CST
• CMA for ELA in grades 3–9, math in grades 3–7 and Algebra I, and science in grades 5, 8, and 10
• STS for RLA and math in grades 2–7, excluding Algebra I
• CAPA proficient: 35 or higher scale score

Post-Test Workshop

Results: Other Performance Levels

M 8−9;Appendix B

• Basic cut score
• CST: 300
• CMA: 300 for ELA in grades 3–9, math in grades 3–7, Algebra I, science in grades 5, 8, and 10
• STS: 300 for RLA and math in grades 2–7 (excluding Algebra I)
• CAPA: 30
• Below basic
• Far below basic
• For each testing program, cut pointsvary for advanced and below basic by
• Subject

Post-Test Workshop

Results: Scale Scores

M 7−11

• Scale scores allow the same score to mean the same thing across test versions within grade and content area
• Accounts for differences in difficulty
• Scale score ranges by program:
• CST, CMA (ELA grades 3–9, mathematics 3–7, Algebra I, science in grades 5, 8, and 10), STS (grades 2–7, excluding Algebra I): 150–600 for each grade and subject
• CAPA: 15–60 for each level and subject

Post-Test Workshop

Results: Equating

M 7

• Psychometric procedure
• Additional information in the technical report on the CDE Web site

Post-Test Workshop

Results: Reporting Clusters(Content Areas)

M 9−11;Appendix A

• Three to six clusters for each subject
• May be useful as indicators of individual or group strengths and weaknesses
• But . . . Reporting clusters should be interpreted with caution

Post-Test Workshop

Results: Cluster Percent Correct Available for 2011

M 9−11;Appendix A

• CST: all
• CMA:
• STS: all

Post-Test Workshop

Results: Reporting ClusterCautions

M 9−11;Appendix A

• Based on small numbers of items; therefore, may not be reliable or generalizable
• NOT equated from year to year
• Should not compare reporting cluster percent correct from year to year

Post-Test Workshop

M 8−12;Appendix A

• Compare to percent-correct range of proficient students statewide

Post-Test Workshop

2011 CST Reporting Clusters: Number of Questions and Average Percent Correct

M Appendix A

2011 Post-Test Guide, Appendices A and C as posted on startest.org will be finalized with complete data on August 15.

Post-Test Workshop

M 11

Post-Test Workshop

Quiz Question 2

What is a scale score?

• Percent correct of all questions
• Mean percent correct of all questions
• An adjustment of this year’s and last year’s raw scores to show changes
• An adjustment of the raw score to account for differences in difficulty

Post-Test Workshop

Quiz Question 2

What is a scale score?

D. An adjustment of the raw score to account for differences in difficulty

Post-Test Workshop

Using Results

M 3

• For instructional decisions with other data
• Used in Academic Performance Index (API) calculations: CSTs, CAPA, CMA (only grades 3–9, Algebra I, Life Science)
• Used in adequate yearly progress (AYP) calculations, ELA and mathematics:
• Grades 3–8 and Algebra I, CMA
• Grades 2–8 and 10, CAPA

Post-Test Workshop

M 12−15

• Mean scale score
• Same content and grade, varying years
• Percent in each performance level
• Same content by grade across years

Post-Test Workshop

M 12−15

• Percentage of students scoring at PROFICIENT and above
• For a given grade and subject, e.g., Percent proficient and above for grade 3 math in 2010 and 2011
• For a given subject and aggregated grades, e.g., Percent proficient and above for grades 2– 6 mathematics in 2010 and 2011
• Across grades and a subject, e.g., Percent proficient and above in all courses and all grades

Post-Test Workshop

Year-to-Year ComparisonsDON’T Compare

M 12−15

• Individual scale scores or statistics based on scale scores for different grades or content areas
• Subjects by grade are independently scaled
• Different content standards are measured in different grades
• Across tests
• CMA grades 6−8, STS grades 5−7, only compare 2011 to 2010
• CAPA, CMA grades 3−5, STS grades 2−4, DON’T compare to years before 2009

Post-Test Workshop

M 14

Post-Test Workshop

Quiz Question 3

What is the best comparison for CST scores of students within a middle school?

• 2010 mean scale scores for ELA of a cohort of grade 7 students with 2011 scale scores for ELA of the same students in grade 8
• 2010 mean scale scores for ELA for grade 8 students with 2011 mean scale scores for ELA for grade 8 students
• 2010 mean percent correct scores for ELA with 2011 percent correct scores for ELA for the same students in grade 8
• 2010 mean percent correct scores for ELA for grade 8 students with 2011 mean percent correct for ELA for grade 8 students

Post-Test Workshop

Quiz Question 3

What is the best comparison for CST scores of students within a middle school?

B. 2010 mean scale scores for ELA for grade 8 students with 2011 mean scale scores for ELA for grade 8 students

Post-Test Workshop

Quiz Question 4

Which is the best comparison of cluster scores for a single student? Compare . . .

• To proficient students statewide
• One cluster to another, same year
• The same cluster to the same cluster, different years
• To the average percent correct of all students in a class

Post-Test Workshop

Quiz Question 4

Which is the best comparison of cluster scores for a single student? Compare…

• To proficient students statewide

Post-Test Workshop

Aggregate (Summary) Reports

M 18−22

• What are they?
• Student Master List Summary
• Student Master List Summary End-of-Course (EOC)
• Subgroup Summary
• Back of reports: guide to abbreviations, score codes
• Report Emphasis: CSTs
• Criterion-referenced tests
• Progress is measured in percent of students scoring proficient and advanced

Post-Test Workshop

Student Master List Summary

M 19–20, M 29−35

• By grade or EOC subject
• CSTs, CMA, CAPA, and STS
• Lists subjects or EOC grades
• % and # at each performance level
• Mean scale score
• CST
• CAPA
• CMA ELA in grades 3–9, math in grades 3–7, Algebra I, and science in grades 5, 8, and 10
• STS RLA and math in grades 2–7 [not Algebra I])

Post-Test Workshop

Student Master List Summary (cont.)
• Reporting cluster: mean percent correct
• CST
• CMA ELA in grades 3–9, math in grades 3–7, Algebra I, and science in grades 5, 8, and 10
• STS

Post-Test Workshop

Student Master List Summary

M 35

Post-Test Workshop

Who Counts? Number Enrolled

M 31

• Total CST/CMA and CAPA multiple-choice answer documents submitted as scorable
• Minus
• Documents marked as “Student enrolled after the first day of testing and was given this test”

Post-Test Workshop

Who Counts? Number Tested

M 31

• All CST, CMA, CAPA, STS answer documents with one or more answers
• Plus
• Z = Tested but marked no answers
• Not included
• A = Students absent
• E = Not tested due to significant medical emergency
• P = Parent/guardian exemptions
• T = Enrolled first day, not tested, tested at previous school
• Non–English learners who took the STS

Post-Test Workshop

Who Counts? Number and Percent Valid Scores

M 31

• Number Valid Scores
• For the subject, number of students tested at grade level who received a score for the test (that is, a scale score or percent correct)
• Percent Valid Scores
• For the subject, number of valid scores divided by the number of students tested
• Not included:
• Incomplete tests
• Modified tests
• Non–English learners who took the STS
• Unknown EOC mathematics (except grade 7 mathematics) or science tests

Post-Test Workshop

Who Counts? Number Tested with Scores

M 31

• All tests taken, including those taken with modifications, that receive a score
• Not included:
• Incomplete tests
• Non–English learners who took the STS
• Unknown EOC mathematics (except grade 7 mathematics) or science tests

Post-Test Workshop

Who Counts? Performance Levels

M 32

• All CSTs, CAPA
• CMA grades 3–9 ELA, grades 2–7 math, Algebra I, and grades 5, 8, and 10 science
• STS grades 2–7 RLA and math (excluding Algebra I)
• Advanced, proficient, basic, below basic
• All valid scores falling in the performance level
• Far below basic
• All valid scores falling in the performance level
• CSTs taken with modifications (in aggregate reporting and accountability only)

Post-Test Workshop

Who Counts?Mean Scale Scores

M 32

• Average of valid scale scores
• Can be used to compare results for the same content/grade across years

Post-Test Workshop

Student Master List Summary:Reporting Clusters

M 32

Compare to:

Average percent correct range for students statewide who scored proficient on the total test(See the Post-Test Guide, Appendix A.)

Post-Test Workshop

B = Blank

C = Copied prompt

I = Illegible

L = Language other than English

R = Refusal

T = Off Topic

W = Wrong prompt (prompt from an earlier administration)

Student Master List Summary:Writing

M 33

Post-Test Workshop

Subgroup Summary:CSTs, CMA, CAPA, and STS

M 42−58

• Disability status
• Based on disability status for CST, CMA, STS
• CAPA: each disability type
• Economic status
• Based on NSLP eligibility or parent education level
• Gender
• English proficiency
• Ethnicity
• Ethnicity for Economic Status (only for CSTs; CMA in grades 3–9 ELA; grades 3–7 math, Algebra I, and grades 5, 8, and 10 science; and CAPA)

Post-Test Workshop

Subgroup Summary: Ethnicity for Economic Status

M 54 −58

Post-Test Workshop

Subgroup Summary: Ethnicity for Economic Status

M 54 −58

Example: Economically disadvantaged for each ethnicity

Post-Test Workshop

### Break — 10 minutes

Internet Reports

M 101−111

• Summaries based on same data as paper reports: CSTs, CMA, CAPA, STS
• Available to the public online for school, district, county, and state
• “Students with Scores” = number tested with scores
• CST Summaries of % Advanced and Proficient
• More subgroups than paper reports
• Parent education
• Special program participation
• Access from http://star.cde.ca.gov/

Post-Test Workshop

Internet Demonstration

Post-Test Workshop

Internet Reports:CST Sample

M 102

Post-Test Workshop

Internet Reports:CST Summary Sample

M 103

Only available for CSTs

Post-Test Workshop

Other Internet Reports
• CMA (M 104−105)
• Same as CST—CMA grades 3−9 ELA, grades 3–7 math, Algebra I, grades 5, 8, and 10 science
• Average percent (%) correct—CMA grades 10–11 ELA, Geometry
• CAPA (M 106−109)
• State level: same as CST; separate Level I
• County, district, school
• Mean scale score
• Percent proficient or above

Post-Test Workshop

Other Internet Reports (cont.)
• STS (M 110−111)
• Same as CST—STS grades 2−7 RLA and math (except Algebra I)
• Average percent (%) correct—STS grades 8−11 RLA and Algebra I and Geometry

Post-Test Workshop

Quiz Question 5

Which subgroup can only be accessed from the Internet?

• Parent Education Level
• CAPA by individual disability status
• Ethnicity for Economic Status
• English Proficiency

Post-Test Workshop

Quiz Question 5

Which subgroup can only be accessed from the Internet?

• Parent Education Level

Post-Test Workshop

Data CDs
• What are they?
• Lists of information from answer documents and scores of every student in the district
• In .txt format: wraps
• What are they used for?
• Searching for specific data
• Creating unique reports
• Verifying paper reports
• What else is needed?
• Text editor
• or Desktop application
• or Student Information System

Post-Test Workshop

View of Data
• As .txt, word wrap on
• With text editor, word wrap off

Post-Test Workshop

Organization of Data
• Two files:
• Demographics, special conditions, and test scores
• Accommodations, modifications, English Learners, and irregularities
• Data Layout = guide to location of data on files
• Position
• Number of characters
• Whether numeric or alpha

Post-Test Workshop

Data Layout Sample

Post-Test Workshop

Individual Reports

M 23−27

• STAR Student Record Label
• Adhesive label to affix to student’s permanent school record
• STAR Student Master List
• Alphabetical list of students and their scores
• Tests listed in order within grade
• CSTs
• CMA
• CAPA
• STS
• STAR Student Report: individual’s scores
• 2 two-sided color copies for each test
• For parents/guardians, school

Post-Test Workshop

Student Record Label Grade 10 Sample:Student Name and Identification

M 59−60

Post-Test Workshop

Student Record Label

M 59−60

Post-Test Workshop

Student Master List

M 61−65

Post-Test Workshop

Student Report

M 66−97

Post-Test Workshop

Student Report

M 75−76

Post-Test Workshop

Student Report

M 76

Student’s name on back

♦ = Percent correct obtained by the student on the reporting cluster/content area

▬ = Average percent-correct range on the reporting cluster for students statewide who scored proficient on the total test scores

Post-Test Workshop

M 7

• Reporting clusters not comparable
• Different difficulty
• Varying number of questions
• Average % correct of clusters  % correct on total test (or total number of test items)
• Scale scores
• Use conversion tables resulting from statistical procedures
• Equating allows scores to have similar meaning (e.g., 350 = lowest score for CSTs proficient)

Post-Test Workshop

Other Student Reports
• CMA (M 77−84)
• Performance levels and scale scores reported for grades 3–9 ELA, grades 3–7 math, Algebra I, and grades 5, 8, and 10 science
• Back, cluster reporting
• Percent correct for grades 10 and 11 ELA and Geometry
• CAPA (M 85−89)

Post-Test Workshop

Other Student Reports (cont.)
• STS (M 90−97)
• In Spanish
• Performance levels and scale scores, grades 2–7 RLA and math (except for Algebra I)
• Back, cluster percent correct
• Percent correct for entire test, grades 8–11 RLA and Algebra I and Geometry
• Back, cluster percent correct
• Back, how to use report

Post-Test Workshop

Unmatched Reports
• CST, CMA—Grs 8, 11
• Only CMA and no CST
• Grades 8 and 11 require CST for History–Social Science

Writing CST, CMA—Grs 4, 7

• Only multiple-choice
• Only writing
• Students receive 2 reports if writing score not matched to multiple-choice score

Post-Test Workshop

Summary
• Do’s and Don’ts
• Do compare mean scale score, percent at performance levels within same grade, same content area
• Do compare cluster scores to the corresponding proficient range provided for this year
• Do not compare cluster scores to each other within test, nor across years, grades, content area
• Do not compare mean scale scores across grades, content area
• Summary and Internet reports
• Data CDs
• Individual reports

Post-Test Workshop

Upcoming Dates

Post-Test Workshop

• http://www.cde.ca.gov/ta/tg/sr/
• http://www.star.cde.ca.gov/
• http://www.startest.org/
• STAR Technical Assistance Center
• 800-955-2954
• CDE Accountability

Post-Test Workshop

To Contact the CDE

Post-Test Workshop

### Early Assessment Program Results

August 10, 2011

Dusty Fox

Early Assessment Program (EAP)
• EAP results are being accepted at participating California Community Colleges (CCC).
• Approximately 55 CCC campuses are accepting EAP results
• List of participating colleges is available on the CCC website at www.CollegeEAP.org. Check the site often for an updated list of campuses.
• PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

Early Assessment Program (EAP)
• An EAP informational web site has been developed. www. CollegeEAP.org.
• The site provides EAP information for both CSU and CCC
• It is a resource for Students, Parents, and Educators
• PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

www.collegeEAP.org

Post-Test Workshop

Early Assessment Program (EAP)
• Reporting Results
• Student Statuses are on the STAR Report
• Ready for CSU and participating CCC college-level mathematics courses – Conditional
• This status in conjunction with a grade of “C” or better in an approved math course or a supervised e-learning program can be exempt from the ELM for CSU or other math placement tests at participating California Community Colleges (CCC)
• A list of courses and e-learning programs can be found at www.csumathsuccess.org
• PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

Early Assessment Program (EAP)
• Sample EAP Box on the STAR Student Report

The EAP box can be found on page 78 of the Post-Test Guide

• PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

Early Assessment Program (EAP)
• Web Sites
• www.CollegeEAP.org
• Links to CSU and CCC informational sites for Students, Parents, and Educators
• www.CalState.edu/EAP
• Provides a list of EAP Coordinators, EAP test blueprints, and informational materials
• www.CCCCO.edu/EAP
• Provides a list of participating Community Colleges
• PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop