2008 star interpreting and using results l.jpg
Advertisement
This presentation is the property of its rightful owner.
1 / 70

2008 STAR Interpreting and Using Results PowerPoint PPT Presentation

2008 STAR Interpreting and Using Results August/September 2008 Objectives Workshop participants will be able to: Describe the purposes of STAR reports Interpret STAR results Explain key statistics Compare and contrast types of reports Identify proper uses of reports Agenda Quiz

Download Presentation

2008 STAR Interpreting and Using Results

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


2008 star interpreting and using results l.jpg

2008 STARInterpreting and Using Results

August/September 2008


Objectives l.jpg

Objectives

Workshop participants will be able to:

  • Describe the purposes of STAR reports

  • Interpret STAR results

  • Explain key statistics

  • Compare and contrast types of reports

  • Identify proper uses of reports

Post-Test Workshop


Agenda l.jpg

Agenda

  • Quiz

  • Post Test Guide

  • What’s New?

  • Results and Statistical Analysis

  • Using Results

  • Summary and Internet Reports

  • Individual Student Reports

  • Teacher Reports

Post-Test Workshop


Post test guide layout l.jpg

Part I: General Info

New in 2008

Interpreting Reports

Comparing Results

Part II: Report Descriptions

Summary

Individual

Teacher

Internet

Part III: Appendices

Reporting clusters

Scale score ranges

Statistical tables

Test Score Verification procedures

Demographic Data Correction procedures

California reading list

Glossary

Post-Test Guide Layout

Post-Test Guide Pages (Pp) i–iii

Post-Test Workshop


What s new in 2008 l.jpg

CMA reports

Percent correct

Grades 3–5

STS reports

Percent correct

Grades 5–7 added

CAPA Science percent correct

Ethnicity for Economic Status subgroup summary (CSTs only)

STAR Student Reports sorted for district

All without addresses

All with addresses

What’s New in 2008

Post-Test Guide P 2

Post-Test Workshop


Results purposes of star reports l.jpg

Results: Purposes of STAR Reports

Post-Test Guide P 3

  • Report progress of students toward proficiency on the state’s academic content standards

  • Inform where improvement needed

    • To help students’ achievement

    • To improve educational programs

  • Provide data for state and federal accountability programs

Post-Test Workshop


Results performance levels l.jpg

Results: Performance Levels

Post-Test Guide P 7

  • State goal: PROFICIENT or higher

    • Scale score of 350 or higher for CSTs

    • Scale score of 35 or higher for CAPA

  • Other performance levels:

    • Advanced

    • Basic (300 CSTs, 30 CAPA)

    • Below Basic

    • Far Below Basic

    • CST and CAPA cut points vary for Advanced, Below Basic, and Far Below Basic by

      • Subject

      • Grade

Post-Test Workshop


Results scale scores l.jpg

Results: Scale Scores

Post-Test Guide P 7–8

  • Scale scores allow same score to mean same thing across test forms within grade and content area and across time

  • Scale score ranges by program:

    • CSTs: 150–600 for each grade and subject

    • CAT/6 Survey: 1–999 for each grade and subject

    • CAPA: 15–60 for each grade and subject

    • CMA, STS, CAPA Science: Not available until 2009

Post-Test Workshop


Results equating l.jpg

Results: Equating

Post-Test Guide Page 7

  • Psychometric procedure

  • Adjusts for test difficulty

  • Additional information in CST Technical Report on CDE Web site

  • For 2008

    • CSTs and CAPA (except science) are equated

    • CMA (3–5),CAPA (science), and STS (2–4) and will be equated starting in 2009

  • CAT/6 nationally norm-referenced; not equated for STAR

Post-Test Workshop


Results reporting clusters content areas l.jpg

Results: Reporting Clusters(Content Areas)

Post-Test Guide P 36

Appendices A & B

  • Four to six clusters for each content area (subject)

  • May be useful as indicators of individual or group strengths and weaknesses

  • But… Reporting clusters should be interpreted with caution

Post-Test Workshop


Results reporting cluster cautions l.jpg

Results: Reporting ClusterCautions

Post-Test Guide P 8–10

Appendices A & B

  • Based on small numbers of items; therefore may not be reliable or generalizable

  • NOT equated from year to year

  • Should not compare reporting cluster percent correct from year to year

Post-Test Workshop


Interpreting reporting clusters or content areas in the same year l.jpg

Interpreting Reporting Clusters or Content Areas in the Same Year

Post-Test Guide P 8–10; 84

Appendices A & B

  • CSTs—Performance on the clusters for students statewide scoring from

    • Lowest score for proficient to

    • Lowest score for advanced

Post-Test Workshop


Csts 2008 reporting clusters number of questions and average percent correct l.jpg

CSTs 2008 Reporting Clusters — Number of Questions and Average Percent Correct

Post-Test Guide Page 138(Appendix A)

Post-Test Workshop


Examples interpreting reporting clusters for cst for geometry l.jpg

Examples—Interpreting Reporting Clusters for CST for Geometry

Post-Test Guide P 9

Post-Test Workshop


Using results l.jpg

Using Results

Post-Test Guide P 3

  • For instructional decisions withother data: CSTs, CMA, CAPA, STS

  • Used in Academic Performance Index (API) calculations: CSTs, CAT/6 Survey,CAPA

  • Used in adequate yearly progress (AYP) calculations for grades 2–8 in ELA and math for CSTs and 2–8 and 10 in ELA and math for CAPA

Post-Test Workshop


Year to year comparisons do compare same grade and same content area l.jpg

Year-to-Year ComparisonsDo Compare Same Grade and Same Content Area

Post-Test Guide Pp 11–14

  • Mean scale score

    • Same content and grade, varying years

  • Percent in each performance level

    • Same content by grade across years

      • e.g., 2007 ELA grade 10 with 2008 ELA grade 10

Post-Test Workshop


Year to year comparisons do compare same grade or same content area l.jpg

Year-to-Year ComparisonsDo Compare Same Grade or Same Content Area

Post-Test Guide Pp 11–14

  • Percentage of students scoring at PROFICIENT and above

    • For a given grade and subject, e.g., Percent proficient and above for grade 3 math in 2007 & 2008

    • For a given subject and aggregated grades, e.g., % proficient and above for grades 2 to 6 math in 2007 & 2008

    • Across grades and a subject, e.g., % proficient and above in all courses and all grades

Post-Test Workshop


Year to year comparisons don t compare l.jpg

Year-to-Year ComparisonsDON’T Compare

Post-Test Guide Pp 11–14

  • Individual scale scores or statistics based on scale scores for different grades or content areas

    • Subjects by grade are independently scaled

    • Different content standards are measured in different grades

  • Cohorts across grades

  • CMA, STS scores because notscaled

Post-Test Workshop


Example using cst results to compare grade results from year to year l.jpg

Example—Using CST Results to Compare Grade Results from Year to Year

Post-Test Guide P 12

Post-Test Workshop


Aggregate summary reports l.jpg

Aggregate (Summary) Reports

Post-Test Guide Pp 17–21

  • What are they?

    • Student Master List Summary

    • Student Master List Summary EOC

    • Subgroup Summary

    • Group Summary: CAT/6 Survey

  • Report Emphasis: California Standards Tests (CSTs)

    • Criterion-referenced tests

    • Progress is measured in percent of students scoring proficient and advanced

Post-Test Workshop


Student master list summary l.jpg

Student Master List Summary

Post-Test Guide Pp 17; 26–37

  • By grade

  • CSTs, CMA, CAPA, and STS

  • Lists subjects

  • % and # at each Performance Level

    • Not for CMA, CAPA Science, STS

  • Mean Scale Score

    • Not for CMA, CAPA Science, STS

  • Reporting Cluster: Mean Percent Correct

    • Not for CMA, CAPA, and STS grades 5–7

Post-Test Workshop


Student master list summary22 l.jpg

Student Master List Summary

Post-Test Guide P 27

Post-Test Workshop


Student master list summary23 l.jpg

Student Master List Summary

Post-Test Guide P 27

Post-Test Workshop


Who counts l.jpg

Who Counts?

Post-Test Guide P 28

  • Number Enrolled

    • All CST, CMA, CAPA, and STS multiple-choice answer documents submitted as scorable

    • Minus

      • Documents marked as “Student enrolled after the first day of testing and was given this test”

Post-Test Workshop


Who counts25 l.jpg

Who Counts?

Post-Test Guide P 28

  • Number, Percent Tested

    • All CST, CMA, CAPA, STS answer documents with one or more answers

    • Plus

      • Z = Tested but marked no answers

    • Not included

      • P = Parent/guardian exemptions

      • A = Students absent

      • T = Enrolled first day, not tested, tested at previous school

      • E = Not tested due to significant medical emergency

      • Students with inconsistent grades

Post-Test Workshop


Who counts26 l.jpg

Who Counts?

Post-Test Guide P 27

  • Number Valid Scores

    • For the subject, number of students tested at grade level who received a score for the test (that is, a scale score or percent correct).

    • Not included:

      • Incomplete tests

      • Modified tests

      • Non-English learners who took the STS

      • Unknown EOC math or science tests

      • Inconsistent grades

Why does STS not have 100% valid scores?

Post-Test Workshop


Who counts27 l.jpg

Who Counts?

Post-Test Guide P 28

  • Number Tested with Scores

    • All tests taken, including those taken with modifications, that receive a score

    • All STS and CMA tests taken

    • Not included

      • Incomplete tests

      • Non-English learners who took the STS

      • Unknown EOC math or science tests

      • Inconsistent grades

Post-Test Workshop


Who counts28 l.jpg

Who Counts?

Post-Test Guide P 29

  • Performance Level Counts for CSTs

    • Advanced, Proficient, Basic, Below Basic

      • All valid scores falling in the performance level

    • Far Below Basic

      • All valid scores falling in the performance level

      • Modified tests (in aggregate reporting and accountability only)

Post-Test Workshop


Student master list summary performance levels l.jpg

Student Master List SummaryPerformance Levels

Post-Test Guide P 30

Post-Test Workshop


Student master list summary reporting clusters l.jpg

Student Master List SummaryReporting Clusters

Post-Test Guide P 30

Compare to:

Average percent correct range for students statewide who scored proficient on the total test

Post-Test Workshop


Student master list summary writing l.jpg

B=Blank

C=Copied Prompt

I=Illegible

L=Language other than English

R=Refusal

T=Off Topic

W=Wrong prompt (Prompt from an earlier administration)

Student Master List Summary:Writing

Post-Test Guide P 30

Post-Test Workshop


Student master list summary end of course l.jpg

Student Master List Summary: End-of-Course

Post-Test Guide Pp 32–37

  • By Subject

  • Lists each grade eligible to take test

  • Math (Grades 7–11)

    • General Math, Algebra I, Geometry, etc.

  • Science (Grades 9–11)

    • Earth Science, Biology, Chemistry, etc.

  • History–Social Science (Grades 9–11)

    • World History

  • Same statistics as grade-level Student Master List Summary

Post-Test Workshop


Subgroup summary csts cma capa and sts l.jpg

Subgroup Summary:CSTs, CMA, CAPA, and STS

Post-Test Guide Pp 38–64

  • Disability Status

    • Based on “Disability Code”

  • Economic Status

    • Based on “NSLP” & “Parent Ed Level”

  • Gender

  • English Proficiency

  • Primary Ethnicity

  • Ethnicity for Economic Status: CSTs and CAPA (ELA and Math) only

Post-Test Workshop


Subgroup summary ethnicity for economic status l.jpg

Subgroup Summary: Ethnicity for Economic Status

Post-Test Guide Pp 60–64

Post-Test Workshop


Subgroup summary ethnicity for economic status35 l.jpg

Subgroup Summary: Ethnicity for Economic Status

Post-Test Guide Pp 60–64

Post-Test Workshop


Subgroup summary ethnicity for economically disadvantaged l.jpg

Subgroup Summary: Ethnicity for Economically Disadvantaged

Post-Test Guide Pages 60–64

Post-Test Workshop


Internet reports l.jpg

Internet Reports

Post-Test Guide Pp 123–134

  • Summaries based on same data as paper reports: CSTs, CMA, CAPA, STS

  • Available to public online

  • More subgroups than paper reports

    • English-language fluency

    • Parent education

    • Special Program Participation

  • Access from http://star.cde.ca.gov

Post-Test Workshop


Internet reports available reports l.jpg

CSTs

Mean scale scores

Percents by performance levels

CST Summary

Percent proficient and advanced

CMA

Average percent correct

Grades 3–5 only

STS

Average percent correct

Grades 2–7 only

CAPA

Mean scale scores

Percents by performance levels

Average percent correct for science

CAT/6 Survey

Mean scale scores

NPR for “Avg. Student”

Percent above percentile cuts

Grades 3 and 7 only

Internet Reports:Available Reports

Post-Test Guide Pp 123-134

Post-Test Workshop


Slide39 l.jpg

Internet ReportsAccess Reports

Post-Test Guide Pp 124

Post-Test Workshop


Internet reports cst sample l.jpg

Internet Reports:CST Sample

Post-Test Guide P 127

Post-Test Workshop


Internet reports cst summary sample l.jpg

Internet Reports:CST Summary Sample

Post-Test Guide P 128

Post-Test Workshop


Internet reports cma correct l.jpg

Internet Reports:CMA% Correct

Post-Test Guide P 129

Post-Test Workshop


Internet reports sts correct l.jpg

Internet Reports:STS% Correct

Post-Test Guide Pp 133

Post-Test Workshop


Exercise l.jpg

Exercise

  • Work with a partner or in small groups

  • Plan what data you’ll present to leaders in your district

  • Remember the Do’s and Don’ts and Cautions

Post-Test Workshop


Break 15 minutes l.jpg

Break — 15 minutes


Individual reports l.jpg

Individual Reports

Post-Test Guide Pp 70–112

  • STAR Student Record Label

    • Adhesive label to affix to student’s permanent school record

  • STAR Student Master List

    • Alphabetical list of students and their scores

    • Tests listed in order within grade

      • CSTs-CAT/6 Survey

      • CMA-CAT/6 Survey

      • CAPA

      • STS

  • STAR Student Report: individual’s scores

    • 2 two-sided color copies

    • For parents/guardians

    • For each test

Post-Test Workshop


Student record label student name and identification l.jpg

Student Record Label:Student Name and Identification

Post-Test Guide Pp 70–73

Post-Test Workshop


Student record label cst for grade 10 results l.jpg

Student Record Label:CST for Grade 10 Results

Post-Test Guide Pp 70–73

Post-Test Workshop


Student master list csts cma cat 6 for grade 3 sample l.jpg

Student Master List:CSTs/CMA-CAT/6 for Grade 3 Sample

Post-Test Guide Pp 74–79

Post-Test Workshop


Student master list csts cma cat 6 for grade 3 sample50 l.jpg

Student Master List:CSTs/CMA-CAT/6 for Grade 3 Sample

Post-Test Guide Pp 74–79

Post-Test Workshop


Student report cst for grade 11 sample l.jpg

Student Report:CST for Grade 11 Sample

Post-Test Guide Pp 81–90

Post-Test Workshop


Student report cst for grade 11 sample52 l.jpg

Student Report:CST for Grade 11 Sample

Post-Test Guide Pp 81–90

Post-Test Workshop


Student report csts for grade 10 sample l.jpg

Student Report:CSTs for Grade 10 Sample

Post-Test Guide Pp 81–90

StudentNameOn Back

♦=Percent correct obtained by the student on the reporting cluster/content area

▬▬▬=Range of percent correct scores on the reporting cluster for students who scored proficient on the total test

Post-Test Workshop


Reporting cluster range for each content area by grade l.jpg

Reporting Cluster Rangefor Each Content Area by Grade

Reporting cluster average percent correct from all minimally advanced scores minus one (e.g., 401, grade 4 math)

Reporting cluster average percent correct from all minimally proficient scores (350)

Post-Test Workshop


Explain to parents scale score average correct cluster score x 600 l.jpg

Explain to ParentsScale Score  Average % correct cluster score x 600

  • Reporting Clusters not comparable

    • Different difficulty

    • Varying number of questions

    • Average % correct of clusters % correct of total scale score

  • Scale scores

    • Use conversion tables and other statistical techniques

    • Equating allows scores to have similar meaning (e.g., 350 = lowest score for CSTs proficient)

Post-Test Workshop


Student report cst for grade 7 with cat 6 sample l.jpg

Student Report:CST for Grade 7 with CAT/6 Sample

Post-Test Guide P 85

Post-Test Workshop


Student report cst for grade 11 with eap sample l.jpg

Student Report:CST for Grade 11 with EAP Sample

Post-Test Guide Pp 81–90

Post-Test Workshop


Other student reports l.jpg

Other Student Reports

  • CMA (Pp 91–98)

    • Percent correct, no performance levels

    • Back, “About the CMA,” P 93

    • No cluster reporting

  • CAPA (Pp 99–104)

    • Back, performance level descriptors for ELA, math P 104

  • STS (Pp 105–112)

    • In Spanish

    • Percent correct, no performance levels

    • Reporting clusters percent correct, grades 2–4

    • Back, how to use report P 108

Post-Test Workshop


Exercise59 l.jpg

Exercise

  • Work with a partner or in small groups

  • Use a Student Report sample and explain content reporting diamonds and bars on back

Post-Test Workshop


Unmatched report l.jpg

Unmatched Report

  • Grade 3

    • CST but no CAT/6 Survey

    • CMA but no CAT/6 Survey

    • CAT/6 Survey but no CSTs or CMA

  • Grades 4 and 7

    • Unmatched Multiple-Choice Report

      • CST Multiple-choice but no writing test

    • Unmatched Writing Report

      • Writing but no CSTs multiple-choice test

Post-Test Workshop


Unmatched report61 l.jpg

Unmatched Report:

Post-Test Workshop


Reports for teachers l.jpg

Reports for Teachers

Post-Test Guide Pp 113–122

  • Summary of results for previous year’s class(es)

  • Two types of summary reports

    • By grade level (e.g., grade 11 ELA) or course (e.g., Algebra I)

    • By teacher name or group name (e.g., GATE)

  • Separate reports for:

    • CST ELA

    • CST Math

  • Not available for:

    • CST science or history–social science

    • CMA

    • CAPA

    • STS

Post-Test Workshop


Reports for teachers63 l.jpg

Reports for Teachers

Post-Test Guide Pp 113 – 122

  • Individualized reports (teacher name or group name)

    • if name on SGID or Pre-ID

    • SGID priority over Pre-ID

  • Group reports (grade level or course): all ELA and math teachers should receive

  • EOC Math

    • Grades 8–11 by subject only

    • Not grade: e.g., All Algebra I in a school

Post-Test Workshop


Report for teachers sample l.jpg

Report for Teachers:Sample

Post-Test Guide Pp 114–122

Post-Test Workshop


Report for teachers 2008 performance by reporting clusters l.jpg

Report for Teachers:2008 PerformanceBy Reporting Clusters

Post-Test Guide Pp 114

Reporting clusters defined under this section.

Post-Test Workshop


Report for teachers cluster reporting l.jpg

Report for Teachers:Cluster Reporting

  • For each reporting cluster, the average percent correct achieved by:

    • Statewide students

    • District students

    • Your students

    • Proficient range

  • Definition of symbols

     = average % correct for teacher’s students

    ▬▬▬ = average % correct range for students statewide who scored proficient on total test

Post-Test Workshop


Report for teachers back l.jpg

Report for TeachersBack

Post-Test Guide P 117

  • School % at each performance level by subgroups

  • % Proficient and above

    • 2007 vs. 2008

    • School; district; state

    • By subgroup

  • Resources

Post-Test Workshop


Early assessment program eap l.jpg

Early Assessment Program (EAP)

  • Reporting Results

    • Student Statuses are on the STAR Report

    • Ready for CSU college-level English/mathematics courses

    • Ready for CSU college-level mathematics courses – Conditional

      • This status in conjunction with a grade of “C” or better in an approved math course or a supervised e-learning program can be exempt from the ELM

      • A list of courses and e-learning programs can be found at www.csumathsuccess.org

    • Not yet demonstrating readiness for CSU college-level English/mathematics courses

Post-Test Workshop


Summary l.jpg

Summary

  • Do’s and Don’ts

    • Do compare mean scale score, percent at performance levels within same grade, same content area

    • Don’t compare across grades, content area

  • Summary reports

  • Individual reports

  • Teacher reports

  • Quiz answers

  • Evaluations

Post-Test Workshop


Quiz answers l.jpg

Quiz Answers

  • 1. B

  • 2. D

  • 3. A

  • 4. C

  • 5. D

Post-Test Workshop


  • Login