2008 star interpreting and using results l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
2008 STAR Interpreting and Using Results PowerPoint Presentation
Download Presentation
2008 STAR Interpreting and Using Results

Loading in 2 Seconds...

play fullscreen
1 / 70

2008 STAR Interpreting and Using Results - PowerPoint PPT Presentation


  • 589 Views
  • Uploaded on

2008 STAR Interpreting and Using Results August/September 2008 Objectives Workshop participants will be able to: Describe the purposes of STAR reports Interpret STAR results Explain key statistics Compare and contrast types of reports Identify proper uses of reports Agenda Quiz

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '2008 STAR Interpreting and Using Results' - albert


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
objectives
Objectives

Workshop participants will be able to:

  • Describe the purposes of STAR reports
  • Interpret STAR results
  • Explain key statistics
  • Compare and contrast types of reports
  • Identify proper uses of reports

Post-Test Workshop

agenda
Agenda
  • Quiz
  • Post Test Guide
  • What’s New?
  • Results and Statistical Analysis
  • Using Results
  • Summary and Internet Reports
  • Individual Student Reports
  • Teacher Reports

Post-Test Workshop

post test guide layout
Part I: General Info

New in 2008

Interpreting Reports

Comparing Results

Part II: Report Descriptions

Summary

Individual

Teacher

Internet

Part III: Appendices

Reporting clusters

Scale score ranges

Statistical tables

Test Score Verification procedures

Demographic Data Correction procedures

California reading list

Glossary

Post-Test Guide Layout

Post-Test Guide Pages (Pp) i–iii

Post-Test Workshop

what s new in 2008
CMA reports

Percent correct

Grades 3–5

STS reports

Percent correct

Grades 5–7 added

CAPA Science percent correct

Ethnicity for Economic Status subgroup summary (CSTs only)

STAR Student Reports sorted for district

All without addresses

All with addresses

What’s New in 2008

Post-Test Guide P 2

Post-Test Workshop

results purposes of star reports
Results: Purposes of STAR Reports

Post-Test Guide P 3

  • Report progress of students toward proficiency on the state’s academic content standards
  • Inform where improvement needed
    • To help students’ achievement
    • To improve educational programs
  • Provide data for state and federal accountability programs

Post-Test Workshop

results performance levels
Results: Performance Levels

Post-Test Guide P 7

  • State goal: PROFICIENT or higher
    • Scale score of 350 or higher for CSTs
    • Scale score of 35 or higher for CAPA
  • Other performance levels:
    • Advanced
    • Basic (300 CSTs, 30 CAPA)
    • Below Basic
    • Far Below Basic
    • CST and CAPA cut points vary for Advanced, Below Basic, and Far Below Basic by
      • Subject
      • Grade

Post-Test Workshop

results scale scores
Results: Scale Scores

Post-Test Guide P 7–8

  • Scale scores allow same score to mean same thing across test forms within grade and content area and across time
  • Scale score ranges by program:
    • CSTs: 150–600 for each grade and subject
    • CAT/6 Survey: 1–999 for each grade and subject
    • CAPA: 15–60 for each grade and subject
    • CMA, STS, CAPA Science: Not available until 2009

Post-Test Workshop

results equating
Results: Equating

Post-Test Guide Page 7

  • Psychometric procedure
  • Adjusts for test difficulty
  • Additional information in CST Technical Report on CDE Web site
  • For 2008
    • CSTs and CAPA (except science) are equated
    • CMA (3–5),CAPA (science), and STS (2–4) and will be equated starting in 2009
  • CAT/6 nationally norm-referenced; not equated for STAR

Post-Test Workshop

results reporting clusters content areas
Results: Reporting Clusters(Content Areas)

Post-Test Guide P 36

Appendices A & B

  • Four to six clusters for each content area (subject)
  • May be useful as indicators of individual or group strengths and weaknesses
  • But… Reporting clusters should be interpreted with caution

Post-Test Workshop

results reporting cluster cautions
Results: Reporting ClusterCautions

Post-Test Guide P 8–10

Appendices A & B

  • Based on small numbers of items; therefore may not be reliable or generalizable
  • NOT equated from year to year
  • Should not compare reporting cluster percent correct from year to year

Post-Test Workshop

interpreting reporting clusters or content areas in the same year
Interpreting Reporting Clusters or Content Areas in the Same Year

Post-Test Guide P 8–10; 84

Appendices A & B

  • CSTs—Performance on the clusters for students statewide scoring from
    • Lowest score for proficient to
    • Lowest score for advanced

Post-Test Workshop

csts 2008 reporting clusters number of questions and average percent correct
CSTs 2008 Reporting Clusters — Number of Questions and Average Percent Correct

Post-Test Guide Page 138(Appendix A)

Post-Test Workshop

examples interpreting reporting clusters for cst for geometry
Examples—Interpreting Reporting Clusters for CST for Geometry

Post-Test Guide P 9

Post-Test Workshop

using results
Using Results

Post-Test Guide P 3

  • For instructional decisions withother data: CSTs, CMA, CAPA, STS
  • Used in Academic Performance Index (API) calculations: CSTs, CAT/6 Survey,CAPA
  • Used in adequate yearly progress (AYP) calculations for grades 2–8 in ELA and math for CSTs and 2–8 and 10 in ELA and math for CAPA

Post-Test Workshop

year to year comparisons do compare same grade and same content area
Year-to-Year ComparisonsDo Compare Same Grade and Same Content Area

Post-Test Guide Pp 11–14

  • Mean scale score
    • Same content and grade, varying years
  • Percent in each performance level
    • Same content by grade across years
      • e.g., 2007 ELA grade 10 with 2008 ELA grade 10

Post-Test Workshop

year to year comparisons do compare same grade or same content area
Year-to-Year ComparisonsDo Compare Same Grade or Same Content Area

Post-Test Guide Pp 11–14

  • Percentage of students scoring at PROFICIENT and above
    • For a given grade and subject, e.g., Percent proficient and above for grade 3 math in 2007 & 2008
    • For a given subject and aggregated grades, e.g., % proficient and above for grades 2 to 6 math in 2007 & 2008
    • Across grades and a subject, e.g., % proficient and above in all courses and all grades

Post-Test Workshop

year to year comparisons don t compare
Year-to-Year ComparisonsDON’T Compare

Post-Test Guide Pp 11–14

  • Individual scale scores or statistics based on scale scores for different grades or content areas
    • Subjects by grade are independently scaled
    • Different content standards are measured in different grades
  • Cohorts across grades
  • CMA, STS scores because notscaled

Post-Test Workshop

example using cst results to compare grade results from year to year
Example—Using CST Results to Compare Grade Results from Year to Year

Post-Test Guide P 12

Post-Test Workshop

aggregate summary reports
Aggregate (Summary) Reports

Post-Test Guide Pp 17–21

  • What are they?
    • Student Master List Summary
    • Student Master List Summary EOC
    • Subgroup Summary
    • Group Summary: CAT/6 Survey
  • Report Emphasis: California Standards Tests (CSTs)
    • Criterion-referenced tests
    • Progress is measured in percent of students scoring proficient and advanced

Post-Test Workshop

student master list summary
Student Master List Summary

Post-Test Guide Pp 17; 26–37

  • By grade
  • CSTs, CMA, CAPA, and STS
  • Lists subjects
  • % and # at each Performance Level
    • Not for CMA, CAPA Science, STS
  • Mean Scale Score
    • Not for CMA, CAPA Science, STS
  • Reporting Cluster: Mean Percent Correct
    • Not for CMA, CAPA, and STS grades 5–7

Post-Test Workshop

student master list summary22
Student Master List Summary

Post-Test Guide P 27

Post-Test Workshop

student master list summary23
Student Master List Summary

Post-Test Guide P 27

Post-Test Workshop

who counts
Who Counts?

Post-Test Guide P 28

  • Number Enrolled
    • All CST, CMA, CAPA, and STS multiple-choice answer documents submitted as scorable
    • Minus
      • Documents marked as “Student enrolled after the first day of testing and was given this test”

Post-Test Workshop

who counts25
Who Counts?

Post-Test Guide P 28

  • Number, Percent Tested
    • All CST, CMA, CAPA, STS answer documents with one or more answers
    • Plus
      • Z = Tested but marked no answers
    • Not included
      • P = Parent/guardian exemptions
      • A = Students absent
      • T = Enrolled first day, not tested, tested at previous school
      • E = Not tested due to significant medical emergency
      • Students with inconsistent grades

Post-Test Workshop

who counts26
Who Counts?

Post-Test Guide P 27

  • Number Valid Scores
    • For the subject, number of students tested at grade level who received a score for the test (that is, a scale score or percent correct).
    • Not included:
      • Incomplete tests
      • Modified tests
      • Non-English learners who took the STS
      • Unknown EOC math or science tests
      • Inconsistent grades

Why does STS not have 100% valid scores?

Post-Test Workshop

who counts27
Who Counts?

Post-Test Guide P 28

  • Number Tested with Scores
    • All tests taken, including those taken with modifications, that receive a score
    • All STS and CMA tests taken
    • Not included
      • Incomplete tests
      • Non-English learners who took the STS
      • Unknown EOC math or science tests
      • Inconsistent grades

Post-Test Workshop

who counts28
Who Counts?

Post-Test Guide P 29

  • Performance Level Counts for CSTs
    • Advanced, Proficient, Basic, Below Basic
      • All valid scores falling in the performance level
    • Far Below Basic
      • All valid scores falling in the performance level
      • Modified tests (in aggregate reporting and accountability only)

Post-Test Workshop

student master list summary performance levels
Student Master List SummaryPerformance Levels

Post-Test Guide P 30

Post-Test Workshop

student master list summary reporting clusters
Student Master List SummaryReporting Clusters

Post-Test Guide P 30

Compare to:

Average percent correct range for students statewide who scored proficient on the total test

Post-Test Workshop

student master list summary writing
B = Blank

C = Copied Prompt

I = Illegible

L = Language other than English

R = Refusal

T = Off Topic

W = Wrong prompt (Prompt from an earlier administration)

Student Master List Summary:Writing

Post-Test Guide P 30

Post-Test Workshop

student master list summary end of course
Student Master List Summary: End-of-Course

Post-Test Guide Pp 32–37

  • By Subject
  • Lists each grade eligible to take test
  • Math (Grades 7–11)
    • General Math, Algebra I, Geometry, etc.
  • Science (Grades 9–11)
    • Earth Science, Biology, Chemistry, etc.
  • History–Social Science (Grades 9–11)
    • World History
  • Same statistics as grade-level Student Master List Summary

Post-Test Workshop

subgroup summary csts cma capa and sts
Subgroup Summary:CSTs, CMA, CAPA, and STS

Post-Test Guide Pp 38–64

  • Disability Status
    • Based on “Disability Code”
  • Economic Status
    • Based on “NSLP” & “Parent Ed Level”
  • Gender
  • English Proficiency
  • Primary Ethnicity
  • Ethnicity for Economic Status: CSTs and CAPA (ELA and Math) only

Post-Test Workshop

subgroup summary ethnicity for economic status
Subgroup Summary: Ethnicity for Economic Status

Post-Test Guide Pp 60–64

Post-Test Workshop

subgroup summary ethnicity for economic status35
Subgroup Summary: Ethnicity for Economic Status

Post-Test Guide Pp 60–64

Post-Test Workshop

subgroup summary ethnicity for economically disadvantaged
Subgroup Summary: Ethnicity for Economically Disadvantaged

Post-Test Guide Pages 60–64

Post-Test Workshop

internet reports
Internet Reports

Post-Test Guide Pp 123–134

  • Summaries based on same data as paper reports: CSTs, CMA, CAPA, STS
  • Available to public online
  • More subgroups than paper reports
    • English-language fluency
    • Parent education
    • Special Program Participation
  • Access from http://star.cde.ca.gov

Post-Test Workshop

internet reports available reports
CSTs

Mean scale scores

Percents by performance levels

CST Summary

Percent proficient and advanced

CMA

Average percent correct

Grades 3–5 only

STS

Average percent correct

Grades 2–7 only

CAPA

Mean scale scores

Percents by performance levels

Average percent correct for science

CAT/6 Survey

Mean scale scores

NPR for “Avg. Student”

Percent above percentile cuts

Grades 3 and 7 only

Internet Reports:Available Reports

Post-Test Guide Pp 123-134

Post-Test Workshop

slide39

Internet ReportsAccess Reports

Post-Test Guide Pp 124

Post-Test Workshop

internet reports cst sample
Internet Reports:CST Sample

Post-Test Guide P 127

Post-Test Workshop

internet reports cst summary sample
Internet Reports:CST Summary Sample

Post-Test Guide P 128

Post-Test Workshop

internet reports cma correct
Internet Reports:CMA% Correct

Post-Test Guide P 129

Post-Test Workshop

internet reports sts correct
Internet Reports:STS% Correct

Post-Test Guide Pp 133

Post-Test Workshop

exercise
Exercise
  • Work with a partner or in small groups
  • Plan what data you’ll present to leaders in your district
  • Remember the Do’s and Don’ts and Cautions

Post-Test Workshop

individual reports
Individual Reports

Post-Test Guide Pp 70–112

  • STAR Student Record Label
    • Adhesive label to affix to student’s permanent school record
  • STAR Student Master List
    • Alphabetical list of students and their scores
    • Tests listed in order within grade
      • CSTs-CAT/6 Survey
      • CMA-CAT/6 Survey
      • CAPA
      • STS
  • STAR Student Report: individual’s scores
    • 2 two-sided color copies
    • For parents/guardians
    • For each test

Post-Test Workshop

student record label student name and identification
Student Record Label:Student Name and Identification

Post-Test Guide Pp 70–73

Post-Test Workshop

student record label cst for grade 10 results
Student Record Label:CST for Grade 10 Results

Post-Test Guide Pp 70–73

Post-Test Workshop

student master list csts cma cat 6 for grade 3 sample
Student Master List:CSTs/CMA-CAT/6 for Grade 3 Sample

Post-Test Guide Pp 74–79

Post-Test Workshop

student master list csts cma cat 6 for grade 3 sample50
Student Master List:CSTs/CMA-CAT/6 for Grade 3 Sample

Post-Test Guide Pp 74–79

Post-Test Workshop

student report cst for grade 11 sample
Student Report:CST for Grade 11 Sample

Post-Test Guide Pp 81–90

Post-Test Workshop

student report cst for grade 11 sample52
Student Report:CST for Grade 11 Sample

Post-Test Guide Pp 81–90

Post-Test Workshop

student report csts for grade 10 sample
Student Report:CSTs for Grade 10 Sample

Post-Test Guide Pp 81–90

StudentNameOn Back

♦ = Percent correct obtained by the student on the reporting cluster/content area

▬▬▬ = Range of percent correct scores on the reporting cluster for students who scored proficient on the total test

Post-Test Workshop

reporting cluster range for each content area by grade
Reporting Cluster Rangefor Each Content Area by Grade

Reporting cluster average percent correct from all minimally advanced scores minus one (e.g., 401, grade 4 math)

Reporting cluster average percent correct from all minimally proficient scores (350)

Post-Test Workshop

explain to parents scale score average correct cluster score x 600
Explain to ParentsScale Score  Average % correct cluster score x 600
  • Reporting Clusters not comparable
    • Different difficulty
    • Varying number of questions
    • Average % correct of clusters % correct of total scale score
  • Scale scores
    • Use conversion tables and other statistical techniques
    • Equating allows scores to have similar meaning (e.g., 350 = lowest score for CSTs proficient)

Post-Test Workshop

student report cst for grade 7 with cat 6 sample
Student Report:CST for Grade 7 with CAT/6 Sample

Post-Test Guide P 85

Post-Test Workshop

student report cst for grade 11 with eap sample
Student Report:CST for Grade 11 with EAP Sample

Post-Test Guide Pp 81–90

Post-Test Workshop

other student reports
Other Student Reports
  • CMA (Pp 91–98)
    • Percent correct, no performance levels
    • Back, “About the CMA,” P 93
    • No cluster reporting
  • CAPA (Pp 99–104)
    • Back, performance level descriptors for ELA, math P 104
  • STS (Pp 105–112)
    • In Spanish
    • Percent correct, no performance levels
    • Reporting clusters percent correct, grades 2–4
    • Back, how to use report P 108

Post-Test Workshop

exercise59
Exercise
  • Work with a partner or in small groups
  • Use a Student Report sample and explain content reporting diamonds and bars on back

Post-Test Workshop

unmatched report
Unmatched Report
  • Grade 3
    • CST but no CAT/6 Survey
    • CMA but no CAT/6 Survey
    • CAT/6 Survey but no CSTs or CMA
  • Grades 4 and 7
    • Unmatched Multiple-Choice Report
      • CST Multiple-choice but no writing test
    • Unmatched Writing Report
      • Writing but no CSTs multiple-choice test

Post-Test Workshop

unmatched report61
Unmatched Report:

Post-Test Workshop

reports for teachers
Reports for Teachers

Post-Test Guide Pp 113–122

  • Summary of results for previous year’s class(es)
  • Two types of summary reports
    • By grade level (e.g., grade 11 ELA) or course (e.g., Algebra I)
    • By teacher name or group name (e.g., GATE)
  • Separate reports for:
    • CST ELA
    • CST Math
  • Not available for:
    • CST science or history–social science
    • CMA
    • CAPA
    • STS

Post-Test Workshop

reports for teachers63
Reports for Teachers

Post-Test Guide Pp 113 – 122

  • Individualized reports (teacher name or group name)
    • if name on SGID or Pre-ID
    • SGID priority over Pre-ID
  • Group reports (grade level or course): all ELA and math teachers should receive
  • EOC Math
    • Grades 8–11 by subject only
    • Not grade: e.g., All Algebra I in a school

Post-Test Workshop

report for teachers sample
Report for Teachers:Sample

Post-Test Guide Pp 114–122

Post-Test Workshop

report for teachers 2008 performance by reporting clusters
Report for Teachers:2008 PerformanceBy Reporting Clusters

Post-Test Guide Pp 114

Reporting clusters defined under this section.

Post-Test Workshop

report for teachers cluster reporting
Report for Teachers:Cluster Reporting
  • For each reporting cluster, the average percent correct achieved by:
    • Statewide students
    • District students
    • Your students
    • Proficient range
  • Definition of symbols

 = average % correct for teacher’s students

▬▬▬ = average % correct range for students statewide who scored proficient on total test

Post-Test Workshop

report for teachers back
Report for TeachersBack

Post-Test Guide P 117

  • School % at each performance level by subgroups
  • % Proficient and above
    • 2007 vs. 2008
    • School; district; state
    • By subgroup
  • Resources

Post-Test Workshop

early assessment program eap
Early Assessment Program (EAP)
  • Reporting Results
    • Student Statuses are on the STAR Report
    • Ready for CSU college-level English/mathematics courses
    • Ready for CSU college-level mathematics courses – Conditional
      • This status in conjunction with a grade of “C” or better in an approved math course or a supervised e-learning program can be exempt from the ELM
      • A list of courses and e-learning programs can be found at www.csumathsuccess.org
    • Not yet demonstrating readiness for CSU college-level English/mathematics courses

Post-Test Workshop

summary
Summary
  • Do’s and Don’ts
    • Do compare mean scale score, percent at performance levels within same grade, same content area
    • Don’t compare across grades, content area
  • Summary reports
  • Individual reports
  • Teacher reports
  • Quiz answers
  • Evaluations

Post-Test Workshop

quiz answers
Quiz Answers
  • 1. B
  • 2. D
  • 3. A
  • 4. C
  • 5. D

Post-Test Workshop