2011 star interpreting and using results
Download
Skip this Video
Download Presentation
2011 STAR Interpreting and Using Results

Loading in 2 Seconds...

play fullscreen
1 / 81

2011 star interpreting and using results - PowerPoint PPT Presentation


  • 275 Views
  • Uploaded on

2011 STAR Interpreting and Using Results. August 10, 2011 Webcast Webcast starts at 9:00 a.m. Objectives. Workshop participants will be able to: Describe the purposes of STAR reports Interpret STAR results Explain key statistics Compare and contrast types of reports

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '2011 star interpreting and using results' - Ava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
2011 star interpreting and using results

2011 STARInterpreting and Using Results

August 10, 2011

Webcast

Webcast starts at 9:00 a.m.

objectives
Objectives

Workshop participants will be able to:

  • Describe the purposes of STAR reports
  • Interpret STAR results
  • Explain key statistics
  • Compare and contrast types of reports
  • Identify proper uses of reports

Post-Test Workshop

agenda
Agenda
  • What’s New?
  • Results and Statistical Analysis
  • Using Results
  • Summary and Internet Reports
  • Data CDs
  • Individual Student Reports
  • Early Assessment Program

Post-Test Workshop

what s new in 2011
California Modified Assessment (CMA) reports

Scale score, performance levels for

ELA grade 9

Life Science, grade 10

Algebra I

Percent correct for

ELA, grades 10 and 11

Geometry

Writing component of grade 4 CST and CMA ELA restored

What’s New in 2011

Manual (M) 2

Post-Test Workshop

quiz question 1
Quiz Question 1

Which of these tests had scale scores reported for the first time in 2011?

  • CST for World History
  • CAPA for Science
  • CMA for Algebra I
  • CMA for Geometry

Post-Test Workshop

quiz question 16
Quiz Question 1

Which of these tests had scale scores reported for the first time in 2011?

C. CMA for Algebra I

Post-Test Workshop

results purposes of star reports
Results: Purposes of STAR Reports

M 3

  • Report progress toward proficiency on the state’s academic content standards
  • Notify where improvement needed
    • To help students’ achievement
    • To improve educational programs
  • Provide data for state and federal accountability programs

Post-Test Workshop

results performance levels
Results: Performance Levels

M 8−9

  • State goal: All students score at proficient or higher
  • 350 or higher scale score
    • All CST
    • CMA for ELA in grades 3–9, math in grades 3–7 and Algebra I, and science in grades 5, 8, and 10
    • STS for RLA and math in grades 2–7, excluding Algebra I
  • CAPA proficient: 35 or higher scale score

Post-Test Workshop

results other performance levels
Results: Other Performance Levels

M 8−9;Appendix B

  • Advanced
  • Basic cut score
    • CST: 300
    • CMA: 300 for ELA in grades 3–9, math in grades 3–7, Algebra I, science in grades 5, 8, and 10
    • STS: 300 for RLA and math in grades 2–7 (excluding Algebra I)
    • CAPA: 30
  • Below basic
  • Far below basic
  • For each testing program, cut pointsvary for advanced and below basic by
    • Subject
    • Grade

Post-Test Workshop

results scale scores
Results: Scale Scores

M 7−11

  • Scale scores allow the same score to mean the same thing across test versions within grade and content area
  • Accounts for differences in difficulty
  • Scale score ranges by program:
    • CST, CMA (ELA grades 3–9, mathematics 3–7, Algebra I, science in grades 5, 8, and 10), STS (grades 2–7, excluding Algebra I): 150–600 for each grade and subject
    • CAPA: 15–60 for each level and subject

Post-Test Workshop

results equating
Results: Equating

M 7

  • Psychometric procedure
  • Adjusts for test difficulty
  • Additional information in the technical report on the CDE Web site

Post-Test Workshop

results reporting clusters content areas
Results: Reporting Clusters(Content Areas)

M 9−11;Appendix A

  • Three to six clusters for each subject
  • May be useful as indicators of individual or group strengths and weaknesses
  • But . . . Reporting clusters should be interpreted with caution

Post-Test Workshop

results cluster percent correct available for 2011
Results: Cluster Percent Correct Available for 2011

M 9−11;Appendix A

  • CST: all
  • CMA:
    • Grades 3–9 ELA, grades 3–7 math, Algebra I, grades 5, 8, and 10 science
  • STS: all

Post-Test Workshop

results reporting cluster cautions
Results: Reporting ClusterCautions

M 9−11;Appendix A

  • Based on small numbers of items; therefore, may not be reliable or generalizable
  • NOT equated from year to year
  • Should not compare reporting cluster percent correct from year to year

Post-Test Workshop

interpreting reporting clusters or content areas in the same year
Interpreting Reporting Clusters or Content Areas in the Same Year

M 8−12;Appendix A

  • Compare to percent-correct range of proficient students statewide

Post-Test Workshop

2011 cst reporting clusters number of questions and average percent correct
2011 CST Reporting Clusters: Number of Questions and Average Percent Correct

M Appendix A

2011 Post-Test Guide, Appendices A and C as posted on startest.org will be finalized with complete data on August 15.

Post-Test Workshop

quiz question 2
Quiz Question 2

What is a scale score?

  • Percent correct of all questions
  • Mean percent correct of all questions
  • An adjustment of this year’s and last year’s raw scores to show changes
  • An adjustment of the raw score to account for differences in difficulty

Post-Test Workshop

quiz question 219
Quiz Question 2

What is a scale score?

D. An adjustment of the raw score to account for differences in difficulty

Post-Test Workshop

using results
Using Results

M 3

  • For instructional decisions with other data
  • Used in Academic Performance Index (API) calculations: CSTs, CAPA, CMA (only grades 3–9, Algebra I, Life Science)
  • Used in adequate yearly progress (AYP) calculations, ELA and mathematics:
    • Grades 2–8, CSTs
    • Grades 3–8 and Algebra I, CMA
    • Grades 2–8 and 10, CAPA

Post-Test Workshop

year to year comparisons do compare csts same grade and same content area
Year-to-Year ComparisonsDo Compare CSTs:Same Grade and Same Content Area

M 12−15

  • Mean scale score
    • Same content and grade, varying years
  • Percent in each performance level
    • Same content by grade across years
      • e.g., 2010 grade 10 ELA with 2011 grade 10 ELA

Post-Test Workshop

year to year comparisons do compare csts percent proficient and advanced
Year-to-Year ComparisonsDo Compare CSTs:Percent Proficient and Advanced

M 12−15

  • Percentage of students scoring at PROFICIENT and above
    • For a given grade and subject, e.g., Percent proficient and above for grade 3 math in 2010 and 2011
    • For a given subject and aggregated grades, e.g., Percent proficient and above for grades 2– 6 mathematics in 2010 and 2011
    • Across grades and a subject, e.g., Percent proficient and above in all courses and all grades

Post-Test Workshop

year to year comparisons don t compare
Year-to-Year ComparisonsDON’T Compare

M 12−15

  • Individual scale scores or statistics based on scale scores for different grades or content areas
    • Subjects by grade are independently scaled
    • Different content standards are measured in different grades
  • Cohorts across grades
  • Across tests
  • CMA grades 9−11, STS grades 8−11 to previous years
    • CMA grades 6−8, STS grades 5−7, only compare 2011 to 2010
  • CAPA, CMA grades 3−5, STS grades 2−4, DON’T compare to years before 2009

Post-Test Workshop

quiz question 3
Quiz Question 3

What is the best comparison for CST scores of students within a middle school?

  • 2010 mean scale scores for ELA of a cohort of grade 7 students with 2011 scale scores for ELA of the same students in grade 8
  • 2010 mean scale scores for ELA for grade 8 students with 2011 mean scale scores for ELA for grade 8 students
  • 2010 mean percent correct scores for ELA with 2011 percent correct scores for ELA for the same students in grade 8
  • 2010 mean percent correct scores for ELA for grade 8 students with 2011 mean percent correct for ELA for grade 8 students

Post-Test Workshop

quiz question 326
Quiz Question 3

What is the best comparison for CST scores of students within a middle school?

B. 2010 mean scale scores for ELA for grade 8 students with 2011 mean scale scores for ELA for grade 8 students

Post-Test Workshop

quiz question 4
Quiz Question 4

Which is the best comparison of cluster scores for a single student? Compare . . .

  • To proficient students statewide
  • One cluster to another, same year
  • The same cluster to the same cluster, different years
  • To the average percent correct of all students in a class

Post-Test Workshop

quiz question 428
Quiz Question 4

Which is the best comparison of cluster scores for a single student? Compare…

  • To proficient students statewide

Post-Test Workshop

aggregate summary reports
Aggregate (Summary) Reports

M 18−22

  • What are they?
    • Student Master List Summary
    • Student Master List Summary End-of-Course (EOC)
    • Subgroup Summary
    • Back of reports: guide to abbreviations, score codes
  • Report Emphasis: CSTs
    • Criterion-referenced tests
    • Progress is measured in percent of students scoring proficient and advanced

Post-Test Workshop

student master list summary
Student Master List Summary

M 19–20, M 29−35

  • By grade or EOC subject
  • CSTs, CMA, CAPA, and STS
  • Lists subjects or EOC grades
  • % and # at each performance level
  • Mean scale score
    • CST
    • CAPA
    • CMA ELA in grades 3–9, math in grades 3–7, Algebra I, and science in grades 5, 8, and 10
    • STS RLA and math in grades 2–7 [not Algebra I])

Post-Test Workshop

student master list summary cont
Student Master List Summary (cont.)
  • Reporting cluster: mean percent correct
    • CST
    • CMA ELA in grades 3–9, math in grades 3–7, Algebra I, and science in grades 5, 8, and 10
    • STS

Post-Test Workshop

student master list summary32
Student Master List Summary

M 35

Grade 7 Example

Post-Test Workshop

who counts number enrolled
Who Counts? Number Enrolled

M 31

  • Total CST/CMA and CAPA multiple-choice answer documents submitted as scorable
  • Minus
    • Documents marked as “Student enrolled after the first day of testing and was given this test”

Post-Test Workshop

who counts number tested
Who Counts? Number Tested

M 31

  • All CST, CMA, CAPA, STS answer documents with one or more answers
  • Plus
    • Z = Tested but marked no answers
  • Not included
    • A = Students absent
    • E = Not tested due to significant medical emergency
    • P = Parent/guardian exemptions
    • T = Enrolled first day, not tested, tested at previous school
    • Students with inconsistent grades
    • Non–English learners who took the STS

Post-Test Workshop

who counts number and percent valid scores
Who Counts? Number and Percent Valid Scores

M 31

  • Number Valid Scores
    • For the subject, number of students tested at grade level who received a score for the test (that is, a scale score or percent correct)
  • Percent Valid Scores
    • For the subject, number of valid scores divided by the number of students tested
    • Not included:
      • Incomplete tests
      • Modified tests
      • Non–English learners who took the STS
      • Unknown EOC mathematics (except grade 7 mathematics) or science tests
      • Inconsistent grades

Post-Test Workshop

who counts number tested with scores
Who Counts? Number Tested with Scores

M 31

  • All tests taken, including those taken with modifications, that receive a score
  • Not included:
    • Incomplete tests
    • Non–English learners who took the STS
    • Unknown EOC mathematics (except grade 7 mathematics) or science tests
    • Inconsistent grades

Post-Test Workshop

who counts performance levels
Who Counts? Performance Levels

M 32

  • All CSTs, CAPA
  • CMA grades 3–9 ELA, grades 2–7 math, Algebra I, and grades 5, 8, and 10 science
  • STS grades 2–7 RLA and math (excluding Algebra I)
  • Advanced, proficient, basic, below basic
    • All valid scores falling in the performance level
  • Far below basic
    • All valid scores falling in the performance level
    • CSTs taken with modifications (in aggregate reporting and accountability only)

Post-Test Workshop

who counts mean scale scores
Who Counts?Mean Scale Scores

M 32

  • Average of valid scale scores
  • Can be used to compare results for the same content/grade across years

Post-Test Workshop

student master list summary reporting clusters
Student Master List Summary:Reporting Clusters

M 32

Compare to:

Average percent correct range for students statewide who scored proficient on the total test(See the Post-Test Guide, Appendix A.)

Post-Test Workshop

student master list summary writing
B = Blank

C = Copied prompt

I = Illegible

L = Language other than English

R = Refusal

T = Off Topic

W = Wrong prompt (prompt from an earlier administration)

Student Master List Summary:Writing

M 33

Post-Test Workshop

subgroup summary csts cma capa and sts
Subgroup Summary:CSTs, CMA, CAPA, and STS

M 42−58

  • Disability status
    • Based on disability status for CST, CMA, STS
    • CAPA: each disability type
  • Economic status
    • Based on NSLP eligibility or parent education level
  • Gender
  • English proficiency
  • Ethnicity
  • Ethnicity for Economic Status (only for CSTs; CMA in grades 3–9 ELA; grades 3–7 math, Algebra I, and grades 5, 8, and 10 science; and CAPA)

Post-Test Workshop

subgroup summary ethnicity for economic status
Subgroup Summary: Ethnicity for Economic Status

M 54 −58

Ethnicity for Economically Disadvantaged CST Grade 6 Example

Post-Test Workshop

subgroup summary ethnicity for economic status45
Subgroup Summary: Ethnicity for Economic Status

M 54 −58

Example: Economically disadvantaged for each ethnicity

Post-Test Workshop

internet reports
Internet Reports

M 101−111

  • Summaries based on same data as paper reports: CSTs, CMA, CAPA, STS
  • Available to the public online for school, district, county, and state
  • “Students with Scores” = number tested with scores
  • CST Summaries of % Advanced and Proficient
  • More subgroups than paper reports
    • Parent education
    • Special program participation
  • Access from http://star.cde.ca.gov/

Post-Test Workshop

internet demonstration
Internet Demonstration

Post-Test Workshop

internet reports cst sample
Internet Reports:CST Sample

M 102

Post-Test Workshop

internet reports cst summary sample
Internet Reports:CST Summary Sample

M 103

Only available for CSTs

Post-Test Workshop

other internet reports
Other Internet Reports
  • CMA (M 104−105)
    • Same as CST—CMA grades 3−9 ELA, grades 3–7 math, Algebra I, grades 5, 8, and 10 science
    • Average percent (%) correct—CMA grades 10–11 ELA, Geometry
  • CAPA (M 106−109)
    • State level: same as CST; separate Level I
    • County, district, school
      • Mean scale score
      • Percent proficient or above

Post-Test Workshop

other internet reports cont
Other Internet Reports (cont.)
  • STS (M 110−111)
    • Same as CST—STS grades 2−7 RLA and math (except Algebra I)
    • Average percent (%) correct—STS grades 8−11 RLA and Algebra I and Geometry

Post-Test Workshop

quiz question 5
Quiz Question 5

Which subgroup can only be accessed from the Internet?

  • Parent Education Level
  • CAPA by individual disability status
  • Ethnicity for Economic Status
  • English Proficiency

Post-Test Workshop

quiz question 555
Quiz Question 5

Which subgroup can only be accessed from the Internet?

  • Parent Education Level

Post-Test Workshop

data cds
Data CDs
  • What are they?
    • Lists of information from answer documents and scores of every student in the district
    • In .txt format: wraps
  • What are they used for?
    • Searching for specific data
    • Creating unique reports
    • Verifying paper reports
  • What else is needed?
    • Text editor
    • or Desktop application
    • or Student Information System

Post-Test Workshop

view of data
View of Data
  • As .txt, word wrap on
  • With text editor, word wrap off

Post-Test Workshop

organization of data
Organization of Data
  • Two files:
    • Demographics, special conditions, and test scores
    • Accommodations, modifications, English Learners, and irregularities
  • Data Layout = guide to location of data on files
    • Position
    • Number of characters
    • Whether numeric or alpha

Post-Test Workshop

data layout sample
Data Layout Sample

Post-Test Workshop

individual reports
Individual Reports

M 23−27

  • STAR Student Record Label
    • Adhesive label to affix to student’s permanent school record
  • STAR Student Master List
    • Alphabetical list of students and their scores
    • Tests listed in order within grade
      • CSTs
      • CMA
      • CAPA
      • STS
  • STAR Student Report: individual’s scores
    • 2 two-sided color copies for each test
    • For parents/guardians, school

Post-Test Workshop

student record label grade 10 sample student name and identification
Student Record Label Grade 10 Sample:Student Name and Identification

M 59−60

Post-Test Workshop

student record label
Student Record Label

M 59−60

CST/CMA Grade 10 Example

Post-Test Workshop

student master list
Student Master List

M 61−65

CST/CMA Grade 3 Example

Post-Test Workshop

student report
Student Report

M 66−97

CST Grade 11 Example

Post-Test Workshop

student report65
Student Report

M 75−76

CST Grade 11 Example

Post-Test Workshop

student report66
Student Report

M 76

CST Grade 11 Example

Student’s name on back

♦ = Percent correct obtained by the student on the reporting cluster/content area

▬ = Average percent-correct range on the reporting cluster for students statewide who scored proficient on the total test scores

Post-Test Workshop

explain to parents scale score average correct cluster score x 600
Explain to ParentsScale Score  Average % correct cluster score x 600

M 7

  • Reporting clusters not comparable
    • Different difficulty
    • Varying number of questions
    • Average % correct of clusters  % correct on total test (or total number of test items)
  • Scale scores
    • Use conversion tables resulting from statistical procedures
    • Equating allows scores to have similar meaning (e.g., 350 = lowest score for CSTs proficient)

Post-Test Workshop

other student reports
Other Student Reports
  • CMA (M 77−84)
    • Performance levels and scale scores reported for grades 3–9 ELA, grades 3–7 math, Algebra I, and grades 5, 8, and 10 science
      • Back, cluster reporting
    • Percent correct for grades 10 and 11 ELA and Geometry
      • Back, “About the CMA”
  • CAPA (M 85−89)
    • Back, “About the CAPA”

Post-Test Workshop

other student reports cont
Other Student Reports (cont.)
  • STS (M 90−97)
    • In Spanish
    • Performance levels and scale scores, grades 2–7 RLA and math (except for Algebra I)
      • Back, cluster percent correct
    • Percent correct for entire test, grades 8–11 RLA and Algebra I and Geometry
      • Back, cluster percent correct
      • Back, how to use report

Post-Test Workshop

unmatched reports
Unmatched Reports
  • CST, CMA—Grs 8, 11
  • Only CMA and no CST
  • Grades 8 and 11 require CST for History–Social Science

Writing CST, CMA—Grs 4, 7

  • Only multiple-choice
  • Only writing
  • Students receive 2 reports if writing score not matched to multiple-choice score

Post-Test Workshop

summary
Summary
  • Do’s and Don’ts
    • Do compare mean scale score, percent at performance levels within same grade, same content area
    • Do compare cluster scores to the corresponding proficient range provided for this year
    • Do not compare cluster scores to each other within test, nor across years, grades, content area
    • Do not compare mean scale scores across grades, content area
  • Summary and Internet reports
  • Data CDs
  • Individual reports

Post-Test Workshop

upcoming dates
Upcoming Dates

Post-Test Workshop

for more information see
For more information see:
  • http://www.cde.ca.gov/ta/tg/sr/
  • http://www.star.cde.ca.gov/
  • http://www.startest.org/
  • STAR Technical Assistance Center
    • 800-955-2954
  • CDE Accountability

Post-Test Workshop

to contact the cde
To Contact the CDE

Post-Test Workshop

early assessment program results

Early Assessment Program Results

August 10, 2011

Dusty Fox

early assessment program eap
Early Assessment Program (EAP)
  • EAP results are being accepted at participating California Community Colleges (CCC).
    • Approximately 55 CCC campuses are accepting EAP results
    • List of participating colleges is available on the CCC website at www.CollegeEAP.org. Check the site often for an updated list of campuses.
  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

early assessment program eap77
Early Assessment Program (EAP)
  • An EAP informational web site has been developed. www. CollegeEAP.org.
    • The site provides EAP information for both CSU and CCC
    • It is a resource for Students, Parents, and Educators
  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

www collegeeap org
www.collegeEAP.org

Post-Test Workshop

early assessment program eap79
Early Assessment Program (EAP)
  • Reporting Results
    • Student Statuses are on the STAR Report
    • Ready for CSU and participating CCC college-level mathematics courses – Conditional
      • This status in conjunction with a grade of “C” or better in an approved math course or a supervised e-learning program can be exempt from the ELM for CSU or other math placement tests at participating California Community Colleges (CCC)
      • A list of courses and e-learning programs can be found at www.csumathsuccess.org
  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

early assessment program eap80
Early Assessment Program (EAP)
  • Sample EAP Box on the STAR Student Report

The EAP box can be found on page 78 of the Post-Test Guide

  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

early assessment program eap81
Early Assessment Program (EAP)
  • Web Sites
    • www.CollegeEAP.org
      • Links to CSU and CCC informational sites for Students, Parents, and Educators
    • www.CalState.edu/EAP
      • Provides a list of EAP Coordinators, EAP test blueprints, and informational materials
    • www.CCCCO.edu/EAP
      • Provides a list of participating Community Colleges
  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop

ad