2011 star interpreting and using results
Download
1 / 81

2011 star interpreting and using results - PowerPoint PPT Presentation


  • 274 Views
  • Updated On :

2011 STAR Interpreting and Using Results. August 10, 2011 Webcast Webcast starts at 9:00 a.m. Objectives. Workshop participants will be able to: Describe the purposes of STAR reports Interpret STAR results Explain key statistics Compare and contrast types of reports

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '2011 star interpreting and using results' - Ava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
2011 star interpreting and using results l.jpg

2011 STARInterpreting and Using Results

August 10, 2011

Webcast

Webcast starts at 9:00 a.m.


Objectives l.jpg
Objectives

Workshop participants will be able to:

  • Describe the purposes of STAR reports

  • Interpret STAR results

  • Explain key statistics

  • Compare and contrast types of reports

  • Identify proper uses of reports

Post-Test Workshop


Agenda l.jpg
Agenda

  • What’s New?

  • Results and Statistical Analysis

  • Using Results

  • Summary and Internet Reports

  • Data CDs

  • Individual Student Reports

  • Early Assessment Program

Post-Test Workshop


What s new in 2011 l.jpg

California Modified Assessment (CMA) reports

Scale score, performance levels for

ELA grade 9

Life Science, grade 10

Algebra I

Percent correct for

ELA, grades 10 and 11

Geometry

Writing component of grade 4 CST and CMA ELA restored

What’s New in 2011

Manual (M) 2

Post-Test Workshop


Quiz question 1 l.jpg
Quiz Question 1

Which of these tests had scale scores reported for the first time in 2011?

  • CST for World History

  • CAPA for Science

  • CMA for Algebra I

  • CMA for Geometry

Post-Test Workshop


Quiz question 16 l.jpg
Quiz Question 1

Which of these tests had scale scores reported for the first time in 2011?

C. CMA for Algebra I

Post-Test Workshop


Results purposes of star reports l.jpg
Results: Purposes of STAR Reports

M 3

  • Report progress toward proficiency on the state’s academic content standards

  • Notify where improvement needed

    • To help students’ achievement

    • To improve educational programs

  • Provide data for state and federal accountability programs

Post-Test Workshop


Results performance levels l.jpg
Results: Performance Levels

M 8−9

  • State goal: All students score at proficient or higher

  • 350 or higher scale score

    • All CST

    • CMA for ELA in grades 3–9, math in grades 3–7 and Algebra I, and science in grades 5, 8, and 10

    • STS for RLA and math in grades 2–7, excluding Algebra I

  • CAPA proficient: 35 or higher scale score

Post-Test Workshop


Results other performance levels l.jpg
Results: Other Performance Levels

M 8−9;Appendix B

  • Advanced

  • Basic cut score

    • CST: 300

    • CMA: 300 for ELA in grades 3–9, math in grades 3–7, Algebra I, science in grades 5, 8, and 10

    • STS: 300 for RLA and math in grades 2–7 (excluding Algebra I)

    • CAPA: 30

  • Below basic

  • Far below basic

  • For each testing program, cut pointsvary for advanced and below basic by

    • Subject

    • Grade

Post-Test Workshop


Results scale scores l.jpg
Results: Scale Scores

M 7−11

  • Scale scores allow the same score to mean the same thing across test versions within grade and content area

  • Accounts for differences in difficulty

  • Scale score ranges by program:

    • CST, CMA (ELA grades 3–9, mathematics 3–7, Algebra I, science in grades 5, 8, and 10), STS (grades 2–7, excluding Algebra I): 150–600 for each grade and subject

    • CAPA: 15–60 for each level and subject

Post-Test Workshop


Results equating l.jpg
Results: Equating

M 7

  • Psychometric procedure

  • Adjusts for test difficulty

  • Additional information in the technical report on the CDE Web site

Post-Test Workshop


Results reporting clusters content areas l.jpg
Results: Reporting Clusters(Content Areas)

M 9−11;Appendix A

  • Three to six clusters for each subject

  • May be useful as indicators of individual or group strengths and weaknesses

  • But . . . Reporting clusters should be interpreted with caution

Post-Test Workshop


Results cluster percent correct available for 2011 l.jpg
Results: Cluster Percent Correct Available for 2011

M 9−11;Appendix A

  • CST: all

  • CMA:

    • Grades 3–9 ELA, grades 3–7 math, Algebra I, grades 5, 8, and 10 science

  • STS: all

Post-Test Workshop


Results reporting cluster cautions l.jpg
Results: Reporting ClusterCautions

M 9−11;Appendix A

  • Based on small numbers of items; therefore, may not be reliable or generalizable

  • NOT equated from year to year

  • Should not compare reporting cluster percent correct from year to year

Post-Test Workshop


Interpreting reporting clusters or content areas in the same year l.jpg
Interpreting Reporting Clusters or Content Areas in the Same Year

M 8−12;Appendix A

  • Compare to percent-correct range of proficient students statewide

Post-Test Workshop


2011 cst reporting clusters number of questions and average percent correct l.jpg
2011 CST Reporting Clusters: Number of Questions and Average Percent Correct

M Appendix A

2011 Post-Test Guide, Appendices A and C as posted on startest.org will be finalized with complete data on August 15.

Post-Test Workshop


Examples interpreting reporting clusters for the cst for geometry l.jpg
Examples—Interpreting Reporting Clusters for the CST for Geometry

M 11

Post-Test Workshop


Quiz question 2 l.jpg
Quiz Question 2

What is a scale score?

  • Percent correct of all questions

  • Mean percent correct of all questions

  • An adjustment of this year’s and last year’s raw scores to show changes

  • An adjustment of the raw score to account for differences in difficulty

Post-Test Workshop


Quiz question 219 l.jpg
Quiz Question 2

What is a scale score?

D. An adjustment of the raw score to account for differences in difficulty

Post-Test Workshop


Using results l.jpg
Using Results

M 3

  • For instructional decisions with other data

  • Used in Academic Performance Index (API) calculations: CSTs, CAPA, CMA (only grades 3–9, Algebra I, Life Science)

  • Used in adequate yearly progress (AYP) calculations, ELA and mathematics:

    • Grades 2–8, CSTs

    • Grades 3–8 and Algebra I, CMA

    • Grades 2–8 and 10, CAPA

Post-Test Workshop


Year to year comparisons do compare csts same grade and same content area l.jpg
Year-to-Year ComparisonsDo Compare CSTs:Same Grade and Same Content Area

M 12−15

  • Mean scale score

    • Same content and grade, varying years

  • Percent in each performance level

    • Same content by grade across years

      • e.g., 2010 grade 10 ELA with 2011 grade 10 ELA

Post-Test Workshop


Year to year comparisons do compare csts percent proficient and advanced l.jpg
Year-to-Year ComparisonsDo Compare CSTs:Percent Proficient and Advanced

M 12−15

  • Percentage of students scoring at PROFICIENT and above

    • For a given grade and subject, e.g., Percent proficient and above for grade 3 math in 2010 and 2011

    • For a given subject and aggregated grades, e.g., Percent proficient and above for grades 2– 6 mathematics in 2010 and 2011

    • Across grades and a subject, e.g., Percent proficient and above in all courses and all grades

Post-Test Workshop


Year to year comparisons don t compare l.jpg
Year-to-Year ComparisonsDON’T Compare

M 12−15

  • Individual scale scores or statistics based on scale scores for different grades or content areas

    • Subjects by grade are independently scaled

    • Different content standards are measured in different grades

  • Cohorts across grades

  • Across tests

  • CMA grades 9−11, STS grades 8−11 to previous years

    • CMA grades 6−8, STS grades 5−7, only compare 2011 to 2010

  • CAPA, CMA grades 3−5, STS grades 2−4, DON’T compare to years before 2009

Post-Test Workshop


Example using cst results to compare grade results from year to year l.jpg
Example—Using CST Results to Compare Grade Results from Year to Year

M 14

Post-Test Workshop


Quiz question 3 l.jpg
Quiz Question 3

What is the best comparison for CST scores of students within a middle school?

  • 2010 mean scale scores for ELA of a cohort of grade 7 students with 2011 scale scores for ELA of the same students in grade 8

  • 2010 mean scale scores for ELA for grade 8 students with 2011 mean scale scores for ELA for grade 8 students

  • 2010 mean percent correct scores for ELA with 2011 percent correct scores for ELA for the same students in grade 8

  • 2010 mean percent correct scores for ELA for grade 8 students with 2011 mean percent correct for ELA for grade 8 students

Post-Test Workshop


Quiz question 326 l.jpg
Quiz Question 3

What is the best comparison for CST scores of students within a middle school?

B. 2010 mean scale scores for ELA for grade 8 students with 2011 mean scale scores for ELA for grade 8 students

Post-Test Workshop


Quiz question 4 l.jpg
Quiz Question 4

Which is the best comparison of cluster scores for a single student? Compare . . .

  • To proficient students statewide

  • One cluster to another, same year

  • The same cluster to the same cluster, different years

  • To the average percent correct of all students in a class

Post-Test Workshop


Quiz question 428 l.jpg
Quiz Question 4

Which is the best comparison of cluster scores for a single student? Compare…

  • To proficient students statewide

Post-Test Workshop


Aggregate summary reports l.jpg
Aggregate (Summary) Reports

M 18−22

  • What are they?

    • Student Master List Summary

    • Student Master List Summary End-of-Course (EOC)

    • Subgroup Summary

    • Back of reports: guide to abbreviations, score codes

  • Report Emphasis: CSTs

    • Criterion-referenced tests

    • Progress is measured in percent of students scoring proficient and advanced

Post-Test Workshop


Student master list summary l.jpg
Student Master List Summary

M 19–20, M 29−35

  • By grade or EOC subject

  • CSTs, CMA, CAPA, and STS

  • Lists subjects or EOC grades

  • % and # at each performance level

  • Mean scale score

    • CST

    • CAPA

    • CMA ELA in grades 3–9, math in grades 3–7, Algebra I, and science in grades 5, 8, and 10

    • STS RLA and math in grades 2–7 [not Algebra I])

Post-Test Workshop


Student master list summary cont l.jpg
Student Master List Summary (cont.)

  • Reporting cluster: mean percent correct

    • CST

    • CMA ELA in grades 3–9, math in grades 3–7, Algebra I, and science in grades 5, 8, and 10

    • STS

Post-Test Workshop


Student master list summary32 l.jpg
Student Master List Summary

M 35

Grade 7 Example

Post-Test Workshop


Student master list summary basic statistics l.jpg
Student Master List SummaryBasic Statistics

M 29−35

Post-Test Workshop


Who counts number enrolled l.jpg
Who Counts? Number Enrolled

M 31

  • Total CST/CMA and CAPA multiple-choice answer documents submitted as scorable

  • Minus

    • Documents marked as “Student enrolled after the first day of testing and was given this test”

Post-Test Workshop


Who counts number tested l.jpg
Who Counts? Number Tested

M 31

  • All CST, CMA, CAPA, STS answer documents with one or more answers

  • Plus

    • Z = Tested but marked no answers

  • Not included

    • A = Students absent

    • E = Not tested due to significant medical emergency

    • P = Parent/guardian exemptions

    • T = Enrolled first day, not tested, tested at previous school

    • Students with inconsistent grades

    • Non–English learners who took the STS

Post-Test Workshop


Who counts number and percent valid scores l.jpg
Who Counts? Number and Percent Valid Scores

M 31

  • Number Valid Scores

    • For the subject, number of students tested at grade level who received a score for the test (that is, a scale score or percent correct)

  • Percent Valid Scores

    • For the subject, number of valid scores divided by the number of students tested

    • Not included:

      • Incomplete tests

      • Modified tests

      • Non–English learners who took the STS

      • Unknown EOC mathematics (except grade 7 mathematics) or science tests

      • Inconsistent grades

Post-Test Workshop


Who counts number tested with scores l.jpg
Who Counts? Number Tested with Scores

M 31

  • All tests taken, including those taken with modifications, that receive a score

  • Not included:

    • Incomplete tests

    • Non–English learners who took the STS

    • Unknown EOC mathematics (except grade 7 mathematics) or science tests

    • Inconsistent grades

Post-Test Workshop


Student master list summary performance levels l.jpg
Student Master List SummaryPerformance Levels

M 32

Post-Test Workshop


Who counts performance levels l.jpg
Who Counts? Performance Levels

M 32

  • All CSTs, CAPA

  • CMA grades 3–9 ELA, grades 2–7 math, Algebra I, and grades 5, 8, and 10 science

  • STS grades 2–7 RLA and math (excluding Algebra I)

  • Advanced, proficient, basic, below basic

    • All valid scores falling in the performance level

  • Far below basic

    • All valid scores falling in the performance level

    • CSTs taken with modifications (in aggregate reporting and accountability only)

Post-Test Workshop


Who counts mean scale scores l.jpg
Who Counts?Mean Scale Scores

M 32

  • Average of valid scale scores

  • Can be used to compare results for the same content/grade across years

Post-Test Workshop


Student master list summary reporting clusters l.jpg
Student Master List Summary:Reporting Clusters

M 32

Compare to:

Average percent correct range for students statewide who scored proficient on the total test(See the Post-Test Guide, Appendix A.)

Post-Test Workshop


Student master list summary writing l.jpg

B = Blank

C = Copied prompt

I = Illegible

L = Language other than English

R = Refusal

T = Off Topic

W = Wrong prompt (prompt from an earlier administration)

Student Master List Summary:Writing

M 33

Post-Test Workshop


Subgroup summary csts cma capa and sts l.jpg
Subgroup Summary:CSTs, CMA, CAPA, and STS

M 42−58

  • Disability status

    • Based on disability status for CST, CMA, STS

    • CAPA: each disability type

  • Economic status

    • Based on NSLP eligibility or parent education level

  • Gender

  • English proficiency

  • Ethnicity

  • Ethnicity for Economic Status (only for CSTs; CMA in grades 3–9 ELA; grades 3–7 math, Algebra I, and grades 5, 8, and 10 science; and CAPA)

Post-Test Workshop


Subgroup summary ethnicity for economic status l.jpg
Subgroup Summary: Ethnicity for Economic Status

M 54 −58

Ethnicity for Economically Disadvantaged CST Grade 6 Example

Post-Test Workshop


Subgroup summary ethnicity for economic status45 l.jpg
Subgroup Summary: Ethnicity for Economic Status

M 54 −58

Example: Economically disadvantaged for each ethnicity

Post-Test Workshop


Subgroup summary ethnicity for economic status46 l.jpg
Subgroup Summary: Ethnicity for Economic Status

M 54-58

Post-Test Workshop


Break 10 minutes l.jpg

Break — 10 minutes


Internet reports l.jpg
Internet Reports

M 101−111

  • Summaries based on same data as paper reports: CSTs, CMA, CAPA, STS

  • Available to the public online for school, district, county, and state

  • “Students with Scores” = number tested with scores

  • CST Summaries of % Advanced and Proficient

  • More subgroups than paper reports

    • Parent education

    • Special program participation

  • Access from http://star.cde.ca.gov/

Post-Test Workshop


Internet demonstration l.jpg
Internet Demonstration

Post-Test Workshop


Internet reports cst sample l.jpg
Internet Reports:CST Sample

M 102

Post-Test Workshop


Internet reports cst summary sample l.jpg
Internet Reports:CST Summary Sample

M 103

Only available for CSTs

Post-Test Workshop


Other internet reports l.jpg
Other Internet Reports

  • CMA (M 104−105)

    • Same as CST—CMA grades 3−9 ELA, grades 3–7 math, Algebra I, grades 5, 8, and 10 science

    • Average percent (%) correct—CMA grades 10–11 ELA, Geometry

  • CAPA (M 106−109)

    • State level: same as CST; separate Level I

    • County, district, school

      • Mean scale score

      • Percent proficient or above

Post-Test Workshop


Other internet reports cont l.jpg
Other Internet Reports (cont.)

  • STS (M 110−111)

    • Same as CST—STS grades 2−7 RLA and math (except Algebra I)

    • Average percent (%) correct—STS grades 8−11 RLA and Algebra I and Geometry

Post-Test Workshop


Quiz question 5 l.jpg
Quiz Question 5

Which subgroup can only be accessed from the Internet?

  • Parent Education Level

  • CAPA by individual disability status

  • Ethnicity for Economic Status

  • English Proficiency

Post-Test Workshop


Quiz question 555 l.jpg
Quiz Question 5

Which subgroup can only be accessed from the Internet?

  • Parent Education Level

Post-Test Workshop


Data cds l.jpg
Data CDs

  • What are they?

    • Lists of information from answer documents and scores of every student in the district

    • In .txt format: wraps

  • What are they used for?

    • Searching for specific data

    • Creating unique reports

    • Verifying paper reports

  • What else is needed?

    • Text editor

    • or Desktop application

    • or Student Information System

Post-Test Workshop


View of data l.jpg
View of Data

  • As .txt, word wrap on

  • With text editor, word wrap off

Post-Test Workshop


Organization of data l.jpg
Organization of Data

  • Two files:

    • Demographics, special conditions, and test scores

    • Accommodations, modifications, English Learners, and irregularities

  • Data Layout = guide to location of data on files

    • Position

    • Number of characters

    • Whether numeric or alpha

Post-Test Workshop


Data layout sample l.jpg
Data Layout Sample

Post-Test Workshop


Individual reports l.jpg
Individual Reports

M 23−27

  • STAR Student Record Label

    • Adhesive label to affix to student’s permanent school record

  • STAR Student Master List

    • Alphabetical list of students and their scores

    • Tests listed in order within grade

      • CSTs

      • CMA

      • CAPA

      • STS

  • STAR Student Report: individual’s scores

    • 2 two-sided color copies for each test

    • For parents/guardians, school

Post-Test Workshop


Student record label grade 10 sample student name and identification l.jpg
Student Record Label Grade 10 Sample:Student Name and Identification

M 59−60

Post-Test Workshop


Student record label l.jpg
Student Record Label

M 59−60

CST/CMA Grade 10 Example

Post-Test Workshop


Student master list l.jpg
Student Master List

M 61−65

CST/CMA Grade 3 Example

Post-Test Workshop


Student report l.jpg
Student Report

M 66−97

CST Grade 11 Example

Post-Test Workshop


Student report65 l.jpg
Student Report

M 75−76

CST Grade 11 Example

Post-Test Workshop


Student report66 l.jpg
Student Report

M 76

CST Grade 11 Example

Student’s name on back

♦ = Percent correct obtained by the student on the reporting cluster/content area

▬ = Average percent-correct range on the reporting cluster for students statewide who scored proficient on the total test scores

Post-Test Workshop


Explain to parents scale score average correct cluster score x 600 l.jpg
Explain to ParentsScale Score  Average % correct cluster score x 600

M 7

  • Reporting clusters not comparable

    • Different difficulty

    • Varying number of questions

    • Average % correct of clusters  % correct on total test (or total number of test items)

  • Scale scores

    • Use conversion tables resulting from statistical procedures

    • Equating allows scores to have similar meaning (e.g., 350 = lowest score for CSTs proficient)

Post-Test Workshop


Other student reports l.jpg
Other Student Reports

  • CMA (M 77−84)

    • Performance levels and scale scores reported for grades 3–9 ELA, grades 3–7 math, Algebra I, and grades 5, 8, and 10 science

      • Back, cluster reporting

    • Percent correct for grades 10 and 11 ELA and Geometry

      • Back, “About the CMA”

  • CAPA (M 85−89)

    • Back, “About the CAPA”

Post-Test Workshop


Other student reports cont l.jpg
Other Student Reports (cont.)

  • STS (M 90−97)

    • In Spanish

    • Performance levels and scale scores, grades 2–7 RLA and math (except for Algebra I)

      • Back, cluster percent correct

    • Percent correct for entire test, grades 8–11 RLA and Algebra I and Geometry

      • Back, cluster percent correct

      • Back, how to use report

Post-Test Workshop


Unmatched reports l.jpg
Unmatched Reports

  • CST, CMA—Grs 8, 11

  • Only CMA and no CST

  • Grades 8 and 11 require CST for History–Social Science

Writing CST, CMA—Grs 4, 7

  • Only multiple-choice

  • Only writing

  • Students receive 2 reports if writing score not matched to multiple-choice score

Post-Test Workshop


Summary l.jpg
Summary

  • Do’s and Don’ts

    • Do compare mean scale score, percent at performance levels within same grade, same content area

    • Do compare cluster scores to the corresponding proficient range provided for this year

    • Do not compare cluster scores to each other within test, nor across years, grades, content area

    • Do not compare mean scale scores across grades, content area

  • Summary and Internet reports

  • Data CDs

  • Individual reports

Post-Test Workshop


Upcoming dates l.jpg
Upcoming Dates

Post-Test Workshop


For more information see l.jpg
For more information see:

  • http://www.cde.ca.gov/ta/tg/sr/

  • http://www.star.cde.ca.gov/

  • http://www.startest.org/

  • STAR Technical Assistance Center

    • 800-955-2954

  • CDE Accountability

    • [email protected]

    • 916-319-0863

Post-Test Workshop


To contact the cde l.jpg
To Contact the CDE

  • CDE STAR office: 916-445-8765

    • English–Language Arts tests, including writing: Jamie Contreras, [email protected], 916-319-0353

    • Mathematics tests: Jane Liang, [email protected], 916-322-1854

    • Science tests: Blessing Mupanduki, [email protected], 916-319-0349

    • CST History–Social Science, CAPA, CMA: Don Killmer, [email protected], 916-319-0350

Post-Test Workshop


Early assessment program results l.jpg

Early Assessment Program Results

August 10, 2011

Dusty Fox


Early assessment program eap l.jpg
Early Assessment Program (EAP)

  • EAP results are being accepted at participating California Community Colleges (CCC).

    • Approximately 55 CCC campuses are accepting EAP results

    • List of participating colleges is available on the CCC website at www.CollegeEAP.org. Check the site often for an updated list of campuses.

  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop


Early assessment program eap77 l.jpg
Early Assessment Program (EAP)

  • An EAP informational web site has been developed. www. CollegeEAP.org.

    • The site provides EAP information for both CSU and CCC

    • It is a resource for Students, Parents, and Educators

  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop


Www collegeeap org l.jpg
www.collegeEAP.org

Post-Test Workshop


Early assessment program eap79 l.jpg
Early Assessment Program (EAP)

  • Reporting Results

    • Student Statuses are on the STAR Report

    • Ready for CSU and participating CCC college-level mathematics courses – Conditional

      • This status in conjunction with a grade of “C” or better in an approved math course or a supervised e-learning program can be exempt from the ELM for CSU or other math placement tests at participating California Community Colleges (CCC)

      • A list of courses and e-learning programs can be found at www.csumathsuccess.org

  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop


Early assessment program eap80 l.jpg
Early Assessment Program (EAP)

  • Sample EAP Box on the STAR Student Report

The EAP box can be found on page 78 of the Post-Test Guide

  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop


Early assessment program eap81 l.jpg
Early Assessment Program (EAP)

  • Web Sites

    • www.CollegeEAP.org

      • Links to CSU and CCC informational sites for Students, Parents, and Educators

    • www.CalState.edu/EAP

      • Provides a list of EAP Coordinators, EAP test blueprints, and informational materials

    • www.CCCCO.edu/EAP

      • Provides a list of participating Community Colleges

  • PowerPoint slides - www.CollegeEap.org, Educators, About EAP.

Post-Test Workshop


ad