2014 caaspp interpreting and using results n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
2014 CAASPP Interpreting and Using Results PowerPoint Presentation
Download Presentation
2014 CAASPP Interpreting and Using Results

Loading in 2 Seconds...

play fullscreen
1 / 65

2014 CAASPP Interpreting and Using Results - PowerPoint PPT Presentation


  • 162 Views
  • Uploaded on

2014 CAASPP Interpreting and Using Results. September 2014 Webcast. Objectives. Workshop participants will be able to: Describe the purposes of CAASPP reports Interpret CAASPP results Explain key statistics Compare and contrast types of reports Identify proper uses of reports.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '2014 CAASPP Interpreting and Using Results' - raven


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
objectives
Objectives

Workshop participants will be able to:

  • Describe the purposes of CAASPP reports
  • Interpret CAASPP results
  • Explain key statistics
  • Compare and contrast types of reports
  • Identify proper uses of reports

2014 Post-Test Workshop

2

slide3

Agenda

  • What’s New?
  • Results and Statistical Analysis
  • Using Results
  • Summary and Internet Reports
  • Data CDs
  • Individual Student Reports
  • Early Assessment Program

2014 Post-Test Workshop

3

slide4

What’s New in 2014

  • Results are available for only those California Assessment of Student Performance and Progress (CAASPP) content areas administered.
  • Results for the Early Assessment Program (EAP) for students in grade eleven who took these tests are not reported by the CAASPP Program.
  • Percent tested data are not reported for the Standards-based Tests in Spanish (STS).
  • Enrollment data—number and percentage—are not reported for the STS.

Post-Test Guide (PTG) 2–3

2014 Post-Test Workshop

4

slide5

What’s New in 2014

  • Since there were no CAASPP end-of-course (EOC) tests administered, there is no Student Master List Summary EOC report.
  • Electronic reporting, formerly Quick-turnaround Reporting, is now offered for all remaining paper-pencil tests. Files have been available since August 8.
    • EAP results are part of electronic reporting.
  • Handscoring of tests is not offered in 2014.

PTG 2–3

slide6

Results:

Purposes of CAASPP Reports

  • Report progress toward proficiency on the state’s academic content standards
  • Notify where improvement needed
    • To help students’ achievement
    • To improve educational programs

PTG 4

2014 Post-Test Workshop

6

slide7

Results: Performance Levels

  • State goal: All students score at proficient or above
  • 350 or higher scale score
    • CST for Science
    • CMA For Science
    • STS for RLA
  • CAPA proficient: 35 or higher scale score

PTG 7–13

2014 Post-Test Workshop

7

slide8

Results:

Other Performance Levels

  • Advanced
  • Basic cut score
    • CST for Science: 300
    • CMA for Science: 300
    • STS for RLA: 300
    • CAPA: 30
  • Below basic
  • Far below basic
  • For each testing program, cut pointsvary for advanced and below basic by
    • Subject
    • Grade

PTG 7 –13;

Appendix B

2014 Post-Test Workshop

8

slide9

Results: Scale Scores

  • Scale scores allow the same score to mean the same thing across test versions within grade and content area.
  • Scale scores account for differences in difficulty.
  • Scale score ranges by program are:
    • CST for Science, CMA for Science, STS for RLA: 150–600 for each grade and subject
    • CAPA: 15–60 for each level and subject

PTG 7–13

2014 Post-Test Workshop

9

slide10

Results: Equating

  • Psychometric procedure
  • Adjusts for test difficulty from year to year (form to form)
  • Additional information in the technical reports on the CDE Technical Reports and Studies Web page at http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp

PTG 7

slide11

Results: Reporting Clusters (Content Area)

  • Three to six clusters for each subject.
  • May be useful as indicators of individual or group strengths and weaknesses.
  • But. . . reporting clusters should be interpreted with caution.

PTG 8–13;

Appendix A

2014 Post-Test Workshop

11

slide12

Results: Reporting Clusters

Cautions

  • Cluster percent correct available for CSTs for Science, CMA for Science, STS for RLA.
  • Clusters are based on small numbers of items; therefore, may not be reliable or generalized.
  • Clusters are NOT equated from year to year.
  • You should not compare reporting cluster percent correct from year to year.

PTG 8–13;

Appendix A

2014 Post-Test Workshop

12

slide13

Interpreting Reporting Clusters or Content Areas in the Same Year

  • Compare to percent-correct range of proficient students statewide

PTG 8–13;

Appendix A

2014 Post-Test Workshop

13

slide14

2014 CST Reporting Clusters:

Number of Questions and

Average Percent Correct

PTG Appendix A

2014 Post-Test Workshop

14

slide16

Using Results

  • For instructional decisions in conjunction with other data
  • CAPA ELA and mathematics in grade ten used in adequate yearly progress (AYP) calculations

PTG 2, 4

2014 Post-Test Workshop

16

slide17

Year-to-Year Comparisons

Do Compare CSTs: Same Grade and Same Content Area

  • Mean scale score
    • Same content and grade, varying years
  • Percent in each performance level
    • Same content by grade across years

PTG 10–13

2014 Post-Test Workshop

17

slide18

Year-to-Year Comparisons

Do Compare CSTs: Percent Proficient and Advanced

  • Percentage of students scoring at PROFICIENT and above
    • For a given grade and subject, e.g., percent proficient and above for grade 5 science in 2013 and 2014
    • For a given subject and aggregated grades, e.g., percent proficient and above for grades 5, 8, and 10 science in 2013 and 2014
    • Across grades and a subject, e.g., percent proficient and above in all courses and all grades

PTG 10–13

2014 Post-Test Workshop

18

slide19

Year-to-Year Comparisons

Don’t Compare

  • Individual scale scores or statistics based on scale scores for different grades or content areas
    • Subjects by grade are independently scaled
    • Different content standards are measured in different grades
  • Cohorts across grades
  • Across tests
  • Scale scores to percent correct scores
  • CAPA to years prior to 2009, because of new standard setting that year

PTG 10–13

2014 Post-Test Workshop

19

slide20

Example—Using CST Results to Compare Grade Results from Year to Year

PTG 10

2014 Post-Test Workshop

20

slide21

Aggregate (Summary) Reports

  • What are they?
    • Student Master List Summary
    • Subgroup Summary
  • Report emphasis: CSTs for Science
    • Criterion-referenced tests
    • Progress is measured in percent of students scoring proficient and advanced
    • Back of reports provides guide to abbreviations and score codes

PTG 16–39

2014 Post-Test Workshop

21

slide22

Student Master List Summary

  • Tests are as follows:
    • CSTs, CMA for Science: Grades 5, 8, and 10
    • CAPA for ELA and Mathematics: Grades 2–11; Levels I–V
    • CAPA for Science: Grades 5, 8, and 10; Level I, III–V
    • STS for RLA: Grades 2–11
  • # and % at each performance level
  • Mean scale score
  • Reporting cluster: Mean percent correct (except CAPA)

PTG 16–18;

PTG 22–25

2014 Post-Test Workshop

22

slide23

Student Master List Summary

Grade 5 Example

PTG 25

2014 Post-Test Workshop

23

slide24

Student Master List Summary

Basic Statistics

PTG 23–25

2014 Post-Test Workshop

24

slide25

Who Counts? Number Enrolled

  • For the content area:
    • Number of multiple-choice answer documents submitted minus the number of answer documents marked to indicate that the student enrolled after the first day and was subsequently testedminus
    • Documents marked as “Student enrolled after the first day of testing and was given this test”
  • Does not apply to the STS

PTG 22–25

2014 Post-Test Workshop

25

slide26

Who Counts? Number Tested

  • For the content area, number of students who responded to any questions on the test or whose answer documents were marked to indicate that the student tested but marked no answers (special condition Z).
  • Not included:
    • A = Students absent
    • E = Not tested due to significant medical emergency
    • P = Parent/guardian exemptions
    • T = Enrolled first day, not tested, tested at previous school
    • Students with inconsistent grades
    • Non–English learners who took the STS

PTG 22–25

2014 Post-Test Workshop

26

slide27

Who Counts? Number and

Percent Valid Scores

  • Number Valid Scores
    • For the subject, number of students tested at grade level who received a score for the test.
    • Not included:
      • Incomplete tests
      • Modified tests
      • Non–English learners who took the STS
      • Inconsistent grades
  • Percent Valid Scores
    • For the subject, number of valid scores divided by the number of students tested.

PTG 24–25

2014 Post-Test Workshop

27

slide28

Who Counts?

Number Tested with Scores

  • All tests taken, including those taken with modifications, that result in a score
  • Not included:
    • Incomplete tests
    • Non–English learners who took the STS
    • Inconsistent grades

PTG 24–25

2014 Post-Test Workshop

28

slide29

Student Master List Summary

Performance Levels

PTG 23–25

2014 Post-Test Workshop

29

slide30

Who Counts?

Performance Levels

  • All CSTs for Science; CMA for Science; CAPA for ELA, Mathematics, and Science; STS for RLA
  • Advanced, proficient, basic, below basic
    • All valid scores falling in the performance level
  • Far below basic
    • All valid scores falling in the performance level
    • CSTs for Science and STS for RLA taken with modifications in aggregate reporting

PTG 22–39

2014 Post-Test Workshop

30

slide31

Who Counts?

Mean Scale Scores

  • Average of valid scale scores
  • Can be used to compare results for the same content/grade across years

PTG 24–25

2014 Post-Test Workshop

31

slide32

Student Master List Summary: Reporting Clusters

Compare to:

Average percent correct range for students statewide who scored proficient on the total test (See the Post-Test Guide, Appendix A.)

PTG 25

2014 Post-Test Workshop

32

slide33

Subgroup Summary:

CSTs, CMA, CAPA, and STS

  • Disability status
    • Based on disability status for CST, CMA, STS
    • CAPA: each disability type
  • Economic status
    • Based on NSLP eligibility or parent education level
  • Gender
  • English proficiency
  • Ethnicity
  • Ethnicity for Economic Status (only for CSTs, CMA, and CAPA)

PTG 26–39

2014 Post-Test Workshop

33

slide34

Subgroup Summary:

Ethnicity for Economic Status

PTG 39

2014 Post-Test Workshop

34

slide35

Subgroup Summary:

Ethnicity for Economic Status

Example: Economically disadvantaged for each ethnicity

PTG 29

2014 Post-Test Workshop

35

slide36

Subgroup Summary:

Ethnicity for Economic Status

PTG 29

2014 Post-Test Workshop

36

slide37

Internet Reports

  • Summaries based on same data as paper reports: CSTs for Science; CMA for Science; CAPA for ELA; Mathematics; and Science, STS for RLA
  • Available to the public online for school, district, county, and state
  • “Students with Scores” = number tested with scores
  • More subgroups than paper reports
    • Parent education
    • Special program participation
  • Access from http://caaspp.cde.ca.gov/

PTG 65–73

2014 Post-Test Workshop

37

slide38

Internet Reports

  • In 2014, neither total enrollment nor percent tested data are included in Internet reports for any test.
  • Aggregate scores will be reported on the Internet in mid-September 2014 after the CALPADS data corrections window.

PTG 65–73

internet demonstration
Internet Demonstration

2014 Post-Test Workshop

39

slide40

Internet Reports:

CST Sample

PTG 65–73

2014 Post-Test Workshop

40

slide41

Other Internet Reports

  • CST (PTG 69)
  • CMA (PTG 69–70)
    • Same as CST
  • CAPA (PTG 70−73)
    • State level: same as CST; separate Level I
    • County, district, school
      • Mean scale score
      • Percent proficient or above
  • STS (PTG 73)
    • Same as CST

2014 Post-Test Workshop

41

slide42

Data CDs

  • What are they?
    • Lists of information from answer documents and scores of every student in the LEA
    • In .txt format
  • What are they used for?
    • Searching for specific data
    • Creating unique reports
    • Verifying paper reports
  • What else is needed?
    • Text editor
    • or Desktop application
    • or Student Information System

2014 Post-Test Workshop

42

slide43

View of Data

  • As .txt, word wrap on
  • With text editor, word wrap off

2014 Post-Test Workshop

43

slide44

Organization of Data

  • Two files:
    • Demographics, special conditions, and test scores
    • Accommodations, modifications, English learners, and irregularities
  • Data Layout = guide to location of data on files
    • Position
    • Number of characters
    • Whether numeric or alpha

2014 Post-Test Workshop

44

slide45

Data Layout Sample

2014 Post-Test Workshop

45

slide46

Individual Reports

  • CAASPP Student Record Label
    • Adhesive label to affix to student’s permanent school record
  • CAASPP Student Master List
    • Alphabetical list of students and their scores
    • Tests listed in order within grade
      • CSTs
      • CMA
      • CAPA
      • STS
  • CAASPP Student Report: individual’s scores
    • Two 2-sided color copies for each test
    • For parents/guardians, school
    • Per California Code of Regulations, Title 5, Section 863, LEA must forward one copy to parent/guardian within 20 business days

PTG 40–64

2014 Post-Test Workshop

46

slide47

Student Record Label Grade 5 Sample: Student Name and Identification

PTG 40–41

2014 Post-Test Workshop

47

slide48

Student Record Label

CST Grade 5 Example

PTG 40–41

2014 Post-Test Workshop

48

slide49

Student Master List

CST/CMA Grade 10 Example

PTG 44

2014 Post-Test Workshop

49

slide50

Student Report

CST Grade 5 Example

PTG 46–50

2014 Post-Test Workshop

50

student report
Student Report

CMA Grade 5 Example

PTG 51–55

student report1
Student Report

STS Grade 2 Example

PTG 60–64

explain to parents scale score average correct cluster score x 600
Explain to ParentsScale Score  Average % correct cluster score x 600
  • Reporting clusters not comparable
    • Different difficulty
    • Varying number of questions
    • Average % correct of clusters  % correct on total test (or total number of test items)
  • Scale scores
    • Use conversion tables resulting from statistical procedures
    • Equating allows scores to have similar meaning (e.g., 350 = lowest score for CSTs proficient)

PTG 7–13

2014 Post-Test Workshop

53

slide54

Student Report Summary

  • CST (PTG 46–50)
    • Front: Scale scores, performance levels, cluster percent correct
    • Back: Interpretation guide
  • CMA (PTG 51−55)
    • Front: Scale scores, performance levels, cluster percent correct
    • Back: Interpretation guide
  • CAPA (PTG 56−59)
    • Front: Scales scores, performance levels
    • Back: Interpretation guide
  • STS (PTG 60-64) (in Spanish)
    • Front: Scale scores, performance levels, cluster percent correct
    • Back: Interpretation guide

2014 Post-Test Workshop

54

slide55

Summary

  • Do’s
    • Do compare mean scale score, percent at performance levels within same grade, same content area.
    • Do compare cluster scores to the corresponding proficient range provided for this year.
  • Don’ts
    • Don’t compare cluster scores to each other within test, nor across years, grades, content areas.
    • Don’t compare mean scale scores across grades, content areas.
  • Summary and Internet reports
  • Data CDs
  • Individual reports

2014 Post-Test Workshop

55

slide56

Upcoming Dates

2014 Post-Test Workshop

56

slide57

2015 Setting Up Test Administrations

  • Return the 2015 CAASPP Superintendent’s Designation form and Security Agreement.
  • Finalize instructional schedule.
  • Set up admin in CAASPP Test Operations Management System prior to December 1.
  • Enter and approve orders prior to December 1.

After order approval, test administration dates should NOT be changed.

2014 Post-Test Workshop

57

slide58

For more information see:

    • CDE CAASPP office
      • 916-445-8765
      • CAASPP Web page: http://www.cde.ca.gov/ta/tg/ca/
      • CAASPP Reporting Web site: http://caaspp.cde.ca.gov/
  • CDE Accountability
    • aau@cde.ca.gov
    • 916-319-0863
  • California Technical Assistance Center
    • 800-955-2954
    • http://caaspp.org/

2014 Post-Test Workshop

58

early assessment program results

Early Assessment Program Results

August 2014

Connie Grueter

slide60

Early Assessment Program (EAP)

  • 2014 EAP Student Report
    • Looks different than 2007 – 2013
    • Limited electronic data provided through Order Management System
    • Electronic data on CST Student Data CD-Rom
  • 2014 Aggregate Data file
    • Available mid-September
  • PowerPoint slides – www.CollegeEAP.org, Educators, About EAP.

2014 Post-Test Workshop

60

slide61

Early Assessment Program (EAP)

  • An EAP informational web site has been developed: www.CollegeEAP.org.
    • The site provides EAP information for both CSU and CCC
    • It is a resource for Students, Parents, and Educators
  • PowerPoint slides – www.CollegeEAP.org, Educators, About EAP.

2014 Post-Test Workshop

61

slide62

www.CollegeEAP.org

2014 Post-Test Workshop

62

slide63

Early Assessment Program (EAP)

  • Reporting Results
    • Ready for CSU and participating CCC college-level English/Mathematics courses.
    • Ready for CSU and participating CCC college-level English/Mathematics courses – Conditional
      • This status in conjunction with a grade of “C” or better in an approved English/math course or a supervised e-learning program can be exempt from the EPT or ELM for CSU or other English or math placement tests at participating California Community Colleges (CCC).
      • A list of courses and e-learning programs can be found at: www.csumathsuccess.org and www.csuenglishsuccess.org .
    • Not yet demonstrating readiness for CSU and participating CCC college-level English/Mathematics courses.

2014 Post-Test Workshop

63

slide64

Early Assessment Program (EAP)

  • Sample 2014 EAP Student Report

2014 Post-Test Workshop

64

slide65

Early Assessment Program (EAP)

  • Web Resources
    • www.CollegeEAP.org
      • Links to CSU and CCC informational sites for Students, Parents, and Educators
    • www.CalState.edu/EAP
      • Provides a list of EAP Coordinators, EAP test blueprints, and informational materials
    • www.CCCCO.edu/EAP
      • Provides a list of participating Community Colleges
  • PowerPoint slides – www.CollegeEAP.org, Educators, About EAP.

2014 Post-Test Workshop

65