Measuring and Reporting Basic Skills Success; You Need a Valid Instrument - CB 21 Coding

1. Measuring and Reporting Basic Skills Success; You Need a Valid Instrument - CB 21 Coding Carole Bogue-Feinour, Retired Vice Chancellor of Academic Affairs, CCCCO Myrna Huffman, Director MIS, CCCCO Janet Fulks, ASCCC Curriculum Chair July 2009 Curriculum Institute

2. Background • BSI directed attention to the ARCC data • Some measures did not make sense • Investigating the coding revealed inaccuracies • Particularly the • Basic Skills Progress Outcome • ESL Progress Outcome • Progress for courses prior to transfer • Needed to define the Supplemental Basic Skills Report for accountability on the BSI funding and noncredit enhanced funding

3. “CB 21-Course Prior to Transfer Course Level” • CB21 is code for the course “level”, in terms of number of levels below the transferrable level How many levels below transfer level is this course? • It is used primarily for basic skills but can be used for non-basic skills, degree-applicable courses • It is used only for English, writing, ESL, reading, or mathematics (TOP codes) • We are reporting credit courses; noncredit in progress

4. MIS Data Element CB21 • Is used for a lot of accountability reporting • Which in turn is used to justify investments and expenditures in basic skills • ARCC Technical Advisory Group: defines metrics for mandated reports • Is necessary to show student progress through basic skills curriculum • 4…3…2…1…transferrable

5. Tracking Progress

6. Old Coding Instructions for Math • Currently, CB21: • A= 1 prereq. for transfer math (Intermediate Algebra) • B= 2 prereq./prep. for “A” (Algebra I/Elem. Algebra) • C= 3 prereq./prep. For “A/B” (Arithmetic) • Y= 4+ >3 levels below transfer level (N/A)

7. Old Coding Instructions for English • Currently, CB21: • A= 1 prereq. for transfer Eng. Comp. (Subject A) • B= 2 prereq./prep. for “A” (Not available) • C= 3 prereq./prep. For “A/B” (Not available) • Y= 4+ >3 levels below transfer level (Not available)

8. Old Coding Instructions for Reading and ESL • Not addressed at all

9. Coding problems • No colleges had a common beginning; college level was different for every college – even colleges within a district • Some college coding had no progress – all courses one level • Some courses were in the wrong order of progression • Some courses were incorrectly identified as transfer

10. Sample Coding – All are Credit Courses

11. Sample Data – All are Credit Courses

12. Sample Data – All are Credit Courses

13. Sample Data

14. CCC MIS Database Emp. Assign. EOPS DSPS Emp. Demo. Matric. Student Demographics (SB) VTEA Calendar Assignments Enrollments (SX) Sessions PBS Sections Pgm. Awds. Fin. Aid Courses Cal- WORKs Assess.

15. Why do we code courses? What are the data uses?

16. Research Questions • Legislative Analyst Office • Department of Finance • California Postsecondary Education Commission • California Student Aid Commission • Public Policy Institute • UC/CSU • Legislature – Committees and individual members • Community College Organizations • Newspapers • Labor Unions • Data Matches • Transfer to UC/CSU/NSC match • Dept. of Social Services • EDD/UI Match/Wage Study • Accountability Reporting • Justification & Funding • Matriculation • EOPS • DSPS • Career Technical Education • Perkins Core Indicator Reports • Perkins Allocations • BOGW Administrative Funding • Federal Integrated Postsecondary • Education Data System • (IPEDS) Reporting • CCC Data Mart • Annual Staffing Report

17. MIS Data Element CB21 • Chancellor’s Office MIS system collects all course information each term • Courses are coded for identification purposes • TOP code, credit status, transfer status, units, basic skills status, SAM/voc code, etc. • CB21=Course Prior to College Level (Current)

18. MIS Data Element CB21 • Last changed in 1994 • Defined number of “codeable” levels at 5 (xfer + 4 below) • Is used across math/English/reading/writing/ESL • Has little curricular definition of levels

19. ARCC Reporting 2009 Statewide ARCC Data

20. What CB21 is used for • Basic Skills Improvement Rate (ARCC) • Credit courses only • Completed (A,B,C,CR, P) any math/Eng basic skills course at 2 or more levels below • Within 3 years, successfully completed a higher level basic skills course of same discipline • Anywhere in the system where SSN’s are reported • Current data range: 24%-62%, • avg 49%.

21. What CB21 is used for • ESL Improvement Rate (ARCC) • Credit ESL courses only • Completed (A,B,C,CR, P) any ESL course at 2 or more levels below • Within 3 years, successfully completed a higher level ESL course • Anywhere in the system where SSN’s are reported Current data range: 0% to 81%, avg. 42%

22. The Strategy - Establishing a Rubric • Is not standardization • Does not drive curricular changes • Is not common course numbering or articulation • IS a mapping exercise designed to maximize our ability to show student progress AND your good work

23. Newly Designed CB 21 – Attached to Rubrics • Each college retains their own curriculum • Does not affect degree applicability • Allows for uniformity throughout state • Will enhance collection of accurate and comparable data • Can be used to collect assessment and placement data

24. The Process • 140 Faculty • Research for standards, outcomes and exit skills • Divided by Discipline • Developed rubrics based on need and curriculum • Universally acceptable, not comprehensive

25. The results • Rubrics created using 4 levels below transfer • Reading • Writing • Math, • ESL created 3 rubrics and needed 6 levels below transfer • ESL Writing • ESL Reading • ESL Listening and Speaking • **ESL Integrated

26. Vetting the Rubrics

27. Vetting the Rubrics

28. Moving on to noncredit • ABE/ASE • Math • English • ESL integrated

29. Next Steps • What will this mean? • How will it change ARCC reporting? • Will the data suddenly show a big difference due to recoding? • How will you proceed?

30. Next Steps & Timeline • Coding instructions • CIOs and Researchers meetings and listservs • Possible web training • Contact people for questions • All recoding must be done by November 30

31. Questions