setting up a postsecondary tracking system role alike groups: data oriented group

1. Kelly Sparks Department of Postsecondary Education and Student Development Chicago Public Schools Jenny Nagaoka Consortium on Chicago School Research at the University of Chicago

3. 1. Understanding Postsecondary Outcomes College aspirations (Senior survey): educational aspirations, parent aspirations for student College application process (Senior Exit Questionnaire): college plans, college applications completed, college acceptance Financial aid (FAFSA tracking system): FAFSA completion, Pell and other grant eligibility, EFC College enrollment (NSC data): college enrollment, college retention, college graduation Employment outcomes (IDES data): employment, sector, income

4. 2. Identifying graduates and tracking cohorts Create rules for identifying high school graduates Based on how your school district tracks graduation data Create rules for how graduates are attached to a cohort Based on how you want to report data Necessary for trending data

5. Create rules for identifying high school graduates Things to consider Who counts as a graduate? Traditional diploma, alternative diploma, program completion, or GED? What are the points in time when a student can graduate? Spring, summer, winter, or some other time period? What time of year should you create your graduates file?

6. Create rules for identifying high school graduates What CPS does A graduate is anyone who Was enrolled in a high school at some point during the year of graduation and Received a HS diploma, graduated from an alternative school or received a GED (based on CPS leave codes tracked in student information system) During January, June, or August graduation The graduates file is created in early October in order to account for August graduates

7. Create rules for how graduates are attached to a cohort Things to consider How do you want to report your data across time? How do you ensure graduates are not included in multiple cohorts?

8. Create rules for how graduates are attached to a cohort What CPS does Cohorts are identified as �Graduating class of [year]� Match school year (2006 cohort is 2005-2006 school year) and Include January, June, and August graduates Students are attached to a cohort the first time they are given a graduation leave code If a student shows up later as a graduate again, the student remains with the first cohort and is removed from the later cohort

9. 3. Identifying groupings of students Identify student groupings that can help to inform practice Things to Consider What data you have in existing data systems that you can use to create student groups?

10. Identifying groupings of students What CPS does Qualifications: GPA, ACT scores, college access categories, advanced course enrollment Programs: IB program, ETC participation, early college Demographics: gender, race/ethnicity School type: selective enrollment, neighborhood, charter

11. Identifying groupings of students Data systems CPS uses to create groupings Data from student information system to group students by qualifications and demographics CPS database to track school type Various mechanisms to track program participation (some are based on course enrollment and some are based on external tracking)

12. 4. Tracking college enrollment Request data from NSC based on graduates and cohorts identified Identify how to determine a student�s first enrollment Connect to external databases for grouping colleges and universities Determine key indicators for reporting

13. Things to consider What time of year should you request data from NSC? What time period should you use for your search dates? What group of students do you start tracking? Do you start with the current graduating class or go back in time? How long should you continue to request data for cohorts?

14. What CPS does Data is requested at the end of February each year (February 08 for 2007 graduates) Timeframe recommended by NSC in order to have most accurate first semester data Use August 15 for the first time requesting a cohort (based on August graduation date) Use February 1 for requesting cohorts previously requested (based on February request date) Initial request to NSC was for cohorts back to 1998 Began reporting for current graduating class at the time (2004) Request data for 10 years for each cohort Request data from NSC based on graduates and cohorts identified

15. Things to consider In what time frame do you expect a student to enroll in college the first time? How do you deal with students that have multiple enrollment records?

16. What CPS does Track student enrollment in first semester of college Include only students that have enrolled in college by November 1 after high school graduation Keep only one record per student For students with multiple records, keep records based on the following rules Different enrollment type (full time, part time, less than part time) Keep the highest enrollment type (Full time over all other types. If no full time, then part time over less than part time.) Same enrollment type, different school type (4 year, 2 year, less than 2 year) Keep the highest type of school (4 year over all other types. If no 4 year, then 2 year over less than 2 year) Same enrollment type and same type of school Keep the last record based on term start

17. Things to consider What external databases are available to use? How do you want to group college and universities to better inform practice?

18. What CPS does Uses college information included with NSC data Uses Barron�s data Uses IPEDS data Creates college groupings based on all three sources

19. Sources used for grouping and getting information about collegesLinking data to other data sources NSC data (includes ope id): two-year v. four-year, in-state v. out-of-state, public v. private Selectivity (hand entered): Barron�s competitiveness categories IPEDS data (linked to NSC by ope id): hbcu, graduation rate, retention rates, Carnegie classification, tuition, average amount of aid received by type, enrollment demographics, percent admitted, etc. http://nces.ed.gov/ipeds/

20. College categories included in NSC data Existing variables in the NSC: Colltype: two-year (less than two-year) v. four-year Collstate: use to create in-state v. out-of-state Collcontrol: public v. private

21. Barron�s Selectivity Ratings CCSR categorizes colleges by their selectivity using categories that are based on 2005 Barron's Profile of American Colleges. This college ranking system rates four-year colleges on the academic qualifications of the students who attend the college (e.g., ACT or SAT scores, GPA, and class rank), as well as the percentage of applicants who are accepted. These data were hand entered and linked by college FICE code.

22. IPEDS Data Linked to NSC by ope id Variables include: hbcu, graduation rate, retention rates, Carnegie classification, tuition, average amount of aid received by type, enrollment demographics, percent admitted Further information at: nces.ed.gov/ipeds/

23. Determine key indicators for reporting What CPS does Overall enrollment Enrollment in 4 year institutions Enrollment in highly selective institutions Full-time enrollment Enrollment by race and gender Enrollment by GPA and ACT

24. 5. Tracking college retention and graduation Use data requested from NSC Develop a definition of retention and graduation Determine key indicators for reporting

25. Use data requested from NSC Things to consider How should you request data for cohorts in years subsequent to the first request? How long should you request data for cohorts?

26. Use data requested from NSC What CPS does After first request, continue to request files annually for every cohort Data is requested for each cohort from February of the prior year to the date of the data request made in February Data is requested for 10 years after the cohort�s graduation year

27. Develop a definition of retention and graduation Things to consider How should you track retention? What is the group of students you want to track forward? How do you account for students that drop out and re-enroll? What does graduation mean? Bachelor�s degree, Associate�s degree, or certificate?

28. Develop a definition of retention and graduation What CPS does Identify students who were enrolled in spring semester as the cohort One year retention: compare them to the fall enrollment of their second year in college Two year retention: compare them to fall enrollment of their third year in college Three year retention: compare them to the fall enrollment of their fourth year in college Students that do not show an enrollment record for a given year are counted as not retained and removed from the cohort for tracking enrollment in subsequent years Current analysis does not account for students re-enrolling Formal graduation reports have not been developed Current analysis looks at completion instead of graduation�any degree or certificate completion is counted

29. Determine key indicators for reporting What CPS does Break out data into retention of students enrolled at 4 year institution and retention of students at 2 year institution Graduates still enrolled in the fall at 4 the same institution, another institution or no longer enrolled Retention rates by gender and race Retention rates by high school GPA and ACT

30. 6. Tracking employment Request data from your state department of employment security Governed by Perkins Act Determine key indicators for reporting

31. Things to consider What student data will you need to provide the department of employment security? What timeframe should you request data for? How do you deal with students with multiple employment records?

32. What CPS does Request data from Illinois Department of Employment Security (IDES) Match based on SSNs Allow students one year to be employed before requesting data Initial request for a cohort made in December one full year after graduation (data for 2006 cohort made in December 2007) Request data for four quarters at a time Dictated by IDES Keep all employment records for students Indicators are based on averages and medians

33. What CPS does Break out data by students enrolled in college and students not enrolled in college Employment in first quarter (October to January) Employment for full year (October to October) Average quarters worked Median annual earning Average number of jobs held

34. 7. Developing an online survey for seniors Develop survey questions Identify online delivery mechanism Determine administration timeline Determine key indicators for reporting

35. Develop survey questions Things to consider What are the most important things you want to know from students? How long should the survey be? What other surveying is being done that you should coordinate with?

36. Develop survey questions What CPS does Used to validate student SSNs stored in student information system Asks students about all postsecondary plans (college, work, military, apprenticeships, and other) Asks students targeted questions based on their identified postsecondary plans Typical survey time is 15 minutes Survey employs numerous skip patterns so students are only asked questions relevant to their situation Coordinates all surveys of seniors being done by various departments into one instrument

37. Sample questions from CCSR senior survey These are examples of questions that were asked on CCSR�s biennial survey. Similar questions also could be asked in another format survey, such as an online survey like CPS�s Senior Exit Questionnaire. The full surveys are available at: ccsr.uchicago.edu/surveys What is the highest level of education you plan to complete? (high school diploma, technical/vocational school certificate, two-year college degree-AA, four-year college degree BA/BS, graduate degree- MD/MA/PhD/MBA What do you think your parents/guardians want you to do next year? (go to college, work full-time, join the military, other, don�t know) Since you started high school have you: (yes/no) Attended college fairs Spoken with college representatives Visited out-of-state college campuses Has anyone in your school discussed the following with you: (did not discuss, discussed briefly, discussed in depth) How to decide which college to attend How to pay for college Different admissions requirements among four-year colleges

38. Identify online delivery mechanism Things to consider What delivery mechanism will be most accessible to students? How do you ensure you can track individual student responses to connect survey data to other datasets? What quality control mechanisms can you put in place to ensure data integrity? What do you count as a complete survey?

39. Identify online delivery mechanism What CPS does Uses unique URL through CPS website Available via internet so students can complete the survey outside of school Students are able to go back and continue their survey if they were not able to finish it the first time through A file of eligible students is preloaded into a database that drives access to the survey Students login to complete the survey using their CPS ID and their birth date Controls are placed on questions to limit students to one answer or to allow for multiple responses Controls are used to make certain questions require an answer before continuing the survey Link to IPEDS data to pre-populate the response set for questions that asks students about the colleges where they have applied or been accepted Students must check a box at the end of the survey to say they have completed the survey Only surveys with the box checked at the end are counted as complete

40. Determine administration timeline Things to consider What is the best time to administer the survey? Who should be eligible to take the survey? How do you ensure a high completion rate?

41. Determine administration timeline What CPS does Survey administration begins on May 2 (after national decision day-May 1) and continues through the first week of June (just before graduation) Same timeline used every year All active seniors at the time of survey administration are eligible to take the survey January graduates can take the survey if schools identify those students Every school has a survey administrator responsible for ensuring students complete the survey Student level reports are available weekly to schools so they can identify who has completed the survey and who has not Schools determine their own process for administration�some schools tie prom tickets or graduation tickets to survey completion District administrators receive weekly reports on school completion rates and follow up with schools that have low completion rates

42. What CPS does Students planning to continue their education Students planning to attend a four year institution College applications College acceptances Scholarship applications Concrete plans Accepted to college they plan to attend

43. 8. Developing a FAFSA tracking system Request FAFSA completion data from agency responsible for overseeing state FAFSA data Develop delivery system that allows real time reporting while keeping student data secure Determine key indicators for reporting

44. Request FAFSA completion data from agency responsible for overseeing state FAFSA data Things to consider What student data will you need to provide to request FAFSA completion data? What student-level data is available for you to request? How frequently can you request updated data?

45. Request FAFSA completion data from agency responsible for overseeing state FAFSA data What CPS does Request data from Illinois Student Assistance Commission (ISAC) ISAC is agency that determines student eligibility for state grants Match is based on SSN only Request FAFSA completion, eligibility for the Pell grant, eligibility for Illinois state grants, and schools listed on FAFSA EFC data was requested but cannot be shared at the student level�aggregate EFC data is provided Updated data is requested weekly

46. Develop delivery system that allows real time reporting while keeping student data secure Things to consider What systems are available within your school district to make student-level FAFSA data available to schools? What solution provides easy access for school staff to access the data?

47. Develop delivery system that allows real time reporting while keeping student data secure What CPS does Use a CPS website where school staff can access the data Data is updated weekly as it is received from ISAC Access to the student data is based on Active Directory so school staff can only access data for the students at their school Active Directory allows users to sign on using their CPS user ID and password and limits data access based on their user ID Report provides student list that can be sorted by student ID, student name, FAFSA completion, or grant eligibility Weekly reports are provided to district administrators that show FAFSA completion rates by school Reports rank schools by percentage increase from prior week

48. Determine key indicators for reporting What CPS does Overall FAFSA completion FAFSA completion by date Grant eligibility Completion by race and gender Completion by GPA and ACT

49. 9. Linking datasets for reporting Link postsecondary datasets to enhance findings Things to consider How can you link your postsecondary datasets to better understand your postsecondary outcomes?

50. Linking datasets for reporting What CPS does Link senior survey data to NSC data when reporting college enrollment Report college enrollment rates of students planning to attend college to those not planning attend college Report college enrollment rates of students with concrete college plans to students without concrete college plans Link NSC data to employment data when reporting employment Report employment of students enrolled in college to employment of students not enrolled in college Link postsecondary datasets to program participation data