NRS Regional Trainings Part 2

1. NRS Regional TrainingsPart 2 Dr. Kristen Corbell Jannai Johnson

2. Agenda • 10:00 – 10:10 Welcome, introductions • 10:10 – 11:30 NRS Reporting (MSG, POP, etc.) • 11:30 – 12:30 Lunch • 12:30 – 1:00 Tracking MSG for 2016-17 • 1:00 - 1:45 Employment outcomes • 1:45 – 2:15 Credential Rates/Handout • 2:15 – 2:30 Break • 2:30 – 2:45 Barriers to employment/reporting • 2:45 – 3:15 How should I design my program? • 3:15 – 3:30 Question/Answer

3. It takes a village • Upper administration/boards/partners • Directors • LEIS Coordinators • Assessment Coordinators • Instructional Coordinators • Instructors • Institutional Research/Institutional Effectiveness • Students

4. Best Data Practices • Programs will need to not only enter data, but regularly evaluate data for program performance and have established processes for ensuring accurate data are entered (system checks as well as other checks). • Regularly record data (recommend a weekly basis). This means turning in the data and recording it. • Record all students you serve with federal and/or state matching funds in the data system. CBO’s should only record students in LACES who are receiving services through funds from NCCCS. This will ensure accurate reporting of participants and reportable individuals.

5. Best Data Practices • Regularly check for any students with 12 or more hours and not initial placement and get a test immediately. Students without an initial test will count against you in federal measures. • For colleges, run XLENIP for no initial placement. The hours will be added to this report in a patch released on January 19. • For CBO’s check for any students who have 12 or more hours and no level defined. • Regularly check hours for post-testing. • Regularly check EFL gains, and keep up with HSE attainment. More on how to do this later in the presentation.

6. WIOA’s Vision • WIOA aims to increase access to and opportunities for employment, education, training, and support services. The act explicitly recognizes the need to coordinate services in order to create a full system of education and training for adults with barriers to employment. http://sites.ed.gov/octae/2015/05/20/wioa-a-vision-to-revitalize-the-workforce-system/

7. NRS Table Coverage

8. WIOA Performance Reporting in the NRS Six Key Issues

9. Six Key Issues • Participants and reportable individuals • Program entry and exit, and periods of participation • Measurable skill gain (MSG) • Credential attainment indicator • Employment performance indicators • Participant exclusions from indicators

10. Participants and Reportable Individuals • Participant: • Only upon achieving 12 contact hours after program entry • Reported on NRS and Statewide Performance Report tables • Count toward performance measures • Reportable Individual: • Provides identifying information • Taken action that demonstrates an intent to use program services • Has less than 12 contact hours • Will report demographic information beginning in 2017-18 to the federal government. • Historically this has been about 17% of all students for colleges. *** What is in LACES for these terms, is not consistent with new WIOA regulations and will be updated in the future. *** Colleges, you will see Reportable Individuals on XL123D

11. Participants and Reportable Individuals Definition of reportable individual* Definition of participant—12-hour rule *Not reported now; but demographic information will be reported in 2017-18 What’s New? What’s Not? See Handout 3: “What’s New, What’s Not.”

12. Six Key Issues • Participants and reportable individuals • Program entry and exit, and periods of participation • Measurable skill gain (MSG) • Credential attainment indicator • Employment performance indicators • Participant exclusions from indicators

13. Program Entry • Program entry—the date on which a reportable individual enrolls in an adult education and family literacy program Program entry

14. Program Exit • Program exit occurs when the participant has not received services for the past 90 consecutive calendar days and has no additional services scheduled (i.e. registered for an upcoming class). The date of exit is the last date on which the participant receives services. • However, the exit date from service cannot be determined until at least 90 consecutive calendar days have elapsed since the participant last received services.

15. Period of Participation (POP) • Every entry is counted as a period of participation, even if it occurs during the same program year • Participants with more than one program entry will have multiple periods of participation in a program year.

16. Periods of Participation (POP) MSG Indicator Example Program Exit (90 days since last service) Program Entry/ Reentry PoP 1 PoP 2 PoP 3 12+ Contact Hours 12+ Contact Hours 12+ Contact Hours • MSG is evaluated. • MSG is evaluated again. • MSG is evaluated again. Jun 30th Jul 1st

17. Periods of Participation (POP) Follow-Up Indicators Example Program Exit (90 days since last service) Program Entry/ Reentry PoP 1 PoP 2 PoP 3 12+ Contact Hours 12+ Contact Hours 12+ Contact Hours • Follow-up must occur for exit-based measures. • Follow-up must occur again for exit-based measures. • Exit based measures are NOT evaluated because there is not an exit. Jun 30th Jul 1st

18. Periods of Participation - Implications

19. Program Entry and Exit, and Periods of Participation Concept of period of participation Report nonduplicativecounting (similar to past reporting) A participant will be counted for each period of participation. Continue to track participant entry and exit dates Employment barriers and outcome data are counted separately for each period of participation. What’s New? What’s Not?

20. Six Key Issues • Participants and reportable individuals • Program entry and exit and periods of participation • Measurable skill gain (MSG) • Credential attainment indicator • Employment performance indicators • Participant exclusions from indicators

21. Measureable Skill Gain (MSG) • Five typesof gain to measure progress toward academic, technical or occupation credential, or employment • Two of the five types of gain apply to adult education: • Educational functioning level gain—three ways to document EFL gain • Receipt of a secondary credential

22. Educational Functioning-Level Gain An educational functioning level (EFL) gain may be measured by the following. • Comparing the participant’s pretest with the participant’s posttest, using an NRS approved test • Awarding of Carnegie Units or credits in an adult high school program • Enrollment in postsecondary education and training after program exit

24. Measurable Skill Gain EFL gain is expanded to include postsecondary entry after exit for all participants. Pre- and posttesting, and assessment procedures and policy, remain the same. Receipt of a secondary diploma counts as type of gain for any participant Entry into postsecondary and receipt of a secondary diploma counted until the end of the program year (June 30). Only one type of gain can count per period of participation What’s New? What’s Not? Adult Secondary Education High (ABE Level 6) is now included in the target for MSG

25. NRS Table 4 Includes Pre-Post Test Gain, AHS, and postsecondary POP

26. NRS Table 4 Includes EFL Gain, AHS, and postsecondary POP

27. Tables 4B and 4C • Look the same as Table 4 • Table 4B: Only includes those who have a post-test • Table 4C: Only includes those participants who have 51% or more of their total contact hours in distance education.

28. MSG • Get Measurable Skills Gain through: • Educational Functioning Level Gain (3 ways) • Attainment of Secondary School Diploma (AHS or HSE) • MSG tracked on Table 4 • Columns B – H are unduplicated and report the first POP for each student. • Columns I – K report each period of participation. • Target is based upon the Grand Total percentage of periods of participation with measurable skill gains. • MSG target of 36% remains in effect for 2016-17 and EFL targets are not renegotiated. The key is to reach 36% overall.

29. Tracking MSG for 2016-17 • Both systems – LACES and LEIS are being developed. LACES and those using Web Advisor in LEIS • Record hours with the actual date the student attends and not a sum for a month. • Regularly update hours and testing to ensure ability to see performance for participants at the 12 hour mark.

30. Discussion Time We expect less than 10%, probably closer to 7% of students have more than one POP statewide. • How do you think Periods of Participation will impact how you design your program? • How will this impact how you evaluate your program?

31. Knowledge Break • A participant enrolls in an IET program. Can this entry be considered enrollment in postsecondary education for MSG? Why or why not? • Scenario: Jane enters ABE Level 3 in July 2016, gets 10 hours, and leaves. She returns in September 2016, gets 40 hours, and exits at the end of the month. She then returns in January 2017, gets 80 hours, and is still there at end of year. • How many periods of participation does Jane have? • When is her exit date for each period of participation?

32. Knowledge Break • A participant enrolls in an IET program. Can this entry be considered enrollment in postsecondary education for MSG? Why or why not? • No: Co-enrollment does not count for enrollment in post-secondary education for MSG. • Scenario: Jane enters ABE Level 3 on July 15, 2016, gets 10 hours, and leaves. She returns in September 2016, gets 40 hours, and exits at the end of the month. She then returns in January 2017, gets 80 hours, and is still there at end of year. • How many periods of participation does Jane have? • When is her exit date for each period of participation?

33. Knowledge Break • A participant enrolls in an IET program. Can this entry be considered enrollment in postsecondary education for MSG? Why or why not? • No: Co-enrollment does not count for enrollment in post-secondary education for MSG. • Scenario: Jane enters ABE Level 3 in July 2016, gets 10 hours, and leaves. She returns in September 2016, gets 40 hours, and exits at the end of the month. She then returns in January 2017, gets 80 hours, and is still there at end of year. • How many periods of participation does Jane have? 2 • When is her exit date for each period of participation?

34. Knowledge Break • A participant enrolls in an IET program. Can this entry be considered enrollment in postsecondary education for MSG? Why or why not? • No: Co-enrollment does not count for enrollment in post-secondary education for MSG. • Scenario: Jane enters ABE Level 3 in July 2016, gets 10 hours, and leaves. She returns in September 2016, gets 40 hours, and exits at the end of the month. She then returns in January 2017, gets 80 hours, and is still there at end of year. • How many periods of participation does Jane have? 2 • When is her exit date for each period of participation? September 30, 2016; no exit date for 2nd POP

35. Questions

36. LEIS • XLETES and XLEARC • Will be updated, but currently it will give you the accurate placement for all students with only one MSG and first POP for other students. It will record gains based upon pre/post and Carnegie units. The one time this is not accurate is if the student doesn’t post-test in the first POP and returns in the second POP it will think the pre-test of the 1st POP is a post-test, but it’s not. You will need to manually track these. • XLET4 • Save the detail report for Table 4 as a .txt file and open it in Excel. Directions provided on this. • Add a column for HSE, postsecondary enrollment, and MSG received.

37. LACES • Programming is being done now.

38. Tracking • Split up in Groups • LEIS • LACES

39. Six Key Issues • Participants and reportable individuals • Program entry and exit, and periods of participation • Measurable skill gain (MSG) • Credential attainment indicator • Employment performance indicators • Participant exclusions from indicators

40. Credential Attainment Indicator • Two components of Credential Attainment Indicator: • Secondary credential attainment • Postsecondary credential attainment • Percentage of participants who obtain a secondary school diploma or recognized equivalent or a recognized postsecondary credential, while enrolled or within one year of exit

41. Credential Attainment Indicator, cont. • But: A participant who has attained a secondary school diploma for the Credential Attainment Indicator is counted only if the participant is employed or enrolled in a postsecondary education or training program while enrolled or within one year of exit

42. Credential Attainment

43. Credential Attainment Indicator Receipt of secondary credential only counts if participant also is employed or in postsecondary education within 1 year after exit Receipt of a secondary credential without employment or postsecondary entry program is still counted, but only as part of MSG Receipt of a postsecondary credential One year follow up needed for secondary credential component What’s New? What’s Not?

44. What is the difference? Counts for any EFL (ABE, ASE, ESL) as a measurable gain Secondary Credential Attainment only includes students at the 9th grade level (either at entry or after post-testing) Must be before June 30 of the program year. Secondary credential only counts when the student also is either employed or enrolled in postsecondary education or training while enrolled or within one year of exit. Co-enrollment counts. Does not require enrollment in post-secondary education or employment to count. Co-enrollment does not count as a measurable skill gain. MSG Credential Attainment

45. How is this tracked? • Enrollment in post-secondary education: • Enrollment in Community College Curriculum Classes (Student ID, SSN) • Enrollment through National Student Clearinghouse (First name, last name, Date of birth) – This should also include NCCCS Curriculum • Enrollment in Occupational Training: • Enrollment in Continuing Education NCCCS courses (Student ID, SSN) • Employment tracked same as with Employment measures with SSN

46. Six Key Issues • Participants and reportable individuals • Program entry and exit, and periods of participation • Measurable skill gain (MSG) • Credential attainment indicator • Employment performance indicators • Participant exclusions from indicators

47. Reporting Quarters for WIOA/NRS Aug Jan Apr Oct Sept Mar June May July Nov Dec Feb July Quarter 1 Quarter 2 Quarter 3 Quarter 4 July 1 – September 30 October 1 – December 31 January 1 – March 31 April 1 – June 30

48. Employment Performance Indicators Employment • Second quarter after exit • The percentage of participantswho are in unsubsidized employment during the second quarter after exit Employment • Fourth quarter after exit • The percentage of participantswho are in unsubsidized employment during the fourth quarter after exit

49. Employment Performance Indicators, cont’d Median Earnings • Second quarter after exit • Median earnings of participants who are in unsubsidized employment during the second quarter after exit from the program

50. Employment Performance Indicators, cont’d Employment measures changed from first- and third-quarter to second- and fourth-quarter follow-up. Participants must betracked after exit. Median earnings added. Employment indicators now measure employment rate apply to all participants. What’s New? What’s Not?