NGLC Wave 1 Grant Evaluation

1. NGLC Wave 1 Grant Evaluation Next Generation Learning Challenges seeks to pinpoint technology solutions that can measurably improve the quality of learning experiences and improve students’ college readiness and chances of completion.

2. NGLC Wave 1 is about • Scaling • Positive student outcomes Wave 1 Grantee-Level evaluation

3. Wave 1 Grant and Evaluation Timeline Grantee Quarterly Report Scheduled Evaluation Report Scheduled Wave 1 Challenge Grant Evaluation Wave 1 Grants Possible Extension Typical Academic Year Typical Academic Year 2011 Q2 2011 Q3 2011 Q4 2012 Q1 2012 Q2 2012 Q3 2012 Q4 2013 Q1 Comparing timelines

4. Our Perspective on the Evaluation • Strong formative component combined with aggregating objective data provided by grantees for summative purposes • Help each grantee use data for continuous improvement and acquire evidence of effectiveness • Attention to balancing the need for common key measures that can be aggregated across sites with the need to respect the unique goals and settings of particular programs Wave 1 Grantee-Level evaluation

5. Overarching Research Questions • Do the funded programs achieve their expected results on student outcomes and progress on scalability? • How strong is each Wave 1 NGLC grant’s evidence of learning effects across contexts and readiness for further scaling? • To what extent and in what ways were NGLC grant proposals implemented as expected? • What enabled or hindered the projects’ ability to achieve results? Wave 1 Grantee-Level evaluation

6. Components of the Evaluation • Early site visit for every grantee organization • Online survey of campus leaders • Online survey of a sample of instructors • Tailored assistance around how to develop evidence of effectiveness and documentation of scaling • Follow-up interviews of selected leaders and service providers as appropriate • Strategic feedback to Grantees Wave 1 Grantee-Level evaluation

7. Initial Site Visits (summer 2011) • 1-2 day visit by a senior evaluation researcher • To the extent possible, scheduled in conjunction with a key project activity • For the grantees, will support refinement of their evaluation and measurement plans • For SRI and NGLC, will help us understand the data we will be receiving and the supports that it will be helpful to provide • Focus on refinement of the grantee-level logic model : • Clarification of expected implementation and outcome measures • What is needed to attain the proposed outcomes? • To what extent are those conditions/practices present? • Identify desired kinds of assistance from NGLC and SRI evaluation team • Clarify reported baseline data (from proposals: what does each measure mean?) Wave 1 Grantee-Level evaluation

8. Grantee Responsibilities Regarding Evaluation • Identify a point-of-contact person to coordinate with SRI • Complete data template showing number of students and staff participating in NGLC activities and performance against benchmarks in Quarterly Reports to EDUCAUSE • Respond to SRI email/phone requests to clarify baseline participation and effectiveness data and grant performance data reported to EDUCAUSE • Help SRI staff to complete the planned site visit • Identify NGLC project leader for each partner institution and encourage all leaders to complete the Grantee Leader Survey • Support SRI’s effort to collect on-line instructor survey data Wave 1 Grantee-Level evaluation

9. QUESTIONS & ANSWERS Contacts Barbara Means Carolyn Dornsife Barbara.means@sri.comCarolyn.Dornsife@sri.com 650-859-4004 650-859-4576 Q&a AND cONTACTDS