ITR COV AC Briefing

1. ITR COV AC Briefing Dr. Lesia Crumption Young ITR COV Member ENG AdCom Member May 11-12, 2005

3. ITR ProgramBackground • 5 years as an NSF Priority Area • Consistent programmatic scope • Interdisciplinary IT research and education • Innovative, high-risk, high-pay-off research and education • Changing Foci • FY00 – Fundamental IT research and education • FY01 – Application of IT to science and engineering challenges • FY02 – Multidisciplinary IT challenges • FY03 – Relationship between acquisition and utilization of knowledge and IT tools • FY04 – IT research for national priorities ITR COV

4. ITR COV Overview • ITR COV Structure: • Chair: Dr. Janie Irwin • And 2 Co-Chairs: Dr. Larry Mayer and Dr. Shenda Baker • 3 Team Leaders (one for each year) overseeing 3 teams of 10 or 11 members each • Dr. Ignacio Grossman • Dr. Jim Beach • Dr. Greg Moses • Fiscal Years covered: 2001, 2002, 2003 • 3 size classes in the ITR competition each year: • Small = Up to $500K total for 3 years • Medium = Up to $1M per year for 5 years • Large = Up to $3M per year for 5 years • Solicitation and management plan were aligned to each year’s scientific opportunities and external demands

5. Demographics of 35 COV Members • Gender:13 females; 22 males. • Geographic Distribution: Northeast: 3; Mid-Atlantic: 6; South: 10; Mid-west: 6, West: 10. • Minority Representation: 4 African Americans; 2 Hispanic Americans; 2 African American-Hispanic Americans; 1 Asian American; (1 American Indian was invited and accepted the invitation, and then became ill the day before the COV). • Academic Institutions: Public: 24; Private: 8 • Federal Labs: 1 • Businesses: 2 large • ITR awardees: 12 ITR awardees • No submission to ITR in past 5 years: 14 • Not currently sitting on an NSF AC: 26

9. ITR COV Agenda • Chunks of time devoted to: • Learning about the ITR program from ITR Program Directors • Learning about the funded projects and their science and education components by talking with Program Directors in poster sessions • Reading ITR awards and declines – small, medium and large • Working in teams to complete the year report • Talking with the ADs about recommendations • Working across teams to synthesize and prepare executive summary

10. ITR COV RecommendationsPart A: ITR Processes & Mgmt • Recognize the problem of assembling a strong, diverse, COI-free pool of reviewers when almost the entire community is submitting ITR proposals • Additional quality mail reviews would help • How to ensure that proposers, reviewers, panels, and NSF PD’s address both merit review criteria • Different interpretations of what is meant by broader impact • Emphasize importance of broadening participation • How to measure (as part of the review process) • Which are high risk, high payoff proposals ? • Which are truly multidisciplinary proposals ? • Evaluation and continuing oversight of large and medium projects

11. ITR COV RecommendationsPart B: ITR Outputs & Outcomes • Concerns about diversity in students, leadership, and participants • Many “best of breed” ideas enabled by ITR • New interdisciplinary NSF areas seeded and fueled by ITR • Bioinformatics, geoinformatics, scientific computing, e-business • Encouraged community building (and reaching across institutional boundaries) by researchers and by NSF PD’s • Many tools developed, best practices beginning to evolve • How are their impacts evaluated and will they be maintained after ITR ? • Are they now – and will they be in the future – broadly accessible ? • Critical to capture lessons learned and incorporate proven business practices to prevent future problems

12. ITR COV RecommendationsITR PART Specific Questions • Made significant research contributions to software design and quality,scalable information infrastructure, high-end computing, IT workforce, and socio-economic impacts of IT • Outstanding nuggets for entire laundry list • Ensured meaningful and effective collaboration across disciplines of science and engineering • Solicitations encouraged interdisciplinary research in all years • Over the years and size classes ~33% of proposals were co-funded across the Foundation • Management plans (always encouraged, required in large proposals) forced PIs to think about & develop plans for collaboration … and reviewers and panels to evaluate these plans

13. ITR COV RecommendationsC: Other Topics • Future large initiatives like ITR should have appropriate, assigned NSF staffing levels • Capture and transfer what PD’s learned about running large, complex, interdisciplinary Priority Area initiatives • Integrated ITR web site of projects • Compromises between success rates and funding levels/cuts • Capture and transfer what PIs learned about managing and coordinating large, interdisciplinary, multi-institutional projects

14. ITR COV RecommendationsC: Other Topics, con’t • ITR has played a key role in launching interdisciplinary projects within NSF • How can projects be sustained after ITR for their productive research lifetime • Maintenance and evolution of ITR products, infrastructures, & virtual organizations necessary to the broader research community (digital repositories, etc.) • Vision for NSF and how interdisciplinary research fits into it for • 2010? • 2015?

15. Issues for Further Discussion • COV process isn’t designed to gather insights that enable “program Improvement” (ie. reactive, not proactive in nature) • COV process doesn’t encourage “critical” feedback from visitors • COV process revealed the lack of good business practices for standardizing program planning, solicitation development, etc. throughout the agency