Panel briefing
1 / 19

Panel Briefing - PowerPoint PPT Presentation

  • Uploaded on

Panel Briefing. CAREER Panel. CISE Organization and Core Research Programs. 70%. CISE Core Programs. CISE Cross-Cutting Programs. 30%. Cross-Foundation Programs. CAREER Program. The Faculty Early Career Development (CAREER) Program

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Panel Briefing' - zariel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Panel briefing

Panel Briefing


CISE Organization and Core Research Programs


CISE Core Programs

CISE Cross-Cutting Programs


Cross-Foundation Programs

Career program
CAREER Program

  • The Faculty Early Career Development (CAREER) Program

    • National Science Foundation's most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through

      • outstanding research

      • excellent education and

      • the integration of education and researchwithin the context of the mission of their organizations.

Career proposal
CAREER Proposal

  • All CAREER proposals must include

    • A Summary explicitly addressing intellectual merits and broader impacts

    • An integrated research and education plan at their core.

    • A description of the proposed research project covering

      • Background and significance

      • Specific objectives

      • Preliminary results where appropriate

      • Methods and procedures to be used

      • Anticipated results

    • A description of the proposed educational activities, including plans to evaluate their impact on students and other participants;

    • A description of how the research and educational activities are integrated with one another; and

    • Results of prior NSF support, if applicable.

Supplementary information
Supplementary Information

  • Letter from the PI’s department head

    • Must support the proposed integrated research and education plan

    • Evaluated as part of the assessment of the overall merits of the proposal

  • Data management plan

  • Postdoctoral mentoring plan (if requesting support for postdocs)

Panel objectives
Panel Objectives

  • Provideinformative feedback to the PIs

    • Even the best proposals can benefit from advice

    • The weaker the proposals are – the more crucial the feedback

    • Informative and constructive feedback is especially important in the case of CAREER proposals

  • Provide advice to NSF Program Directorsfor making award recommendations

  • Provide basis for decision making by the NSF Division Directorin approving (or not) funding or declination recommendations

Conflicts of interest
Conflicts of Interest


  • remove or limit the influence (or appearance of influence) of ties to an institution or investigator

  • preserve the trust of the scientific community, Congress, and the general public

  • Institutional (“statutory”)

    • Current, previous (12 months) or possible future employment

    • Spousal or off-spring employment or enrollment

    • Received payment for participation in advisory committees, honorarium, etc. (12 months)

  • Personal(“regulatory”)

    • Thesis advisor or student relationship

    • Coauthor, research collaborator within past 48 months

    • Family member or close friend of the PI

      Panelists must not be present in the room when proposals are discussed with which they have a COI.


  • Your participation is confidential!

    • OK to say you participated in an NSF panel

      • Granularity: IIS Division, year 2009

    • Not OK to say which panel, which day!

  • Panel recommendations are confidential!

    • Do not discuss recommendations and panel proceedings, panel membership, proposals, …

  • Proposals contain sensitive information

    • Proposals contain sensitive information; not in public domain

    • Do not copy, distribute or quote from them

    • Leave them and any notes in the room after the panel

  • Panel makes recommendations, not decisions

Proposal evaluation
Proposal Evaluation

  • NSF Evaluation Criteria:

    • Intellectual Merit

    • Broader Impacts

  • CAREER proposals:

    • Integration of Research and Education

    • Letter from the department chair (required)

  • Data Management Plan

  • Postdoctoral Mentoring Plan (if applicable)

  • Results from Prior NSF Support (if applicable)

  • Human Subjects Considerations (if applicable)

  • Facilities and Resources

  • Budget

Standard nsf evaluation criteria intellectual merit
Standard NSF Evaluation Criteria: Intellectual Merit

  • Importance of proposed activity

    • To advance knowledge and understanding

    • Within the field and across fields

  • How well conceived and organized is the proposed activity?

    • Soundness and feasibility of approach, evaluation, research plan

  • Significance of expected contributions

  • Creative, original, potentially transformative research

  • Qualifications of the investigator

  • Data Management Plan

  • Facilities and Resources

    • Equipment, facilities, etc

    • Requested Support (budget)

Standard NSF Evaluation Criteria:

Broader Impacts

  • Advance discovery and understanding while promoting teaching, training, and learning

  • Broaden the participation of underrepresented groups

    • Computer Science education, computer systems workforce

    • Gender, ethnicity, disability, geographic, age, etc.

  • Enhance the infrastructure for research and education?

  • Disseminate results (data, software, …) to enable further research, education, technology transfer

  • Societal benefits

Integration of research and education
Integration of Research and Education

  • All CAREER proposals must have an integrated research and education plan at their core.

  • Can be at any level – K-12, undergraduate, graduate, professional practice, community at large

  • Examples of activities

    • Development of innovative curricula, pedagogy

    • Outreach (community, profession)

    • Mentoring

    • Broadening participation

    • Enquiry-based learning

    • Integration of a research component into undergraduate courses

  • Must advance the PIs own disciplinary and educational interests and goals, as well as the needs and context of his or her organization

Career integration of research education
CAREER: Integration of Research & Education

  • There is a broad range of possible activities

    • Innovative curriculum and pedagogy

    • Outreach and mentoring

    • Students or the general public

    • At any level: K-12, undergraduate, graduate

    • Designing new materials and practices

  • Activities must be integrated

Writing good reviews
Writing Good Reviews

  • Imagine yourself in the role of the PI

  • Write the kind of review that you would like to receive on your own proposals

  • Reviews should be

    • Informative, substantive

    • Non-inflammatory

    • Anonymous

    • Substantiated

    • Unbiased

    • Constructive

  • Avoiding bias

    • Beware of sources of implicit bias

    • Evaluate the proposal as written

    • Focus on strengths and weaknesses under each review criterion

    • Weigh the strengths and weaknesses to arrive at an overall rating

Writing good reviews1
Writing Good Reviews

  • Fill out all sections

    • Rating (use single rating if possible; if you must use a double rating, say why: e.g., I rate the proposal VG on intellectual merits but E on broader impacts”)

    • Proposal summary

    • Intellectual Merits

      • Strengths & Weaknesses

    • Educational Activities, including Integration of Research and Education

      • Strengths & Weaknesses

    • Broader Impacts

      • Strengths & Weaknesses

    • Summary statement

      • Justify the rating (E, VG, G, F, P) in terms of your overall assessment of intellectual merits, broader impacts AND integration of research and education activities

Writing good reviews2
Writing Good Reviews

  • You are on the panel because you expertise relevant to the review of proposals being reviewed

    • Do not say: “I am not qualified to review this proposal ..”.

    • Such comments prompt PIs to question the entire review:

      • Reviewer’s qualifications to conduct a proposal review

      • Assignment of proposals to panels

      • Assignment of reviewers to proposals

  • Avoid unsubstantiated statements

    • Do not say “There is nothing novel about this work”

    • Instead, you might say “The novelty of this work relative to the work of XYZ et al. (see <citation>) is unclear”

  • Avoid inflammatory language

Minimizing Bias in Evaluation

  • Implicit bias toward a group (“schemas”):

    • Non-conscious hypotheses/stereotypes, often about competence

  • Lack of critical mass leads to greater reliance on schemas

    • Few women and minorities in sciences

  • Accumulation of disadvantage

    • Small bias in same direction has large effect (in salary, promotion, and prestige) over time [Valian1998]

Grades & Recommendations

  • Panelist grades: E, V, G, F, P

    • Avoid being overly harsh (“I never give an E”) or overly generous.

    • Be discriminative & use the entire spectrum P .. E

  • Panel recommendations:

    • Highly Competitive (HC): Solid proposal, deserves funding.

    • Competitive (C): Good proposal, but some portions unconvincing.

    • Low Competitive (LC): PI is encouraged to revise and resubmit

    • Not Competitive (NC): PI is discouraged from resubmitting

  • Panel recommendation is based on insights gained during discussion

  • Funding a project with F or P rating and declining one with E rating requires explanation by PD.

Panel Summaries

  • Please follow the format in writing panel summaries.

    • Do not merely list reviewers’ comments.

    • Should reflect the whole discussion to be useful to NSF and PI

    • Crisp comments to help unsuccessful PIs improve their proposals for the next competition

  • Comments in reviews and panel summaries should be constructive, informative, non-inflammatory, and non-discriminatory

  • Ignore letters of endorsement except from organizations contributing funds, data/traces, use of facilities, etc.