Panel briefing
1 / 19

Panel Briefing - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Panel Briefing. CAREER Panel. CISE Organization and Core Research Programs. 70%. CISE Core Programs. CISE Cross-Cutting Programs. 30%. Cross-Foundation Programs. CAREER Program. The Faculty Early Career Development (CAREER) Program

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Panel Briefing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Panel Briefing


CISE Organization and Core Research Programs


CISE Core Programs

CISE Cross-Cutting Programs


Cross-Foundation Programs

CAREER Program

  • The Faculty Early Career Development (CAREER) Program

    • National Science Foundation's most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through

      • outstanding research

      • excellent education and

      • the integration of education and researchwithin the context of the mission of their organizations.

CAREER Proposal

  • All CAREER proposals must include

    • A Summary explicitly addressing intellectual merits and broader impacts

    • An integrated research and education plan at their core.

    • A description of the proposed research project covering

      • Background and significance

      • Specific objectives

      • Preliminary results where appropriate

      • Methods and procedures to be used

      • Anticipated results

    • A description of the proposed educational activities, including plans to evaluate their impact on students and other participants;

    • A description of how the research and educational activities are integrated with one another; and

    • Results of prior NSF support, if applicable.

Supplementary Information

  • Letter from the PI’s department head

    • Must support the proposed integrated research and education plan

    • Evaluated as part of the assessment of the overall merits of the proposal

  • Data management plan

  • Postdoctoral mentoring plan (if requesting support for postdocs)

Panel Objectives

  • Provideinformative feedback to the PIs

    • Even the best proposals can benefit from advice

    • The weaker the proposals are – the more crucial the feedback

    • Informative and constructive feedback is especially important in the case of CAREER proposals

  • Provide advice to NSF Program Directorsfor making award recommendations

  • Provide basis for decision making by the NSF Division Directorin approving (or not) funding or declination recommendations

Conflicts of Interest


  • remove or limit the influence (or appearance of influence) of ties to an institution or investigator

  • preserve the trust of the scientific community, Congress, and the general public

  • Institutional (“statutory”)

    • Current, previous (12 months) or possible future employment

    • Spousal or off-spring employment or enrollment

    • Received payment for participation in advisory committees, honorarium, etc. (12 months)

  • Personal(“regulatory”)

    • Thesis advisor or student relationship

    • Coauthor, research collaborator within past 48 months

    • Family member or close friend of the PI

      Panelists must not be present in the room when proposals are discussed with which they have a COI.


  • Your participation is confidential!

    • OK to say you participated in an NSF panel

      • Granularity: IIS Division, year 2009

    • Not OK to say which panel, which day!

  • Panel recommendations are confidential!

    • Do not discuss recommendations and panel proceedings, panel membership, proposals, …

  • Proposals contain sensitive information

    • Proposals contain sensitive information; not in public domain

    • Do not copy, distribute or quote from them

    • Leave them and any notes in the room after the panel

  • Panel makes recommendations, not decisions

Proposal Evaluation

  • NSF Evaluation Criteria:

    • Intellectual Merit

    • Broader Impacts

  • CAREER proposals:

    • Integration of Research and Education

    • Letter from the department chair (required)

  • Data Management Plan

  • Postdoctoral Mentoring Plan (if applicable)

  • Results from Prior NSF Support (if applicable)

  • Human Subjects Considerations (if applicable)

  • Facilities and Resources

  • Budget

Standard NSF Evaluation Criteria: Intellectual Merit

  • Importance of proposed activity

    • To advance knowledge and understanding

    • Within the field and across fields

  • How well conceived and organized is the proposed activity?

    • Soundness and feasibility of approach, evaluation, research plan

  • Significance of expected contributions

  • Creative, original, potentially transformative research

  • Qualifications of the investigator

  • Data Management Plan

  • Facilities and Resources

    • Equipment, facilities, etc

    • Requested Support (budget)

Standard NSF Evaluation Criteria:

Broader Impacts

  • Advance discovery and understanding while promoting teaching, training, and learning

  • Broaden the participation of underrepresented groups

    • Computer Science education, computer systems workforce

    • Gender, ethnicity, disability, geographic, age, etc.

  • Enhance the infrastructure for research and education?

  • Disseminate results (data, software, …) to enable further research, education, technology transfer

  • Societal benefits

Integration of Research and Education

  • All CAREER proposals must have an integrated research and education plan at their core.

  • Can be at any level – K-12, undergraduate, graduate, professional practice, community at large

  • Examples of activities

    • Development of innovative curricula, pedagogy

    • Outreach (community, profession)

    • Mentoring

    • Broadening participation

    • Enquiry-based learning

    • Integration of a research component into undergraduate courses

  • Must advance the PIs own disciplinary and educational interests and goals, as well as the needs and context of his or her organization

CAREER: Integration of Research & Education

  • There is a broad range of possible activities

    • Innovative curriculum and pedagogy

    • Outreach and mentoring

    • Students or the general public

    • At any level: K-12, undergraduate, graduate

    • Designing new materials and practices

  • Activities must be integrated

Writing Good Reviews

  • Imagine yourself in the role of the PI

  • Write the kind of review that you would like to receive on your own proposals

  • Reviews should be

    • Informative, substantive

    • Non-inflammatory

    • Anonymous

    • Substantiated

    • Unbiased

    • Constructive

  • Avoiding bias

    • Beware of sources of implicit bias

    • Evaluate the proposal as written

    • Focus on strengths and weaknesses under each review criterion

    • Weigh the strengths and weaknesses to arrive at an overall rating

Writing Good Reviews

  • Fill out all sections

    • Rating (use single rating if possible; if you must use a double rating, say why: e.g., I rate the proposal VG on intellectual merits but E on broader impacts”)

    • Proposal summary

    • Intellectual Merits

      • Strengths & Weaknesses

    • Educational Activities, including Integration of Research and Education

      • Strengths & Weaknesses

    • Broader Impacts

      • Strengths & Weaknesses

    • Summary statement

      • Justify the rating (E, VG, G, F, P) in terms of your overall assessment of intellectual merits, broader impacts AND integration of research and education activities

Writing Good Reviews

  • You are on the panel because you expertise relevant to the review of proposals being reviewed

    • Do not say: “I am not qualified to review this proposal ..”.

    • Such comments prompt PIs to question the entire review:

      • Reviewer’s qualifications to conduct a proposal review

      • Assignment of proposals to panels

      • Assignment of reviewers to proposals

  • Avoid unsubstantiated statements

    • Do not say “There is nothing novel about this work”

    • Instead, you might say “The novelty of this work relative to the work of XYZ et al. (see <citation>) is unclear”

  • Avoid inflammatory language

Minimizing Bias in Evaluation

  • Implicit bias toward a group (“schemas”):

    • Non-conscious hypotheses/stereotypes, often about competence

  • Lack of critical mass leads to greater reliance on schemas

    • Few women and minorities in sciences

  • Accumulation of disadvantage

    • Small bias in same direction has large effect (in salary, promotion, and prestige) over time [Valian1998]

Grades & Recommendations

  • Panelist grades: E, V, G, F, P

    • Avoid being overly harsh (“I never give an E”) or overly generous.

    • Be discriminative & use the entire spectrum P .. E

  • Panel recommendations:

    • Highly Competitive (HC): Solid proposal, deserves funding.

    • Competitive (C): Good proposal, but some portions unconvincing.

    • Low Competitive (LC): PI is encouraged to revise and resubmit

    • Not Competitive (NC): PI is discouraged from resubmitting

  • Panel recommendation is based on insights gained during discussion

  • Funding a project with F or P rating and declining one with E rating requires explanation by PD.

Panel Summaries

  • Please follow the format in writing panel summaries.

    • Do not merely list reviewers’ comments.

    • Should reflect the whole discussion to be useful to NSF and PI

    • Crisp comments to help unsuccessful PIs improve their proposals for the next competition

  • Comments in reviews and panel summaries should be constructive, informative, non-inflammatory, and non-discriminatory

  • Ignore letters of endorsement except from organizations contributing funds, data/traces, use of facilities, etc.

  • Login