Skip directly to content

Quality & Transparency of the Merit Review Process

NSF's task of identifying and funding work at the frontiers of science and engineering is not a "top-down" process. NSF operates from the "bottom up," meaning it is the job of the NSF program officer to protect the integrity of the process, keep close track of research around the United States and the world, maintain constant contact with the research community to identify ever-moving horizons of inquiry, monitor the areas most likely to result in spectacular progress and choose the most promising people to conduct the research.

Nearly every proposal, whether solicited or unsolicited, receives the same rigorous and objective treatment, and it is the program officers role to ensure that this takes place. Proposals are evaluated by independent reviewers consisting of scientists, engineers and educators, who do not work at NSF or for the institution that employs the proposing researchers. NSF selects the reviewers from among a pool of experts in each field and their evaluations are anonymous. On average, about 50,000 experts now give their time to serve on review panels each year.

The reviewers job is to provide advice on which projects are the very highest priorities. This competitive process, called "merit review," ensures that many voices are heard and that only the best projects make it to the funding stage. An enormous amount of research, deliberation, thought and discussion goes into the final recommendations of the independent reviewers.

The Program Officer’s role is to provide an accurate summary of how a decision was reached to either fund or decline a given proposal and justify the expenditure of Federal funds.

Merit review policy and guidance is provided in the Proposal and Award Manual (PAM). The PAM is an internal compendium of NSF policies and procedures related to the proposal and award process. In addition to being the authoritative source of information regarding the NSF proposal process, it supplements other internal and external guidelines relevant to grants and other NSF assistance agreements. The PAM should be the first document that NSF staff refer to when there is a question related to NSF proposal and award process.

The NSF program officer is responsible for shaping the nations science and engineering (S&E) enterprise, developing the S&E workforce, developing scientific infrastructure, and ensuring that underrepresented groups and diverse institutions across all geographic regions are included in the scientific enterprise of the nation. NSF Program Officers are the heart and soul of this process and it is essential that they understand that their role is to protect the integrity of the merit review process while at the same time contributing to its quality and transparency.

Merit Review Criteria

All NSF proposals are evaluated through the use of the two NSB approved merit review criteria: Intellectual Merit and Broader Impacts. NSF program staff should also give careful consideration to the following in making funding decisions: Integration of Research and Education and integrating diversity into NSF Programs, projects, and activities as is indicated in the Proposal & Award Policies & Procedures Guide (PAPPG) Chapter III.A.

The criteria include considerations that help define them. These considerations are suggestions, and not all will apply to any given proposal. While proposers must address both merit review criteria, reviewers will be asked to address only those considerations that are relevant to the proposal being considered and for which the reviewer is qualified to make judgments.

What is the intellectual merit of the proposed activity?

  • How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields?
  • How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, the reviewer will comment on the quality of prior work.)
  • To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts?
  • How well conceived and organized is the proposed activity?
  • Is there sufficient access to resources?

In addition to addressing the questions below related to Broader Impacts, mentoring activities provided to postdoctoral researchers supported on the project, as described in a one-page supplementary document, will be evaluated under the Broader Impacts criterion.

What are the broader impacts of the proposed activity?

  • How well does the activity advance discovery and understanding while promoting teaching, training, and learning?

  • How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)?

  • To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships?

  • Will the results be disseminated broadly to enhance scientific and technological understanding?

  • What may be the benefits of the proposed activity to society?

In some instances, NSF will employ additional criteria as required to highlight the specific objectives of certain programs and activities.

Reviewers are asked to provide an overall rating of the individual proposal, as well as separate statements addressing each of the two criteria. The statements also should include comments about the relative importance of the two criteria in assigning the rating. Note that the criteria need not be weighed equally. For more information on merit review criteria, please see PAM Chapter V.A.

Transformative Research

NSF must continually focus on the frontier. A core value for the Foundation is the support of potentially transformational research - research that revolutionizes disciplines, creates new fields, or disrupts accepted theories and perspectives. All reviewers and all NSF staff are expected to search for proposals that have the potential for transformational advances in science and engineering research and education.

By its very nature, transformative research often is challenging and frequently crosses disciplines. It questions the status quo by proposing new (sometimes radically new) ways of approaching a fundamental scientific question.

NSF has adopted the following working definition of transformative research:

Transformative research involves ideas, discoveries, or tools that radically change our understanding of an important existing scientific or engineering concept or educational practice or leads to the creation of a new paradigm or field of science, engineering, or education. Such research challenges current understanding or provides pathways to new frontiers.

To further the opportunities for transformative research, NSF has created two funding opportunities: Grants for Rapid Response Research (RAPID); and EArly-concept Grants for Exploratory Research (EAGER).

The RAPID funding mechanism is used for proposals having a severe urgency with regard to availability of, or access to data, facilities or specialized equipment, including quick-response research on natural or anthropogenic disasters and similar unanticipated events. Investigators must contact the NSF Program Officer(s) whose expertise is most germane to the proposal topic before submitting a RAPID proposal. This will facilitate determining whether the proposed work is appropriate for RAPID funding. For further information on the RAPID funding mechanism, see PAM Chapter VIII.E.

The EAGER funding mechanism may be used to support exploratory work in its early stages on untested, but potentially transformative, research ideas or approaches. This work may be considered especially "high risk-high payoff" in the sense that it, for example, involves radically different approaches, applies new expertise, or engages novel disciplinary or interdisciplinary perspectives. Investigators must contact the NSF Program Officer(s) whose expertise is most germane to the proposal topic prior to submission of an EAGER proposal. This will aid in determining the appropriateness of the work for consideration under the EAGER mechanism; this suitability must be assessed early in the process. For further information about the EAGER funding mechanism, see PAM Chapter VIII.F.

For additional information about transformative research see the NSF Transformative Research website.

The Program Officer's Recommendation

NSF Program Officers are experts in the scientific areas that they manage. They have advanced educational training in science, engineering, and/or education plus experience as appropriate in research, education and/or administration. They are therefore expected to exercise their professional judgment in their recommendations for funding. NSF is unique in this respect.

Program Officers are expected to produce and manage a balanced portfolio of awards that addresses a variety of considerations and objectives. When making funding decisions, in addition to information contained in the external proposal reviews, NSF program officers evaluate proposals in the larger context of their overall portfolio and consider issues such as:

  • Support for high-risk proposals with potential for transformative advances in a field;

  • Different approaches to significant research questions;

  • Capacity building in a new and promising research area;

  • Potential impact on human resources and infrastructure;

  • NSF core strategies, such as the integration of research and education; and broadening participation;

  • Achievement of special program objectives and initiatives;

  • Other available funding sources; and

  • Geographic distribution.

Communicating Rationale for Decision to Principal Investigator

As previously stated, the Program Officer must justify the decision to expend Federal funds for projects supported by NSF. This is accomplished as a public record and justification in the form of an award abstract, and as a communication between the program officer and the principal investigator in the form of the PI Notice of Award. The Notice of Award would contain a description of the context in which the proposal was reviewed, copies of reviews and a panel summary (if the proposal was reviewed by a panel).

It is also important that program officers provide the rationale for declined proposals. In view of the seriousness of a declination by NSF, it is essential that declination notices are phrased with utmost care. The Program Officer will provide access to the reviews of the proposal as well as a description of the context in which it was reviewed. This information is provided to the PI in order to assist with preparing a revised proposal. PIs are also informed that the NSF Program Officer is available to discuss the decision either by phone or e-mail. For further information on communicating a decision to the PI, see PAM Chapter VI.F.