Rigor and Reproducibility in Research

Scientific rigor, transparency and reproducibility are integral to the responsible conduct of research.  Research sponsors expect researchers to conduct research in a manner that promotes scientific integrity, enhances confidence and reliability in research, and improves research rigor and transparency.  Outlined below are the expectations set forth by the NIH and NSF with regard to rigor and reproducibility in research, data sharing policies for each of the agencies, and guidelines and resources for achieving best practices in the conduct of scientific research.

National Institutes of Health (NIH)

The NIH has updated grant application instructions and review language in order to promote rigor, transparency, and reproducibility in NIH-funded research, focusing on the following four areas:

  • Scientific premise of proposed research: the NIH expects applicants to describe the general strengths and weaknesses of prior research being cited by the applicant as crucial to support the application.
  • Rigorous experimental design for robust and unbiased results: the NIH expects strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation, and reporting of results.  This includes full transparency in reporting experimental details so that other may reproduce and extend findings.
  • Consideration of relevant biological variables: biological variables such as sex, age, weight, and underlying health conditions should be considered in animal study designs and analysis.  In particular, the NIH expects that sex as a biological variable (SABV) will be factored into research designs, analyses, and reporting.
  • Authentication of key biological and/or chemical resources: authentication of biological and chemical resources include, but are not limited to, cell lines, specialty chemicals, antibodies, and other biologics.  Authentication is critical to the ability to reproduce the results of research.

New Grant Guidelines for Rigor and Reproducibility for NIH Applications

Rigor and Reproducibility in NIH Applications – Resource Chart

Reviewer Guidance on Rigor and Transparency

Reviewer Guidance to Evaluate SABV in NIH Applications

NIH webpages for rigor and reproducibility:

https://www.nih.gov/research-training/rigor-reproducibility/principles-guidelines-reporting-preclinical-research

NIH grants webpage for rigor and reproducibility:

https://grants.nih.gov/reproducibility/index.htm

NIH Data Sharing Policy:

https://grants.nih.gov/grants/policy/data_sharing/

National Science Foundation (NSF)

The NSF has set forth a framework to improve reproducibility, replicability, and robustness in NSF-funded research, intending to improve quality of question formulation, data collection and analysis, and dissemination of studies yielding negative results.  

NSF Framework

NSF Data Sharing Policy:

https://www.nsf.gov/bfa/dias/policy/dmp.jsp

Rigor and Reproducibility Resources

Below are additional resources and guidance for enhancing rigor and reproducibility in research:

Guidelines for Scientific Record Keeping in the Intramural Research Program at the NIH

Online learning tool for research integrity and image processing (ORI):

http://ori.hhs.gov/education/products/RIandImages/default.html

Methods and techniques for integrating sex into research (NIH):

https://orwh.od.nih.gov/research/sex-gender/methods-and-techniques/

Enhancing Research Reproducibility:  Recommendations from the Federation of American Societies for Experimental Biology (FASEB)

Center for Open Science Reproducible Research and Statistics Training Online Materials (COS): https://osf.io/7gqsi/

Global Biological Standards Institute (GBSI) website (contains information on material authentication and other reproducibility initiatives):  https://www.gbsi.org/

Social, Behavioral, and Economic Science Perspectives on Robust and Reliable Science (NSF)