CAMARADES impact statement

We aim to improve the reproducibility and quality of preclinical research, by using meta-research methods to inform and improve the standards and processes used by publishers, funders, pharmaceutical companies, and preclinical researchers.

Impact on publishing standards

Following changes in the Nature Publishing Group’s (NPG) editorial policy in 2016 aimed at improving the consistency and quality of reporting of its articles, we conducted a before-and-after study to investigate whether this policy change had improved reporting. The study concluded that full compliance with the policy increased from 0% to 16%; a record improvement not seen in matched articles from other publishers [1].

Development of a Minimal standards framework for all major publishers

In 2018, Macleod joined a 9-person working group to develop minimal reporting standards for all life sciences research [2a]. This group developed a checklist of materials, design, analysis, and reporting (MDAR), which was tested in 13 journals in 2019, and found that 80% of users (including both editors and authors) would it helpful and quick to complete [2b].

Revision of ARRIVE guidelines

Our IICARUS randomised controlled trial found that after 9 years and endorsement from 430 journals, the ARRIVE guidelines had not made a substantial impact on reporting quality [3]. The prompted the creation of an international expert working group to revise the ARRIVE guidelines, of which Macleod and Sena were included. The ARRIVE 2.0 guidelines were launched in July 2020 with re-prioritised items, extra clarifications, and specific examples [4].

BMJ Open Science

Launched in 2017 with a mission to “improve the transparency, integrity, and reproducibility of biomedical research closely aligned to medicine”, the new journal BMJ Open Science has emphasised the importance of robust and rigorous research to publishers. Sena is the journal’s inaugural Editor-in-Chief [5].

Impact on research funders

In 2015, the NC3Rs coordinated an effort to align policies between UK funders, crediting the research conducted by our research group as being the driving force:

CAMARADES […] work has been instrumental in revealing issues with the quality of animal experiments, estimating the scale of the problem, and highlighting its impact on the reliability of published research. Awareness of these issues amongst the funders prompted a greater focus on the rigour of the science described in funding applications, with major UK funders such as the MRC, BBSRC, Wellcome and Cancer Research UK now requesting explicit experimental design and statistics information, providing training for panel members to assess the reliability of the research proposed, and including statisticians on funding panels.

Impact on the pharmaceutical industry

As a partner in the European Quality in Preclinical Data (EQIPD) Consortium, the our research is impacting the pharmaceutical industry’s policies and practices, by defining and providing training on a shared framework of quality management for both industry and academia [6].

The CEO of BioCurate, an Australian not-for-profit company aiming to generate high-quality preclinical candidates from academic research for the bio-pharmaceutical industry, said of us:

Research from the CAMARADES group was pivotal in establishing a quantitative approach to identify research reports that are at risk of bias. In addition, they have provided important insights as to how these issues can be improved.

Impact on research quality

Research from our group has made significant contributions to increased robustness and rigour of publisher and funder models, and research policies and practices in the pharmaceutical industry. Consequentially, this allows for improvements in the overall standard and quality of preclinical research. Not only will these improvements result in more efficient use of public funds, but they will also increase the likelihood of clinical trial success and the production of approved therapies.

References

[1] The NPQIP Collaborative group, MacLeod M, Sena E & Howells DW 2019, 'Did a change in Nature journals’ editorial policy for life sciences research improve reporting?' BMJ Open Science doi: 10.1136/bmjos-2017-000035

[2] Minimal standards. a. Minimal standards blog b. PLoS Blog with pilot results

[3] Hair K, Macleod M, & Sena E 2019, ‘A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus)’ Research Integrity and Peer Review doi: 10.1186/s41073-019-0069-3

[4] Percie du Sert N et al. 2020, ‘The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research’ PLOS Biology doi: 10.1371/journal.pbio.3000410

[5] BMJ open Science Editorial Board. Web archive

[6] European Quality in Preclinical Data (EQIPD), Our Vision.