We are excited to announce the launch of Registered Reports on our journal Royal Society Open Science.

Registered Reports are a new type of empirical research article in which editorial decisions are based, in part, on peer review that takes place before authors conduct research.


writing-104091_1920Why review before results?

Registered Reports stem from the philosophy that the publishable quality of science should be judged according to the importance of the research question and rigour of the methodology, and never based on whether or not the hypothesis was supported. Once the methods and proposed analyses are provisionally accepted, Royal Society Open Science will commit to publishing the results regardless of the outcome, provided the final study conforms to the initially approved proposal and meets pre-agreed quality checks. This model eliminates publication bias while minimising, as much as possible, various forms of researcher bias.

Royal Society Open Science is now offering the opportunity to submit and publish registered reports alongside its traditional format research articles.

Registered Reports are not argued to be a superior form of publishing compared with standard (unregistered) research articles but they are designed to improve the transparency and reproducibility of studies that have a clear a priori hypothesis. They are not designed for purely exploratory studies that have no predictions or for approaches that focus on developing or optimising techniques.

How to submit?

Registered Reports are now an option available on our submission system. Full guidelines on how the Registered Report review process works can be found here.


Meet the Editor

Registered Reports are being handled by our new editor, Professor Chris Chambers from Cardiff University. His main interests include the cognitive neuroscience of self-control, strategies for improving the transparency and reproducibility of research, and the relationship between science, the media, and public policy. He also writes for the Guardian psychology blog Head quarters and serves on the advisory committee of the Science Media Centre.

 

What’s the attraction of Registered Reports?

For authors there are two major selling points. The first is that it offers the opportunity to have your paper accepted for publication before the data are even collected. This provides an ideal incentive to undertake methodologically rigorous and ambitious projects, including both original studies and replications. In many sciences there is little incentive for authors to conduct replication studies. Regardless of their outcome, traditional journals usually find replication attempts boring and lacking a substantial scientific contribution. But with Registered Reports, authors can have their replication studies accepted before they invest resources into conducting them. What better starting point could you have for testing whether an important result in your field is reproducible?

Second, and linked to this, is that Registered Reports eliminate the pressure on authors to obtain “good results”, over and above executing robust and reproducible methods. The career pressure on scientists to generate attractive and “publishable” results is a bane of the modern academic career model. It grows out of publication bias, where in judging whether a manuscript is worthy of publication, reviewers and editors typically assess not only the scientific question and methodology but whether the results make a sufficient contribution to knowledge. Outcomes that are novel, or eye-catching are generally seen as more attractive and competitive than those that are null or ambiguous, even when the methodologies that produce them are the same. And when we reward scientists for producing “good results” we encourage a host of biased practices to produce them – practices such as reinventing history to “predict” results that were, in fact, unexpected, or selectively reporting analyses that allow more publishable narratives. Registered Reports not only reduce or eliminate bias, they also eliminate the need for biased practices in the first place. We therefore aim to liberate hypothesis-driven science from the grip of pernicious incentives, saying once and for all: the results of your hypothesis-tests are of interest to other scientists but they are irrelevant to whether we will publish your paper – and so irrelevant that we make editorial decisions before the results are even known. All that matters – all that should ever matter – is that your question is worth answering and that your methods are rigorous and reproducible.

 

What will be your biggest challenge as editor?

I find it immensely exciting to be launching this format across more than 200 scientific disciplines, many of which will be experiencing Registered Reports for the first time. Such a wide remit will no doubt bring challenges for us as editors, but I welcome them because Registered Reports are an evolving project with a broad appeal, and Royal Society Open Science is equipped with an outstanding array of specialist editors. Authors can be assured that their submissions are in very safe hands.

recent Nature article called Registered Reports “radical”, which reflects another type of challenge because the scientific community are naturally cautious about new publication models. Many scientists have approached Registered Reports with interest but also trepidation. I often get asked: if editors accept studies before results are known how can they be sure the authors will apply due care and attention when conducting them? The answer is that authors must pre-specify quality checks, such as positive controls or other vital tests. These checks must be independent of the main hypotheses, and if they fail then the manuscript can be rejected. This process is extremely rigorous – from my editorial experience at other journals it is a lot more rigorous than for regular unregistered articles.

Perhaps the most common concern for us to overcome is that Registered Reports could restrict scientific creativity by requiring authors to adhere to a fixed research methodology. In fact – and this is important to emphasise – Registered Reports place no restrictions whatsoever on creativity, flexibility or the reporting of serendipitous findings. If anything, Registered Reports safeguard serendipity because, in contrast to conventional publishing, papers can’t be rejected because reviewers (or editors) find the results surprising or unappealing. Also, while it is true that the pre-specified methods in a Registered Report must be followed, there are no bounds on the reporting of additional unregistered analyses. The only requirement is that such additional material is labelled transparently so that readers know which analyses were pre-registered and which were exploratory. More answers to frequently asked questions about Registered Reports can be found here.

 

What advice would you give to someone who wants to try Registered Reports for the first time?

The most striking lesson I learned from pre-registering my research is how effectively it protects us from our own fallibilities. When we started pre-registering in my laboratory a few years ago, something fascinating happened. About 12 months after our first protocol, when the results were in, we were exploring the data in a meeting and found a striking effect. A literature search then turned up more reasons to suppose this effect could be both fascinating and real, and we somehow convinced ourselves that we must have therefore predicted it all along. But when we went back to the pre-registered protocol we found that we hadn’t. Not by a mile. Knowledge of the results led us to (almost) fool ourselves, and it was only our own foresight that prevented us from making a biased entry in the literature. Of course we will still report the finding, but it will now be accurately labelled as an exploratory unregistered analysis. Richard Feynman was absolutely right when he said, “The first principle is that you must not fool yourself and you are the easiest person to fool.”

So my advice to researchers would be to simply give it a try because you won’t be disappointed. If you are a senior scientist with post-docs or PhD students, you will find no better to way to teach them the power of deductive science than by having them complete a Registered Report, and the reward for them is substantial in having their work provisionally accepted in advance. Or perhaps there is a particular hypothesis you have always wanted to test but are afraid that it might return a negative result that would be hard to publish. If so, this format is designed for you. Or perhaps you just want to do hypothesis-driven science in a way that allows you to focus on theoretical rationale and methodological precision, free from the usual pressure to obtain “good results”. The scientific community knows well the dangers of bias in research and I am confident Registered Reports will be held up as highly credible contributions to the literature.

 

What do you do in your spare time?

As a recent dad, any spare time I once had has been commandeered by our rambunctious little boy. He’s 15 months old and enjoys shouting “haroo” to people on the seafront, chasing the cats around the garden and playing with building blocks (speciality: demolition). He’s a great life balancer. One time after seeing me reading email on my phone, I turned around to find that he’d posted it through the cat flap. Bravo.

 

Leave a Reply

  • (will not be published)