Pre-registration and Registered Reports

Pre-registration

Pre-registration is a mechanism in which one places a detailed description of a planned study into a repository where it is timestamped and will remain immutable.
For more background information, see here.

• All empirical projects will be pre-registered, preferably using the Open Science Framework.

  • Pilot studies need not be pre-registered, but should then be subject to a pre-registered replication or extension.

• Pre-registration should include as much detail as possible. Pre-registration must include at minimum:

  • Sample size (with justification)

  • Inclusion/exclusion criteria

  • Criteria and procedures for outlier exclusion and data transformation

  • Primary hypotheses or outcomes to be tested (or an explicit statement that the study is simply exploratory)

• In cases where you can’t say exactly how you will do something (usually because the choices are data-dependent), outline your procedures for determining how to do the thing, and in particular what aspects of the data will go into making that decision.
  

• In some cases, multi-stage pre-registration may be appropriate

  • E.g. when an initial discovery sample is used to determine hypotheses for subsequent testing in a validation sample
  • In this case, the initial pre-registration should lay out the sampling plan and procedures for data splitting, as well as the plan for followup pre-registrations.

• For fMRI studies, pre-registration should specify:

  • Any anatomical regions of interest to be used (with a specific definition and/or image mask for the region)
  • Motion modeling strategies (including trial- or subject-level exclusion criteria)
  • Confound modeling strategies at the trial-, subject- and group-level (including the specific design of response time modeling strategies)

• Deviations from pre-registration

  • Pre-registration should not be viewed as handcuffs. If a detail of the pre-registration is clearly suboptimal, then the rationale for using a more appropriate method should be noted, and the optimal method should be used.
  • Publications should include an explicit “Deviations from pre-registration” section that outlines any deviations and their rationale.

Registered Reports

Whenever possible, we support the submission of Registered Reports.

Registered reports are a publication mechanism in which one submits a manuscript that contains an Introduction and Methods section that describe a study prior to execution of the research; it is essential a written version of a pre-registration.
The manuscript is reviewed on the strength of the rationale and methods, and if deemed sufficient it is given an “in-principle acceptance” such that the journal guarantees to publish the paper regardless of the results, as long as the approved methods were followed.

There are currently over 300 journals that accept Registered Report submissions.
In addition, Registered Reports can be submitted to PCI-RR, which is a community project that reviews preprint Registered Reports.