Specifying preregistration content

Source: vignettes/specifying_prereg_content.Rmd

The preregr package enables specifying (pre)registration content from within R. This information can then be exported to a human-readable HTML file with embedded machine-readable JSON, which can be imported again with preregr. This has three benefits:

  • It decentralizes specification of (pre)registration forms, contributing to the democratization and inclusiveness aims of the open science movement;
  • It enables harvesting (pre)registration information from arbitrary sources in a machine-readable format, making (pre)registrations more findable, accessible, interoperable, and reusable;
  • It facilitates integrating (pre)registration specification in an R Markdown-based reproducible research workflow.

Integrating preregistration in project planning

One of the advantages of integrating preregistration forms in R Markdown is that it becomes possible to produce R Markdown preregistration templates that include a number of parameters that then automatically conduct the sample size computations and insert those into the preregistration form specification.

Similarly, such templates can integrate R packages such as ganttrify to facilitate straightforward but thorough planning of a project, documenting the results in a preregistration with little extra effort.

An example

In this example, we will load the “inclusive systematic review registration form”, a (pre)registraton form that was intended to be useable for all types of systematic reviews in all scientific fields. We will store the example in our R session, so we can add more item content later on.


preregExample <-
  preregr::prereg_initialize(
    "inclSysRev_v0_92"
  );

To see which items to complete next, we can use preregr::prereg_next_item().


preregr::prereg_next_item(
  preregExample,
  nrOfItems = 4
);
#> 
#> ℹ Form: Inclusive Systematic Review Registration Form
#> ℹ Version: 0.92
#> 
#> The next 4 items to complete are:
#> 
#> ✔ Target discipline:  [target_discipline]
#> 
#> ✔ Title: Prisma: 1. [title]
#> 
#> ✔ Author(s) / contributor(s):  [authors]
#> 
#> ✔ Tasks and roles: Describe the expected tasks and roles of each author/contributor, for example using the Contributor Roles Taxonomy (CRediT). [tasks_and_roles]
#> 
#> To specify preregistration content, you can use the following command:
#> 
#> preregr::prereg_specify(preregExample, target_discipline = "The content to
#> specify");
#> 
#> Note that in this example, the item identifier for the first item was used;
#> replace this with the item identifiers for the other items to specify content
#> for those (shown between square brackets after each item's description).
#>

We can then follow this suggestion to specify content. For example:


preregExample <-
  preregExample |>
  preregr::prereg_specify(
    title = "Example Study",
    authors = "Littlebottom, C., Dibbler, C., & Aching, T."
  );
#> 
#> ── Specifying content for (pre)registration items ──────────────────────────────
#> ✔ Content specified for item Title passed validation!
#> ✔ Specified content for Title [title]
#> ✔ Content specified for item Author(s) / contributor(s) passed validation!
#> ✔ Specified content for Author(s) / contributor(s) [authors]

Erroneous item identifiers

preregr checks whether users don’t try to pass content for fields that weren’t specified in the form that was initialized:


preregExample <-
  preregExample |>
  preregr::prereg_specify(
    nonExistent_item = "This can't be stored anywhere"
  );
#> 
#> ── Specifying content for (pre)registration items ──────────────────────────────
#> ! You specified content for item 'nonExistent_item', but this item does not exist in the prereg form specification that you initialized (Inclusive Systematic Review Registration Form). Ignoring these items.

Item Content Validation

In addition, preregr form specifications can contain item content validation information with custom error messages. By default, validation is performed and item content isn’t stored unless the validation passes (although this can be overwritten):


preregExample <-
  preregExample |>
  preregr::prereg_specify(
    start_date = "2021-9-01"
  );
#> 
#> ── Specifying content for (pre)registration items ──────────────────────────────
#> ! Item Start date did not pass validation: The specified date does not conform to the ISO 8601 standard (YYYY-MM-DD)..
#> ✖ The content you specified for Start date [start_date] failed to pass validation with message 'The specified date does not conform to the ISO 8601 standard (YYYY-MM-DD).', so did not set it.

If we do comply, preregr happily saves the specified content:


preregExample <-
  preregExample |>
  preregr::prereg_specify(
    start_date = "2021-09-01"
  );
#> 
#> ── Specifying content for (pre)registration items ──────────────────────────────
#> ✔ Content specified for item Start date passed validation!
#> ✔ Specified content for Start date [start_date]

Specifying validation expressions

Although the format for specifying (pre)registration forms for preregr was designed to be usable by users without R experience, more experienced users can integrate validation of item content by including bits of R code in the prergistration form.

Viewing the preregistration

Viewing specific content

(Pre)registrations forms are divided into sections, enabling viewing the specified content by section:


preregExample |>
  preregr::prereg_show_item_content(
    section="metadata"
  );
#> 
#> ── Specified item content in (pre)registration specification ───────────────────
#> 
#> ℹ Form: Inclusive Systematic Review Registration Form
#> ℹ Version: 0.92
#> 
#> ── Section: Metadata ──
#> 
#> ✖ Target discipline [target_discipline]
#> ✔ Title [title]: Example Study
#> ✔ Author(s) / contributor(s) [authors]: Littlebottom, C., Dibbler, C., & Aching, T.
#> ✖ Tasks and roles [tasks_and_roles]

Viewing completion only

It is also possible to just view which fields have been completed:


preregExample |>
  preregr::prereg_show_item_completion();
#> 
#> ── Items in (pre)registration specification ────────────────────────────────────
#> 
#> ℹ Form: Inclusive Systematic Review Registration Form
#> ℹ Version: 0.92
#> 
#> ── Section: Metadata ──
#> 
#> ✖ Target discipline [target_discipline]
#> ✔ Title [title]
#> ✔ Author(s) / contributor(s) [authors]
#> ✖ Tasks and roles [tasks_and_roles]
#> 
#> ── Section: Review methods ──
#> 
#> ✖ Type of review [type_of_review]
#> ✖ Review stages [review_stages]
#> ✖ Current review stage [current_stage]
#> ✔ Start date [start_date]
#> ✖ End date [end_date]
#> ✖ Background [background]
#> ✖ Primary research question(s) [primary_research_question]
#> ✖ Secondary research question(s) [secondary_research_question]
#> ✖ Expectations / hypotheses [expectations_hypotheses]
#> ✖ Dependent variable(s) / outcome(s) / main variables [dvs_outcomes_main_vars]
#> ✖ Independent variable(s) / intervention(s) / treatment(s) [ivs_intervention_treatment]
#> ✖ Additional variable(s) / covariate(s) [additional_variables]
#> ✖ Software [software]
#> ✖ Funding [funding]
#> ✖ Conflicts of interest [cois]
#> ✖ Overlapping authorships [overlapping_authorships]
#> 
#> ── Section: Search strategy ──
#> 
#> ✖ Databases [databases]
#> ✖ Interfaces [interfaces]
#> ✖ Grey literature [grey_literature]
#> ✖ Inclusion and exclusion criteria [inclusions_exclusion_criteria]
#> ✖ Query strings [query_strings]
#> ✖ Search validation procedure [search_validation_procedure]
#> ✖ Other search strategies [other_search_strategies]
#> ✖ Procedures to contact authors [procedure_for_contacting_authors]
#> ✖ Results of contacting authors [results_of_contacting_authors]
#> ✖ Search expiration and repetition [search_expiration_and_repetition]
#> ✖ Search strategy justification [search_strategy_justification]
#> ✖ Miscellaneous search strategy details [misc_search_strategy_details]
#> 
#> ── Section: Screening ──
#> 
#> ✖ Screening stages [screening_stages]
#> ✖ Screened fields / masking [screened_fields_masking]
#> ✖ Used exclusion criteria [used_exclusion_criteria]
#> ✖ Screener instructions [screener_instructions]
#> ✖ Screening reliability [screening_reliability]
#> ✖ Screening reconciliation procedure [screening_reconciliation_procedure]
#> ✖ Sampling and sample size [sampling_and_sample_size]
#> ✖ Screening procedure justification [screening_procedure_justification]
#> ✖ Data management and sharing [screening_data_management_and_sharing]
#> ✖ Miscellaneous screening details [misc_screening_details]
#> 
#> ── Section: Extraction ──
#> 
#> ✖ Entities to extract [entities_to_extract]
#> ✖ Extraction stages [extraction_stages]
#> ✖ Extractor instructions [extractor_instructions]
#> ✖ Extractor blinding [extractor_blinding]
#> ✖ Extraction reliability [extraction_reliability]
#> ✖ Extraction reconciliation procedure [extraction_reconciliation_procedure]
#> ✖ Extraction procedure justification [extraction_procedure_justification]
#> ✖ Data management and sharing [extraction_data_management_and_sharing]
#> ✖ Miscellaneous extraction details [misc_extraction_details]
#> 
#> ── Section: Synthesis and Quality Assessment ──
#> 
#> ✖ Planned data transformations [planned_data_transformations]
#> ✖ Missing data [missing_data]
#> ✖ Data validation [data_validation]
#> ✖ Quality assessment [quality_assessment]
#> ✖ Synthesis plan [synthesis_plan]
#> ✖ Criteria for conclusions / inference criteria [criteria_for_conclusions]
#> ✖ Synthesist masking [synthesis_masking]
#> ✖ Synthesis reliability [synthesis_reliability]
#> ✖ Synthesis reconciliation procedure [synthesis_reconciliation_procedure]
#> ✖ Publication bias analyses [publication_bias]
#> ✖ Sensitivity analyses / robustness checks [sensitivity_analysis]
#> ✖ Synthesis procedure justification [synthesis_procedure_justification]
#> ✖ Synthesis data management and sharing [synthesis_data_management_and_sharing]
#> ✖ Miscellaneous synthesis details [misc_synthesis_details]

R Markdown partials

The (pre)registration specification can also be knitted into an R Markdown document directly. The full preregistration item content will be stored machine-readably in JSON, and can be imported later on again with preregr.


preregr::prereg_knit_item_content(
  preregExample,
  section="metadata"
);

Inclusive Systematic Review Registration Form

Section: Metadata

Target discipline
target_discipline
Unspecified
Title
title
Example Study
Author(s) / contributor(s)
authors
Littlebottom, C., Dibbler, C., & Aching, T.
Tasks and roles
tasks_and_roles
Unspecified

Justifying decisions

preregr integrates the justifier package, allowing immediate specification of the decision and its justification. These justifier specifications will also be stored machine-readably in the R Markdown document.