Introduction

Systematic reviews and meta-analyses play a critical role in informing point-of-care decisions, clinical guideline development teams, policy decisions, and directions for new research to fill gaps in knowledge. The importance and usefulness of systematic reviews and meta-analyses across stakeholder groups is reflected in their place at the top of the hierarchy of evidence as well as the high rates of citations for systematic reviews and meta-analyses (SRs/MAs) [1]. As such, the quality of the conduct and reporting of systematic reviews and meta-analyses has clinical, ethical, sociopolitical, and financial implications [2, 3]. The perceived and codified importance of systematic reviews and meta-analyses may also drive researchers to produce them without regard or sufficient attention to methodological and reporting quality, as indicated in studies of the increasing publication rates for systematic reviews and meta-analyses [4–8].

The methodological quality of systematic reviews and meta-analyses has been a long-standing concern [9–12], with considerable effort invested in creating and updating methodological guidelines and recommendations to improve rigor by organizations such as those from the Cochrane Collaboration and the Institute of Medicine, among others. Likewise, tools to assess methodological quality of published evidence syntheses, such as AMSTAR (A MeaSurement Tool to Assess systematic Reviews) [13] and risk of bias assessments for systematic reviews, such as ROBIS [14] were developed to assess published evidence syntheses. The early concerns about the methodological quality of SRs/MAs were quickly succeeded by concerns related to the reporting quality thereof, with the understanding that, without thorough and transparent reporting, neither methodological rigor nor reliability of results and recommendations could be assessed.

The concerns about the reporting quality of published evidence syntheses in the health fields led to the creation and dissemination of reporting guidelines for meta-analyses of randomized controlled trials, known as the Quality of Reporting of Meta-Analyses (QUORUM), in 1999 [15]. In 2009 the guideline was updated to include more criteria for reporting quality and renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) out of a need to address advances in the science of systematic reviews [16]. The PRISMA statement was updated again and republished with expanded reporting guidelines in March 2021 [17]. PRISMA extensions have also been created to guide reporting for specific review types and key pieces of reviews including abstracts, diagnostic test accuracy, equity, protocols, and searching, among others [18].

The QUORUM and PRISMA 2009 and 2020 guidelines were published in multiple, well-known English language journals (e.g., PLOS Medicine, The Lancet, BMJ) and translated into other languages. The PRISMA recommendations alone have been endorsed by many journals and publishers and have been widely cited [17]. Despite QUORUM and PRISMA 2009 being widely disseminated at the time of publication, studies of systematic reviews following the guidelines’ publication continued to note low methodological and reporting quality [19, 20]. The issues in methodological and reporting quality were not confined to specific clinical or disciplinary areas. Reviews of methodological and reporting quality in SRs and MAs from internal medicine [21], oncology [22], nursing [23], nephrology [24], rehabilitation [25], dental medicine [26, 27], and complementary and alternative medicine fields [28, 29] all note shortcomings in the conduct and reporting for systematic reviews.

The continued low reporting quality of SRs/MAs might be attributed to low levels of awareness of PRISMA, yet citation data from Scopus as well as a pooled analysis of studies evaluating uptake and impact of PRISMA indicate that it has wide acceptance and adoption, though imperfect adherence [30]. Many factors related to use of PRISMA guidelines may enable or inhibit use of the guidelines, such as study team size and primary author country of affiliation, with mixed results for both of these characteristics [31–33]. Likewise, the endorsement and requirement of adherence to PRISMA by disciplinary journals in Instructions for Authors (IA) is important and ought to prompt authors to improve their methodological reporting. However, the number of journals including PRISMA in IA is noted as being low, ranging between 12%-50%- of a given profession’s journals [23, 32, 34, 35].

While the inclusion of study-specific reporting guidelines (beyond SRs/MAs) in requirements or instructions for authors has been found to improve some reporting practices for primary research [36, 37], it is worth noting that research examining journals’ inclusion of PRISMA in IA, either as an endorsement or requirement, could not find sufficient evidence that there was a strong improvement in reporting [30, 38]. These same findings appear to be true for other reporting guidelines [38–40]. This may either be due to lack of clear instructions for peer reviewers on utilization of reporting guidelines during the peer review process, or authors’ openness to peer review comments related to needed improvements in reporting, or other pressures [41–44] Published evidence indicates that author uptake of the guidelines, evidenced through the statement of adherence to PRISMA within the abstract or full text of SRs/MAs, has a greater effect on reporting quality than the inclusion of PRISMA in IA [45].

Although studies elucidating reporting quality and methodological quality of SRs/MAs have been noted across the health disciplines for some time, the examination of these issues in dental medicine has only been a focus over the last decade, with greater focus on the methodological quality of SRs/MAs for this profession, rather than the reporting quality. Few studies have examined how well PRISMA reporting standards are adhered to within the full text of published SRs/MAs in dental medicine. The majority of studies assessing the impact of PRISMA on published dental medicine SRs/MAs focus on the effect of PRISMA for Abstracts (PRISMA-A) on the reporting quality of abstracts of systematic reviews [32, 46–50]. However, abstracts are so abbreviated that they cannot, for instance, provide a reproducible search strategy or fully describe selection processes, which, among other PRISMA checklist items, provide the critical information to allow the assessment of the methodological quality of the review and enable reproducibility and replicability.

Assessments of adherence to PRISMA in the full text of dental medicine SRs/MAs have utilized small sets of articles, though the articles sampled have never been those published in the highest impact journals for dental medicine [51]. The relationship between dental medicine journal inclusion of PRISMA in its IA is unknown. Strikingly, only a quarter (24.1%) of dental medicine journals in 2018 required adherence to PRISMA for published systematic reviews and meta-analyses [52]. In light of these lacunae in the literature, this study sought to improve the understanding of the effect of PRISMA 2009 on the reporting quality of the full text of SRs/MAs in the highest impact journals for dental medicine.

Materials and methods

Data collection and sources

This study used published SRs/MAs from a compiled set of the twelve highest impact journals categorized as dental medicine journals in Journal Citation Reports (JCR) and Scimago (https://www.scimagojr.com/). The methods for determining the journals, searching for systematic reviews and meta-analyses within the selected journals, and assessing the resulting systematic reviews and meta-analyses are reported in detail in a previously published study [27] and supporting data and data documentation are available on the project’s Open Science Framework site [https://osf.io/hgkmq/].

Briefly, the top ten highest impact journals in the dental medicine categories from JCR and Scimago were combined and deduplicated, leaving a set of twelve journals (S1 File). SRs/MAs published in the twelve journals were identified via a targeted search in PubMed, run on June 18, 2018, available in S2 File. After the initial search was run, two authors (KS and JS) reviewed all the results to determine which records were legitimately systematic reviews or meta-analyses using a blinded screening method in Rayyan. Conflicting decisions about the records were resolved by consensus, after which 913 articles remained. Subsequent assessment of the size of the corpus of systematic reviews and meta-analyses published in journals classified as Dental journals in PubMed up to the date of October, 2018 indicates that the sample from the journals used for this study represents ~14% of the systematic reviews and meta-analyses for Dental journals included in PubMed published prior to 2019.

The articles from the selected journals were initially evaluated for key methodological and reporting characteristics using an evaluation instrument adapted from the instrument used by Rethlefsen et al. [21]. The evaluation instrument assessed the provision of details of blinding, study selection, and risk of bias, as well as search strategy inclusion in the included SRs and MAs. For the purposes of this study, additional data were added to the dataset from two sources: PubMed and Scopus. Additional data elements for this study include: the publication year for each record, specification of the study as a ‘systematic review’ or ‘meta-analysis’ in the title, number of authors, international collaboration of authors, country of the corresponding author and the first author, world region for those two positions (when they differed), reference to PRISMA in the abstract, reference to PRISMA in the full text, and whether the reference to PRISMA indicated use of PRISMA to guide reporting or conduct of the SR or MA. To add these data points, the complete set of PMIDs for the SRs and MAs in this sample were entered into PubMed, publication date information was downloaded and added to the existing dataset, and then a search combining the PMIDs with terms for systematic reviews or meta-analyses limited to the title field was run to find just the records meeting that PRISMA criterion. Additionally, within PubMed, Reference to PRISMA in abstracts was assessed using a search for PRISMA or “Preferred Reporting Items for Systematic Reviews and Meta-Analyses” in the abstract field of included records. Reference to PRISMA within the full text and the stated use of PRISMA to guide the methodological conduct or reporting was collected through assessment of the full text of the document. Additional details on these processes may be found on the Open Science Framework project space for this study.

The data related to number of authors, first and corresponding author country of affiliation, associated world region, along with presence/absence of international collaboration were all derived from publication data in Scopus. For author affiliations missing from the data provided by Scopus, author-supplied affiliations were determined from affiliation data from the author associated with publications from the same time period, along with general searches for scholarly profiles or public profiles for institutions for those authors, focusing on affiliation data for the time period of the publication. This data collection was performed by two authors (JB and KS) All first author countries were mapped to United Nations "Standard Country or Area Codes for Statistical Use" [53], along with the corresponding World Region and World SubRegion Codes from that publication.

Because the dataset for this study included articles published in high-impact dental medicine journals between 1993–2018, the PRISMA 2009 checklist was used to assess reporting quality. To determine how reporting items fit within the PRISMA 2009 requirements, the initial evaluation instrument’s assessment items are mapped to the PRISMA 2009 checklist items in Table 1, along with the additional datapoint of specification of SR or MA in the title. The authors note that several key methodological items were applicable to a single PRISMA item, namely for reporting eligibility criteria and study selection processes. The mapped items from the assessment instrument used in this study were kept separate to determine the extent of author reporting for those compound reporting items. It is also notable that several of these items are largely unchanged between the QUORUM, PRISMA 2009, and PRISMA 2020 checklists, such as the identification of the report as a systematic review in its titling, the specification of inclusion and exclusion criteria, and reporting study selection results for each phase of the review, though clarity for some of those items has improved between the reporting guidelines. Therefore, studies not meeting many of the assessed checklist items for PRISMA 2009 would typically fail to meet the same or similar reporting items for QUORUM and PRISMA 2020.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Assessment instrument items mapped to 2009 PRISMA items.

https://doi.org/10.1371/journal.pone.0295864.t001

The publications from 2010–2011 were removed from the dataset to account for publication timelines for SRs/MAs written prior to or just following the publication of the 2009 PRISMA guidelines, as well as to account for a potential lag in awareness of PRISMA based upon diffusion of knowledge. Additionally, SRs/MAs affiliated with the Cochrane Organization were removed from the dataset. The removal of SRs and MAs published in 2010–2011, along with Cochrane-affiliated SRs/MAs left 829 articles for analysis. The process of selecting studies is outlined in the PRISMA flow diagram in S3 File. Variables were recoded for this study to allow for calculated variables (total number of PRISMA-mapped items met in each included publication) and for combining assessment items related to search strategy reporting. The refined, re-coded dataset, along with its data dictionary and data cleaning documentation are in the Open Science Framework project space for this study.

The websites for all journals selected for this study were examined to determine the inclusion of PRISMA in IA. This website survey was conducted between April and May, 2022. For all journals with PRISMA mentioned or required in IA, the journals were contacted to determine the date when PRISMA endorsement was added to IA. For journals not responding or providing insufficient information about the date of addition of PRISMA in IA, the Internet Archive (https://archive.org/web/) was used to attempt to access archived websites for those journals.

Data analysis

All data were re-coded and analyzed in SPSS (version 28.0). For all items mapped to PRISMA items for this study, descriptive statistics were run. Crosstabs were run to assess the percent of SRs and MAs providing key methodological details before and after the publication of PRISMA. A Chi-square test of independence was run to determine the significance of differences between the groups and the effect size was determined using Cohen’s d. Significance was determined using p < 0.05.

For questions other than the first research question, this study analyzed the overall level of reporting for each publication using a calculated variable (total PRISMA-mapped items), derived from the sum of all key methodological detail items assessed for this study. This calculated sum for PRISMA-mapped items reported was used as the dependent variable when assessing the effect of author team characteristics on reporting, including author team size, international collaboration in author teams, and world region of first author. The sum total of PRISMA-mapped items was also used to assess the effect of described use of PRISMA in the abstract or full text on reporting level. For questions related to rate of inclusion of PRISMA in the abstract or full text of publications, as well as the described use of PRISMA in the full text, frequencies were run. The variables, maps to PRISMA items, statistical analysis methods chosen, and other key details are provided in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Data analysis.

https://doi.org/10.1371/journal.pone.0295864.t002

Results

Characteristics of included SRs/MAs

SRs/MAs by year and journal: The number of SRs/MAs in the included journal set increased threefold between the time periods studied, with 211 SRs/MAs published between 1998–2009 (pre-PRISMA) and 618 published between 2012–2018 (post-PRISMA). The majority of the included records were published in the Journal of Clinical Periodontology, followed by Clinical Oral Implants Research and Journal of Dentistry.

Bibliometric data findings: For the purposes of this study, we assessed the number of authors per paper, the country, world region, and world sub-region of first authors, along with the country income classification, and the presence of international collaboration based upon author affiliation data. Our findings indicate that most SR/MA teams with publications in the selected high-impact dental medicine journals include 3–6 authors, with a mean number of 4.51. About one third of all included studies had international collaboration (35.9%).

First authors for the SRs/MAs were affiliated with institutions or dental practices primarily in Europe (n = 476, 57.4%), the Americas (n = 232, 28.0%, or Asia (n = 92, 11.1%). By world subregion, most authors were affiliated with institutions in Western Europe (n = 241, 29.1%), followed by Northern Europe (n = 137, 16.5%), and Northern America (n = 122, 14.7%). The vast majority of first authors were from institutions in high-income countries (n = 660, 79.6%) or upper-middle income countries (n = 163, 19.6%). Major characteristics of included studies are presented in Table 3. The full frequencies for world region, sub-region, country, and country income classification are in a S1 Table.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Characteristics of included studies.

https://doi.org/10.1371/journal.pone.0295864.t003

What was the effect of PRISMA 2009 on the level of reporting of key methodological details for systematic reviews and meta-analyses in high impact dental medicine journals?

There was a statistically significant difference in the number of key methodological details reported among articles published after 2011 (M = 10.55, SD = 1.4) as compared to those published prior to the release of PRISMA 2009 (M = 7.83, SD = 3.267); t(827) = -12.915, p < .001. Examining the individual methodological reporting items tracked, crosstabs with Chi-Square revealed that reporting of each individual item improved following publication of PRISMA in 2009. Results of the crosstabs with Chi-Square are presented in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Reporting by methodological item, pre- and post-PRISMA 2009.

https://doi.org/10.1371/journal.pone.0295864.t004

Following the publication of the PRISMA 2009 guidelines and checklist, how frequently did authors in the selected dental medicine journals refer to PRISMA and what effect did described use of PRISMA have on reporting quality?

How frequently did SRs and MAs in the selected, high-impact dental medicine journals refer to PRISMA in their abstracts or full-text?

Among SRs/MAs published after 2011, PRISMA is rarely mentioned in the abstracts of author-identified SRs/MAs in the highest-impact dental medicine journals. For the journals assessed, over 90% of the abstracts did not mention PRISMA, with only 8.9% explicitly describing use of PRISMA in the abstract for the published SR/MA. Within the full text of the SR/MAs published after 2011, authors explicitly mention PRISMA in the manuscript body in roughly 60% of publications (n = 370, 59.87%). The examination of the reference list to identify studies that referenced PRISMA but did not name it explicitly in the manuscript body, does not greatly improve the number of publications in which authors indicate knowledge of PRISMA (n = 376, 60.84% of sample). Data provided in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Authors’ explicit description of the use of PRISMA in dental medicine systematic reviews, 2012–2018.

https://doi.org/10.1371/journal.pone.0295864.t005

For authors referring to PRISMA, did they distinguish between PRISMA as a reporting guideline rather than a methodological guideline?

The intended application of PRISMA is not well reflected in the described use of PRISMA within dental medicine SR/MAs that mention the guidelines. Of the articles that mentioned or cited PRISMA, the guidance was most frequently described as being used to guide the methodological conduct of the study (35.6%), followed by its correct application as a guideline to improve the reporting of the SR/MA being described in the publication (27.4%). In other cases, the guidelines were referenced as guiding both conduct and reporting (6.6%) or protocol development (9.6%). The authors note that the application of PRISMA was unclear for 15.4% of the articles assessed for this study. The descriptive statistics, including the language used to describe the use of PRISMA for those publications coded as ‘unclear,’ is presented in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Authors’ described Use of PRISMA within published dental medicine systematic reviews and meta-analyses, 2012–2018.

https://doi.org/10.1371/journal.pone.0295864.t006

Post-PRISMA, what effect did reference to PRISMA in the full text have upon the reporting quality of articles?

For SR/MA articles published post-PRISMA, a one-way ANOVA revealed there was a significant effect of description of PRISMA application within the full text on the number of items meeting PRISMA reporting requirements [F(7,610) = 13.81, p < .001]. The results of the One-Way ANOVA are presented in Table 7 and means and standard deviations for number of items reported by described use of PRISMA are presented in Table 8. Post hoc comparisons using the Tukey HSD indicated that any explicitly described application of PRISMA is significantly associated with improved reporting of the key methodological items tracked in this study compared to articles not explicitly describing the application of PRISMA to the SR/MA process (M = 9.55, SD = 2.867); t(616) = -9.5, p < .001. There was a non-significant difference between the absence of PRISMA in the full text and only mentioning PRISMA through the titling of the flow diagram as a PRISMA flow diagram (p = 1.0) or use of PRISMA guidelines as a supporting reference without explicit naming of the guidelines in the full text (p = .99). Likewise, there was a non-significant difference between authors explicitly describing the guidelines as being used for the conduct of the study (p = .98), the reporting of it (p = 1.0), both conduct and reporting (p = 1.0), protocol development (p = 1.0), or an unclear application of PRISMA (p = 1.0). Taken together, the results indicate that any explicit description of PRISMA in the full text improves the level of reporting of items required by PRISMA.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. One-Way analysis of variance of PRISMA item reporting by use of PRISMA 2009 within the full text of published manuscript.

https://doi.org/10.1371/journal.pone.0295864.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Descriptive statistics for PRISMA reporting based upon described application of PRISMA 2009 in article full text.

https://doi.org/10.1371/journal.pone.0295864.t008

Assessing the predictors of levels of reporting of key methodological details

To understand the impact of total number of authors and mention of PRISMA 2009 in the full text, while controlling for the other variables tested (total authors, mentioning PRISMA in the abstract, international collaboration, and world region of first author) on the level of transparent reporting for dental medicine SRs from 2011–2018, a linear regression analysis was run. The overall regression was statistically significant (R2 = 13.9, F(8, 609) = 12.24, p < .001). The results of the linear regression indicate that the total number of authors and mention of PRISMA in the full text account for 13.9% of the variation in the number of key methodological items reported. This means that 86.4% of the variation in key methodological items correctly reported cannot be explained by the number of authors or direct mention of PRISMA alone. The results of the linear regression are reported in Table 9.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Regression analysis summary for predictors of number of key methodological items reported (post-PRISMA).

https://doi.org/10.1371/journal.pone.0295864.t009

Discussion

Although the QUORUM statement and checklist were published nearly a decade prior to the release of PRISMA in 2009, the PRISMA guidelines have served as a more widely recognized set of reporting guidelines for the exponentially increasing number of systematic reviews published across the disciplines. They are intended to serve the critical purpose of providing sufficient information to allow readers to assess the methodological quality of the conduct of the SR/MA and decide if the findings are trustworthy for the purposes of clinical, policy, or personal health decisions. As Davidoff [54, p. 231] notes, “accurate and transparent reporting is like turning the light on before you clean up a room: It doesn’t clean it for you but does tell you where the problems are.” This is increasingly important as umbrella reviews (including meta-umbrella reviews) are undertaken to better understand the benefits and harms of an array of interventions or exposures. Low quality reporting in the systematic reviews and meta-analyses meeting inclusion criteria in the umbrella then create additional difficulties for the team involved in the umbrella review.

The development and publication of high-quality reporting guidelines is the critical first step to addressing sub-optimal reporting quality. However, in order for reporting issues to be addressed, authors must be aware of and understand the correct application of those reporting guidelines. Only through dissemination of reporting guidelines, paired with author, editor, and peer reviewer education on how to use them, can reporting issues in studies be improved. These same observations have been made in the assessment of the impact of other reporting guidelines, such as the ARRIVE reporting guidelines [39, 55]. This study sought to assess the impact of PRISMA guidelines on the level of reporting of a subset of key methodological details in SRs/MAs in high-impact dental medicine journals. In assessing the frequency of described use of PRISMA in the full text of articles published after 2011, it also provides something of a measure of awareness and uptake of the guidelines, though awareness and utilization of guidelines may not track neatly with one another.

The study findings indicate that, following the widespread dissemination of PRISMA 2009 guidelines, there was a statistically significant improvement in the number of key methodological details in the SRs/MAs in the set of high-impact dental medicine journals we sampled. Although there are no other studies examining whether the PRISMA guidelines improved methodological reporting in dental medicine as a whole, one study has examined the effect of PRISMA on the reporting quality of systematic reviews in orthodontics [31]. Researchers note that the search strategies and selection processes in dental medicine SRs are generally not reproducible [26, 27]. Limited studies have examined whether PRISMA improved reporting in other health-related fields. Our results are consistent with those from the fields of pain medicine [56], psychology [57], nursing [58], vascular surgery [59], and radiology [60]. Our results are also consistent with studies indicating that PRISMA for Abstracts has improved the quality of dental medicine abstracts in SRs since it was published [61, 62].

Our assessment of the mention of PRISMA in the abstract and full text of the SRs/MAs indicate that PRISMA is explicitly mentioned in the full text of SRs/MAs far more frequently than in abstracts (59.9% vs. 8.9%, respectively). This may be due to word count limits for the journals selected, an issue which inhibits the reporting of methodological information for all study types. It is also far less critical for PRISMA to be mentioned in abstracts than in full texts. While PRISMA was explicitly mentioned in the full text of almost 60% of the SRs/MAs sampled, an assessment of the language used to describe its application indicates that authors’ textually expressed awareness of PRISMA and their understanding of the intended application of those guidelines are not necessarily aligned. Less than a quarter (21.7%) of the published SRs/MAs published after 2011 correctly referred to the guidelines as guiding the reporting of the review. While there is a relationship between the conduct of a study and its record [37], the PRISMA Statement is clear in its intended application as guiding the reporting of methodological details rather than the methodological conduct of the review. There are key differences in these two considerations. For example, while the guidelines’ requirement to report risk of bias assessment details may cue authors to undertake risk of bias assessment for included studies, that cue does not guarantee that the risk of bias assessment instrument was appropriately chosen or properly applied. Likewise, a reproducible search may neither be rigorous in its conceptualization, nor appropriately mapped across relevant databases. Such methodological quality assessments are better determined through assessment instruments such as AMSTAR [13] or ROBIS [14], and are the considerations that should ideally, ultimately drive the level of trust in the results and recommendations of the SR/MA. In spite of the apparent misunderstanding of the intended application of PRISMA, the authors note that articles that explicitly mentioned PRISMA in the full texts assessed improved reporting, even when authors described PRISMA as having guided the conduct of the SR/MA.

The study also assessed the effect of author team characteristics on the level of adherence to PRISMA for the selected reporting items. Our findings indicate that the number of authors involved is the sole author team factor associated with improved adherence to PRISMA, with mid-sized and large study teams having a higher number of items reported than small teams. One possible reason is that a larger team brings together people with different skill sets and experience levels, and that the chances of a larger team having someone who is well-versed at reporting methodology and/or being familiar with PRISMA are higher. The lack of correlation between first author world region or international collaboration with level of key methodological items reported indicates that the linguistic and geographic characteristics of teams matter far less than those individuals’ awareness of PRISMA. This is notable, given that the guidelines are primarily in English, with limited translations for other languages. Although the country classification by income of first authors was coded for the dataset, this study did not assess how the country income classification of first authors might influence the level of adherence to PRISMA. The inclusion of that data point did, however, indicate that SRs/MAs in the selected journals are primarily authored by those in high-income or upper-middle income countries. The ramifications of that are outside of the scope of this review but worth further study.

The lack of inclusion of PRISMA in the IA for the set of journals sampled in this study as of spring 2021, is of note. The ideal is certainly for authors to be aware of the reporting guidelines applicable to the type of study their manuscript reports, but IA have a potential to address awareness gaps by explicitly stating that SRs/MAs submitted must adhere to PRISMA and provide the checklist along with the manuscript. Panic, et al., noted that there is evidence that these recommendations improve the reporting of SR/MA methodology. As such, and out of a concern related to broader reproducibility issues, many journals have adopted the practice of recommending or requiring reporting that follows PRISMA guidelines for publication. Among dental medicine journals, Ruy Carneiro et al. [52] noted that only a quarter of journals in the field required adherence to PRISMA for published systematic reviews and meta-analyses. That said, this recommendation is only effective if journals and peer reviewers choose to enforce the recommendations and turn away articles that do not follow PRISMA reporting guidelines. Peer reviewers who are well-versed in PRISMA and SR methodology should be selected to review SRs and MAs.

Although this study found larger author team sizes and explicitly described use of PRISMA in the full text improved the level of reporting of the key methodological items in published SRs/MAs, it only explained about 14% of the variability in the number of items correctly reported. The authors conclude that the awareness of the guidelines and an understanding of each reporting item must ultimately come down to the level of education about the reporting guideline among the individuals involved in the SR/MA. This same need to promote meaningful implementation of reporting guidelines will remain true for PRISMA 2020. Previous studies note the critical role of librarians and statisticians in review teams for adequate reporting of methodological details within their respective purview [21, 27, 63, 64].

It is also notable that many of the reporting items are complex and compound, such as the requirement to provide a reproducible search. For example, although authors may provide a search, an understanding of what makes a search reproducible would require more extensive training in search syntax, along with the record-keeping required. Likewise, details related to selection processes ask authors to describe any automation tools used in study selection processes; authors may not assume that deduplication features or screening artificial intelligence models built into review software would fall into the category of automation tools needing clear description. These issues are likely yet more difficult for authors faced with the increased level of search reporting required in the PRISMA 2020 statement. The PRISMA 2020 Statement and PRISMA-S extension address some of the shortcomings noted in study selection and search processes, respectively. Authors would ideally utilize the Explanation and Elaboration publications to better understand the meaning and best practices attached to checklist items.

Conclusion

While conduct and reporting of research are distinct concepts, they are closely intertwined for systematic reviews [2]. This study of the implementation of PRISMA 2009 reporting guidelines in dental SRs and MAs indicates that there is a need for increased use of and education related to PRISMA, particularly following the publication of PRISMA 2020. We suggest that those who are conducting SRs in dental medicine should become familiar with the PRISMA 2020 Statement in order to understand the complex nature of the individual reporting items and the methodological concerns informing the reporting items, prior to the start of the review process. Author examination of the Explanation and Elaboration publication accompanying PRISMA 2020 [65] is highly recommended, beyond just using the checklist as a standalone item. The employment of methodological guidelines for SRs/MAs might also help elucidate the meaning and importance of the reporting items. We further suggest that study teams should become familiar with the PRISMA extensions that apply to their reviews. We believe that the requirement by journals for SR teams to comply with PRISMA will improve methodological reporting in SRs, but that journals should either select peer reviewers who are trained in PRISMA to review submitted SRs/MAs or provide training materials on applicable reporting guidelines for peer reviewers less familiar with those guidelines. In an ideal world, requiring peer reviewers to assess manuscripts against applicable reporting guidelines, along with the methodological and conceptual merit of the submission, would address some shortcomings in reporting. We continue to advocate for librarians and information specialists to be fully-credited members of the study team, as many have been trained in PRISMA requirements. The results of this study indicate forward progress toward having rigorous and reproducible SRs/MAs, but areas for improvement are clear as well, and we believe these actions will improve research products in dental medicine. We further believe that these actions will improve reporting for SRs/MAs in all other areas of healthcare.

Supporting information

Acknowledgments

The authors wish to acknowledge Carrie Forbes and Tristan Ebron for their role in data collection in the initial data collection process. They contributed to data collection for the previously published study [21]. The dataset from the initial study contained twelve of the thirteen methodological reporting items used for this study. The authors also acknowledge Dr. Hui Bian for her initial consultation that informed the design of the data collection instrument(s) and approaches to data analysis.