Design

This is the study protocol for a cross-sectional analysis of reporting guidelines indexed in the web-based EQUATOR Network Library. The EQUATOR Network Library provides an up-to-date online collection of guidelines and policy documents related to health research reporting. These reporting guidelines are aimed mainly at authors of research papers, journal editors, peer reviewers, reporting guideline developers, educators and research funders. At present, more than 300 guidelines and related documents are available on the EQUATOR Network website (http://www.equator-network.org/library/). The EQUATOR Network Library provides a searchable database of all reporting guidelines (published since 1996) identified through systematic literature searches that focus on/provide specific guidance for reporting of various types of research designs, components of research reports or specific medical conditions or procedures. The EQUATOR Network Library provides guidance for different types of health research studies, for example, experimental studies including randomised trials (eg, CONSORT statement19 20), systematic reviews and meta-analyses (eg, PRISMA statement21 22), diagnostic and prognostic studies (eg, STARD guidelines23 and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis statement35 36), observational studies (eg, STROBE statement24), economic evaluations (eg, Consolidated Health Economic Evaluation Reporting Standards statement37), case reports (eg, CAse REport guidelines38 39), study protocols (eg, Standard Protocol Items: Recommendations for Interventional Trials statement40 and PRISMA-Protocols41 42), animal preclinical research (eg, ARRIVE guidelines25) and other reporting guidelines (eg, mixed-methods studies, reliability and agreement studies and clinical practice guidelines). In addition to these reporting guidelines, the EQUATOR Network Library lists guidance that relates to publication of health research for sections of research reports (eg, Statistical Analyses and Methods in the Published Literature for statistical methods and analysis43), guidance for specific medical conditions, procedures or clinical areas (eg, cardiovascular medicine, clinical oncology, paediatrics, public health or genetics), but also reporting guidelines under development. Currently, the EQUATOR Network Library website lists only references to journals that published the reporting guideline first, not later republications (often in a national or local journal, usually in a local language edition).29

Eligibility criteria

To be eligible for inclusion, published papers will have to meet the following definition of a reporting guideline related to health research: ‘A reporting guideline is a checklist, flow diagram or explicit text to guide authors in reporting a specific type of research, developed using explicit methodology’.26 27 We will include reporting guidelines with no restrictions by study design, medical specialty, disease or condition. We will include published papers of reporting guidelines (ie, ‘completed’ reporting guidelines) written in English. Translations of any existing reporting guideline in other non-English languages will be excluded (eg, CONSORT translation into Spanish or Chinese). Meeting abstracts, proceedings paper (journals, book based), editorials, book chapters, corrections, retracted publications and other items (eg, data paper, news and notes) will be excluded. Reporting guidelines under development will be also excluded.

Searching

To provide a reliable summary of the reporting guideline literature, we will search for completed reporting guidelines indexed in the EQUATOR Network Library from inception to 31 July 2018 (this date will represent 10 years after the official launch of the EQUATOR Network in 200844). We will first identify all the published papers catalogued in the EQUATOR Network Library. For all reporting guidelines, an information scientist will obtain from the EQUATOR website the following: the title (or reference), standard identifier for each paper (eg, PubMed ID or a digital object identifier, if the paper has one), study design and date of record last updated. Additionally, we will contact the EQUATOR Network Library team for clarifications and further data (if necessary). To ensure consistency in the searching process, a second researcher will perform complementary hand-searches (eg, websites of reporting guidelines catalogued in the EQUATOR Network Library) (see examples in table 1) to check for potential indexing omissions or errors. We will use all the standard identifiers retrieved for published papers to carry out a search in PubMed/MEDLINE in order to have access to full references, titles and abstracts. With this information, we will perform a search of the Web of Science (Clarivate Analytics, Philadelphia, Pennsylvania, USA) based on the title or the standard identifiers of the papers. Intended date for the searches in PubMed/MEDLINE and Web of Science will be 15 April 2019. The search results will be imported into a Microsoft Access (Microsoft, Seattle, Washington , USA) database.

Screening and selection

All titles and abstracts will be screened against our eligibility criteria by one researcher and verified by a second researcher. We will retrieve the full text of any paper(s) meeting our eligibility criteria or for which eligibility remains unclear. Any discrepancies in the screening of titles and abstracts and selection of full-text papers will be resolved via discussions by the research team, if necessary.

Data extraction and data normalisation

Raw data from each paper will be exported (downloaded online) from the Web of Science (Clarivate Analytics) by one researcher (senior information specialist). Data will be collected in Microsoft Access (Microsoft) containing the following: title, publication year, journal, subject category, keywords, citations (the number of times a paper has been cited by other publications), source, language, document type (eg, article or review according to Document Type Field from the Web of Science) and the authors’ names, author’s affiliated institution (eg, institution name, department and institution address including city), and country. All data will be verified by a second researcher to minimise potential omissions or information errors. Any discrepancies in the extracted data will be resolved via discussion or adjudication by a third researcher if necessary.

To ensure consistency in the data extraction, a process of data cleaning and normalisation will be conducted by two members of the research team to bring together the different names of an author, institution, country, journal and keywords. Specifically, one researcher will check the names by which an individual author appeared in two or more different forms (eg,  ‘Douglas G Altman’ or ‘Douglas Altman’ or ‘Doug Altman’), using coincidence in that author’s affiliated institution as the basic criterion for normalisation (eg, Oxford University, UK). In the case of author’s affiliated institution, names in many records may include two or more institutions (eg, university hospitals, research centres and academic institutions).45 So, we will proceed to distinguish between institution names by recording all variations of any individual macroinstitution as could be identified for each record (eg, for the institutional address ‘The Ottawa Hospital and University of Ottawa, Canada’, the normalisation approach will be to present ‘University of Ottawa, Canada’ separately from ‘The Ottawa Hospital, Canada’). Missing data (eg, missing country information of an affiliated institution) will be substituted and completed by manual web search.

Keywords (‘author keywords’ and ‘keyword plus’) which are automatically assigned by the Web of Science (Clarivate Analytics) from the title of the paper will be extracted and used to represent the conceptual content of the publications.

Typographical, transcription and/or indexing errors will be corrected or removed. All potential discrepancies will be resolved via discussion.

Data analysis

The analysis will be descriptive, with data summarised as frequencies and percentages for categorical variables or median and IQR for continuous variables.

Our primary analysis of production, citations and collaboration patterns will be based on single papers (paper-level analysis). The description of the scientific production and citation impact will be carried for authors, institutions, countries and journals (eg, total number of papers and total number of citations). Tables will be used to present the most prolific authors (5 or more published papers), institutions (5 or more papers), countries (10 or more papers), journals (10 or more papers) and the most cited papers and reporting guidelines (‘top 50 citation classics’).

We will describe collaboration patterns with respect to authors, institutions and countries: number of signatures (total number of authors included in all the papers of each author), number of collaborators (and main collaborators in terms of number of collaborations), number of papers with coauthors, collaborations between authors (‘interindividual collaboration’), and collaboration index which is the mean number of signatories per paper. We will identify collaborations between institutions (‘interinstitutional collaboration’) and countries (‘international collaboration’). For institutions, we will differentiate between institutional collaborations in the same country (‘domestic interinstitutional collaboration’) and institutional collaborations from different countries (‘international interinstitutional collaboration’).

Network analyses46–48 will be carried to study the structure of collaborations between authors, institutions and countries. Collaboration networks will be constructed by identifying the regularities in the patterning of relationships among entities (eg, which might be authors, institutions and countries) for each single paper. We will present network diagrams to illustrate the structure of scientific collaborations. A network diagram is a visual representation of a collection of nodes or vertices (eg, authors) and the relational ties connecting them, called links or edges (eg, numbers of papers or collaborations). Particularly, we will identify clusters of more intense collaboration networks by applying a priori thresholds (‘collaboration intensity’) based on the numbers of papers signed in collaboration between authors (5 or more papers), institutions (5 or more papers) and countries (10 or more papers). All the network diagrams will be presented as undirected-weighted graphs (where node sizes are proportional to the number of papers, links are bidirectional, and the link thicknesses are proportional to the number of collaborations). Key standard measures and metrics for quantifying network structure (centrality measures such as ‘degree of centrality’, ‘closeness’ and ‘betweenness)49 50 will also be calculated to complement the cluster analysis. Centrality measures identify the most prominent (‘key’) members of a scientific community, that is, those who are extensively involved in relationships with other network members. Degree of centrality51 52 is a simple centrality measure that counts how many connected links a node has (eg, the number of coauthors of a given author). If a node has many ties compared with nodes, this indicates that this node has a central position in the network. In a collaboration network, it seems reasonable to suppose that a node (eg, author) having connections to many others might have more activity or more prestige than those having fewer connections. Centrality can also characterise the shape of a whole network. Closeness46 48 is based on the notion of distance. Closeness measures the mean distance from a node to other nodes (eg, an author that has high centrality closeness is likely to receive information more quickly than others in the network). Closeness measures efficiency or independence. Betweenness46 52 measures the extent to which a node (eg, author) lies on paths between other nodes or how much a node connects other nodes with each other (eg, an author who is high in betweenness is able to control the flow of information or resources passing between other coauthors).

We will conduct topical (also called linguistic, semantic or textual) data analyses and visualisations to determine the topic coverage of a body of text. The most frequently used keywords will be identified and presented for each journal. Word clouds displaying the frequency of terms (with larger words depicting the frequency of occurrence) will be generated for the main keywords to identify major topics. We will also present the ‘co-words network’ of keywords representing the co-occurrence phenomenon of highly frequent words in the papers.53

Multiple publication of identical paper(s)

It is important to note that for a paper presenting a reporting guideline may have been published simultaneously by multiple journals to support wider dissemination and uptake.26 Some of these reporting guidelines published in multiple journals include CONSORT statement19 20 and PRISMA statement.21 22 Secondary analysis will consider and characterise production, citations and collaboration patterns by integrating data for multiple (identical) papers of a single reporting guideline (guideline-level analysis). This will involve the following: (1) authorship and collaboration data will be counted once and (2) citation impact will be computed by grouping the total number of citations across the multiple (identical) papers published in different journals.

Additional analyses

We will undertake a subgroup analysis to compare papers and reporting guidelines in terms of scientific productivity, citation and collaboration patterns before and after the official launch of the EQUATOR Network in 200844: ‘before 2008 (pre-EQUATOR Network)’ versus ‘after 2008 (post-EQUATOR Network)’.

Depending on the productivity and collaboration patterns, we plan to explore whether sensitivity analyses related to thresholds (‘collaboration intensity’) are feasible using different numbers of papers in collaboration (eg, between authors, institutions and countries) to see how stable our results are.

Software considerations

Relational structure (network) visualisations and metrics will be produced by using PAJEK (University of Ljubljana, Slovenia),54 a free software package for large network analyses. Word clouds will be drawn using Wordle,55 a free online tool, for graphical representation of words frequency. We also plan to explore additional visualisation tools, considering this is a constantly evolving field.56–58

Patient and public involvement

No patients and/or public were involved in setting the research question, nor were they involved in developing plans for design (or implementation) of this study protocol. No patients and/or public will be asked to advise on the interpretation or writing up of results. There are no specific plans to disseminate the results of the research to the patient community.

Ethics and dissemination

To the best of our knowledge, this cross-sectional analysis will be the first attempt to systematically map the content of the EQUATOR Network Library using methods and tools from network science.46–48 The study will serve to identify reporting guidelines of health research, key actors producing and disseminating them, citation impact and scientific collaborations in the area. It will be helpful to determine the most intense collaboration patterns, most productive authors, institutions and countries developing reporting guidelines, most prolific journals disseminating them and the ‘citation classics’ of guidelines for reporting health research studies.

Any amendments made to this protocol when conducting the analyses will be outlined and reported in the final manuscript. Findings from this study will be published in peer-reviewed journals. All data underlying the findings reported in the final manuscript will be deposited in a cross-disciplinary public repository, such as the Open Science Framework (https://osf.io/) or Zenodo (https://zenodo.org/).

We anticipate the study could be relevant to a variety of audiences including journal editors, peer reviewers, research authors, guideline developers, research funders, educators and other potential key stakeholders. Moreover, the study findings could further be used to strengthen scientific capacity for international, national and subnational collaborations in order to increase value and reduce waste from incomplete or unusable reports of health research studies.12 15