History and publication trends in the diffusion and early uptake of indirect comparison meta-analytic methods to study drugs: animated coauthorship networks over time
======================================================================================================================================================================

* Joann K Ban
* Mina Tadrous
* Amy X Lu
* Erin A Cicinelli
* Suzanne M Cadarette

## Abstract

**Objective** To characterise the early diffusion of indirect comparison meta-analytic methods to study drugs.

**Design** Systematic literature synthesis.

**Data sources** Cochrane Database of Systematic Reviews, EMBASE, MEDLINE, Scopus and Web of Science.

**Study selection** English language papers that used indirect comparison meta-analytic methods to study the efficacy or safety of three or more interventions, where at least one was a drug.

**Data extraction** The number of publications and authors was plotted by year and type: methodological contribution, review or empirical application. Author and methodological details were summarised for empirical applications, and animated coauthorship networks were created to visualise contributors by country and affiliation type (academia, industry, government or other) over time.

**Results** We identified 477 papers (74 methodological contributions, 42 reviews and 361 empirical applications) by 1689 distinct authors from 1997 to 2013. Prior to 2002, only three applications were published, with contributions from the USA (n=2) and Canada (n=1). The number of applications gradually increased annually with rapid uptake between 2011 and 2013 (n=254, 71%). Early diffusion occurred primarily in Europe with the first application credited to the UK in 2003. Application spread to other European countries in 2005, and may have been supported by regulatory requirements for drug approval. By the end of 2013, contributions included 49% credited to Europe (22% UK, 27% other), 37% credited to North America (11% Canada, 26% USA) and 14% from other regions.

**Conclusion** Indirect comparison meta-analytic methods are an important innovation for health research. Although Canada and the USA were the first to apply these methods, Europe led their diffusion. The increase in uptake of these methods may have been facilitated by acceptance by regulatory agencies, which are calling for more comparative drug effect data to assist in drug accessibility and reimbursement decisions.

*   diffusion of innovations
*   indirect comparisons
*   methodological innovation
*   network meta-analysis
*   social networks

### Strengths and limitations of this study

*   Our paper walks through the development and history of indirect comparison meta-analytic methods and its early diffusion in the study of drugs using coauthorship networks.

*   Our animated coauthorship networks are innovative by allowing the visualisation of social structures and collaboration trends through authors and their connections with each other over time.

*   Mapping contributions based on first and last authors may miss some important contributions by coauthors, yet including second authors had little impact in a prior study.

*   We conducted an extensive systematic literature search that identified 477 (361 empirical applications) eligible English language papers that used indirect comparison meta-analytic methods to study drugs through to December 2013, yet some relevant papers such as non-English language papers and grey literature may have been missed.

*   We did not consider methodological and reporting quality, since focus was on early uptake by country and institutional affiliation, yet our results set the stage for further research that may consider quality.

## Introduction 

Randomised controlled trials (RCT) are essential for bringing novel pharmaceutical products to market. RCTs for drug approval typically compare new treatment efficacy to placebo and provide safety data for only common adverse effects. However, RCTs are often not powered to identify all important drug efficacy and safety endpoints and thus meta-analytic methods were developed. Meta-analysis is a statistical method that combines the results of two or more studies to evaluate the same intervention in comparison to a control such as placebo, to obtain a more precise estimate of the intervention’s effects relative to that control.1–3 The term *meta-analysis* was first coined by GV Glass in 1976, yet use of statistical methods to combine the results of multiple studies dates back to the early part of the 20th century, with early methodological techniques proposed by R Fisher and W Cochran in the 1930s.1 2 

When completed using high-quality RCTs, meta-analyses are regarded as providing the highest level of evidence.4 However, traditional pairwise meta-analysis is limited by only being able to combine and estimate the benefits or harms of two treatments if they have been compared directly. In addition, meta-analysis cannot compare more than two treatments at a time.3 5 This presents a challenge to policymakers, clinicians and patients who often need to select the most optimal treatment from several competing options.6 Indirect comparisons have been made informally using point estimates and 95% CIs of treatments.7 However, this informal approach does not provide a precise estimate of the relative difference between two treatments because the relative effects are not measured.

In 1997, the *adjusted indirect comparison* method was proposed by Bucher *et al*, as an innovative meta-analytic approach that uses indirect evidence to estimate the relative benefits and risks between two treatments.8 Unlike traditional pairwise meta-analysis, adjusted indirect comparisons estimate the relative effects of two treatments that have not been compared directly by leveraging results from each treatment that has been compared with a common comparator, such as a placebo.6 8 9 However, the adjusted indirect comparison method ignores direct evidence, even when available. In 2002, *network meta-analysis* was proposed as an extension of the adjusted indirect comparison method that combines direct and indirect comparative data across several sets of pairwise treatment comparisons.5 10 The combination of direct and indirect data yields more precise effect estimates.6 A similar method, coined *mixed treatment comparison*, was proposed in 2004,11 and the term *multiple treatment meta-analysis* was also introduced to describe concepts of combining both direct and indirect evidence in 20055 (table 1).

View this table:
[Table 1](http://bmjopen.bmj.com/content/8/6/e019110/T1)

Table 1 
Timeline of meta-analytic methodological innovations

Indirect comparison meta-analytic methods have become valuable tools in clinical and policy decision-making, and have thus been rapidly adopted since their introduction.7 12–14 However, application of these methodological innovations varies widely.6 12 15 Rogers’ Diffusion of Innovations Model defines *diffusion* as the process by which an innovation is communicated across individuals within a social system, particularly during the initial stages of its use.16 17 Our study sought to characterise the early diffusion of indirect comparison meta-analytic methods used to study drugs.16 We interpreted diffusion and uptake relative to the social system by creating coauthorship networks to examine the speed of uptake (number of publications) and spread of these methods (collaboration between authors, authors’ countries and across institutions) over time.

## Materials and methods

We recently examined the diffusion of two confounder summary score methods and illustrate the importance of innovation attributes (*relative advantage*, *compatibility*, *simplicity*, *trialability* and *observability*) and seminal author engagement on the uptake of methodological innovations using Rogers’ Diffusion of Innovations Model.16 In addition to innovation attributes, Rogers’ Model identifies key aspects of the social system that may impact the rate of adoption.17 In particular, a methodological innovation will have a quicker rate of adoption if members within the social system (eg, researchers, clinicians and policymakers) share similar system norms. For example, regulatory agencies make decisions for drug approval and formulary coverage. Regulatory agencies are therefore well positioned to influence the uptake of methodological innovations that support the drug approval process. If novel methods become a requirement for drug approval, pharmaceutical companies, which share a vested interest in the drug approval process, may willingly adopt the methodological innovation in question. We examined the diffusion and early uptake of indirect comparison meta-analytic methods used to study drugs, and interpreted contributions by country and affiliation type using Rogers’ Diffusion of Innovations Model.

### Systematic search

We completed a systematic literature search to identify all papers that used indirect comparison meta-analytic methods to study drug effects in humans. We searched the Cochrane Database of Systematic Reviews, EMBASE and MEDLINE from their dates of inception to 31 December 2013 using keywords based on a recent search (online supplementary appendix table A).18 We then used SCOPUS and Web of Science to perform a citation search to identify papers that referenced key seminal papers,8 10 major methodological contributions19–21 and reviews7 13–15 22 23 on indirect comparison meta-analytic methods.18 

### Supplementary file 3

[[bmjopen-2017-019110-SP3.pdf]](pending:yes) 

All English language papers that used indirect meta-analytic methods to compare the clinical efficacy or safety of three or more interventions among humans were eligible if at least one intervention was a drug. We excluded abstracts, letters, commentaries, cost-effectiveness studies, overviews of systematic reviews, protocols and papers with no identifiable authors. Papers that used informal indirect comparisons (eg, simply compared point estimates with 95% CIs) or did not clearly describe the techniques used to perform the indirect comparison in the title, abstract, introduction or methods sections were also excluded. Two authors (JKB and MT) independently searched and screened all titles and abstracts for eligibility. Discrepancies following full-text review were resolved by a third author (SMC).

The number of papers and cumulative authors was plotted by calendar year and type: methodological contribution, review paper or empirical application; and important social system events (eg, publication of seminal papers) were added to the graph. We then focused exclusively on empirical applications. A proportional Venn diagram was used to illustrate the yield of each database search strategy that contributed to the identification of eligible empirical applications. We abstracted: author(s), journal, year of publication, area of study, primary outcomes (efficacy, safety or both), first and last author institutional affiliations, terminology used to describe methods, and presence and details of network diagrams. If no primary outcome was explicitly stated, all outcomes were considered primary. When multiple diagrams were present, the total number of unique comparators across all network diagrams was taken. Two authors (JKB and EAC) abstracted all the data, and another (MT) verified the data.

### Coauthorship network of empirical applications

An Excel macro was used to generate a coauthorship matrix from author names downloaded into Microsoft Excel 2010 from Endnote X5 (Thomson Reuters, 2011). Names of authors presented in multiple forms were collapsed into the most common presentation or, in the event of a tie, the one with more initials. Publication (authors and order) and paper characteristics (country and institutional type ascribed) were imported into R, V.3.3.1 (R Foundation for Statistical Computing, 2016), leveraging RStudio, V.0.99.887 (RStudio, 2009), to generate directed coauthorship networks, and identify components. Coauthorship networks depict authors as ‘nodes’ with ‘ties’ between nodes denoting coauthorship. Directed coauthorship networks clarify network structure by sending ‘ties’ depicted as arrows, from first authors to coauthors. A component is a group of authors connected directly as coauthors on the same paper, or indirectly through a mutual coauthor on separate papers. A disconnected coauthorship network is based on the total number of components. The more components found in a coauthorship network, the more disconnected authors are from each other as a result of isolated publishing. Institutional affiliations and corresponding countries of the first and last authors of each empirical application were used to ascribe credit to each application and the network.16 Institutions were categorised by country and type (academia, government, industry or other). Node size was created proportional to the number of publications by that author. Node colour was created, first based on country affiliation attributed to each paper, and second based on institutional type. The networks were animated by calendar year of publication to visualise growth in application and contributions over time.

### Patient and public involvement

No patients or the public were involved in the development and design of this research.

## Results

### Systematic search

We identified 477 eligible papers: 74 methodological contributions (online supplementary appendix B), 42 review papers (online supplementary appendix C) and 361 empirical applications (online supplementary appendix D) (figure 1); published by 1691 distinct authors between 1997 and 2013. A steady increase in the number of eligible papers was seen over time, and proportionally more were published in recent years (figure 2). Focusing exclusively on the 361 empirical applications, the keyword search strategy identified most applications (n=314, 87%; 30% unique). EMBASE identified the most (n=282, 78%; 6% unique), followed by MEDLINE (n=239, 66%; 3% unique), and relatively few were identified by the Cochrane Database of Systematic Reviews (n=20, 6%; <1% unique) (online supplementary appendix A figure A). The citation search identified an additional 47 (13%) papers outside keyword searches (online supplementary appendix A figure B).

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/8/6/e019110/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/8/6/e019110/F1)

Figure 1 
Flow diagram of systematic search results.

![Figure 2](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/8/6/e019110/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/8/6/e019110/F2)

Figure 2 
Number of publications on indirect comparison meta-analytic methods by year of publication, n=477. Methodological contributions (chequered bar), review papers (horizontal stripes) and empirical applications (solid). Cumulative number of unique authors represented by the solid grey line, n=1689. †Innovators by seminal publication8: Bucher *et al* 1997 (Canada)8; Lumley 2002 (USA)10; Lu and Ades 2004 (UK).11 Early adopters: §government-sponsored academic groups and health technology and reimbursement assessment agencies (National Institute for Health and Clinical Excellence Guidelines Technical Support Unit 2002 (UK)30; Pharmaceutical Benefits Advisory Committee 2005 (Australia)37 38; Canadian Agency for Drugs and Technologies in Health 2009 (Canada)19; Haute Autorité de Santé 2009 (France)35; Institute for Quality and Efficiency in Health Care 2013 (Germany)36); #independent research organisations: Indirect Treatment Comparisons Good Research Practices Task Force 2011 (Canada, The Netherlands, USA, UK).6 15 

The indirect comparison meta-analytic applications were published in 188 different journals. The most common areas of study were cardiovascular disorders (22%), cancers (12%), musculoskeletal disorders (12%), infectious diseases (10%) and psychiatry (9%) (table 2). Sixty-nine per cent of primary outcomes assessed therapeutic efficacy, 25% assessed efficacy and drug safety, and 6% assessed drug safety alone. Of the 361 empirical applications, only 161 (45%) published network diagrams illustrating the direct or indirect comparisons. The median number of interventions compared was 7 (IQR 5–10, min=3, max=145). The most common terminology used was network meta-analysis (38%), followed by mixed treatment comparison (26%), Bucher’s method (24%) and adjusted indirect comparison (21%). The sum of these percentages is greater than 100% due to an overlap in the terminology used. More specifically, 18% (n=65) of all eligible empirical applications used two or more terms to describe the methods used.

View this table:
[Table 2](http://bmjopen.bmj.com/content/8/6/e019110/T2)

Table 2 
Characteristics of empirical indirect comparison meta-analytic applications in the study of drugs, n=361

### Coauthorship network of empirical applications

Figure 3 (A: country, B: affiliation) summarises the final coauthorship networks, and online supplementary appendices E and F map the growth of each network by country affiliation and institution type over time. The largest component included 143 (40%) papers and 567 (37%) authors, including innovators Guyatt GH, Lu G and Ades AE (online supplementary appendix D1-143). Of the remaining 128 components, 90 (70%) included only a single paper (25% of all applications made up single-paper components), demonstrating a relatively disconnected network.

![Figure 3](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/8/6/e019110/F3.medium.gif)

[Figure 3](http://bmjopen.bmj.com/content/8/6/e019110/F3)

Figure 3 
Directed coauthorship network of the 361 indirect comparison meta-analytic applications, 129 components, 1513 authors, 2000–2013. The lines represent the relationships (coauthorship) between authors, with arrows directed from first author to coauthors of each paper. Node size is proportional to the number of published articles. (A) Colour based on country: Canada (red), the USA (blue), the UK (yellow), all other Europe (light yellow) and all other regions (white). Authors publishing on papers with more than one country affiliation were coloured based on combinations of the primary colours and white. For example, authors on papers with affiliations from Canada and the USA were coloured purple (a combination of red and blue), while authors on papers affiliated with Canada, the USA and the UK were coloured grey (a combination of red, blue and yellow). (B) Colour based on affiliation type: academic (red), government (yellow), industry (blue) and all other affiliation types (white). Authors publishing on papers with more than one affiliation type were coloured based on combinations of the primary colours and white. For example, authors on papers with affiliation types from academia and government were coloured orange (a combination of red and yellow), while authors on papers affiliated with academic, industry and other were coloured light purple (a combination of red, blue and white).

Early application of these methods started in 2000, with three papers published by 200224–26; and each referencing the innovator paper.8 Authors were from Canada (red) and the USA (blue), and published in isolation of each other (online supplementary appendix E). In 2003, five papers were published in isolation of each other, with two credited to the USA (blue), and three credited to the UK (yellow). The majority referenced innovator Bucher *et al*,8 yet one paper referenced innovator Lumley.10 By 2004, an increase in collaboration between authors from different countries was noted, with the first multipaper component (France) published in 2004, and the first single-paper component with institutional affiliations from two countries (USA and Belgium) published in 2005. By 2006, another 13 papers were published: 11 papers referenced innovator Bucher with institutional affiliations credited to many countries worldwide (Belgium, Canada, France, Germany, India, USA), and two papers referenced two innovator papers,10 11 with one paper credited to the USA, the UK and Greece, and the other credited to the UK. From 2007 to 2013, we noted an increase in the number of applications published over time, with fastest uptake noted in 2011, and an increase in authors publishing from a broad range of countries depicted by the increase in colours observed in the animated networks (online supplementary appendices E and F and online supplementary files 1 and 2). In particular, a rapid increase in collaboration between authors was noted in 2009, as demonstrated by the merging of smaller components into larger components. Europe led the diffusion with node colours of yellow (UK), light yellow (all other Europe) and combinations of yellow with other primary colours comprising the majority of nodes in the coauthorship network.

Overall, institutional credit was given to 358 unique institutions around the world: 77% of contributions came from academic institutions, 18% from industry, 1% from government and 4% from other institutions (table 3). Europe led the diffusion with 49% of credited papers (22% UK, 27% other); 37% were credited to North America (26% USA, 11% Canada), and 14% to other regions.

View this table:
[Table 3](http://bmjopen.bmj.com/content/8/6/e019110/T3)

Table 3 
Institutional affiliations by country (n=35) and institution type (n=7) for the entire indirect comparison meta-analytic applications network

## Discussion

Indirect comparison meta-analytic methods are an important methodological innovation that has become valuable in providing comparative drug effect data in the absence of head-to-head trials. In this paper, we found that uptake was concentrated primarily in Europe (49%) with further contributions from North America (37%). Despite initial development from Canada (1997) and the USA (2002),8 10 our results are not surprising given that refined methods were published by core innovators from the Universities of Bristol and Washington.10 11 Early use of indirect comparison meta-analytic applications predominated from the UK, and may have been the result of an increase in demand by the UK government for more comparative effectiveness research to assist with clinical practice guideline development and to guide drug funding decisions. Indeed, the need for clinical practice guideline development was one of the major reasons for the establishment of the National Institute for Health and Clinical Excellence (NICE) in 1999,27 which has since become a world leader in providing guidance on the clinical and cost-effectiveness of new and established health technologies (including drugs). NICE decisions are made by independent committees of researchers, clinicians, industry and lay representatives; and have included innovator Ades, and early adopters from the NICE Guidelines Technical Support Unit, University of Bristol.5 20 28–30 

The steady increase in the use of indirect comparison meta-analytic methods, and effective diffusion to Europe and North America, may also be partially explained by consideration of the five key innovation attributes described in Rogers’ Diffusion of Innovations Model (*relative advantage*, *compatibility*, *simplicit*y, *trialability* and *observability*).16 The Multi-Parameter Evidence Synthesis (MPES) Research Group (from which the NICE Guidelines Technical Support Unit is based) has offered introductory short courses and workshops to facilitate understanding and application of these methods to health economists, statisticians and policymakers worldwide in collaboration with other academic institutions in the UK (Universities of Sheffield and York) since 2002 (*observability*, *simplicity*, *trialability*).30 Active workshops demonstrating the use of this methodological innovation likely provided a vehicle for peer observation to occur, so that the results and benefits of using this innovation were visible to potential adopters (*observability*). The provision of sample data sets and statistical code, as well as the integration of these methods into established software and software packages, may have also eased the use of these methods (*simplicity*), and allowed potential adopters the chance to try using these methods with direct guidance from the innovators and early adopters themselves (*trialability*). In addition, the MPES Research Group published tutorials and case studies highlighting the advantages of using pairwise, indirect comparison, and network meta-analyses for evidence synthesis (*advantage*), and highlighting the validity of these methods to inform clinical and policy decision-making (*compatibility*).28 31–34 We noted rapid uptake since 2011, coinciding with the publication of guidelines and reviews on these methods by health technology assessment and reimbursement agencies (eg, Canadian Agency for Drugs and Technologies in Health (CADTH), Haute Autorité de Santé, Institute for Quality and Efficiency in Health Care, Pharmaceutical Benefits Advisory Committee and Scottish Medicines Consortium) from many countries around the world.19 35–41 

Given the economic pressure on payers to better allocate healthcare resources, many regulatory agencies have been calling for the use of comparative effectiveness research to assist in drug accessibility and reimbursement decisions.40 41 In addition, applications focused on drug efficacy tie into payer demands for more cost-effectiveness analyses of newly marketed drugs in comparison with competing or existing therapies. For example, the CADTH has published guidance documents to facilitate best practices in the use of indirect comparison meta-analytic methods to assess clinical and economic value of drugs and other health technologies in Canada, including how to best incorporate these methods to inform clinical parameters in these types of evaluations.19 42 Consequently, many pharmaceutical companies and contract research organisations have started to apply these methods. For example, the International Society for Pharmaceutical Outcomes Research Indirect Comparisons Good Research Practice Task Force adopted methods and statistical code from the MPES Research Group to publish a two-part report to guide researchers, clinicians and policymakers on good research practices for indirect comparisons; given its value and increasing acceptance by regulatory agencies.6 15 Coauthors mainly comprised research experts from pharmaceutical companies and contract research organisation (including JP Jansen who collaborated with innovator AE Ades and coauthors from the MPES Research Group), which may have helped disseminate use of these methods into industry. In addition, publication of this report may partially explain rapid and large uptake from 2011 since coauthors from the two-part report were from multiple countries (Belgium, Canada, the Netherlands, the UK, the USA). We believe that this observation may have been a response to requests by these agencies, as we noted collaboration with core innovators from academia, and an increase in the number of industry-sponsored applications published from 2009.

Our findings demonstrated rapid increase in the use of indirect comparison meta-analytic methods in recent years, with contributions increasing worldwide. With 70% (n=90) of the coauthorship network comprising  single-paper components, and 81% (n=1121) of authors having published only a single paper, use of indirect comparison meta-analytic methods has indeed spread to many distinct research groups. However, uptake of these methods has been diffuse and highly disconnected when compared with the diffusion and early uptake of other methodological innovations,16 since many authors are publishing in isolation of each other (ie, smaller, single-paper components). In a prior study that examined the diffusion and early uptake of two confounder summary scores (the disease risk score and high-dimensional propensity score), only 19% and 11% of all eligible applications made up single-paper components in their respective coauthorship networks in comparison with 25% of all indirect comparison meta-analytic applications.16 Rapid and widespread use by academics, and more recently, government and industry, suggests that use of these methods has become diffuse and are no longer in the early stages of adoption, but rather, mainstream and accepted methods. As we also noted a lack of standardisation in the terminology used to describe the indirect comparison meta-analytic methods used, we encourage use of the term, *network meta-analysis*, as it is clearer than mixed treatment or multiple treatments meta-analysis, which may be assumed to indicate the concomitant administration of two or more drug therapies (eg, adjuvant therapy).

Our results are subject to some limitations. First, our analysis limited the coauthorship of empirical applications to English language papers identified in select bibliographic databases: the Cochrane Database of Systematic Reviews, EMBASE, MEDLINE, Scopus and Web of Science. The limitation of our search to these databases may have resulted in missed articles that were published in other languages, or identifiable in other bibliographic databases, such as Google Scholar, JSTOR, PubMed and RevMan5. Articles that did not clearly describe the techniques used to perform these methods were also excluded, since we could not assume that these methods were used. While we acknowledge that this may have resulted in the exclusion of some applications, we included articles that clearly described these methods in the title, abstract, introduction or methods section to allow for as much inclusion as possible. Consequently, we believe that our systematic search is both comprehensive and robust, as this is the largest and only search completed to date that examines the diffusion of indirect comparison meta-analytic methods in the study of drugs. However, it is worth noting that the term *matching-adjusted indirect comparison*, an extension of the adjusted indirect comparison which was introduced in 2010 and uses individual patient data from single-comparator RCTs to adjust for differences in patient characteristics across studies, was not considered in our analysis.43 However, six eligible papers were published using this term, and we expect to see an increase in the future.

Second, our study only ascribed country and institutional credit to the first and last authors of each paper. Although the first (principal) and last (often senior) authors traditionally contribute the most, and thus receive the most credit, for papers in the biomedical sciences, other coauthors in the authorship order may also help drive use of novel methods. Consequently, mapping contributions based on first and last authors may have resulted in missed contributions by other coauthors. Nonetheless, inclusion of the second authors in a previous study that examined the diffusion of two confounder summary scores found little impact on country and institutional credit.16 

Finally, our work did not examine the quality of eligible empirical articles, or explore the correlation and impact of early diffusion on the quality of indirect comparison meta-analytic methods. Given the large number of authors who published in isolation of each other, it is possible that the degree of interconnectedness between authors in the network may have influenced the quality of eligible applications, as inconsistencies in methodological and reporting quality of indirect comparison meta-analytic methods have been documented.18 Similar to traditional pairwise meta-analysis, limitations related to the quality of the search conducted, quality and heterogeneity of studies included and publication bias can all influence the quality of the study. Uniquely, indirect comparison meta-analytic methods have additional limitations that should be accounted for, such as issues with transitivity and inconsistency of networks, as well as the presentation of results.44 A recent systematic review of network meta-analyses in clinical research demonstrated improvement in methodological and reporting quality over time.45 However, we acknowledge that this is an important area of future research that should be explored.

In conclusion, prior research identified challenges with integrating new statistical methods into practice.46 47 We recently identified the importance of considering the five innovation attributes from Rogers’ Diffusion of Innovations Model to facilitate knowledge translation of new methods for rapid integration.16 In this paper, we used indirect comparison meta-analytic methods to examine the impact of social systems on the diffusion of novel methods. We demonstrated rapid adoption by effective consideration of innovation attributes by innovators, and rapid adoption due to collaboration between innovators from the UK and a large number of early adopters from many countries around the world. Although speculative, and while there are likely multiple reasons for the relatively rapid adoption of these methods, we believe that adoption by government agencies may have contributed to more rapid uptake, and is worth noting; though further research should be explored. We believe that the social system can play a major role in facilitating the adoption of innovative methods, here through regulation, and by the increase in demand by government for more comparative effectiveness research. As many health technology assessment and regulatory agencies have started to call for more evidence synthesis methods to assist in drug accessibility and reimbursement decisions,41 use of indirect comparison meta-analytic methods has become more widely accepted, and will likely continue to be a key tool for policy decision-making. We encourage authors of novel methods to consider the five innovation attributes when integrating new methods into practice (*relative advantage*, *compatibility*, *simplicity*, *trialability* and *observability*), with emphasis on early collaboration with potential adopters, such as government regulatory bodies.

### Supplementary file 1

[[bmjopen-2017-019110-SP1.mp4]](pending:yes) 

### Supplementary file 2

[[bmjopen-2017-019110-SP2.mp4]](pending:yes) 

## Acknowledgments

The authors thank Dave Dubins, PhD, for initial work in creating Excel macros for coauthorship matrix generation; and Gina Matesic, MLIS, for helping with search strategies.

## Footnotes

*   Contributors JKB, MT and SMC were responsible for the conception and design of this study. JKB, EAC and MT collected the data and verified the results with SMC. Data analysis was performed by JKB and AXL. JKB led the drafting of the manuscript in consultation with MT and SMC. All authors provided critical feedback and helped shape the research, analysis and manuscript.

*   Funding JKB was supported in part by the Canadian Institute of Health Research (CIHR) Drug Safety and Effectiveness Cross-Disciplinary Training Program. MT was supported in part by a CIHR Frederick Banting and Charles Best Canada Graduate Scholarship (GSD-11342). EAC was supported in part by a CIHR Health Professional Student Research Award. Participation at CAHSPR was supported by a CAHSPR Student Travel Bursary, and participation at ICPE was supported by an ICPE Travel Scholarship, the Leslie Dan Faculty of Pharmacy Student Experience Fund, and a University of Toronto School of Graduate Studies Conference Grant.

*   Competing interests None declared.

*   Patient consent Not required.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement Online technical appendix and supplemental materials are available at [/doi: 10.1136/bmjopen-2017-019110]

*   Presented at This research was presented at the Canadian Association for Health Services and Policy Research (CAHSPR) Annual Conference in Toronto, ON, Canada, May 2016, and at the International Conference on Pharmacoepidemiology and Therapeutic Risk Management (ICPE) in Dublin, Ireland, August 2016.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:©[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  1. Friedman HP , Goldberg JD . Meta-analysis: an introduction and point of view. Hepatology 1996;23:917–28.[doi:10.1002/hep.510230437](http://dx.doi.org/10.1002/hep.510230437) 
    
    

2.  2. O’Rourke K . An historical perspective on meta-analysis: dealing quantitatively with varying study results. J R Soc Med 2007;100:579–82.[doi:10.1177/0141076807100012020](http://dx.doi.org/10.1177/0141076807100012020) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1258/jrsm.100.12.579&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18065712&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000252093800019&link_type=ISI) 

3.  3. Jansen JP , Naci H . Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med 2013;11:159.[doi:10.1186/1741-7015-11-159](http://dx.doi.org/10.1186/1741-7015-11-159) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1741-7015-11-159&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23826681&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

4.  4. Burns PB , Rohrich RJ , Chung KC . The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg 2011;128:305–10.[doi:10.1097/PRS.0b013e318219c171](http://dx.doi.org/10.1097/PRS.0b013e318219c171) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/PRS.0b013e318219c171&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21701348&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

5.  5. Caldwell DM , Ades AE , Higgins JP . Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005;331:897–900.[doi:10.1136/bmj.331.7521.897](http://dx.doi.org/10.1136/bmj.331.7521.897) 
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMzEvNzUyMS84OTciO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vOC82L2UwMTkxMTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

6.  6. Jansen JP , Fleurence R , Devine B , et al . Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR task force on indirect treatment comparisons good research practices: part 1. Value Health 2011;14:417–28.[doi:10.1016/j.jval.2011.04.002](http://dx.doi.org/10.1016/j.jval.2011.04.002) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jval.2011.04.002&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21669366&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000299080800004&link_type=ISI) 

7.  7. Edwards SJ , Clarke MJ , Wordsworth S , et al . Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract 2009;63:841–54.[doi:10.1111/j.1742-1241.2009.02072.x](http://dx.doi.org/10.1111/j.1742-1241.2009.02072.x) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1742-1241.2009.02072.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19490195&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000266025900007&link_type=ISI) 

8.  8. Bucher HC , Guyatt GH , Griffith LE , et al . The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol 1997;50:683–91.[doi:10.1016/S0895-4356(97)00049-8](http://dx.doi.org/10.1016/S0895-4356(97)00049-8) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0895-4356(97)00049-8&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9250266&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1997XH67100008&link_type=ISI) 

9.  9. Song F , Altman DG , Glenny AM , et al . Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 2003;326:472.[doi:10.1136/bmj.326.7387.472](http://dx.doi.org/10.1136/bmj.326.7387.472) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjYvNzM4Ny80NzIiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vOC82L2UwMTkxMTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. 10. Lumley T . Network meta-analysis for indirect treatment comparisons. Stat Med 2002;21:2313–24.[doi:10.1002/sim.1201](http://dx.doi.org/10.1002/sim.1201) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.1201&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12210616&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000177458600002&link_type=ISI) 

11. 11. Lu G , Ades AE . Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004;23:3105–24.[doi:10.1002/sim.1875](http://dx.doi.org/10.1002/sim.1875) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.1875&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15449338&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000224307300002&link_type=ISI) 

12. 12. Lee AW . Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. J Clin Epidemiol 2014;67:138–43.[doi:10.1016/j.jclinepi.2013.07.014](http://dx.doi.org/10.1016/j.jclinepi.2013.07.014) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2013.07.014&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24090930&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

13. 13. Song F , Loke YK , Walsh T , et al . Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 2009;338:b1147.[doi:10.1136/bmj.b1147](http://dx.doi.org/10.1136/bmj.b1147) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzgvYXByMDNfMS9iMTE0NyI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi84LzYvZTAxOTExMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

14. 14. Donegan S , Williamson P , Gamble C , et al . Indirect comparisons: a review of reporting and methodological quality. PLoS One 2010;5:e11054.[doi:10.1371/journal.pone.0011054](http://dx.doi.org/10.1371/journal.pone.0011054) 
    
    

15. 15. Hoaglin DC , Hawkins N , Jansen JP , et al . Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR task force on indirect treatment comparisons good research practices: part 2. Value Health 2011;14:429–37.[doi:10.1016/j.jval.2011.01.011](http://dx.doi.org/10.1016/j.jval.2011.01.011) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jval.2011.01.011&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21669367&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000299080800005&link_type=ISI) 

16. 16. Cadarette SM , Ban JK , Consiglio GP , et al . Diffusion of Innovations model helps interpret the comparative uptake of two methodological innovations: co-authorship network analysis and recommendations for the integration of novel methods in practice. J Clin Epidemiol 2017;84:150–60.[doi:10.1016/j.jclinepi.2016.12.006](http://dx.doi.org/10.1016/j.jclinepi.2016.12.006) 
    
    

17. 17. Rogers EM . Diffusion of innovations. 5th edn. Toronto: Free Press, 2003.
    
    

18. 18. Bafeta A , Trinquart L , Seror R , et al . Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review. BMJ 2013;347:f3675.[doi:10.1136/bmj.f3675](http://dx.doi.org/10.1136/bmj.f3675) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDcvanVsMDFfMS9mMzY3NSI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi84LzYvZTAxOTExMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. 19. Wells GA , Sultan SA , Chen L , et al . Indirect evidence: indirect treatment comparisons in meta-analysis. Ottawa: Canadian Agency for Drugs and Technologies in Health, 2009.
    
    

20. 20. Lu G , Ades AE , Sutton AJ , et al . Meta-analysis of mixed treatment comparisons at multiple follow-up times. Stat Med 2007;26:3681–99.[doi:10.1002/sim.2831](http://dx.doi.org/10.1002/sim.2831) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.2831&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17285571&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000249032700001&link_type=ISI) 

21. 21. Salanti G , Higgins JP , Ades AE , et al . Evaluation of networks of randomized trials. Stat Methods Med Res 2008;17:279–301.[doi:10.1177/0962280207080643](http://dx.doi.org/10.1177/0962280207080643) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/0962280207080643&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17925316&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

22. 22. Glenny AM , Altman DG , Song F , et al . Indirect comparisons of competing interventions. Health Technol Assess 2005;9:1–134.[doi:10.3310/hta9260](http://dx.doi.org/10.3310/hta9260) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15763038&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

23. 23. Salanti G , Ades AE , Ioannidis JP . Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol 2011;64:163–71.[doi:10.1016/j.jclinepi.2010.03.016](http://dx.doi.org/10.1016/j.jclinepi.2010.03.016) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2010.03.016&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20688472&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000287281300007&link_type=ISI) 

24. 24. Wilhelmus KR . The treatment of herpes simplex virus epithelial keratitis. Trans Am Ophthalmol Soc 2000;98:505–32.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11190039&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

25. 25. Packer M , Antonopoulos GV , Berlin JA , et al . Comparative effects of carvedilol and metoprolol on left ventricular ejection fraction in heart failure: results of a meta-analysis. Am Heart J 2001;141:899–907.[doi:10.1067/mhj.2001.115584](http://dx.doi.org/10.1067/mhj.2001.115584) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1067/mhj.2001.115584&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11376302&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000169033500005&link_type=ISI) 

26. 26. Sauriol L , Laporta M , Edwardes MD , et al . Meta-analysis comparing newer antipsychotic drugs for the treatment of schizophrenia: evaluating the indirect approach. Clin Ther 2001;23:942–56.[doi:10.1016/S0149-2918(01)80082-5](http://dx.doi.org/10.1016/S0149-2918(01)80082-5) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0149-2918(01)80082-5&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11440294&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

27. 27. Chalkidou K . Comparative effectiveness review within the U.K.’s National Institute for Health and Clinical Excellence. Issue Brief 2009;59:1–11.
    
    

28. 28. Caldwell DM , Welton NJ , Ades AE . Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency. J Clin Epidemiol 2010;63:875–82.[doi:10.1016/j.jclinepi.2009.08.025](http://dx.doi.org/10.1016/j.jclinepi.2009.08.025) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2009.08.025&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20080027&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000279439900012&link_type=ISI) 

29. 29. Dias S , Welton NJ , Caldwell DM , et al . Checking consistency in mixed treatment comparison meta-analysis. Stat Med 2010;29:932–44.[doi:10.1002/sim.3767](http://dx.doi.org/10.1002/sim.3767) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.3767&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20213715&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000276219000023&link_type=ISI) 

30. 30.University of Bristol. Multi-parameter Evidence Synthesis Research. Multi-parameter Evidence Synthesis Research Group | School of Social and Community Medicine | University of Bristol. 2017 [http://www.bristol.ac.uk/social-community-medicine/centres/cresyda/mpes/](http://www.bristol.ac.uk/social-community-medicine/centres/cresyda/mpes/) (accessed 23 Jul 2017).
    
    

31. 31. Dias S , Sutton AJ , Ades AE , et al . Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Making 2013;33:607–17.[doi:10.1177/0272989X12458724](http://dx.doi.org/10.1177/0272989X12458724) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/0272989X12458724&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23104435&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000320986600002&link_type=ISI) 

32. 32. Sutton A , Ades AE , Cooper N , et al . Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 2008;26:753–67.[doi:10.2165/00019053-200826090-00006](http://dx.doi.org/10.2165/00019053-200826090-00006) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2165/00019053-200826090-00006&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18767896&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000259794300006&link_type=ISI) 

33. 33. Dias S , Sutton AJ , Welton NJ , et al . Evidence synthesis for decision making 3: heterogeneity--subgroups, meta-regression, bias, and bias-adjustment. Med Decis Making 2013;33:618–40.[doi:10.1177/0272989X13485157](http://dx.doi.org/10.1177/0272989X13485157) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/0272989X13485157&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23804507&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000320986600003&link_type=ISI) 

34. 34. Dias S , Welton NJ , Sutton AJ , et al . Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials. Med Decis Making 2013;33:641–56.[doi:10.1177/0272989X12455847](http://dx.doi.org/10.1177/0272989X12455847) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/0272989X12455847&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23804508&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000320986600004&link_type=ISI) 

35. 35.Haute Autorité de Santé. Indirect comparisons: methods and validity. Saint-Denis: Haute Autorité de Santé, 2009.
    
    

36. 36.Institute for Quality and Efficiency in Health Care. General methods (Version 4.1). Cologne: Institute for Quality and Efficiency in Health Care, 2013.
    
    

37. 37.Indirect Comparisons Working Group. Report of the indirect comparisons working group to the pharmaceutical benefits advisory committee: assessing indirect comparisons. Canberra: Australian Government Department of Health and Ageing, 2013.
    
    

38. 38.Pharmaceutical Benefits Advisory Committee. Guidelines for preparing submissions to the pharmaceutical benefits advisory committee (version 4.3). Canberra: Australian Government Department of Health and Ageing, 2008.
    
    

39. 39.National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013, process and method guides: National Institute for Healh and Care Excellence, 2013.
    
    

40. 40. Lebioda A , Gasche D , Dippel FW , et al . Relevance of indirect comparisons in the German early benefit assessment and in comparison to HTA processes in England, France and Scotland. Health Econ Rev 2014;4:31.[doi:10.1186/s13561-014-0031-5](http://dx.doi.org/10.1186/s13561-014-0031-5) 
    
    

41. 41. Laws A , Kendall R , Hawkins N . A comparison of national guidelines for network meta-analysis. Value Health 2014;17:642–54.[doi:10.1016/j.jval.2014.06.001](http://dx.doi.org/10.1016/j.jval.2014.06.001) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jval.2014.06.001&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25128059&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 

42. 42.Use of network meta-analysis to inform clinical parameters in economic evaluation. Ottawa: Canadian Agency for Drugs and Technologies in Health.
    
    

43. 43. Signorovitch JE , Wu EQ , Yu AP , et al . Comparative effectiveness without head-to-head trials: a method for matching-adjusted indirect comparisons applied to psoriasis treatment with adalimumab or etanercept. Pharmacoeconomics 2010;28:935–45.[doi:10.2165/11538370-000000000-00000](http://dx.doi.org/10.2165/11538370-000000000-00000) 
    
    

44. 44. Cipriani A , Higgins JP , Geddes JR , et al . Conceptual and technical challenges in network meta-analysis. Ann Intern Med 2013;159:130–7.[doi:10.7326/0003-4819-159-2-201307160-00008](http://dx.doi.org/10.7326/0003-4819-159-2-201307160-00008) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-159-2-201307160-00008&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23856683&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000322112000019&link_type=ISI) 

45. 45. Petropoulou M , Nikolakopoulou A , Veroniki AA , et al . Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol 2017;82:20–8.[doi:10.1016/j.jclinepi.2016.11.002](http://dx.doi.org/10.1016/j.jclinepi.2016.11.002) 
    
    

46. 46. Altman DG , Goodman SN . Transfer of technology from statistical journals to the biomedical literature. Past trends and future predictions. JAMA 1994;272:129–32.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.1994.03520020055015&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8015123&link_type=MED&atom=%2Fbmjopen%2F8%2F6%2Fe019110.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1994NV42400013&link_type=ISI) 

47. 47. Pullenayegum EM , Platt RW , Barwick M , et al . Knowledge translation in biostatistics: a survey of current practices, preferences, and barriers to the dissemination and uptake of new statistical methods. Stat Med 2016;35:805–18.[doi:10.1002/sim.6633](http://dx.doi.org/10.1002/sim.6633)