Article Text
PDF
Abstract
Objective Cluster randomised trials (CRTs) are used for evaluating health-related interventions in low-income and middle-income countries (LMICs) but raise complex ethical issues. To inform the development of future ethics guidance, we aim to characterise CRTs conducted exclusively in LMICs by examining the types of clusters, settings, author affiliations and primary clinical focus and to evaluate adherence to trial registration and ethics reporting requirements over time.
Design A systematic scoping review using the Preferred Reporting Items for Systematic Review and Meta-Analyses Extension for Scoping Reviews.
Data sources We searched MEDLINE between 1 January 2017 and 17 August 2022.
Eligibility criteria for selecting studies We included primary reports of CRTs evaluating health-related interventions, conducted exclusively in LMICs and published in English between 2017 and 2022.
Data extraction and synthesis Data were extracted by one reviewer; a second reviewer verified accuracy by extracting data from 10% of the reports. Results were summarised overall and categorised by country’s economic level or publication year.
Results Among 800 identified CRTs, 400 (50.0%) randomised geographical areas and 373 (46.6%) were conducted in Africa. 30 (3.7%) had no authors with an LMIC affiliation, and 246 (30.8%) had neither first nor last author with an LMIC affiliation. The relative frequency of first or last authors holding an LMIC affiliation increases as a country’s economic level increases. Most CRTs focused on reducing maternal and neonatal disorders (106, 13.3%). 670 (83.8%) CRTs reported trial registration, 786 (98.2%) reported research ethics committee review and 757 (94.6%) reported consent statements. Among the 757 CRTs, 46 (6.1%) reported a waiver or no consent and, among these, 10 (21.7%) did not provide a rationale. Gatekeepers were identified in 403 (50.4%) CRTs. No meaningful trends were observed in adherence to trial registration or ethics reporting requirements over time.
Conclusion Our findings suggest existing inequity in authorship practices. There is high adherence to trial registration and ethics reporting requirements, although greater attention to reporting a justification for using a waiver of consent is needed.
- Systematic Review
- MEDICAL ETHICS
- Randomized Controlled Trial
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This is the largest review of cluster randomised trials (CRTs) to date and it provides a comprehensive overview of CRTs conducted exclusively in low-income and middle-income countries.
We used the Preferred Reporting Items for Systematic Review and Meta-Analyses Extension for Scoping Reviews to report our methods and results.
Stratifying results allowed for comparison across low-income, lower-middle-income and upper-middle-income country categories.
Our search was limited to MEDLINE and the English language and using other databases and languages may have identified additional trials but our substantive conclusions would likely not have changed.
Single data extraction was used, although there was high inter-reviewer agreement on a subset of 80 trials used to ensure accuracy and consistency of data extraction.
Introduction
Cluster randomised trials (CRTs) are an important research design for evaluating interventions to address acute and chronic health and public health issues.1 As opposed to individually randomised trials that randomly allocate individuals to study interventions, CRTs randomly allocate groups of people or ‘clusters’ to study interventions. CRT designs are often used when the intervention must be delivered to a group rather than an individual (eg, vector control interventions), when evaluating the direct and indirect effects of interventions such as vaccines, and when there is substantial risk of spillover effects across study arms resulting in attenuation of intervention effects. These designs might also be particularly useful in low-income and middle-income counties (LMICs), as they offer logistical and practical advantages over individual randomisation, such as facilitating the fieldwork and intervention delivery across large geographical areas and lowering implementation costs.2
Besides the well-documented methodological issues with CRTs, such as their statistical inefficiency and increased risks of bias compared with individually randomised trials,3–8 these trials raise complex ethical issues.9 The Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials—the most comprehensive international ethics guidance document specific to CRTs—proffers 15 recommendations to address ethical issues raised by CRTs (see online supplemental table 1 for Ottawa Statement recommendations).10
Supplemental material
One of the recognised limitations of the Ottawa Statement is the under-representation of LMIC perspectives. The authors ‘recommend that subsequent versions include greater LMIC representation,’10 and, since its publication, various efforts have focused on greater collaboration with LMIC representatives to identify issues specific to CRTs in LMICs in need of further guidance.1 11–14 To further assist with the identification of unique ethical and methodological issues and inform the forthcoming update of the Ottawa Statement, we conducted a systematic scoping review to describe CRTs in LMICs and create a database of trials that can serve as a rich resource for further in-depth analyses. Specific objectives are: (1) to characterise the types of clusters, settings, author affiliations and primary clinical focus of CRTs in LMICs; (2) to describe adherence to trial registration and ethics reporting requirements—specifically the Ottawa Statement recommendations on research ethics committee review, informed consent and gatekeeping; and (3) to explore variations across income strata and over time.
Methods
We report our methods and results according to the Preferred Reporting Items for Systematic Review and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR).15 Our PRISMA-ScR checklist is available in online supplemental table 2.
Search strategy
Our search filter (shown in online supplemental table 3) was adapted from a search used in a previously published systematic review of CRTs evaluating public health interventions in LMICs.16 The adaptation involved removing public health terms, adding the names of LMICs (as of 16 August 2023), and, for efficiency, superimposing the Cochrane Highly Sensitive Search Strategy for identifying randomised controlled trials.17 We implemented the search in MEDLINE to identify trials published between 1 January 2017 and 17 August 2022 (the date of the search). We limited our search to MEDLINE, which was considered suitable for capturing a large, representative sample of CRTs in LMICs. We chose 1 January 2017 as the start date because this marks about 5 years since the publication of the Ottawa Statement, providing ample opportunity for the dissemination of and adherence to ethics guidance and reporting requirements.
Inclusion and exclusion criteria
Studies were eligible for inclusion if they met the following criteria: (1) published in English language, (2) primary reports of CRTs, (3) evaluated health-related interventions and (4) conducted exclusively in LMICs, classified by the World Bank as low-income, lower-middle-income or upper-middle-income countries.18 Primary reports were defined as presenting the primary outcomes of the trial. Our strategy for identifying primary reports is reported elsewhere.19 Health-related interventions were defined as health, healthcare or public health interventions or implementation strategies. Studies were excluded if they met any of the following criteria: (1) no human participants or evaluated a medical education intervention with simulated patients; (2) clusters were households, dyads or families; (3) clusters were individuals with measures on multiple body parts; (4) there was further random or non-random allocation of participants within randomised clusters; or (5) described a study within a trial. Conference abstracts, design papers, editorials, protocols or design papers, pilot and feasibility studies, secondary analyses and process evaluations were also excluded. The full list of inclusion and exclusion criteria is available in online supplemental table 4.
Study selection
All identified records were uploaded into Covidence software.20 Title and abstract screening of each record was completed independently by two of four reviewers (AS, CEG, MT and YM) with discrepancies being resolved during regular consensus meetings. Six reviewers (AS, CEG, JFS, LQ, MT and YM) proceeded with full-text screening of all potentially eligible studies, with two independent reviewers screening each record and discrepancies being resolved during regular consensus meetings.
Data extraction
A data extraction form was developed and then pilot tested by 3 reviewers (CEG, MT and YM) using 25 randomly selected eligible records. The final extraction form was uploaded into Covidence software. The form is available in online supplemental table 5. One reviewer (YM) proceeded to extract data from all records, while a second reviewer (CEG) extracted data from 80 records in batches of 10 or 15 records every 2 weeks over 12 weeks to ensure accurate and consistent data extraction. If differences could not be resolved between reviewers, a third reviewer (MT) was considered the final arbitrator.
To address objective 1 (characterise the types of clusters, settings, author affiliations and primary clinical focus of CRTs in LMICs), data were extracted on type of CRT design (eg, crossover, factorial, parallel arm, stepped wedge), type of cluster (eg, primary care clinics or settings, hospitals or specialised care settings, geographical areas) and country or countries of trial conduct. We reviewed authors’ affiliations to determine whether authors of the primary CRT report held LMIC affiliations, and whether first and last authors were affiliated with high-income country institutions, LMIC institutions or both. We recorded detailed information about the primary focus of each CRT using the Institute for Health Metrics and Evaluation’s Global Burden of Disease 2019 Cause and Risk Summaries.21 We classified the stated primary trial objective in each CRT into one of the three Global Burden of Disease Level I categories; namely, whether the primary objective was (1) to reduce the prevalence or incidence of a disease of health problem, (2) to reduce or prevent health risks or (3) to manage impairments. Trials that could not be classified as (1), (2) or (3) were classified as ‘other’. Within each level 1 category, we selected the most relevant level 2 and level 3 subcategories from a drop-down list.
To address objective 2 (examine adherence to trial registration and ethics reporting requirements), data were extracted on whether trials were reported to be registered anywhere in the abstract or full text and, if so, in which trial registries. We extracted whether the study reported research ethics committee approval, exemption or non-submission, and justifications for receiving an exemption from or not submitting a protocol for research ethics committee review. If research ethics committee approval was obtained, we documented the location of the research ethics committee (ie, host country, sponsor county or both). We identified whether a clear statement about participant informed consent was reported. If reported, we extracted from whom consent was obtained (ie, individual-level participants or professional-level participants) or if consent was waived or otherwise not obtained. If consent was reported as waived or otherwise not obtained, any justifications or rationales provided were documented and subsequently categorised. We extracted information on whether a gatekeeper had a reported role and, if so, we documented and post hoc categorised their role in the trial by inductive analysis.
Data analysis
A descriptive analysis was used to summarise results using frequencies and percentages for all categorical variables, and medians and interquartile ranges for continuous variables with skewed distributions. Results were presented overall and stratified by World Bank classifications (ie, low income, lower-middle income, upper-middle income) and publication year (2017–2018, 2019–2020, 2021–2022). For all our analyses, we used R (V.4.3.1).
Patient and public involvement
Patients and members of the public were not involved in the design, conduct, reporting or dissemination plans of this study.
Results
Our search strategy identified 3381 records. After duplicates were removed, 3355 records underwent title and abstract screening. Inter-rater agreement on which studies met the criteria for full-text screening was high (raw percentage agreement 87.4%; kappa statistic 0.83, 95% CI: 0.91 to 1.00) and resulted in 2226 records being excluded. After full-text screening of the remaining 1129 records, 800 CRTs were included in our review. The study flow diagram is presented in figure 1.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.
Trial characteristics
Types of CRT designs included parallel arm (716, 89.5%), stepped wedge (45, 5.6%), factorial (30, 3.8%), crossover (8, 1.0%) and parallel adaptive (1, 0.1%). Clusters were most commonly geographical areas (400, 50.0%), primary care clinics or settings (167, 20.9%) and schools or classrooms (110, 13.8%). The three countries in which most CRTs were conducted were India (83, 10.4%), China (78, 9.8%) and Kenya (53, 6.6%). Table 1 presents descriptive characteristics of the 800 CRTs included in our review, online supplemental table 6 presents a full breakdown of CRT conduct by country.
Descriptive characteristics of n=800 cluster randomised trials included in the review
Among 800 CRTs, 30 (3.7%) had no authors affiliated with an institution based in an LMIC, while 124 (15.5%) were published by authors who were all affiliated with an institution based in an LMIC. First author affiliations were exclusively within high-income countries in 338 (42.2%) CRTs, exclusively with LMICs in 336 (42.0%) and both high-income country and LMIC affiliations in 126 (15.8%). Last author affiliations were exclusively with high-income countries in 410 (51.2%) CRTs, exclusively with LMICs in 303 (37.9%) and both high-income country and LMIC affiliations in 87 (10.9%). The prevalence of first and last author LMIC affiliation increases as country’s economic level increases. In 246 (30.8%) CRTs, neither first nor last author had an LMIC affiliation. Table 2 provides the descriptive summary of author affiliations.
Overview of author affiliations among n=800 cluster randomised trials included in the review
The most commonly reported primary focus among the 800 CRTs was maternal and neonatal disorders (106, 13.3%), followed by HIV/AIDS and other sexually transmitted infections (91, 11.4%) and malaria and other neglected tropical diseases (88, 11.0%). Figure 2 presents the top 10 primary foci of CRTs in LMICs. Online supplemental table 7 presents a frequency distribution of the primary focus of each CRT, including the Global Burden of Disease level 1, 2 and level 3 categories.21
Top 10 primary clinical focus of n=800 cluster randomised trials in low-income and middle-income countries.* Categories taken from the Institute for Health Metrics and Evaluation’s Global Burden of Disease 2019 Cause and Risk Summaries.
Trial registration and ethics reporting
Table 3 presents information relevant to trial registration and ethics reporting. Among 800 CRTs, 670 (83.8%) reported registration in 19 different trial registries. Among the 670 CRTs reporting registration, 642 (95.8%) reported registration in one registry and 28 (4.2%) in two different registries.
Adherence to trial registration and ethics reporting requirements in n=800 cluster randomised trials included in the review
A statement about research ethics committee approval was reported in 786 (98.1%) CRTs. Of these, 452 (57.5%) reported review from both host and sponsor country, 315 (40.1%) from host country only, 18 (2.3%) from sponsor country only and 1 (0.1%) reported review from a for-profit ethics committee not associated with either the host or sponsor country.
A statement about consent was reported in 757 (94.6%) CRTs. Among studies with a consent statement, 683 (90.2%) pertained to individual-level participants (eg, patients, students), 56 (7.4%) pertained to professional-level participants (eg, health providers, teachers), 16 (2.1%) pertained to both individual-level and professional-level participants and 2 (0.3%) were unclear.
Informed consent was reported as being obtained for all aspects of the trial in 711 (93.9%) CRTs, obtained for only some aspects of the trial (eg, data collection procedures) in 20 (2.7%) and not obtained or waived for all aspects of the trial in 25 (3.3%). One (0.1%) CRT reported obtaining written informed consent from participants enrolled at three study sites, while a waiver of consent was granted at 30 study sites.
For CRTs in which consent was not obtained or waived for some or all aspects of the trial, 36 (78.3%) provided a rationale. Rationales included the use of deidentified routinely collected data, administrative data or registry data (23, 63.9%), the research involves a standard of care or usual care intervention (6, 17.7%), consent was obtained from gatekeepers (4, 11.1%), the research involves a cluster-level intervention (3, 8.3%) and other (14, 38.9%).
A gatekeeper’s role was identified in 403 (50.4%) CRTs. Identified gatekeeper roles included: assisting with study implementation or intervention development; facilitating or involved in consultations, engagement activities or public events; identifying eligible clusters or participants; providing or withholding access to data or study intervention(s); providing or withholding permission for study conduct or to approach cluster members; providing proxy consent on behalf of cluster members; and reviewing or approving study protocols.
No meaningful changes over time were observed with respect to reporting of trial registration or consent.
Discussion
Summary of key findings
This systematic scoping review characterises the types of clusters, settings, author affiliations and primary clinical focus of CRTs in LMICs. Geographical areas were the most common units of randomisation. We found that most CRTs were conducted in India, China and Kenya. Almost all CRTs included at least one author with an LMIC affiliation, but a substantial minority had neither first nor last author with an LMIC affiliation. We also found that the relative frequency of all authors having exclusively LMIC affiliations increases as a country’s economic level increases and that the prevalence of the first or last author having exclusively LMIC affiliations increases as a country’s economic level increases. The most common primary clinical focus of CRTs in LMICs were to reduce the prevalence or incidence of maternal and neonatal disorders, HIV/AIDS and other sexually transmitted diseases and malaria or other neglected tropical diseases.
We found that most CRTs in LMICs report trial registration. We also found that adherence to ethics recommendations was high, although not ideal. Nearly all CRTs report a statement about research ethics committee review and approval. Almost all CRTs reported a statement about consent and, when reported, it was almost always clear from whom consent was obtained. However, when consent was reported as not obtained or waived, a substantial minority of CRTs did not report a justification. Many of the provided justifications were not consistent with or explicitly rejected by the Ottawa Statement recommendations.
Implications of key findings
One of the most common justifications for adopting a CRT design over an individually randomised design is to avoid contamination between study arms.22 When geographical areas are selected as the unit of randomisation—which we found to be typical of CRTs in LMICs—there is often a substantial risk of spillover effects caused by migration of individuals between clusters that can undermine the scientific validity of a CRT. Future in-depth analyses should explore any efforts to avoid or mitigate contamination between geographical clusters in these trials.
First and last authorship positions are considered prestigious and are often used by funding agencies reviewing grants and by institutions evaluating applications for promotion. Yet, researchers affiliated with LMIC institutions are often ‘stuck in the middle when it comes to global health authorship resulting from international partnerships’.23 Our findings suggest that this holds true for many CRTs in LMICs. To take steps towards rectifying inequities in authorship, researchers affiliated with high-income-country institutions should, at the very least, create more opportunities for local LMIC-affiliated collaborators to contribute to CRTs in ways that lead to first and last authorship.24
The primary focus of CRTs in LMICs are oriented towards communicable, maternal, neonatal and nutritional diseases, which corresponds to the focus of the global donor community during the Millennium Development Goals period (2000–2015).25 Since we did not explore this issue in our data, we can only speculate that many of these studies are donor driven. Priorities should evolve, if they have not started to do so already, to address the ‘rapid rise in non-communicable diseases and disabilities’ that currently account for much of the burden of disease worldwide (with the exception of countries early in the epidemiological transition).26 The focus of future CRTs in LMICs should correspond to local burden of disease that are of national importance in the Sustainable Development Goals period (2016–2030).27
Finally, almost all CRTs in LMICs were compliant with trial registration and ethics reporting recommendations. Yet, there is room for improved reporting of justifications for not obtaining consent or using a waiver of consent. A probable reason for poor reporting is that it is not required by most journals or trial reporting guidelines. However, the Consolidated Standards of Reporting Trials extension for stepped-wedge CRTs states, ‘When a waiver or modification of consent has been granted by a research ethics committee, it should be reported and a justification given. It should be clear whose consent was waived and whether the waiver pertains to study participation, data collection or both’.28 Other reporting guidelines ought to be updated to improve reporting of consent practices in CRTs.
Comparison with other studies
This is the first study to describe the characteristics of CRTs conducted exclusively in LMICs. Two previous studies have also examined the characteristics of CRTs and ethics reporting, although none is specific to LMICs: a review of 300 primary CRT reports published between 2000 and 200829; and a review of 173 articles reporting primary or secondary analysis of CRTs published in 2008.30 These reviews found that 15%–25% of CRTs are conducted in LMICs. Consistent with our findings, these reviews also found that geographical areas are commonly the units of randomization in CRTs; yet, CRTs in LMICs are more likely to randomise geographical areas (50%) than CRTs in general (15%).29
In terms of trial registration requirements, Odutayo et al’s cross-sectional study of 1122 primary reports of randomised controlled trials published in December 2012 found that only 593 (52.9%) reported trial registration.31 Among the 1122 trials, 31 (2.8%) were CRTs and, of these, 17 (54.8%) reported trial registration. Our review suggests that CRTs in LMICs are more likely to report trial registration in the primary report than randomised controlled trials and CRTs in general or that a substantial improvement in trial registration has occurred over the last 10 years.
Finally, in terms of ethics reporting, the two aforementioned reviews found that 73%–90% of CRTs include a statement about research ethics committee approval and 69%–83% include a statement about informed consent.29 30 Compared with these earlier reviews, our review found a higher percentage of reporting research ethics committee approval (98.2%) and informed consent (94.6%), which suggests that adherence to ethics reporting requirements may be higher in CRTs conducted exclusively in LMICs or that an improvement has occurred over time.
Strengths and limitations
Our study is the largest review of CRTs conducted to date and provides a comprehensive overview of CRTs in LMICs. It has also created a large database of CRTs that can serve as a rich recourse for further in-depth analysis. Our study has some limitations. First, by excluding CRTs conducted in high-income countries from the search and only included trials conducted exclusively in LMICs, we cannot compare the two sets of trials. This limits our ability to accurately identify ethical issues that are unique to trials conducted in LMICs. However, stratifying results allowed for some comparison between LMIC categories. Second, our literature search was limited to MEDLINE and the English language; using other databases and languages may have identified additional trials. However, since our study objectives relate to a general characterisation of CRTs conducted in LMICs and a description of their adherence to trial registration and ethics reporting requirements, capturing a large, representative sample of such trials was considered adequate. Searching other databases and including non-English primary reports of CRTs would have been unlikely to yield substantively different conclusions. Third, single data extraction was used since the scope of this review would render duplicate review unmanageable. To limit misclassification, the primary extractor underwent multiple rounds of training prior to commencing data extraction and the extraction form provided clear guidance and examples. There was also high inter-reviewer agreement on the subset of 80 trials that were used to ensure accuracy and consistency of data extraction.
Conclusion
In this review, which is part of a larger project to update the Ottawa Statement, we compiled a large database of 800 CRTs conducted exclusively in LMICs, characterised the types of clusters, settings, author affiliations and primary clinical focus of these trials and evaluated their adherence to trial registration and ethics reporting requirements. Our findings suggest existing inequity in authorship practices. We also found high adherence to trial registration and ethics reporting requirements, although greater attention to reporting a justification for using a waiver of consent is needed. Future secondary analyses will examine specific ethical and methodological issues in more detail to ensure that the updated Ottawa Statement recommendations are applicable to all CRTs irrespective of location, while also providing recommendations to address unique issues raised by CRTs in LMICs.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors would like to thank Laura Quinn (LQ) for assisting with full-text screening and Spencer Phillip Hey (SPH) for assisting with ClincalTrials.gov data extraction and linkage.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @charlesweijer
Contributors CEG, CW and MT conceived of the project and developed the extraction form with MJ, SGN and YM. Record screening was completed by AS, CEG, JFS, LQ, MT and YM. YM completed data extraction and the statistical analysis with assistance from CEG and MT. CEG wrote the first draft of the manuscript with input from CW, JFS, MT and YM and was responsible for revisions of subsequent drafts. AS, CW, FA, JFS, KH, LM, MJ, MT, RAF, RvdG, SGN, ST, VAW and YM provided feedback on subsequent drafts, had full access to all data and approved the decision to submit for publication. CEG is the guarantor of the manuscript.
Funding This work was supported by a Canadian Institutes of Health Research (CIHR) project grant (PJT-479757). CEG is supported by a CIHR Fellowship Award. These funding sources had no role in the study design, data collection, data analysis, data interpretation, writing of the report or decision to publish.
Competing interests CW receives consulting income from Cardialen and Eli Lilly & Company. Other authors declare no competing interests.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.