Article Text
Abstract
Objective We aim to assess which variables are associated with recruitment failure of obstetrical and gynaecological randomised controlled trials (RCTs), leading to an extension of the study period.
Design Nationwide study.
Setting A cohort of RCTs supported by the trial centre of the Dutch Consortium of Obstetrics and Gynaecology.
Population We included 83 RCTs that recruited patients between 1 March 2003 and 1 December 2023.
Main outcome measures Main outcome was recruitment target not achieved within 6 months after the preplanned recruitment period. Secondary outcomes were recruitment target not achieved within an extension period of at least 12 months and premature termination of the trial. In all RCTs, we collected information on variables with a potential effect on recruitment failure, recorded at five levels; patient, doctor, participating centre, study organisation and study design.
Results In total, 46 of 83 RCTs (55%) did not achieve their targeted recruitment within the preplanned study period with a maximal extension period of 6 months. The most relevant variables for recruitment failure in multivariable risk prediction modelling were presence of a no-treatment arm (where treatment is standard clinical practice), a compensation fee of less than €200 per included patient, funding of less than €350 000, while a preceding pilot study lowered this risk.
Conclusions We identified that the presence of a no-treatment arm, low funding and a low compensation fee per included patient were the most relevant risk factors for recruitment failure within the preplanned period, while a preceding pilot study lowered this risk. Awareness of these variables is important when designing future studies.
- obstetrics
- gynaecology
- randomized controlled trial
- reproductive medicine
- epidemiology
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Recruitment failure was assessed in a nationwide collection of randomised controlled trials (RCTs) performed within a standardised setting with support and monitoring by the same clinical trial centre.
This study was able to assess all infrastructural variables with a potential association with poor recruitment as described in literature.
The study is limited by the number of trials.
The standardised setting may limit the generalisability as many RCTs are conducted in settings without such an infrastructure.
A limitation of the study was that it did not include patients’ or practitioners’ perspectives, which may affect recruitment.
Introduction
Randomised controlled trials (RCTs) are widely regarded as the gold standard for assessing the effectiveness of medical interventions and hold a leading position in the hierarchy of medical evidence.1 RCT outcomes are most often adopted into (inter) national clinical guidelines and have great influence on daily routine clinical practice. Unfortunately, obtaining evidence from RCTs is often hampered by failure to recruit enough patients within the preplanned study period, leading to premature termination of the trial or extension of the study period.2
Overall, a longer recruitment period may result in a shortage of resources possibly impacting the quality of the trial, limit the institutional capacity to start new RCTs, can postpone the availability of beneficial interventions, permit harmful or ineffective interventions to remain in use for longer than ethically warranted, thus hindering a conclusion with sufficient statistical power.3
Premature termination due to poor recruitment has been estimated to occur in 9%–10% of all RCTs.4–6 Variables that have been associated with poor recruitment leading to premature termination are an overestimation of the number of eligible patients, a preference for one of the interventions by the patients, a high burden of the tested intervention for the patients, an unclear trial design, strict eligibility criteria, a lack of logistic support or a lack of funding.7–10
While the variables that may result in poor recruitment leading to premature termination of the trial are known, much less is known on variables related to recruitment failure within the preplanned study period, leading to extension of the study period.
The one study to investigate this matter explored factors associated with recruitment in a cohort of 114 multicentre RCTs in more than nine clinical areas, including cancer, cardiology, and obstetrics and gynaecology (18 RCTs had a clinical area classified as ‘other’), and was funded by two public bodies in the UK; the UK Medical Research Council (MRC) and the Health Technology Assessment (HTA) Programme.6 RCTs that were funded by the MRC (as compared with the HTA) and were in the clinical area ‘cancer’, had better chances of good recruitment, which was a marginally statistically significant association. The vast heterogeneity of RCTs included in that study hampered the identification of other variables associated with poor recruitment and did not allow the authors to provide useful advice for improvement.
To assess factors that are associated with recruitment failure within the preplanned study period, we performed a nationwide cohort study of RCTs within the homogeneous setting of the Dutch Consortium of Obstetrics and Gynaecology in the Netherlands. Such knowledge is crucial for researchers, trial centres and funding agencies to prevent this type of recruitment failure.
Methods
Study design
This study was designed as a nationwide cohort study and included all multicentre RCTs carried out within the Dutch Consortium for Women’s Health Research, embedded within the professional society, that is, Dutch Society of Obstetrics and Gynaecology.11 The Dutch Consortium for Women’s Health Research facilitated studies in obstetrics, gynaecology and reproductive medicine.
Within the consortium, participating clinical centres are both academic and non-academic hospitals. RCTs conducted within the consortium are supported by a clinical trial centre (https://zorgevaluatienederland.nl/), a multidisciplinary trial bureau with methodologists, data managers, contract managers and trial managers. The trial centre staff supports research groups by advising on the budget, logistics, methods and ethics approval, developing electronic case record forms, performing contract management and monitoring, creating the interim reports for the data safety and monitoring board, and providing advice on the statistical analyses. The findings in our manuscript were reported according to the Strengthening the Reporting of Observational Studies in Epidemiology guideline.12
Study population
We included finalised multicentre RCTs supported by the clinical trial centre and performed within the Dutch Consortium for Women’s Health Research, between 1 March 2003 and 1 December 2023. We excluded studies with an observational design, single-centre RCTs, RCTs initiated outside the Netherlands, RCTs with a cluster or parallel study design, RCTs that never actually started, RCTs in which inclusion of patients was still ongoing and RCTs prematurely discontinued for other reasons than poor recruitment, for example, due to safety issues after an interim analysis.
Outcome measures
Main outcome was recruitment target not achieved within 6 months after the preplanned recruitment period. These RCTs were defined as RCTs with recruitment failure. The preplanned recruitment period was documented by the principal investigator before the start of the trial. Secondary outcomes included recruitment target not achieved within an extension period of at least 12 months and premature termination of the trial (defined as stopping with including patients before the recruitment target was achieved). All studies that recruited during the COVID-19 pandemic received 6 months extension of their recruitment period.
In all RCTs, we collected information on variables with a potential effect on recruitment failure, identified after a scoping review. We recorded variables at five levels; patient, doctor, participating centre, study organisation and study design (online supplemental appendix 1).
Supplemental material
Statistical analysis
For the primary outcome, we used the planned recruitment period as documented in the General Assessment and Registration form, a form that needs to be submitted to the ethical committee before actual start of the study. If we could not get access to this form, we retrieved this information from the main investigator and/or used the data mentioned in the protocol of the study. The actual recruitment period was calculated as the time between the first and last inclusion dates.
We checked the continuous potential variables with spline curve analysis. We dichotomised on the basis of the spline curve and used the median when the spline curve suggested a straight line. We used univariable logistic regression to evaluate the association between potential variables of recruitment failure and expressed these as ORs with corresponding 95% CIs.
To further explore the most relevant risk factors for recruitment failure, multivariable risk prediction modelling was done by using both forward and backward stepwise logistic regression including all predictors at once (entry p=0.2 and exclusion p=0.1) and expressed these as adjusted ORs with 95% CI.
We used SPSS V.25 (IBM 2019, USA) software for all statistical analyses.
Ethics approval
Our study focused on logistics and design issues and did not include patients as study participants. Consequently, we did not need ethical approval for this study.
Transparency statement
All authors had full access to all the data in the study and the corresponding author had final responsibility for the decision to submit for publication. The manuscript is an honest, accurate and transparent account of the study being reported, no important aspects of the study have been omitted, and any discrepancies from the study as originally planned have been explained.
Role of the funding source
This study was supported by a small departmental grant from the Centre for Reproductive Medicine, Amsterdam University Medical Centres, location AMC.
Public and patient involvement
No patients or members of the public were involved in this study.
Results
Between 1 March 2003 and 1 December 2023, 189 studies started recruitment and were assessed for eligibility. Of these, 106 studies did not fulfil our inclusion criteria, such that in total 83 RCTs were included in the analyses (figure 1). Characteristics of the included studies are summarised in table 1. Fifteen RCTs did not have funding at all (18%). A more detailed list of all RCTs can be found as online supplemental appendix 2.13–89
Supplemental material
Flow diagram of studies. *In four studies on advice of the Data Safety Monitoring Board due to potential safety issues, and in one study because of revised insights based on new evidence. **One study was a follow-up study of an RCT, three were implementation studies, one was a study to develop a decision tool, and one was a preference study. RCT, randomised controlled trial.
Characteristics of the included studies
Primary and secondary outcomes
In total, 46 of 83 RCTs (55%) did not achieve their targeted recruitment within the preplanned study period with a maximal extension period of 6 months (table 2). Recruitment was not achieved within the preplanned study period with a maximal extension period of 12 months in 41 RCTs (49%). Of these 41 RCTs, 29 studies had a total recruitment period of up to 5 years, and 12 RCTs finished their recruitment within 5–10 years.
Recruitment details in the studies with recruitment failure and those with successful recruitment
Nineteen RCTs (23%) stopped prematurely due to recruitment issues. Of these 19 RCTs, 4 studies reached 0%–10% of their recruitment target, 6 studies 10%–20%, 2 studies 20%–30%, 5 studies 30%–60% and 2 studies reached 70%–80% of their planned recruitment target.
The mean recruitment period was 50 months (range 12–96 months) for RCTs with recruitment failure versus 31 months (range 12–91 months) for RCTs without recruitment failure. Twenty-two RCTs had a recruitment period of over 48 months. The actual absolute recruitment rate was 4.5 inclusions per month in RCTs with recruitment failure compared with 18.5 inclusions per month in RCTs without recruitment failure (p<0.001).
Potential variables of recruitment failure
The association of the potential variables with RCTs with recruitment failure, that is, RCTs that did not achieve their recruitment target within the preplanned study period with a maximal extension period of 6 months, is shown in table 3.
Association with potential variables
Variables associated with higher chances of recruitment failure were presence of a no-treatment arm, having a design with more than two arms, a compensation fee of less than €200 per included patient, funding of less than €350 000 and having more than four inclusion criteria. One variable associated with lower chances on recruitment failure was a preceding pilot study. The most relevant variables for recruitment failure in multivariable risk prediction modelling were presence of a no-treatment arm (OR 4.95, 95% CI 1.18 to 20.80), a compensation fee of less than €200 per included patient (OR 2.90, 95% CI 1.02 to 8.25), funding of less than €350 000 (OR 2.99, 95% CI 1.05 to 8.51), while a preceding pilot study lowered the risk for treatment failure (OR 0.21, 95% CI 0.05 to 0.83).
When we compared the 41 RCTs that did not achieve their recruitment target within the preplanned study period with a maximal extension period of 12 months, with the 42 RCTs that completed recruitment within that period, the described associations with treatment failure remained comparable in direction and size.
The most relevant variables for stopping prematurely were the absence of a preceding pilot study and having a no-treatment arm. None of the 19 RCTs that stopped prematurely had performed a pilot study (0%), compared with 17 of the 62 RCTs that completed recruitment (27%). Ten of the 19 RCTs that stopped prematurely had a no-treatment arm (52%), compared with 8 of the 64 RCTs that completed recruitment (12.5%) (OR 6.13, 95% CI 1.98 to 19.06).
Discussion
Main findings
In this nationwide cohort study, 46 of 83 included RCTs (55%) did not achieve their recruitment target within the preplanned study period with a maximal extension period of 6 months. RCTs that had a no-treatment arm, low funding and low financial compensation per included patient were at risk to experience this type of recruitment failure, while a preceding pilot study lowered this risk. On extension of the preplanned study period from 6 months to 12 months, 41 RCTs (49%) still did not achieve the preplanned recruitment target. Nineteen RCTs (23%) were stopped prematurely because of recruitment issues.
Strengths and limitations
Our study has a number of strengths. First, we investigated recruitment failure in 83 RCTs embedded within the infrastructure of the Dutch Consortium for Women’s Health Research—and thus within one homogeneous discipline—with support and monitoring by the clinical trial centre. This allowed us to standardise several important aspects, like trial management and logistics, data collection and data monitoring. Second, we were able to assess all variables with a potential association with poor recruitment as described in literature; type of investigation, placebo-controlled study, treatment versus no treatment, whether the intervention was new or only available in the trial, whether the study was blinded or if there were any competing RCTs, number of study arms, number of inclusion and exclusion criteria, whether a pilot study was performed, number of participating centres, and funding and compensation per included patient.
The main limitation of our study is the number of trials. Obviously, if we could have accessed an even larger cohort of trials, we might have been able to identify more potential variables for recruitment failure. Furthermore, our study was done within a standardised setting which may limit the generalisability as many RCTs are conducted in settings without such an infrastructure. A further limitation may be that within our study we focused on objective variables, such as trial logistics and design issues. Other aspects, like patients’ or practitioners’ perspectives, which may affect recruitment as well were beyond the scope of our study.
In our trials, when the target number of patients was high, the prevalence was high as well. When writing up our protocol, it was decided that this prevalence should not be an input variable. We did a post hoc analysis and found no impact of target number on failure.
Interpretation
The design of a no-treatment arm where treatment is standard clinical practice was associated with recruitment failure. This design is particularly relevant, since we may be overtreating patients while we are actually in equipoise on whether the intervention is effective at all. Possibly, in this design specifically, the preference of the doctor or patient might play a role in the laborious recruitment. A no-treatment arm was also associated with stopping prematurely, supporting its relevance as a risk factor. In our study, 10 (52%) of 19 RCTs that stopped prematurely had a no-treatment arm where current clinical practice treatment is expected.
Not very surprisingly, the lack of funding and compensation fee per included patient (lack of funding and low funding) were associated with recruitment failure. Twelve studies with recruitment failure had no funding at all, compared with three studies without recruitment failure. Along with our finding that extending the recruitment period from 6 months to 12 months did only slightly increase the number of RCTs achieving their preplanned sample size, this has significant clinical, logistical and financial implications. RCTs may reach their recruitment target, but in 12 RCTs in our study, recruitment took up to 10 years. It implies that when recruitment is doomed to fail, it may reach its required sample size in the end, but at the expense of a lot of endurance and extra funding by a willing sponsor. On the other hand, RCTs can still be of extreme clinical importance if the research question is—and remains—relevant. This is shown by a trial that investigated low-molecular-weight heparin in women with recurrent pregnancy loss and inherited thrombophilia, which took seven and a half years years to recruit, but results were eagerly awaited and eventually published in a high impact journal.15
A preceding pilot study lowers recruitment failure, while a study design with more than two arms or more than four inclusion criteria might increase the chance of recruitment failure, although with a wide CI, perhaps due to small numbers. We believe that conducting a preliminary pilot study can help identify and address potential challenges before the actual study begins. Our results furthermore suggest that a study design involving more than two arms or over four inclusion criteria may complicate the recruitment process excessively. In a review of the literature on factors limiting the quality and progress of RCTs not hampered by recruitment failure, a straightforward study protocol and data collection as well as careful planning were also identified as key factors for completion.90
A competing study was not associated with a lower chance on recruitment failure, which is the opposite of what we expected. We hypothesise that when more RCTs in the same field are recruiting patients at the same time, clinicians are more aware of the possibility of including patients in a particular RCT, or when one RCT recruits rapidly, this might be ‘contagious’ for the other RCTs.
It is important to note that our results should not withhold clinicians from conducting RCTs. Investigating the efficacy and safety of treatments and providing robust evidence can be of the utmost importance. Although it is known that the results of randomised and non-randomised studies have a good correlation, non-randomised studies tend to show larger treatment effects, and thus observational studies can be a good adjunct to RCTs, but they cannot replace them.91 92 More importantly, our study also shows that RCTs with recruitment that takes many years may answer highly relevant clinical questions and can truly make a big difference in the clinical field. Principal investigators, sponsors and all who are participating in an RCT should be aware of the variables associated with poor recruitment, and that with dedication and persistence the RCT could be successfully completed and published.
Further research on how to improve recruitment efforts and increase the success of obstetrical and gynaecological RCTs is needed. It would also be relevant to explore differences in infrastructure and funding rules and whether these influence recruitment success. Additionally, future research should investigate the perspectives of both patients and practitioners on why participants decline to join RCTs. This research could consider factors such as treatment preferences, as well as patients’ fear, anxiety, mistrust in research, and challenges faced by low-income and non-English-speaking groups.
Conclusion
To conclude, RCTs with a no-treatment arm, low funding and low financial compensation per included patient are more likely to experience recruitment failure, while a preceding pilot study lowers this chance. We propose that investigators and grant providers consider these issues before the start of the actual recruitment of the study, to improve the chances of recruitment success. If a relevant trial is destined to have a suspected long recruitment period, it seems wise to ponder on the question whether to start the trial, or to accept a longer recruitment period with all its consequences.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors thank all the clinical and principal investigators of the included RCTs for cooperating in data collection, and Maya Kruijt, policy advisor Zorgevaluatie Nederland, for her advice.
References
Footnotes
X @n/a
Collaborators NVOG Consortium group: J Huirne, S Middeldorp, J B Derks, J A M van der Post, M A Oudijk, I M Custers, E Pajkrt, C Willekes, H C J Scheepers, A Kwee, M P Lambregtse-van den Berg, J J Duvekot, V Mijatovic, F J M Broekmans, A Hoek, J P de Bruin, C B Lambalk, M H Mochtar, S Mastenbroek, L Ramos, J P W R Roovers, M J E Mourits, H J van Beekhuizen, F Mol, A Vollebregt, C H van der Vaart, M Y Bongers, K B Kluivers, M E Vierhout, R C Painter, P M A J Geomini.
Contributors JR, MvW, MCvdW and RD conceived the study. JR and RC performed the scoping review, selected the potential variables and collected the data. Differences of opinion and questions regarding the data were resolved with MvW. JR was responsible for the data. JR, RC and MvW analysed the data. JR, MvW, MG and FvdV drafted the manuscript, supported by BWM. All authors contributed to the critical revision of the paper and approved the final manuscript. MvW is responsible for the overall content as guarantor.
The authors in the Consortium Group were project leaders on the included RCTs and all critically revised and approved the final mansucript.
Competing interests MG reports department research and educational grants from Guerbet and Ferring (location VUmc) outside the submitted work. BWM reports grants from NHMRC, personal fees from ObsEva, personal fees from Merck KGaA, personal fees from Guerbet, personal fees from iGenomix, outside the submitted work. The other authors report no competing interests.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.