You are here

  • Home
  • Archive
  • Volume 12, Issue 9
  • Embedded point of care randomisation for evaluating comparative effectiveness questions: PROSPECTOR-critical care feasibility study protocol

Article Text

Article menu
Health informatics
Protocol
Embedded point of care randomisation for evaluating comparative effectiveness questions: PROSPECTOR-critical care feasibility study protocol
  1. Matthew G Wilson1,
  2. Folkert W Asselbergs1,2,
  3. Ruben Miguel3,
  4. David Brealey4,5,
  5. Steve K Harris1,5
  1. 1Institute of Health Informatics, University College London, London, UK
  2. 2Department of Cardiology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
  3. 3Clinical Research Informatics Unit, Institute of Health Informatics, University College London, London, UK
  4. 4Bloomsbury Institute for Intensive Care Medicine, University College London, London, UK
  5. 5Critical Care Department, University College London Hospitals NHS Foundation Trust, London, UK
  1. Correspondence to Dr Matthew G Wilson; matthew.wilson8{at}nhs.net

Abstract

Introduction Many routinely administered treatments lack evidence as to their effectiveness. When treatments lack evidence, patients receive varying care based on the preferences of clinicians. Standard randomised controlled trials are unsuited to comparisons of different routine treatment strategies, and there remains little economic incentive for change.

Integrating clinical trial infrastructure into electronic health record systems offers the potential for routine treatment comparisons at scale, through reduced trial costs. To date, embedded trials have automated data collection, participant identification and eligibility screening, but randomisation and consent remain manual and therefore costly tasks.

This study will investigate the feasibility of using computer prompts to allow flexible randomisation at the point of clinical decision making. It will compare the effectiveness of two prompt designs through the lens of a candidate research question—comparing liberal or restrictive magnesium supplementation practices for critical care patients. It will also explore the acceptability of two consent models for conducting comparative effectiveness research.

Methods and analysis We will conduct a single centre, mixed-methods feasibility study, aiming to recruit 50 patients undergoing elective surgery requiring postoperative critical care admission. Participants will be randomised to either ‘Nudge’ or ‘Preference’ designs of electronic point-of-care randomisation prompt, and liberal or restrictive magnesium supplementation.

We will judge feasibility through a combination of study outcomes. The primary outcome will be the proportion of prompts displayed resulting in successful randomisation events (compliance with the allocated magnesium strategy). Secondary outcomes will evaluate the acceptability of both prompt designs to clinicians and ascertain the acceptability of pre-emptive and opt-out consent models to patients.

Ethics and dissemination This study was approved by Riverside Research Ethics Committee (Ref: 21/LO/0785) and will be published on completion.

Trial registration number NCT05149820.

  • STATISTICS & RESEARCH METHODS
  • Adult intensive & critical care
  • Health informatics
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi-org.ezproxy.u-pec.fr/10.1136/bmjopen-2021-059995

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Randomised trials integrated into clinical workflows have shown promise but require further feasibility testing to determine acceptability to patients and clinicians.

    • A mixed-methods approach allows combination of quantitative outcomes with explanatory qualitative data, increasing understanding of reasons underpinning success or failure of the intervention.

    • Testing study feasibility allows estimation of randomisation compliance, judges acceptability of the candidate research question, and allows optimisation of electronic prompt design prior to embarking on an adequately powered main trial.

    • As the study examines variation at an individual patient and clinician level, it is unclear how generalisable future study results will be outside the study centre.

    Introduction

    Every day, clinicians collectively make hundreds of thousands of decisions regarding the application of treatments and interventions in the care of patients. While some of these treatments will be guided by robust evidence from randomised controlled trials (RCTs), many ‘routine’ aspects of clinical care continue to lack a strong evidence base.1 Braithwaite et al describe this as the ‘60–30–10’ challenge—approximately 60% of administered treatments conform to evidence, 30% may be wasted or ineffective and 10% result in harm.2

    When evidence for an intervention is absent, clinicians vary in their decision making according to their experience and preferences.3 This variation is manifestly observable and can be seen across multiple domains from choice of surgical procedure,4 5 management of heart failure or diabetic ketoacidosis6 7 or administration of antibiotics and intravenous fluids.8 9

    Another commonly used treatment which varies in practice is the administration of supplemental magnesium for the prophylaxis of atrial fibrillation in critical care patients. While this practice is commonplace, the only evidence as to its effectiveness comes from the cardiac surgery population.10 Over time, this has been extrapolated to all critical care patients, without additional evidence of benefit. As such, clinicians vary in their threshold for routinely supplementing magnesium. Clinician behaviour will be consistent at extremes of serum magnesium measurements (never/always supplement), but within a ‘normal’ range the decision to supplement will have a random component linked to the clinician’s preference.11

    Variation in practice does not necessarily imply substandard care—it may be that the clinician’s experience offers benefits in optimising treatment delivery, or it may be that there is no meaningful difference between treatment choices. Under ideal conditions, clinicians would be able to learn from variation and improve the quality and coverage of evidence for future patients. Ineffective yet costly treatments could be minimised, and strategies demonstrating effectiveness targeted to ever smaller subgroups of patients.

    Unfortunately, generating new evidence from routine clinical decision making has proven difficult using existing research methodologies. RCTs, whilst well suited to demonstrating treatment efficacy in homogenised cohorts, under rigid treatment protocols, have proven costly and difficult to conduct in more pragmatic settings.12 While the classical RCT remains ideal for evaluating novel therapies, for treatments already in widespread use, with likely small effect sizes, the expense of conducting comparative effectiveness trials becomes untenable. In most cases, researchers rely on observational methods, which lack the validity derived from prospective randomisation.13 Therefore, to properly evaluate the comparative effectiveness of multiple treatment strategies, an element of randomisation is essential, together with a mechanism to deploy this efficiently.14

    Electronic health record systems (EHRSs) offer a potential solution. Increasingly widespread and comprehensive, they have renewed interest in integrating clinical trials into routine care.15 While embedding trial infrastructure has improved efficiency, the requirement for point-of-care consent and randomisation remains. Predominantly, this continues to be delivered by a research nurse, partnered with the treating clinician, a process which remains time intensive and financially costly.16

    Two barriers to implementing routine comparative effectiveness research standout—(1) how to fully integrate randomisation into EHRSs, ensuring that patient safety and the scientific integrity of the study is maintained; and, (2) what is the correct way for patients to consent to the randomised delivery of routine treatments? Central to these issues is the principle of clinical equipoise—the idea that without evidence, every clinical decision comes with a degree of uncertainty. When the benefits and risks of the treatment are balanced (or unknown), then it becomes justifiable to randomise, in order to learn what decision is best.17 18

    Flexible electronic point-of-care randomisation

    To learn effectively from clinical decisions, a rapid and responsive randomisation mechanism is required. To achieve this, we propose a two-stage innovation: (1) to embed the randomisation process into the EHRS and link randomisation to the moment of clinical decision making and (2) to make that randomisation optional for the clinician. The first step ensures that the prompt to randomise is presented to the clinician at the point of potential equipoise, ensuring relevance and minimising disruption to normal care processes. The second step means that the clinician–patient dyad only access the randomisation process if they share equipoise with the trial.

    Our design builds on Vickers and Scardino’s concept of the clinically integrated randomised trial, as well as work by Fiore and colleagues in point-of-care trial design.19 20 To this we add concepts from preference trials, which are designed to explicitly acknowledge treatment preferences to minimise bias.21 While most of these trials target patient preferences, we believe that the concepts are equally applicable to clinicians.

    A preference approach has the advantage of allowing clinicians to follow their preferred course of action when they feel strongly, while simultaneously allowing randomisation under conditions of equipoise. In this manner, the clinician retains overall responsibility and control over the patient’s treatment—ensuring safety is maintained. This is key for integrated trials where, by definition, oversight from research teams is minimised.

    We propose to modify existing functionality within the EHRS to intercede at the point of clinical decision making. Many EHRSs use clinical decision support systems (CDSSs), based on series of logical rules, to deliver information to clinicians under pre-defined circumstances. These logical rules may be used to emulate inclusion and exclusion criteria within a trial. Once designated conditions are met, an electronic prompt can be displayed to the clinician, at the point of clinical decision making to highlight both the opportunity to randomise and the predetermined treatment group allocation.

    Our design of electronic point-of-care randomisation (ePOCR) prompt will invite the clinician to consider whether they have equipoise for the treatment decision. In this way, the prompt simply externalises and makes explicit the normal decision-making process.

    If the clinicians have equipoise, the ePOCR prompt allows them to view the randomised allocation, which can then be followed, and the patient contributes data to the randomised arm of the study. However, if the clinician lacks equipoise, they remain free to follow their preference. In a classical RCT, declining to follow randomisation may represent a protocol violation and result in the participant being excluded from the final analysis. However, in a preference approach, the participant continues to contribute data into the parallel observational study arms determined by the clinician’s preference.

    Where the clinician declines randomisation, the parallel observational arm of the study continuously evaluates external validity and can identify previously unrecognised subgroups where clinicians have strong preferences that may require modification of the trial. In addition, where preferences are known, these observational arms may be used to identify preference and selection effects, adding extra information to that gained from the treatment effect estimation in the randomised arm.22 This flexible approach to delivering randomisation is depicted in figure 1.

    Figure 1
    Figure 1

    Flexible randomisation as an expression of clinical equipoise. ePOCR, electronic point-of-care randomisation; RCT, randomised controlled trial.

    To integrate randomisation into clinical workflows requires understanding of how clinicians interact with EHRS and how data are used to make decisions. While the use of interruptive prompts based on modified CDSS is an attractive method for accomplishing this, the possible disruption to care processes must be considered. The concept of alert fatigue in this setting is well documented.23 As such, ePOCR prompts must be designed to be minimally disruptive, while permitting the data collection required by the study. To this end, our feasibility study will compare two designs of interruptive prompt, a simple ‘Nudge’ design, and a more complex ‘Preference’ design. The Nudge prompt encapsulates the simplest version of the study design, while the Preference design allows the collection of additional treatment preference data for use in the observational study arm.

    Pre-emptive and opt-out consent

    There is ongoing debate as to the most appropriate consent mechanisms for facilitating comparative effectiveness research, specifically, for treatments with demonstrable variation already present in their routine use. Faden and colleagues highlight the strong ethical arguments in favour of streamlining consent procedures in this area and the acceptability to stakeholders of the same.24–26

    In this study, we will investigate moving the point at which consent is obtained proximally, away from the final application of eligibility criteria and randomisation. A future model might see patients routinely consented for a range of potential trials (under a specific operational framework such as that suggested by Fiore and Lavori14) on admission to hospital, before it is known whether or not they will be eligible. This single point of contact would decrease the burden of identifying and consenting patients and minimise disruption to clinical workflows.

    Study objectives

    The overall study aim is to ascertain the feasibility of conducting a future clinical trial using infrastructure integrated into the EHRS and using a system of ePOCR. Feasibility will be judged by combining outcome data related to (1) the effectiveness of the ePOCR system and (2) the acceptability of ePOCR to clinicians. Since the feasibility of scaling future large scale trials using ePOCR and preference design approaches is reliant on a streamlined consent model, we will also evaluate the acceptability of both pre-emptive and opt-out consent models to patients. Finally, we will collect pilot data specific to the candidate research question of magnesium supplementation to inform design of a future trial.

    Methods and analysis

    Study design and setting

    This single centre, mixed-methods feasibility study will follow an explanatory-sequential design, which allows supplementation of quantitative data on the effectiveness of ePOCR with qualitative data to aid interpretation.27 The study will run across four critical care units within University College London Hospitals (UCLH) NHS Trust from January to August 2022. These critical care units care for a mix of surgical patients including colorectal, urology and thoracics but excluding cardiac and neurosurgery. UCLH has used the Epic EHRS since 2018.

    We will recruit patients aged 18 years and over, undergoing elective surgery of sufficient complexity to warrant postoperative admission to critical care. This cohort was selected opportunistically to facilitate obtaining informed consent pre-emptively during hospital visits prior to surgery. Potentially eligible participants will be identified through a combination of algorithmic screening of the EHRS by surgical procedure code, and by manual identification from booked critical care admissions.

    We will recruit a cohort of critical care clinicians to undertake the qualitative interview programme. The intervention is targeted to bedside critical care nurses. There are approximately 300 critical care nurses working across all the study sites. Neither clinicians nor patients will be compensated for participating in the study.

    Exclusion criteria will be applied at two stages. Patients unable to provide written informed consent, or who are pregnant will be excluded. Following postoperative admission to critical care, patients whose initial documented heart rhythm is atrial fibrillation will be excluded. Prior to the deployment of the ePOCR prompt, the EHRS will screen against the following criteria: (1) no documented allergy or intolerance to any preparation of supplemental magnesium, (2) no active treatment for bronchospasm (defined as active treatment administration indicating bronchospasm and screening of active problem list) and (3) the most recent serum magnesium result prior to prompt deployment lies between 0.5 and 1.5 mmol/L. This final criterion ensures that the prompt does not facilitate randomisation for magnesium values outside the scope of reasonable clinical equipoise. For example, serum magnesium values <0.5 mmol/L would normally always be supplemented, and vice versa for values >1.5 mmol/L.

    On successful conclusion of the screening process, the ePOCR prompt will display to the bedside critical care nurse. The screening process repeats for each new serum magnesium result received. Screening and overall participant flow through the study are illustrated in figure 2.

    Figure 2
    Figure 2

    Anticipated participant flow through study. EHRS, electronic health record system; ePOCR, electronic point-of-care randomisation.

    Qualitative assessments will be conducted in three stages. A random sample of all critical care clinicians involved in routinely caring for this patient cohort will be invited to undertake an initial interview. Two further interviews will focus specifically on the bedside critical care nurses exposed to the ePOCR prompts.

    Patient and public involvement

    We sought opportunities to engage with patients and the public from study inception. To this end, two focus groups were conducted. The first addressed utilisation of electronic clinical data for research and the presence of naturally occurring variation in practice for evidence-light treatments. The second focused on the premise of flexible ePOCR and the need to investigate alternative consent models for comparative effectiveness research. These groups highlighted a general lack of awareness regarding evidence gaps for routine treatments. Both groups agreed that this is a priority area for future research. The authors are grateful to members of both groups for their feedback in improving the clarity of communication regarding a complex study design. We plan to disseminate study results to consenting participants on completion.

    Interventions

    Electronic point-of-care randomisation prompts

    This study will compare two ePOCR prompts—Nudge versus Preference designs, illustrated in figure 3. The Nudge design is characterised by its passive nature and requires minimal interaction from the clinician. The intention is to ‘nudge’ the clinician to consider their level of equipoise for the decision to supplement magnesium and follow the randomised treatment where they have no preference. In contrast, the Preference design facilitates the explicit recording of the clinician’s treatment preference, while simultaneously allowing randomisation under conditions of equipoise. Preference options are presented as three possible choices—a strong preference for or against administering supplemental magnesium, and no preference. If no preference is selected, the randomised action is provided. If the clinician selects a strong preference, they are advised to continue with their preferred treatment. While this design is more burdensome because it requires interaction, it will allow the derivation of preference and selection effects as described above.

    Figure 3
    Figure 3

    Examples of Nudge (top) and Preference (bottom) ePOCR prompt designs. ePOCR, electronic point-of-care randomisation.

    Both prompt designs will be constructed using the Epic build module designed for ‘Best Practice Advisory’ creation, essentially a form of clinical decision support. Construction of a system of logical rules will allow screening of eligibility criteria as described. The technical aspects of both prompt designs will be tested in a sandbox environment prior to live deployment.

    Deployment follows the same pathway for both ePOCR prompt designs. Following recruitment, participants will be randomised to either Nudge or Preference design. They will then be randomised again to either liberal or restrictive magnesium supplementation strategies (figure 4). The liberal magnesium arm will encourage supplementation at a serum magnesium value <1.0 mmol/L. The restrictive arm will encourage supplementation at a serum magnesium value <0.75 mmol/L. These values were determined from an observational study of supplementation practices at the study centre and fall within the boundaries of observed variation in practice.11

    Figure 4
    Figure 4

    Two stage randomisation process

    Randomisation will be conducted using the EHRS, which conducts simple randomisation using an internal number rule.28 For this feasibility study, basic randomisation without additional covariate balancing will be used. Randomisation will remain the same for both prompt design and magnesium strategy throughout study participation.

    Both prompts will display to the bedside critical care nurse under either of two conditions: (1) accessing of the participant’s blood test results or (2) accessing the supplemental magnesium prescription within the EHRS. The prompt will deploy once for each new serum magnesium result, for five postoperative days or the end of the participant’s critical care admission, whichever is sooner.

    This study has been designed to be highly pragmatic. At our institution, it is normal practice for all patients admitted to critical care to be issued with an ‘as required’ prescription for either intravenous or oral magnesium. In both study arms, the method and frequency of magnesium supplementation, and frequency of serum magnesium measurement remain at the discretion of the clinical team.

    Qualitative assessments

    Both clinician and patient interviews follow a semistructured design. Prior to ePOCR deployment, critical care clinicians will be invited to undertake an interview exploring general attitudes towards EHRS research and their current interactions with existing electronic alerts. The interview will feature guided simulation introducing both prompt designs and encouraging initial feedback. The use of simulation to introduce the prompts acknowledges the logistical difficulty in ensuring that each critical care nurse participating in the study is exposed to each prompt design at least once during the study period.

    Critical care nurses will undertake a further interview following exposure to an ePOCR prompt to gather immediate feedback. A final follow-up interview will invite nurses to give a preference on prompt design, having experienced the intervention in a clinical setting.

    Patients participating in the study will undertake a semistructured interview following discharge from critical care. This will explore attitudes towards pre-emptive and opt-out consent models. Interview schedules are included in online supplemental material S1.

    Outcome, data collection and analysis

    We will collect descriptive data on ePOCR performance. The primary study outcome will be the proportion of prompts of either design which result in compliance with randomisation by the clinician. Estimates of prompt compliance will be generated for both liberal and restrictive magnesium strategies in addition. Compliance is defined as either: (1) the appropriate administration of magnesium following prompt deployment, where the measured serum magnesium is less than the randomised threshold or (2) the appropriate withholding of supplemental magnesium following prompt deployment, where the serum magnesium is greater than the randomised threshold. The potential outcomes following prompt deployment are illustrated in figure 5, using the Nudge design as an example.

    Figure 5
    Figure 5

    Derivation of compliance with randomisation from observed clinician action.

    For the Preference design, descriptive data will be presented across the range of possible responses. We will link events where the clinician declines randomisation and expresses a strong preference to the subsequent action (administration of magnesium or not). The observation period for assessing compliance will be defined as the time from exposure to prompt to the subsequent shift change in clinical team. We will assess between group differences in proportion using a χ2 test.

    All quantitative data pertinent to addressing the study outcomes will be extracted from the EHRS. The study will not require any additional documentation or data entry by clinical teams. We will extract routinely collected clinical data from the EHRS for study patients to aid planning a future main study. This will include baseline rates of atrial fibrillation in the study population, frequency of serum magnesium measurement, frequency of supplementation and estimates of treatment group separation (difference in mean serum magnesium values between liberal and restrictive groups).

    The semistructured interview programme will contribute qualitative data to both primary and secondary outcomes. Overall study feasibility will be judged through a combination of ePOCR prompt compliance rates and acceptability to clinicians. ePOCR compliance data will contribute to further simulation work designed to estimate plausible ranges of samples sizes for a main study, which will be used to further demonstrate study design feasibility.

    We will use a thematic analysis approach to analyse interview data as described by Braun and Clarke and illustrated in a recent analysis by McNulty et al.29 30

    The primary objective of the study is to determine the feasibility of using ePOCR prompts. Overall feasibility will be judged using technical aspects around design and implementation and the experience of clinicians, which will both be assessed qualitatively.

    While overall study feasibility will be contingent on the rate of prompt compliance (the proportion of alerts where the clinician complies with randomisation), we will also estimate differences in compliance between nudge and preference designs to add to our qualitative assessments. We propose a non-inferiority approach based on the premise that the preference design has improved research utility relative to the nudge design through estimation of treatment and selection effects. Therefore, if the preference prompt proves non-inferior in terms of observed compliance, and qualitatively acceptable, it would be demonstrably the preferred design. We will not seek to test between group differences for each magnesium strategy in addition.

    To this end, we estimate the required sample size based on hypothesised equal compliance rate of 50% in both groups. We hypothesise a non-inferiority margin of −25% to be justifiable relative to the additional data the preference design would provide. This produces a sample size of 50 prompts per design, with a power of 80% and a 5% significance level.31 Using an average of two prompts per patient, this results in a sample size of 25 patients per group.

    We will aim to recruit 20 clinicians identified as key informants relevant to the study question to undertake baseline interviews.32 We will employ a purposive sampling strategy as used by Connell et al to evaluate a complex digital intervention in a similar healthcare setting and justified by international consensus guidance for mixed-methods research.33 34 We will aim to interview all bedside nurses who receive a prompt and use guided simulation to aid the evaluation of preference for either prompt design.

    Ethics and dissemination

    This study protocol was approved by the NHS Riverside Research Ethics Committee (Ref: 21/LO/0785) and sponsored by University College London (Ref: 142382).

    Potentially eligible patients will be approached during their anaesthetic pre-assessment clinic visit. After confirming initial eligibility, a member of the research team will discuss the study and issue the Participant Information Sheet (online supplemental material S2), which includes research team contact information and mechanisms to withdraw from the study at any point. The patient will be able to give written consent (online supplemental material S3) at any point from initial approach to immediately prior to surgery.

    By approaching patients in pre-assessment clinic, we evaluate a pre-emptive approach to providing consent which may be transitioned to an opt-out approach in the future if acceptable. We justify obtaining consent at the initial visit in three ways. First, the study premise and intervention carry minimal risk to the participant, second, the burden on the participant is low (one follow-up interview following surgery). Third, we provide multiple routes to discontinue participation with multiple checks throughout the perioperative journey.

    We ensure that participant data are protected by extracting on data pertinent to the study from the EHRS. All clinical data obtained during the study will remain within UCLH computer infrastructure and firewall. Data extracted from the EHRS will only be accessible by designated members of the research team and presented as summary level data. Interview data will be audio recorded and uploaded via secure email to UCLH computer systems for analysis. Only anonymised quotes will be used in the study reports. We plan to disseminate results by publication in peer-reviewed journals and will also prepare reports for patients and clinicians involved in the study on completion.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    The authors would like to acknowledge the help of Professor Matthew Sydes in advising on the study design and supporting MGW’s PhD work in this field. We also thank Ms Nausheen Saleem for her ongoing support in the development of the ePOCR prompts together with the support of the UCL Clinical Research Informatics Unit. MGW is supported through a doctoral training partnership funded by the Medical Research Council. FWA and SKH are supported by University College London Hospitals National Institute for Health Research Biomedical Research Centre. SKH is supported by a Health Foundation Improvement Science Fellowship.

    References

    1. ↵
      1. Casey JD,
      2. Courtright KR,
      3. Rice TW, et al
      . What can a learning healthcare system teach us about improving outcomes? Curr Opin Crit Care 2021;27:527–36.doi:10.1097/MCC.0000000000000857pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/34232148
      OpenUrlPubMed
    2. ↵
      1. Braithwaite J,
      2. Glasziou P,
      3. Westbrook J
      . The three numbers you need to know about healthcare: the 60-30-10 challenge. BMC Med 2020;18:102.doi:10.1186/s12916-020-01563-4pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/32362273
      OpenUrlCrossRefPubMed
    3. ↵
      1. Mercuri M,
      2. Gafni A
      . Examining the role of the physician as a source of variation: are physician-related variations necessarily unwarranted? J Eval Clin Pract 2018;24:145–51.doi:10.1111/jep.12770pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28556526
      OpenUrlPubMed
    4. ↵
      1. Glover JA
      . The incidence of tonsillectomy in school children. Int J Epidemiol 2008;37:9–19.doi:10.1093/ije/dym258pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/18245048
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. McCulloch P,
      2. Nagendran M,
      3. Campbell WB, et al
      . Strategies to reduce variation in the use of surgery. Lancet 2013;382:1130–9.doi:10.1016/S0140-6736(13)61216-7pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/24075053
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Safavi KC,
      2. Dharmarajan K,
      3. Kim N, et al
      . Variation exists in rates of admission to intensive care units for heart failure patients across hospitals in the United States. Circulation 2013;127:923–9.doi:10.1161/CIRCULATIONAHA.112.001088pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/23355624
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Gershengorn HB,
      2. Iwashyna TJ,
      3. Cooke CR, et al
      . Variation in use of intensive care for adults with diabetic ketoacidosis*. Crit Care Med 2012;40:2009–15.doi:10.1097/CCM.0b013e31824e9eaepmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/22564962
      OpenUrlCrossRefPubMed
    8. ↵
      1. Peltan ID,
      2. Mitchell KH,
      3. Rudd KE, et al
      . Physician variation in time to antimicrobial treatment for septic patients presenting to the emergency department. Crit Care Med 2017;45:1011–8.doi:10.1097/CCM.0000000000002436pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28426466
      OpenUrlPubMed
    9. ↵
      1. Lilot M,
      2. Ehrenfeld JM,
      3. Lee C, et al
      . Variability in practice and factors predictive of total crystalloid administration during abdominal surgery: retrospective two-centre analysis. Br J Anaesth 2015;114:767–76.doi:10.1093/bja/aeu452pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/25586725
      OpenUrlCrossRefPubMed
    10. ↵
      1. Garg J,
      2. Chaudhary R,
      3. Krishnamoorthy P, et al
      . Role of prophylactic magnesium supplementation in prevention of postoperative atrial fibrillation in patients undergoing coronary artery bypass grafting: a meta-analysis of 23 randomized controlled trials. J Am Coll Cardiol 2016;67:689.doi:10.1016/S0735-1097(16)30690-8
    11. ↵
      1. Wilson MG,
      2. Rashan A,
      3. Klapaukh R
      . Prophylactic magnesium supplementation and new-onset atrial fibrillation in a general critical care population: a prescribing preference instrumental variable analysis. Intensive Care and Critical Care Medicine 2021.doi:10.1101/2021.11.25.21266861
    12. ↵
      1. Gattinoni L,
      2. Giomarelli P
      . Acquiring knowledge in intensive care: merits and pitfalls of randomized controlled trials. Intensive Care Med 2015;41:1460–4.doi:10.1007/s00134-015-3837-7pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/26077049
      OpenUrlPubMed
    13. ↵
      1. Vetter TR,
      2. Mascha EJ,
      3. Bias MEJ
      . Bias, confounding, and interaction: lions and tigers, and bears, OH my! Anesth Analg 2017;125:1042–8.doi:10.1213/ANE.0000000000002332pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28817531
      OpenUrlCrossRefPubMed
    14. ↵
      1. Fiore LD,
      2. Lavori PW
      . Integrating randomized comparative effectiveness research with patient care. N Engl J Med 2016;374:2152–8.doi:10.1056/NEJMra1510057pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/27248620
      OpenUrlCrossRefPubMed
    15. ↵
      1. van Staa T-P,
      2. Klungel O,
      3. Smeeth L
      . Use of electronic healthcare records in large-scale simple randomized trials at the point of care for the documentation of value-based medicine. J Intern Med 2014;275:562–9.doi:10.1111/joim.12211pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/24635449
      OpenUrlPubMed
    16. ↵
      1. Morris ZS,
      2. Wooding S,
      3. Grant J
      . The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011;104:510–20.doi:10.1258/jrsm.2011.110180pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/22179294
      OpenUrlCrossRefPubMed
    17. ↵
      1. London AJ,
      2. systems Lhealth
      . Learning health systems, clinical equipoise and the ethics of response adaptive randomisation. J Med Ethics 2018;44:409–15.doi:10.1136/medethics-2017-104549pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29175968
      OpenUrlAbstract/FREE Full Text
    18. ↵
      1. Faden R,
      2. Kass N,
      3. Whicher D, et al
      . Ethics and informed consent for comparative effectiveness research with prospective electronic clinical data. Med Care 2013;51:S53–7.doi:10.1097/MLR.0b013e31829b1e4bpmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/23793051
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Vickers AJ,
      2. Scardino PT
      . The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost. Trials 2009;10:14.doi:10.1186/1745-6215-10-14pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/19265515
      OpenUrlCrossRefPubMed
    20. ↵
      1. Fiore LD,
      2. Brophy M,
      3. Ferguson RE, et al
      . A point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen. Clin Trials 2011;8:183–95.doi:10.1177/1740774511398368pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/21478329
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Salkind N
      . Encyclopedia of Research Design. 2455 Teller Road, Thousand Oaks California 91320 United States: SAGE Publications, Inc, 2010..
    22. ↵
      1. Wasmann KA,
      2. Wijsman P,
      3. van Dieren S, et al
      . Partially randomised patient preference trials as an alternative design to randomised controlled trials: systematic review and meta-analyses. BMJ Open 2019;9:e031151.doi:10.1136/bmjopen-2019-031151pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/31619428
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Backman R,
      2. Bayliss S,
      3. Moore D, et al
      . Clinical reminder alert fatigue in healthcare: a systematic literature review protocol using qualitative evidence. Syst Rev 2017;6:255.doi:10.1186/s13643-017-0627-zpmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29237488
      OpenUrlCrossRefPubMed
    24. ↵
      1. Faden RR,
      2. Beauchamp TL,
      3. Kass NE
      . Informed consent, comparative effectiveness, and learning health care. N Engl J Med 2014;370:766–8.doi:10.1056/NEJMhle1313674pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/24552325
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Faden RR,
      2. Kass NE,
      3. Goodman SN, et al
      . An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. Hastings Cent Rep 2013;Spec No:S16–27.doi:10.1002/hast.134pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/23315888
      OpenUrlPubMed
    26. ↵
      1. Morain SR,
      2. Tambor E,
      3. Moloney R, et al
      . Stakeholder perspectives regarding alternate approaches to informed consent for comparative effectiveness research. Learn Health Syst 2018;2:e10047.doi:10.1002/lrh2.10047pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/31245580
      OpenUrlPubMed
    27. ↵
      1. Edmonds WA,
      2. Kennedy TD
      . An applied guide to research designs: quantitative, qualitative and mixed method. 2455 Teller Road, Thousand Oaks California 91320: SAGE Publications, Inc, 2017.
    28. ↵
      1. Wilson FP,
      2. Greenberg JH
      . Acute kidney injury in real time: prediction, alerts, and clinical decision support. Nephron 2018;140:116–9.doi:10.1159/000492064pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/30071528
      OpenUrlPubMed
    29. ↵
      1. Braun V,
      2. Clarke V
      . Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101.doi:10.1191/1478088706qp063oa
      OpenUrlCrossRef
    30. ↵
      1. McNulty C,
      2. Ricketts EJ,
      3. Rugman C, et al
      . A qualitative study exploring the acceptability of the McNulty-Zelen design for randomised controlled trials evaluating educational interventions. BMC Fam Pract 2015;16:169.doi:10.1186/s12875-015-0356-0pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/26577832
      OpenUrlPubMed
    31. ↵
      1. Ltd SE
      . Power calculator for binary outcome non-inferiority trial [Online]. Sealed Envelope Ltd. Available: https://www.sealedenvelope.com/power/binary-noninferior/ [Accessed 19 May 2022].
    32. ↵
      1. Marshall MN
      . The key informant technique. Fam Pract 1996;13:92–7.doi:10.1093/fampra/13.1.92pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/8671109
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Connell A,
      2. Black G,
      3. Montgomery H, et al
      . Implementation of a digitally enabled care pathway (Part 2): qualitative analysis of experiences of health care professionals. J Med Internet Res 2019;21:e13143.doi:10.2196/13143pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/31368443
      OpenUrlCrossRefPubMed
    34. ↵
      1. Vasileiou K,
      2. Barnett J,
      3. Thorpe S, et al
      . Characterising and justifying sample size sufficiency in Interview-Based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol 2018;18:148.doi:10.1186/s12874-018-0594-7pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/30463515
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @MWilson1987

    • Contributors SKH, FWA and MGW conceived the research idea and developed the study design. SKH is the chief investigator for the study and MGW the principal investigator. MGW prepared the initial manuscript, and all authors were involved in contributing major edits. RM and MGW designed the ePOCR prompt system. DB assisted with trial design, implementation, clinical oversight and manuscript review. All authors have read and approved the final protocol and this manuscript.

    • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.