Study design

ADAPT will be a Multiphase Optimization STrategy (MOST)35 study of CDS for the treatment of OUD in the ED focusing on buprenorphine initiation. We will apply an empirical, iterative, learning health system36 37 approach (figure 1). We will add reproducible best measurement and intervention optimisation and evaluation to rapid plan–do–study–act cycles following best practices for CDS refinement using the 10-step process of McCoy et al’s Clickbusters initiative.38–40

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Learning Health System framework used to optimise clinical decision support (CDS) for opioid use disorder care. This iterative process integrates reproducible measurement, the Multiphase Optimization Strategy (MOST) framework and rapid improvement cycles.

The national Epic build of the EMBED CDS will be expanded from a stand-alone CDS to a multicomponent intervention including evidence-based, sustainable implementation strategies33 34 41 with iterative refinement by applying an MOST framework (figure 2).35 42–44 The MOST framework includes three phases: preparation, optimisation and evaluation. For the ADAPT trial, the optimisation phase will be conducted in two stages: (1) a randomised factorial trial focused on increasing CDS use and (2) rapid, serial randomised testing37 with iterative usability refinement to enhance interface and workflow.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Application of the Multiphase Optimization Strategy framework to the Adaptive Decision support for Addiction Treatment trial. CDS, clinical decision support; EMBED, EMergency department-initiated BuprenorphinE for opioid use Disorder

MOST preparation phase

In the preparation phase, the EMBED intervention will be refined into a multicomponent CDS intervention to improve ED initiation of buprenorphine in patients with OUD via increased CDS uptake, usability and equity (additional details provided in the Intervention section below). In addition, the investigative team will prepare for the trial across three distinct data needs: (1) CDS measure development, (2) phenotype refinement and (3) data coordination. Measure development will involve implementing measurement science to EHR audit logs to conceptualise, specify, test and evaluate novel, reproducible, scalable outcome measures for assessing CDS uptake and usability to implement ED initiation of buprenorphine for OUD.45–47 The existing EMBED EHR phenotype will be expanded to optimise the identification of ED patients experiencing OUD or having both opioid-specific and opioid-related visits.48 49 The data coordination team will plan for the management of trial data collection, retrieval, compliance with NIH and NIDA (National Institute on Drug Abuse) HEAL (Helping to End Addiction Long-term Initiative) regulations, and analysis.

MOST optimisation phase, stage 1: factorial trial

With no interruptive alerts in the EMBED trial, CDS was used in only 9.4% of encounters with eligible patients who met the EMBED EHR phenotype criteria. Buprenorphine was initiated in 61.7% of cases when the intervention was used.25 Therefore, the overarching objective of the optimisation phase, stage 1 is to increase CDS use (ie, patient reach and clinician adoption) as a means to increase buprenorphine initiation rates to a larger portion of ED patients with OUD. With a goal of 50% CDS use and a corresponding 30% buprenorphine initiation rate for encounter with eligible patients, starting in February 2025, we will conduct a 2×2×2 factorial trial by expanding the EMBED intervention to include sustainable implementation strategies.33 34 41 The factorial trial design allows efficient and simultaneous testing of the main effects and interaction of several intervention components on CDS use in its real-world implementation.

Stage 1 of the MOST optimisation phase will aim to evaluate three elements amenable to encounter-level randomisation, each with its own reference and comparator level in a randomised factorial trial. This 2×2×2 factorial trial with 2³=8 conditions (figure 3) with component-dependent definitions of reference and comparator is easily amenable to changing or adding components at the start of the trial. We anticipate that components identified for randomisation in the MOST preparation phase will fall under the categories of (1) patient-facing materials, (2) prompts for clinicians to engage with the CDS and (3) prompts for nurses to engage with the CDS.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Depiction of the eight arms of the 2×2×2 factorial trial with 2³=8 unique combinations.

MOST optimisation phase, stage 2: rapid, randomised testing

We will aim to improve CDS usability via serial randomised testing to inform iterative refinement of the CDS interface and workflow to minimise user errors, task disruption and abandonment by identifying specific targets for improvement via applying novel CDS outcome measures in serial randomised tests.

In this stage, the EMBED CDS interface and workflow will be incrementally improved based on rapid, serial randomised testing using novel, validated EHR use metrics developed to evaluate CDS performance. These metrics will measure CDS performance details including the time to completion of the intervention, increasing the likelihood and efficiency of use and task completion. This method will allow systematic, rapid learning of what aspects of the EMBED interface and workflow work and deimplementation of what is not.37 Once an assigned intervention change or component in one randomisation group is noted to be superior to another, it will become the new standard. Of note, while the EMBED CDS interface and workflow will be optimised, the specific treatment approaches (eg, medication dosing and adjunct medications) promoted by EMBED will not change unless best practices change, so front-line staff will not have to alter their clinical practices aside from hopefully increasing the uptake of recommended practices.

MOST evaluation phase

The objective of this third phase is to compare the effectiveness of the optimised, multicomponent CDS package to the original EMBED CDS on ED initiation of buprenorphine rates in patients with OUD in a randomised trial. During this confirmatory phase, the optimised treatment package will be tested in a fully powered two-arm randomised trial comparing the efficacy of the optimised, multicomponent intervention package to the original EMBED CDS on rates of ED initiation of buprenorphine in patients with OUD. Based on the results of the optimisation phase factorial trial in conjunction with findings from the rapid, serial randomised testing, the optimised treatment package will include individual components found to increase CDS uptake and all usability and workflow improvements found to increase use and efficiency in serial randomised testing. This phase will be our trial’s final phase, expected to conclude in March 2028.

Participants

Smalley et al distinguish between direct, indirect and collateral participants in pragmatic research in which: (1) direct participants are intervened on and/or identifiable data are collected for research purposes, (2) indirect participants are not direct participants but may have their rights or welfare affected by the intervention and (3) collateral participants are members of other stakeholder communities who may be affected by the research.50 For the ADAPT trial, the investigative team will not collect identifiers for patients or clinicians and the intervention will be focused at the patient level, with patients considered direct participants and clinicians indirect participants, with process improvement (non-research, such as feedback emails) interventions directed at the clinician level, without containing identifiable information. These indirect participants will include approximately 80 attendings (67 faculty and 13 fellows), 74 residents and 55 advanced practice providers (physician associates and advanced practice registered nurses) at one academic ED, one community ED and one free-standing ED.

In the EMBED trial, patients with probable OUD were identified using a validated EHR-based algorithm to identify patients aged 18 years or older who were not pregnant and not currently receiving medication for OUD.25 51 52 For the ADAPT trial, we will expand this algorithm to include additional opioid-related visits and implement it in real time through the EHR (figure 4). Patients will be screened for eligibility using deidentified data processed securely.48 53 Additional details and phenotype validation will be reported separately.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Illustration of Adaptive Decision support for Addiction Treatment’s target population: patients who are ready for treatment but are not currently offered treatment. OUD, opioid use disorder.

Intervention

To improve the EMBED intervention using a more comprehensive, multicomponent strategy, the ADAPT trial intervention will include key updates to adapt the CDS for broader use, addressing potential risks such as precipitated opioid withdrawal after buprenorphine administration and enhancing the intervention with audit and feedback.34 41 54 Practitioners will retain all control of their practice and will have the option whether or not to use the intervention (ie, can opt out of use). The patient-facing material team will design and formatively evaluate educational resources that encourage patient engagement and readiness for treatment, with a focus on equitable access to OUD medication through deliberate community engagement and advisory board involvement, as demonstrated in previous successful work with personalised risk communication.55 Components amenable to patient-level randomisation (patient-facing materials and targeted prompts to nurses and clinicians based on patient characteristics) will be identified and prepared for implementation.24 41

Randomisation and allocation of intervention

The randomisation unit for each phase will be the individual ED patient encounter. Randomisation will depend on study phase. For phase 2 stage 1, the 2×2×2 factorial trial, encounters will be randomised in a 1:1:1:1:1:1:1:1 ratio to each of the eight intervention combinations. Therefore, the probability of receiving each intervention will be 0.5 and the probability of receiving a given intervention combination will be 0.125. Simple (ie, not stratified or blocked) randomisation will be implemented in Epic where the allocation sequence will be generated and concealed. To ensure equal intervention distribution, a random sequence will be generated with a computer-generated algorithm within the Epic EHR.43 Allocation will be concealed from participants using Epic by turning these options on/off in EMBED according to the assignment for an encounter. Once a participant is identified as eligible based on an automated phenotypic evaluation within Epic, they will be randomised and intervention components will be automatically turned on or off depending on their assignment. Phase 2 stage 2 will require serial randomised testing. The precise nature of the randomisation will depend on the specific component targeted for improved workflow; however, the ratio of the assignments to each individual component intervention will be 1:1 to maximise statistical efficiency (ie, power). During the evaluation phase in which the effectiveness of the optimised multicomponent CDS package will be compared with the original EMBED CDS, we will randomise 1:1 to the two interventions. As in the optimisation phase, simple (ie, not stratified or blocked) randomisation will be implemented within Epic where the allocation sequence will be generated and concealed until the participant is enrolled.

Outcomes

As an implementation study of a well-established, effective treatment strategy that has been shown to be highly efficacious in the ED setting,15 56 the overall primary outcome across phases of the ADAPT trial will be encounter-level buprenorphine initiation in the ED, defined as the proportion of eligible patients who receive buprenorphine in the ED or as a prescription at discharge, including those interested in buprenorphine treatment but not yet in a stage of withdrawal that would warrant ED buprenorphine administration who will receive instruction for after visit initiation. In the optimisation stage 1 factorial trial, we will also examine, as a primary outcome, the proportion of encounters in which the CDS is used, as the focus of this phase is increasing CDS uptake, regardless of actions taken through the CDS. During the serial, rapid randomised testing cycles of optimisation phase, stage 2, the primary outcome may vary depending on testing cycle goals. As in the EMBED trial, outcomes will be evaluated using the Reach, Effectiveness, Adoption, Implementation, and Maintenance25 57 58 framework. In all trial phases, we will examine the following secondary outcomes: (1) attending physician initiating buprenorphine in the ED for at least one eligible patient, (2) naloxone prescription on ED discharge, (3) attending physician prescribing naloxone on ED discharge, (4) patient receipt of appropriate opioid-related discharge instructions, (5) attending physician giving appropriate opioid-related discharge instructions, (6) patient referral to ongoing medication for OUD treatment and (7) attending physician placing patient referral to ongoing medication for OUD treatment.

In addition to these outcomes, in optimisation phase stage 1, while focusing on increasing CDS use in the factorial trial, we will also measure meaningful engagement with the intervention (reach to patients and adoption by clinicians) with implementation outcomes regarding both the CDS being launched from the alert and use without the alert. In the optimisation phase stage 2 serial randomised testing, we will use novel CDS measures developed in the MOST preparation phase using measurement science to conceptualise, specify, implement, test and validate reproducible, scalable outcome measures for assessing uptake and usability of CDS for ED initiation of buprenorphine for patients with OUD.41–43 Our approach will be multidisciplinary and iterative, incorporating multiple leadership, advisory and interprofessional partner codesign meetings to ensure thorough metric conceptualisation, specification and validation.

Sample size

Sample size calculations were performed using PASS V.2019 (Kaysville, Utah, USA) and will vary according to the phases of the study.

• Optimisation phase, stage 1: factorial trial of multiple components for CDS uptake.

As each component is allocated to half the participants, the sample size to detect main effects in a full factorial design depends not on the number of components evaluated, but rather the smallest clinically important difference between the presence and absence of a component. Our goal is to determine whether the individual intervention components (OUD patient-facing materials, nurse prompt for Clinical Opiate Withdrawal Scale, clinician prompt for CDS) will improve CDS use by at least 10%. A sample size of 136 per group (total n=1088, across eight arms) will provide 90% power at the two-sided 0.05 significance level to detect differences of 10% (the smallest clinically important difference agreed on by the investigative team)28 between the presence and absence of each of these components. This sample size was maximised to assure at least 90% power to detect an absolute difference of 10% regardless of the proportion using the CDS in the absence of the intervention component. We did not inflate the sample size for clustering by provider, given the crossed design, where we expect variance inflation to be negligible. We will investigate whether the effect of a component is dependent on the levels of other components (ie, interactions), but our trial is not powered with the intent to detect these interactions.

• Optimisation phase, stage 2: serial randomised testing for usability (user errors and efficiency).

Our goal is to rapidly identify interface and workflow solutions to reduce errors and time to completion. Given this goal, we will design these serial randomised experiments to have high power to detect large differences.37 59 We anticipate using a variety of parallel group randomised designs (such as two-arm, factorial). A two-sided 0.10 significance level will be chosen, given the benign consequences of false positive errors. While the sample size for a given serial test will depend on the relevant outcome, we expect to require between 80 and 150 encounters with CDS uptake to rapidly evaluate various solutions. These sample sizes will provide 90% power at the two-sided 0.10 significance level to detect medium-to-large standardised differences in proportions (such as errors, abandonment) of h=0.48–0.65. We did not inflate the sample size for clustering by provider, given the crossed design, where variance inflation is expected to be negligible.60

• Evaluation phase: randomised trial to evaluate optimised package.

In the previous EMBED trial, ~12% of patients with OUD had buprenorphine initiated. We will conservatively assume the proportion initiating buprenorphine in the original EMBED CDS arm to be 20%. A sample size of 412 per group (Total n=824) will be required to detect an improvement of 10% (absolute increase from 20% to 30%)25 28 with 90% power at the two-sided 0.05 significance level. We did not inflate the sample size for clustering by provider given the crossed design, where we expect variance inflation is expected to be negligible.

Data collection

All clinical data will be retrieved from our institution’s Epic Clarity database. The data will include Epic User Action Log (UAL) and Access Log—audit logs hosted locally, which will be used in the CDS use measure calculation.

Data management

For the ADAPT project, the investigative team will not collect identifiers for patients or clinicians. A third-party honest broker61 with access to protected health information (PHI) as part of their health system operational digital technology role will use study codes on data documents instead of recording identifying information. They will keep a separate document that links the study code to subjects’ identifying information locked in an alternative location and restrict access to this document. Additionally, any identifiable data will be encrypted prior to collection and storage. All data documents will be securely stored in encrypted, HIPAA-compliant digital environments within locked locations. The third-party honest broker will destroy identifiable data within 6 months of study completion.

Data monitoring

As a minimal-risk implementation study of established best practices, an independent study monitor (ISM) will be used in place of a formal Data Safety Monitoring (DSM) Board. Interim monitoring will focus on adherence to the protocol, completeness of data retrieval from each ED’s EHR and uptake of the CDS intervention. A set of monitoring tables will be generated for this purpose. The ISM will report directly to the study DCC. The principal investigator (PI) will be responsible for monitoring the safety and efficacy of this trial, executing the DSM plan and complying with reporting requirements. The PI will provide a summary of the DSM report to NIDA annually as part of the progress report. The DSM report to NIDA will also include, when available, the results of any efficacy data analysis conducted.

Data analytics team and plan: advisory board

To augment and complement the investigative team’s expertise and optimise their scientific approach, the ADAPT investigators will be advised and counselled by an advisory board of national methodological experts who will meet quarterly. It will be composed of two specialists for each of the following categories: EHR use measurement, study design for pragmatic evaluation and optimisation trials in routine care, and CDS for OUD in the ED and one in health equity.

Data analytics team and plan: analysis plan

Baseline data will be reported with frequencies, percentages, means, SD, medians, and IQRs as appropriate for categorical and continuous measures. All statistical tests will be two sided, with a 0.05 significance level.

• Optimisation phase, stage 1: evaluation of components in the 2×2×2 factorial design will be performed using multivariable random effects logistic regression with random effects to accommodate clustering of patients by clinician and repeated measures from the same patients.60 For this full-factorial design, the model will include main effects for each of the three components as well as all two-way and three-way interactions. The regression will also include baseline covariates: age, sex, race, ethnicity, insurance type and status, and reason for ED visit (such as overdose, withdrawal, soft tissue infection). The mixed logistic regression model will be used to estimate marginal outcome proportions to compare intervention components. Differences in proportions for the presence and absence of each component (ie, main effects) will be estimated along with 95% CIs. A p value of 0.10 will be used as a guide to flag potentially important interactions which will be explored graphically with a particular emphasis on identifying substantial synergistic effects or qualitative (ie, where the impact of 1 component changes direction depending on the presence or absence of another component) effect modification.

• Optimisation phase, stagee 2: given the smaller sample sizes required, only first patient encounters will be randomised. Random effects logistic regression will be used for the analysis of serial randomised experiments for outcomes such as proportion with errors or abandonment. The models will include a fixed effect for intervention and a random effect for clinicians. Differences in proportions for the intervention will be estimated along with 95% CIs. Time-based variables (such as time to completion of CDS) will be evaluated using survival analyses. Log-rank tests will be used to compare cumulative time to event between interventions, and median completion times will be presented.

• Evaluation phase: comparison of the optimised multicomponent package to the original EMBED CDS will be performed using multivariable random effects logistic regression with random effects to accommodate the clustering of patients by clinician and repeated measures from the same patients.60 The model will include the same baseline covariates from the optimisation phase, stage 2 (above) for this two-arm parallel group trial design. The mixed logistic regression model will be used to estimate marginal outcome proportions to compare intervention components. Differences in proportions for the optimised EMBED compared with the original EMBED CDS will be estimated along with a 95% CI.

Patient and public involvement

Creating and refining the novel outcome measures for optimisation phase, stage 2 of this project will involve reaching a consensus among stakeholders representing emergency medicine physicians and administrators, addiction medicine experts, nurses, clinical pharmacists, clinical informaticists and data scientists, professional organisation representatives, EHR analysts and health promotion advocates. This group will be recruited to assemble a representative group with different perspectives. Final participant selection will balance age, gender, race, ethnicity and practice settings for diverse representation with oversampling of historically marginalised groups.62–65

With the goal of equitable treatment for people experiencing moderate to severe OUD,31 32 the ADAPT project will seek community stakeholder perspectives and input from community members with lived experience with OUD as well as experience in study conceptualisation and design, CDS design and execution, and patient-facing communication. This community advisory board will inform intervention refinement by helping to identify priorities and concerns of patients with lived experience with OUD, understand social will drivers of health and identify opportunities to expand intervention reach. In particular, involving the community advisory board will help and ensure that community members can see themselves represented appropriately in patient-facing materials.

Ethics and dissemination

We will secure all required approvals and protocols for regulatory compliance and human subjects protection. The protocol has received approval from our institution’s institutional review board (protocol #2000038624) with a waiver of informed consent for collecting non-identifiable information only. Any relevant modifications in protocol will be submitted to our institution’s institutional review board as amendments. Consistent with the initial EMBED trial,25 28 we have obtained a waiver of informed consent under the common rule (45 code of federal regulations 46.116),66 67 since this research involves observation of routine care that would be altered if consent was obtained. Any patient who has opted out of research in their personal health record (Epic MyChart) will not be included in this study. Additionally, posters with information on how patients may opt out will be placed at study sites. We will also have informational meetings with nurses, residents and attending physicians describing the study, informing them of this research activity. We plan to anonymise clinical staff, reducing the only risk to staff (ie, individual identification). Paradoxically, opting out would create the greatest likelihood of identification. Therefore, instead of providing a standard opt-out for staff, we will use an embedded consent model, where all practitioners are informed that data from routine practice may be analysed in anonymised form. This approach (clear communication about the study purpose, data handling and confidentiality) will ensure that practitioners are aware of the study without requiring an explicit opt-out, which could draw attention to those who choose to opt out. Moreover, there will be no collection or analysis of identifiable health information, and there is minimal risk in implementing evidence-based best practices for OUD.

No private information for patients with OUD will be collected. Patients will instead be distinguished by a unique study identifier. It will be mandatory that the identifier that is used is not an identifiable piece of PHI and that special administrative access is required to the local EHR to use this identifier to link back to patient data. Data collected from healthcare providers will include Epic Access Log, UAL, flowsheet data, alert data and medication order data associated with ED workflow for physicians, advanced practice providers, nurses and pharmacists involved in the encounters in question. This ‘audit log’ data, including time spent on specific activities, will allow for a granular analysis of CDS use. Additional provider data may be obtained from the NPPES (National Plan and Provider Enumeration System) National Provider Identifier (NPI) Registry, publicly available for download. An honest broker will then link providers in Epic to this data set via NPI and provide a deidentified report containing provider gender, role, title/degree and number of years since NPI was issued as a proxy for clinician’s age and years since medical school graduation, binned by decade. Throughout this study, it is important to note that physicians and other healthcare providers will have access to all standard-of-care medication for OUD and other interventions they normally would and retain full autonomy to make clinical decisions, including not using the proposed intervention (ie, opt-out).

Findings from this study will be published in open-access, peer-reviewed journals, as well as publicised through additional strategies, such as dissemination to the community via channels suggested by the community advisory board and informing EHR vendors.