Article Text
PDF
Abstract
Introduction Headache is a common chief complaint of children presenting to emergency departments (EDs). Approximately 0.5%–1% will have emergent intracranial abnormalities (EIAs) such as brain tumours or strokes. However, more than one-third undergo emergent neuroimaging in the ED, resulting in a large number of children unnecessarily exposed to radiation. The overuse of neuroimaging in children with headaches in the ED is driven by clinician concern for life-threatening EIAs and lack of clarity regarding which clinical characteristics accurately identify children with EIAs. The study objective is to derive and internally validate a stratification model that accurately identifies the risk of EIA in children with headaches based on clinically sensible and reliable variables.
Methods and analysis Prospective cohort study of 28 000 children with headaches presenting to any of 18 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). We include children aged 2–17 years with a chief complaint of headache. We exclude children with a clear non-intracranial alternative diagnosis, fever, neuroimaging within previous year, neurological or developmental condition such that patient history or physical examination may be unreliable, Glasgow Coma Scale score<14, intoxication, known pregnancy, history of intracranial surgery, known structural abnormality of the brain, pre-existing condition predisposing to an intracranial abnormality or intracranial hypertension, head injury within 14 days or not speaking English or Spanish. Clinicians complete a standardised history and physical examination of all eligible patients. Primary outcome is the presence of an EIA as determined by neuroimaging or clinical follow-up. We will use binary recursive partitioning and multiple regression analyses to create and internally validate the risk stratification model.
Ethics and dissemination Ethics approval was obtained for all participating sites from the University of Utah single Institutional Review Board. A waiver of informed consent was granted for collection of ED data. Verbal consent is obtained for follow-up contact. Results will be disseminated through international conferences, peer-reviewed publications, and open-access materials.
- accident & emergency medicine
- paediatric A&E and ambulatory care
- migraine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We are prospectively enrolling a diverse group of children with headaches from one of 18 paediatric emergency departments participating in the Pediatric Emergency Care Applied Research Network (PECARN), which collectively sees more than 1.1 million patients annually and has geographic representation across the USA.
The eligibility criteria identify a spectrum of children with chief complaints of headaches for whom clinicians have greater uncertainty regarding the presence of emergent intracranial abnormalities.
Clinicians perform standard evaluations to prospectively collect patient history and physical examination findings that are potential predictors of emergent intracranial abnormalities.
We will analyse the data using binary recursive partitioning and multiple logistic regression analyses to derive the risk stratification model.
Study results may not be generalisable to children with pre-existing medical or neurosurgical conditions or those with neurological or developmental conditions for whom history or physical examination may be unreliable.
Introduction
More than 400 000 children present annually to emergency departments (EDs) in the USA with a chief complaint of headache.1–3 Most of these children have headaches that are primary (eg, migraines) or secondary to conditions such as respiratory infections. Approximately 0.5%–1%, however, are associated with intracranial abnormalities requiring emergent identification, such as brain tumours, haemorrhages or strokes.2 4–7
Although emergent neuroimaging has a role for a small subset of children, as many as 36% of children presenting to EDs in the USA with headaches or migraines receive neuroimaging in the ED.1 3 4 8 9 Overuse of CT, the most commonly used emergent neuroimaging for headaches, exposes children to unnecessary radiation, with an estimated lifetime risk of inducing lethal malignancies between 1 per 1000 and 1 per 5000 CT scans, depending on radiation dose and patient age.1 5 8 10–13 The main alternative, MRI, is not always available in the ED, is not time efficient in the ED setting and may require procedural sedation with its associated risks and time intensiveness.14–16 Finally, neuroimaging of any type may identify inconsequential findings that lead to unnecessary testing and interventions and unwarranted patient and parental concerns.17 18
The overuse of ED neuroimaging for complaints of headaches reflects the concern among clinicians for life-threatening, intracranial abnormalities requiring emergent interventions. Neuroimaging overuse also reflects the lack of clarity regarding which clinical characteristics, or ‘red flag findings’, can be used to accurately identify children with headaches who may have emergent intracranial abnormalities (EIAs). Red flag findings in current use were derived from research studies that were methodologically limited (eg, retrospective studies, biased study populations) and/or of insufficient sample size.6 19–30 The current frequency of emergent neuroimaging and the relative lack and limitations of prior research highlight the clear need for well-designed, large prospective studies to identify the risk of EIAs in children with headaches based on specific clinical factors.
Younger children with headaches require special consideration. Prior studies suggest that young age is a risk factor for EIAs; the relationship between age and risk of EIA, however, is unclear and needs to be systematically defined.8 14 29 31–36 In addition, risk factors for EIAs may differ based on age, partly due to challenges in eliciting signs and symptoms in younger children. These issues may lead to diagnostic uncertainty and increased rates of neuroimaging in younger children, who are at greater risk of radiation-induced lethal malignancies from CT and adverse events during sedation.11 12 37 38
Objectives
ED clinicians require specific recommendations based on precise estimates of the risk of EIAs to facilitate appropriate use of emergent neuroimaging for children presenting with headaches. Therefore, the objective of the study is to generate the definitive evidence that will allow clinicians to identify the risk of EIAs in otherwise healthy children presenting to EDs with chief complaints of headaches. The aims of the study are as follows: (1) to derive and internally validate a stratification model for children presenting to the ED with headaches that identifies the specific risk of EIAs based on clinically sensible and reliable variables, and (2) to determine whether the prevalence of EIAs and association between risk factors and EIAs differs by age.
Methods and analysis
Study overview
We are conducting a prospective, multicentre cohort study titled, ‘Headache Assessment of Children for Emergent Intracranial Abnormalities (HEADACHE). This study is enrolling children with headaches evaluated in any of the EDs in the Pediatric Emergency Care Applied Research Network (PECARN). PECARN is a federally funded multi-institutional network consisting of 18 paediatric EDs with geographic representation across the USA. Collectively, the EDs in PECARN have approximately 1.1 million paediatric visits annually.39 Enrolment started in February 2021 and is anticipated to end in 2024.
Study population
Inclusion criteria
Children are eligible for inclusion from the age of 2–17 years old (ie, before their 18th birthday) if they present to the ED with headache as a chief complaint (or per parent or the clinician). Headache may be present either by itself or in conjunction with other chief complaints and includes patients who do not have a headache at the time of ED evaluation.
Exclusion criteria
The eligibility criteria identify a spectrum of children with headaches for whom clinicians have greater uncertainty regarding the presence of EIAs. As such, we exclude patients for whom there are very low concerns for EIAs, predisposing conditions that would potentially render the study results less generalisable, or substantial concerns based on pre-existing conditions such that ED neuroimaging is more clearly necessary (Box 1).
Eligibility criteria
Inclusion criteria
Age 2–17 years old.
Headache is a chief complaint of the patient/parent or per clinician.
Exclusion criteria
Clear, non-intracranial alternative diagnosis or aetiology at presentation.
Documented temperature of ≥38°C within prior 24 hours.
Neuroimaging performed within previous year.
Neurological or developmental condition such that patient history or examination may be unreliable.
Glasgow Coma Scale score<14.
Intoxication.
Known pregnancy.
History of intracranial surgery.
Known structural abnormality of the brain.
Known pre-existing condition predisposing to an intracranial abnormality or intracranial hypertension.
Head injury within previous 14 days.
Prior enrolment in study.
Foster child or ward of the state.
Patient and/or the parent/legal guardian do not speak English or Spanish.
Study procedures
Participant screening and consent
Participants are screened for eligibility at all hours of the day at all participating sites. Screening criteria include any patient meeting age and chief complaint criteria. We have a waiver of informed consent from the University of Utah single Institutional Review Board (IRB) to collect information related to the patient history and physical examination. Verbal consent is obtained from the parent/legal guardian to conduct text messaging and/or telephone follow-up.
Data collection
Eligible participants undergo a standard evaluation that includes a medical history and physical examination. The attending physician, fellow, nurse practitioner or physician assistant caring for the patient completes a standardised case report form prior to knowledge of neuroimaging results (if performed). Participants are considered enrolled if any part of the case report form is completed. For a subset of participants, a second attending physician, fellow, nurse practitioner or physician assistant performs an independent medical history evaluation and physical examination and completes the same standard case report form within 60 min of the primary evaluation to determine the inter-rater reliability of predictors that will be considered for use in the risk stratification model.40
Data are collected to characterise the enrolled population, their clinical course and symptom severity. These data include patient demographics, vital signs, weight, Emergency Severity Index, neuroimaging performed, treatments administered, procedures performed, consultations obtained, ED disposition, return ED visits within 72 hours and diagnoses. We are collecting data to evaluate reasons clinicians ordered neuroimaging (if applicable) and the clinician’s assessment of risk of EIA for each child.
Participant follow-up
Follow-up procedures depend on whether participants undergo neuroimaging in the ED, the type of neuroimaging they receive and the results of the neuroimaging (when applicable). Participants who receive a regular cranial MRI that is interpretable—irrespective of findings—do not undergo follow-up because outcome determination has been completed. Participants who receive a cranial CT or rapid MRI that identifies an EIA do not undergo follow-up. However, participants undergo follow-up if CT or rapid MRI from the ED reveals normal findings, sinus findings or any other non-EIA intracranial abnormality.
For participants who consent to follow-up and do not have neuroimaging performed in the ED or have neuroimaging with only limited or uninterpretable results, we perform follow-up using monthly text messaging for up to 6 months after the index ED visit and a telephone call, if needed, based on the results of the texting. The text messages sent during months 1–5 ask a single question assessing whether the participant underwent neuroimaging after the index ED visit. The text message sent at month 6 asks two questions: one assessing whether the participant underwent neuroimaging after the index ED visit, and the second assessing if the participant had a healthcare visit for headache-related reasons after the index ED visit. If the parent/legal guardian answers ‘no’ to both 6 month text message questions, no further follow-up is conducted. If the parent/legal guardian answers ‘yes’ to any of the monthly text messages, or if either of the 6 month text messages are unanswered, study personnel perform a medical record review. If any new neuroimaging, or treatment or intervention indicative of an intracranial abnormality is identified in the medical record, no further follow-up will be conducted. If no such record is identified, a telephone follow-up is performed to ascertain if the patient received any neuroimaging, or underwent any treatments or interventions indicative of an intracranial abnormality. We consider patients lost to follow-up if a telephone call was indicated but unable to be completed. Patients who did not consent to follow-up procedures undergo 6 month medical record reviews and are considered lost to follow-up if no relevant medical record is identified. The primary analysis considers patients with an unknown outcome (ie, did not respond to both 6 month text message questions, could not complete telephone call and no medical record identified) as being negative for the primary outcome. A sensitivity analysis will be performed that considers patients with unknown outcomes as missing.
Missed eligible patients
An assessment of patients who were eligible but not enrolled (ie, missed eligible) is being performed to evaluate for biased patient enrolment. We perform medical record reviews for patients presenting to participating EDs on two randomly selected days per week to assess for missed eligibility. Data collected for these patients include demographics, disposition, any neuroimaging (and associated findings), neurosurgical procedures or interventions performed during index ED visit, and hospitalisation directly from index ED visit and/or within 6 months after the index ED visit. These patients will be compared with enrolled patients to assess for biased enrolment.
Potential predictor variables
Potential predictors of EIA to be evaluated were selected through extensive literature review and expert consensus. Table 1 lists examples of potential predictors; a full list of the variables collected is found in the online supplemental table.
Supplemental material
Examples of potential predictors of emergent intracranial abnormalities in children with headaches
Outcome measures
The primary outcome is the presence of an EIA, defined as an intracranial finding for which one of the following interventions is indicated at the index ED visit: (1) neurosurgical intervention; (2) directed medical intervention (eg, chemotherapy); (3) interventional radiological procedure (eg, endovascular thrombectomy) or (4) hospital admission to monitor for potential clinical deterioration, obtain additional diagnostic evaluation or perform another intervention specifically targeting the intracranial abnormality. Based on these criteria, the specific diagnoses included as EIAs are listed in table 2. This definition and the diagnoses included were determined by an expert consensus group consisting of paediatric emergency medicine physicians, neurologists, neuro-oncologists, and neurosurgeons.
Emergent intracranial abnormalities
Secondary outcomes include the presence of serious intracranial abnormalities (SIA) or incidental intracranial abnormalities (IIA). We defined SIAs as intracranial findings that the consensus panel did not consider emergent but have the potential to be the cause of the headache (depending on characteristics such as size or location of the finding) and potentially require an intervention as above. We defined an IIA as a finding that is neither an EIA nor SIA, may or may not require outpatient follow-up and is unlikely to be the cause of the headache. Although termed incidental in accordance with prior literature, these findings may still be concerning to patients and families and elicit further evaluation. Diagnoses assigned to the categories of SIA and IIA are listed in table 3.
Serious and incidental intracranial abnormalities
A centralised review by the lead study investigators (DST, PSD and NK) and neurologist coinvestigator (LR) makes the determination by consensus of whether an abnormality is an EIA, SIA or IIA. This review is based on available radiology reports, medical records, follow-up text message responses and information gathered from telephone calls. It is conducted without any knowledge of associated clinical variables. If the classification of the abnormality cannot be determined by this centralised review process, the case is referred to an independent adjudication panel, who determines the classification of the abnormality by consensus.
Data analysis
Sample size and power
The derivation of the risk stratification rule will be conducted in two parts with two goals: a near-zero risk classification and a risk stratification model for patients not at near-zero risk. The sample size was determined based on the desired sensitivity of the near-zero risk classification model. We used the presence of an EIA as the main outcome to determine the sample size, because a risk stratification model for EIA must have a nearly perfect sensitivity to identify those at near-zero risk of EIAs. Specifically, we aim to enrol at least 140 patients with EIAs, such that a model with a minimum 99.3% sensitivity (ie, at most one missed EIA of the 140) will have a lower boundary of the 95% CI for sensitivity greater than 95%. After univariable screening (p<0.1), the number of variables we will consider for inclusion in a multivariable logistic regression analysis to derive a risk score without needing to employ lasso penalisation is 15 (approximately one-tenth of the expected 139 patients with observed EIAs who do not meet near-zero risk criteria).
To enrol 140 patients with EIAs, we aim to enrol 28 000 eligible patients with headaches over a period of at least 3.5 years. Using the PECARN clinical registry, we estimated that 1%–1.5% of ED visits met eligibility criteria across all sites between 2012 and 2014.41 With 1.1 million annual visits to PECARN EDs and expected enrolment of 80% of eligible patients, we estimated enrolling 8000 eligible patients with headaches annually. We expected to have outcome data (ie, neuroimaging or follow-up results) for at least 80% of the 28 000 patients enrolled (ie, 22 400 patients). Based on prior literature describing children presenting to EDs with headaches, we conservatively assumed that 0.7% of enrolled patients in whom follow-up is completed will have EIAs.2 4–7 This would result in having 156 patients with EIAs in 3.5 years, which is greater than our desired 140 patients.
Statistical analysis plan
To derive the near-zero risk component of the risk stratification model, we will use binary recursive partitioning.42 Patients with missing predictors will be included by substituting surrogate variables that partition patients in a way similar to the missing variables. However, if more than 20% of the data for any variable are missing across all sites, that variable will be excluded. We will also exclude variables with kappa statistics less than 0.5, calculated on those patients with two assessments.
In the construction of the decision tree, we will assign misclassification costs to specific misclassification errors. We will vary the assigned value of the relative misclassification cost of not identifying a patient with an EIA from 100 to 1000 relative to misclassifying a patient who is at low risk for having an EIA and assess how this impacts tree creation. We will use classification and regression tree software (CART; SPM Salford Predictive Modeler®; Minitab) to perform the recursive partitioning analysis and will internally validate the risk stratification model using 10-fold cross validation. We will also enter each PECARN site as a dummy variable into the analysis to explore whether any site exerts disproportionate influence in model generation. For the primary analysis, patients lost to follow-up will be considered not to have an EIA. Sensitivity analyses will be performed by excluding patients for whom the primary outcome could not be determined.
To complement the near-zero risk component of the risk stratification model created by recursive partitioning, we will use the same candidate variables to perform multiple logistic regression analyses to derive a risk score model for those patients who do not meet near-zero risk criteria. We will first conduct single variable logistic regression to identify all variables with associations (p<0.1) with EIAs and include these for consideration in the multivariable model. The multivariable model will be based on a combination of best subsets and bidirectional stepwise selection at p<0.1 if there are at most 15 candidate variables; otherwise, we intend to use lasso estimation but may use forward selection (p<0.1) if lasso estimation is unwieldy given the multiple imputation of missing data.
We will also perform the multivariable logistic regression approach on the entire cohort (including near-zero risk patients) and compare the performance (ie, concordance-statistic) and prediction calibration with the model that best assesses near-zero risk. We will conduct these analyses for our primary outcome (ie, presence of EIA) and our secondary outcomes of the presence of an EIA or SIA. We will use SAS software V.9.4 or higher (SAS Institute, Cary, NC) or other statistical software to perform all regression analyses. As an exploratory analysis, we will also use random forests (and possibly other machine learning algorithms) to derive a prediction algorithm for EIA. We will use SPM Salford Predictive Modeler® and R software (www.R-project.org) to perform the random forests analyses.
To assess how age relates to prevalence of EIA among children presenting to the ED with headaches as chief complaints, a logistic regression model will be fit to the primary outcome with only age as a predictor. In one model, age will be categorised using thresholds determined by Schwarz’s Bayesian Information Criterion from among candidates deemed clinically relevant by study PIs. In another model, age will be entered with linear trends (and if warranted, higher order polynomial trends or even cubic splines with up to two interior knots). In the event anything more than a quadratic trend is included, graphical depiction with 95% pointwise confidence bands will be used to summarise the relationship between age and log-odds of the presence of EIA.
To explore the effect of age on risk stratification, we will include age as a potential predictor in all stages of model derivation. We will also examine the performance characteristics (eg, sensitivity and specificity) of our final risk stratification model as a function of age and will perform multivariable logistic regression analyses using our derived risk score and age as predictors. We will consider age as both a continuous variable (possibly including a quadratic term) and a categorical variable. If we suspect an age-specific relationship, we will explore the derivation of separate risk stratification models in different age groups. Furthermore, recognising that it may be more difficult for clinicians to ascertain some assessments in younger patients, we will examine missingness of potential predictors by age category (with the age categories not necessarily prespecified). If key predictors from the primary analysis have widely variable missingness rates across age groups, we will consider deriving age-specific rules.
Patient and public involvement
This research was planned without patient involvement. Patients did not comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.
Ethics and dissemination
This study poses minimal risk to participating children and their families. Ethics approval was obtained from the University of Utah single IRB, whose determination was reviewed and accepted by the local IRBs of participating sites. Patients receive standard care in the ED. There is no change in the ED care provided for study purposes, and patients are not subjected to any interventions. In particular, neuroimaging performed in the ED is at the discretion of the clinician caring for the patient. Children are enrolled irrespective of whether ED neuroimaging is obtained. The only possible risk is a minor risk of loss of confidentiality. Local sites store identified data necessary for participant tracking and follow-up procedures in locked filing cabinets and/or in secured electronic data systems in locked offices. A waiver of informed consent was granted for collection of ED data because the study is minimal risk, and to avoid patient enrolment bias from incomplete enrolment that would lead to invalid and non-generalisable results. Verbal consent is required for follow-up text and/or telephone contact. Written consent was not required because the study is minimal risk and does not involve any procedures for which written consent is normally required outside of the research context. Families can withdraw at any time without explanation.
Results will be disseminated at regional, national and international conferences and through peer-reviewed research publications. PECARN social media and creation of free open-access materials will also be used for dissemination of results.
Limitations
We anticipate several limitations of this study. First, study results may not be generalisable to children with pre-existing medical or neurosurgical conditions who have increased risk of EIA or those with neurological or developmental conditions for whom history or physical examination may be unreliable. Similarly, study results may not be applicable to children with headaches for whom clinicians may have a lower suspicion for an EIA (eg, children with documented fever or a clear non-intracranial alternative diagnosis or aetiology). However, our rule will provide important information for the cohort of children with headaches who pose the greatest degree of diagnostic uncertainty for clinicians. Second, we will not be obtaining definitive neuroimaging on all participants because we could not ethically justify exposing children to the risks associated with neuroimaging if the clinician did not think it was indicated. However, we have an extensive follow-up plan that accounts for symptom intervals (ie, duration of symptoms before diagnosis) for EIAs such as brain tumours. This type of follow-up is an acceptable alternative for outcome determination when definitive testing is not feasible or ethical.
Discussion
This study will create a robust and precise stratification model that will enable clinicians to accurately determine the risk of EIAs in children with headaches based on clinical findings. The data will fundamentally improve how children with headaches presenting to the ED are managed by providing definitive evidence to facilitate the clinician’s decision to obtain or forgo emergent neuroimaging. Future implementation of this risk stratification tool will facilitate the safe reduction of unnecessary emergent CT and MRI scans and decrease exposure to risks associated with neuroimaging in children with headaches.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
NK and PSD are joint senior authors.
Contributors DST, NK and PSD conceived the study and wrote the initial draft of the manuscript. TCC and BJB are the methodologists that contributed to study design and plan and will execute the statistical analysis plan. LR contributed substantive intellectual input into the design of the study. DST, DBL, PJO, CRM, SM, JKS, RDM, LB, SPS, MDJ, EK, KSQ, DS, ATC, AJR, DT, JMG-P and TJ are the initial site investigators for this study. All authors contributed to study design and execution, critically reviewed and edited the initial protocol draft and approve of the final manuscript. DST takes responsibility for this manuscript as a whole.
Funding This work is supported by the National Institute of Neurological Disorders and Stroke (NINDS) grant number R01NS110826. In addition, PECARN is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS), in the Maternal and Child Health Bureau (MCHB), under the Emergency Medical Services for Children (EMSC) program through the following cooperative agreements: EMSC Data Center (EDC)-University of Utah (UJ5MC30824), GLACiER-Nationwide Children’s Hospital (U03MC28844), HOMERUN-Cincinnati Children’s Hospital Medical Center (U03MC22684), PEMNEWS-Columbia University Medical Center (U03MC00007), PRIME-University of California at Davis Medical Center (U03MC00001), CHaMP node- State University of New York at Buffalo (U03MC33154), STELAR - Seattle Children's Hospital (U03MC33156), and SPARC- Rhode Island Hospital/Hasbro Children's Hospital (U03MC33155). This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government. HRSA/ EMSC Contact: Patty Fanflik (pfanflik@hrsa.gov)
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; peer reviewed for ethical and funding approval prior to submission.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.