Article Text
PDF
Abstract
Objectives We aim to determine, using routinely collected data and common scoring systems, whether parameters seen at intensive care unit (ICU) discharge can be predictive of subsequent clinical deterioration.
Design/setting A single-centre retrospective study located in a tertiary hospital in the south of England.
Participants 1868 patients who were admitted and discharged from ICU between 1 April 2023 and 31 March 2024 were screened for eligibility. A total of 1393 patients were included in the final analysis, including 122 patients who were classified in the ‘deteriorated’ subgroup.
Interventions Assessment of vital signs, blood markers of infection and inflammation and three scoring systems (National Early Warning Score 2 (NEWS2), Acute Physiology and Chronic Health Evaluation II Score and Sequential Organ Failure Assessment (SOFA) score) taken within 24 hours prior to ICU discharge.
Primary outcomes Assessment of predictors of deterioration after ICU discharge.
Secondary outcomes Reasons for readmission to ICU, hospital mortality, ICU length of stay and time before readmission to ICU.
Results Heart rate, conscious level (alert, voice, pain, unresponsive scale) and SOFA score were independent predictors of deterioration after ICU discharge (under the curve 0.85, CI 0.79 to 0.90, specificity 82.3%, sensitivity 79.7%) in multivariable models. Of these, a reduced level of consciousness was the most significant predictor of clinical deterioration (OR 19.6, CI 11.4 to 35.0). NEWS2 was an independent predictor for deterioration on univariable analysis. Mortality was significantly increased in patients who experienced deterioration after ICU discharge, as was ICU length of stay.
Conclusions Predictive models may be useful in assisting clinicians with ICU discharge decisions. Further research is required to develop patient-tailored scoring systems that incorporate other factors that are needed for decisions around ICU discharge.
- Clinical Decision-Making
- Clinical Protocols
- Adult intensive & critical care
- Observational Study
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Large data set of over 1300 patients.
Adjusted for collinearity between vital sign values and scoring systems.
Assessed individual parameters and vital signs before forming overall predictive models.
Single centre.
Comorbidities not assessed.
Introduction
The decision to discharge patients from intensive care is complex and often based on multiple factors. For most, an ideal combination of clinical, nursing and management factors leads to a timely discharge to the most appropriate ward facility. However, some patients experience unanticipated clinical deterioration after discharge, leading to unplanned readmission and adverse outcomes.1–3 Currently, there is no ideal clinical variable or scoring system available to guide suitability of patient discharge from the intensive care unit (ICU). While there are several methods used to identify potential patients who could deteriorate, there is no clear consensus.4 5 Some studies suggest scoring systems, such as the National Early Warning Score 2 (NEWS2), a system with widespread usage already in the UK and worldwide, could be an easy and effective method of screening for patients at risk of deterioration prior to ICU discharge.6–8
Developed by the Royal College of Physicians, NEWS2 is a system used to quantitatively score routine physiological parameters to identify those acutely ill or with deteriorating clinical status.9 10 Although its use is almost universal in UK hospital wards to predict patients at risk of deterioration,9 the use of NEWS2 to assess suitability of ICU discharge has not been validated in ICU patients and as such is not routinely used in ICU. In comparison, the Sequential Organ Failure Assessment score (SOFA) is validated in ICU cohorts.11 12 Based on organ systems, the SOFA score uses a multisystem-based approach to assess acute morbidity and mortality of critical illness. In recent years, this has been applied to the identification and monitoring of sepsis through the work of Sepsis-3.13 A further score often used in ICU is the Acute Physiology and Chronic Health Evaluation II Score (APACHE II). APACHE II estimates disease severity based on physiological measurements, including blood markers, along with considerations for age and chronic health conditions.14 15 It is used at the time of admission and recalculated daily in ICU for prognostic scoring, having been shown to be an accurate measurement of illness severity with correlations to clinical outcomes.15
Given their potential use, we aim to evaluate these scoring systems, along with routinely collected variables such as vital signs and blood markers, to determine if measurements taken before ICU discharge can predict those who will unexpectedly deteriorate after leaving intensive care.
Materials and methods
Study design and setting
This is a retrospective analysis of a 31-bed general intensive care (ICU) admissions and discharges between 1 April 2023 and 31 March 2024 based at a large tertiary hospital in the south of England. Patients were identified using databases that are routinely used by the ICU auditing team performed as part of the Intensive Care National Audit and Research Centre (ICNARC) data collection.16 This study is part of a wider study investigating outcomes of critical illness in intensive care (CRIT-CO). CRIT-CO has approval from the NHS Health Research Authority (HRA, UK: IRAS 232922, 26 November 2018). This study was also registered as a quality improvement project by the University Hospital Southampton Service Evaluation Team (Ref: QI/0272). This study follows local ethical standards, and no identifiable data is presented here. Given its retrospective nature and no additional information required, consent was waived.
Patient and public involvement
Patients and the public were not directly involved in this research study. However, the foundations of this research were developed after discussions with patients who had experienced unexpected deterioration after discharge from intensive care.
Data collection
Patients were identified using the Intensive Care database (MetaVision-iMDsoft, Israel). The inclusion criteria were patients 18 years and older, admitted and discharged from the ICU during a single hospital admission and had vital sign monitoring undertaken before ICU discharge. The exclusion criteria were patients who died in ICU during the first admission, discharged on an end-of-life care pathway (palliation), discharged to another ICU department (same hospital), transferred to another hospital, transferred to another care provider on ICU discharge (eg, rehabilitation unit, care home), discharged directly home or self-discharged. Patients who deteriorated were separated from the main data set through review of the ICU records. As part of routine audit data, a patients discharge location, hospital outcome and ≤48 hours/>48 hours ICU readmission are recorded. Deterioration was defined as anyone who was either readmitted back to the ICU during the same hospital admission or who died on the ward after being discharged with active treatment ongoing. Any patient who had more than one admission to the ICU over the year period, but which occurred in separate hospital admissions, was included as individual entries.
We collected standard variables such as age, gender and body mass index (BMI). Individual vital sign data (heart rate (HR), respiratory rate (RR), systolic blood pressure (SBP), temperature, oxygen saturation (SpO2) and conscious level (alert, voice, pain, unresponsive scale (AVPU))) was collected using the last recorded set of vital signs before ICU discharge. For all, these measurements were within the preceding 4 hours before discharge. For assessment of SOFA and APACHE II scores, these were calculated using the most abnormal values in the preceding 24 hours prior to discharge.12 15 NEWS2 was calculated using the vital sign data as described previously. Biochemical data (total white cell count (WCC), lymphocyte count, neutrophil count and C-reactive protein (CRP)) was taken using the last recorded value, which for all was within 24 hours prior to ICU discharge. A scoring matrix for NEWS2, SOFA and APACHE II can be seen in online supplemental appendix 1. Other data, such as length of ICU stay, hospital mortality, timing and reason for readmission, were determined via review of ICU records.
Supplemental material
Outcomes
The main outcome of this study was to identify patients discharged from ICU who had unexpected clinical deterioration to determine if there were any predictors of subsequent deterioration at ICU discharge. Secondary outcomes included determining reasons for readmission to ICU, timing of ICU readmission, hospital mortality and ICU length of stay (LOS).
Statistical and data analysis
Differences in baseline characteristics between groups were described with median and 25th–75th percentile for continuous variables and counts with percentages for categorical groups. Each vital sign parameter was analysed individually and ICU discharge data (this included: LOS, APACHE II score on admission, SOFA Score on discharge, laboratory markers of infection/inflammation). Comparisons between groups were made using Mann-Whitney U and Fisher’s exact test for continuous and binary variables, respectively.
Logistic regression models were constructed for the prediction of re-escalation. Variables with a significance threshold of p<0.25 within univariable models were included in multivariable analysis. Subsequently, backwards selection was performed using the Akaike information criterion (AIC) to produce the final models. Overall models were further described with receiver-operating characteristic curves (ROC) and McFadden’s pseudo-R2. On significance testing, p values<0.05 were deemed significant throughout the analysis. All analyses were performed using R (V.4.2.2). All figures were formed using BioRender.com.
Results
We screened 1868 discharges for this 12-month period. After exclusions, the final data set included a total of 1393 patients, with 122 patients who clinically deteriorated (8.76%). Of these 122, 74 patients were readmitted to ICU (5.3% readmission rate) (figure 1).
Flowchart showing the formulation to the final data set.38 ICU, intensive care unit.
Patient characteristics and readmission profile
The average age was similar between the two groups and the proportion of male patients was also similar. Both groups had a mean BMI within the ‘overweight’ group according to National Health Service guidance17 (table 1). Of those readmitted, 41.9% required treatment for acute hypoxia and 18.9% for unplanned postoperative care. Other reasons for readmission included hypotension (5/74 (6.8%)), bleeding (leading to haemodynamic compromise) (4/74 (5.4%)), renal support (5/74 (6.8%)), reduced level of consciousness (6/74 (8.1%)) and seizure activity (5/74 (6.8%)). 15 patients had more than one reason (eg, hypoxia and hypotension) and so were included in both categories. 10 patients (13.5%) developed sepsis on the ward requiring organ support and 3 patients had a cardiac arrest on the ward leading to ICU readmission. Four patients (5.4%) became more unwell from medical and nursing perspectives without one defining factor and so were classified as ‘increasing acuity needs’. Similar proportions of each group were discharged out of hours, including over the weekend.
Patient characteristics, scores and outcomes
Comparison of vital signs at the point of discharge
Median RR was 18 per min for both groups with differences in 25th–75th percentile distinguishing the two (stable: 16.0–20.0, deterioration 17.0–20.0). HR, SpO2 and SBP were different in those that deteriorated compared with those that remained stable. A higher proportion of patients were discharged with supplemental oxygen in the deterioration group, 54.9% versus 39.8%, p=0.01. AVPU for both groups had a majority rated ‘Alert’ (stable: 88.7%, deterioration 82.8%) and temperature did not differ for both groups (table 2).
Statistical analysis of vital signs included in the NEWS2 score taken before discharge
Analysis of individual vital signs prior to ICU discharge showed statistically significant differences in RR, HR, SpO2, AVPU and oxygen requirement between the two groups (table 2). Although differences in RR are statistically significant, they are not clinically significant given that the median value is 18.0 for both groups. Moreover, despite the statistical difference, the HR and the SpO2 in the deteriorated group remained within normal range.
Comparison of blood markers for infection and inflammation
For those that remained stable, WCC, neutrophils, lymphocytes and CRP were similar to those who deteriorated.
There was no significant difference in WCC, neutrophils, lymphocytes or CRP (table 3). No blood marker was predictive of ICU readmission on univariable regression analysis (table 4).
Statistical analysis of blood markers taken before discharge
Analysis using univariable and multivariable regression
Comparison of common scoring systems at discharge
National Early Warning Score 2
NEWS2 taken before discharge from ICU was significantly different between the two groups. For those in the stable group, the average NEWS2 was 2.0 (1.0–4.0) compared with 4.0 (2.0–6.0), in those who clinically deteriorated. NEWS2 on readmission was significantly increased to 7.0 (5.0–10.0) as expected when patients returned to the unit critically unwell. On univariable logistic regression (table 4), higher NEWS2 values were associated with deterioration after discharge (OR 1.1, CI 1.0 to 1.2). However, NEWS2 alone was a poor predictor of readmission with area under the curve (AUC): 0.6 (0.5–0.6). As NEWS2 is formed from the individual vital signs, we did not include it in the multivariable models to limit collinearity.
Acute Physiology and Chronic Health Evaluation II
The APACHE II on first admission was similar between the two groups (stable 16.0 (12.0–19.0), deterioration 16.0 (14.0–19.3)). When APACHE II was considered in regression analysis, it did not show correlations to clinical deterioration after ICU discharge (OR: 1.1, CI: 1.0 to 1.1).
Sequential Organ Failure Assessment
SOFA score at first discharge was similar between the groups (stable: 4.0 (3.0–5.0), deterioration: 4.0 (3.0–6.0)). SOFA score showed no correlation with readmission in univariable logistic regression (OR: 1.1, CI: 1.0 to 1.2). However, using AIC to form multivariable models, SOFA score improved the overall model when included with HR and AVPU (figure 2B).
Receiver operator characteristic curves: (A) Curve for model 1 which included heart rate and AVPU scoring. (B) Curve for model 2 which included heart rate, AVPU and SOFA scoring. Area under the curve (AUC) is reported on each graph with their respective CIs. Point 0.0 represents the point of optimum specificity (%), sensitivity (%).39 AVPU, Alert, voice, pain, unresponsive; SOFA, Sequential Organ Failure Assessment.
Multivariable modelling and receiver operator characteristic curves
ROC curves produced from backwards step elimination using AIC produced two multivariable models. The first model (figure 2A) considered all vital sign parameters (HR, RR, etc) included within the NEWS2 score. After backwards step elimination, HR and AVPU were predictive of clinical deterioration (AUC: 0.84, CI 0.8 to 0.9, specificity 86.2%, sensitivity 75.4%, Pseudo R2 0.2). The second model (figure 2B) additionally considered SOFA and APACHE II with HR and AVPU included as predictors of deterioration after ICU discharge. After backwards step elimination, SOFA, HR and AVPU were included in the model (AUC 0.84, CI 0.8 to 0.9, specificity 89.9%, sensitivity 71.0%, Pseudo R2 0.2). Of the final predictors, reduced conscious level at discharge (VPU on AVPU scale) held the strongest predictive power of post-ICU deterioration.
Outcomes: duration of ICU stay
Patients who deteriorated stayed an average of 1 day longer in ICU (4.0 (2.0–8.0)), before discharge compared with those who remained stable (3.0 (1.8–5.0)). Readmissions occurred at a median of 94.5 hours (50.0–250.3) after discharge from ICU.
Outcomes: hospital mortality
The hospital mortality was 287/1614 (17.7%), which includes 193 patients who died in ICU, 28 palliative patients who died expectantly on the ward and 66 patients who unexpectedly deteriorated and died. This total patient number includes all those included in the final data set plus those that died during their first ICU admission or were discharged palliatively. Mortality data was not available for those who were excluded due to other discharge destinations or those with incomplete data. The hospital mortality was 54.1% (66/122) in those that deteriorated after ICU discharge. Of those, the subgroup that required readmission had an overall hospital mortality of 24.3% (18/74). All three patients who required readmission after cardiac arrest died within 72 hours of the event. They all experienced asystolic cardiac arrests with evidence of hypoxic–ischaemic encephalopathy on subsequent testing.
Discussion
In this study, we identified 122 patients from 1393 ICU discharges who experienced unexpected deterioration during the 1-year study period. Secondary deterioration after ICU discharge was associated with higher hospital mortality and a longer first admission ICU LOS. While the evaluation of clinical variables and scoring systems at discharge suggests HR, oxygen requirement and NEWS2 are predictive on univariable analysis, multivariable modelling suggests HR, AVPU and SOFA score to be predictive of clinical deterioration after ICU discharge (table 4). Although our readmission rate is higher (5.3%) than the UK-wide audit data from the ICNARC (1.2%), which only includes unplanned readmission rate within the first 48 hours of discharge, our data is inclusive of all readmissions at any time point post-discharge.16 However, our patient demographics and readmission rates were similar to previously published work.18–20 To our knowledge, this is the largest study in the UK assessing the predictive capabilities of routinely collected individual vital signs and NEWS2 scoring at discharge from a general ICU.
Previous studies7 21–23 have suggested that NEWS scoring at discharge can be predictive of clinical deterioration after ICU discharge. Of these studies, two report similar NEWS values to our results, with average NEWS for stable patients of 2.5 and 2.3 compared with 3.7 and 5.5 for those that showed clinical deterioration.7 23 The other two studies report much higher average NEWS values of 3.0 and 4.0 for stable patients and 9.1 and 10.0 in those that deteriorated after ICU discharge.21 22 These differences may have been due to other factors such as population, resources and clinical pressures. We found NEWS2 scoring to be predictive of clinical deterioration after ICU discharge on univariable analysis. On the breakdown of NEWS2 components, only HR and AVPU were predictive in our final models. The predictivity of HR has been assessed in only a few studies, with all including it within multivariable models.24–26 In comparison, acute changes to the level of consciousness have been shown to be a sign of clinical deterioration in isolation.27–29 In this study, patients with a conscious level to voice, pain or unresponsive were 19× more likely to deteriorate after leaving ICU compared with those who were classified as alert at ICU discharge. Compared with the other factors in our final model, AVPU was the most sensitive marker of subsequent unexpected deterioration. Oh et al also reported a predictive ability of altered conscious level, yet in their study only 10% of patients discharged from ICU had a Glasgow Coma Scale (GCS) below 13 and an average GCS of 14.4 at discharge.27 This supports our data with an over 80% majority in both groups reported as ‘alert’ on an AVPU scale. It is possible that reducing conscious level as a marker of patient deterioration may be limited by the broad categories of AVPU within the NEWS2 score.
In our study, the median NEWS2 score was 4 for those subsequently deteriorated and 2 for those that remained stable. Although statistically significant, according to the NEWS2 protocol, both scores would classify in the ‘low clinical risk’ category (if no more than 3 in any one category). Although HR, RR and SpO2 were different between the two groups, these were only marginal differences, and so the clinical relevance of this should be considered. In comparison to other UK hospitals, studies reporting average NEWS values are limited. Chiu et al assessed discharge NEWS after cardiac ICU discharge.30 Although no value is reported, only 33% of patients reached a threshold of NEWS >3 after 24 hours postdischarge, suggesting a low average NEWS value. Scott et al used NEWS2 to assess all-cause deterioration and found 50% to have low values of 1–2 on admission.18 This therefore highlights that subtle changes in NEWS2/vital sign values are likely to be the early clues for acute deterioration. We therefore suggest that an awareness of trends is likely to be a better clinical representation of overall physiology.
The median time from discharge to readmission was 94 hours, which is similar to previous work by Johnson et al who found an average 4-day stay before readmission.31 Studies assessing EWS often use time cut-offs, most commonly 48 hours, before readmission to determine their cohorts.32 33 Although they highlight patients who quickly deteriorated, our findings of an average ward stay of over 3 days before ICU readmission suggest that limits of under 2 days are likely to exclude a large proportion of patients in subsequent analysis and models. It also emphasises the need for careful patient monitoring within the first 4 days after ICU discharge. At our centre, patients discharged from critical care have daily surveillance by critical care outreach teams (CCOT) up until clinically stable or placed on palliative care pathways. The use of CCOT is recommended by the National Institute for Health and Care Excellence for patients at risk of acute deterioration, but not standardised within the UK, with CCOT provision varying greatly between centres.34
There are several notable limitations to our study. This was a single centre, retrospective, observational study, and the NEWS2 and vital sign parameters were taken as a single snapshot of physiology at discharge. As the electronic systems used within our hospital change at ICU discharge, we were unable to track NEWS2 changes over time which may have provided valuable trend analysis. The data presented here are related to general ICU admissions and are not specific to specialist ICU patients such as cardiac. Consequently, the results may not be transferable to other specialist centres. We have not included comorbidities or the initial diagnosis in our analysis due to limitations with data collection, and as such, we are unable to identify patient-specific factors that may increase the risk of readmission. Despite these limitations, this is one of the largest studies to explore the predictive variables of readmission risk captured at ICU discharge. Moreover, there were no exclusions, and we included all ICU admissions and discharges with the intention for subsequent escalation if there is clinical deterioration.
At present, there is no consensus for a risk stratification tool prior to ICU discharge. The Society for Critical Care Medicine suggests that ‘discharge parameters should be based on ICU admission criteria, the admitting criteria for the next lower level of care, institutional availability of these resources, patient prognosis, physiological stability and ongoing active interventions’ and that ‘severity of illness scores should not be used as a sole reason for discharge’.35 This is supported by the work into predictive modelling whereby single models are outperformed by those trained with targeted approaches or machine learning.36 37 Together, these suggest that although physiological factors on discharge may be useful in determining the predictability of post-ICU deterioration, further work is required to develop decision aids that combine multiple predictors tailored to each patients’ individual risks, of which scoring systems may play a role. While this study highlights that there are some helpful predictive markers of deterioration at discharge, the variables may still be regarded as within the range of physiological normality. Moreover, there are several other factors such as patient-specific variables, disease-specific variables, availability of CCOT and the frequency of monitoring that will all require consideration, and as such, no single scoring system is comparable to clinical judgement.
Conclusion
Our study found predictive ability of HR, AVPU and SOFA score at discharge between those that were subsequently deteriorated after ICU discharge and those who remained physiologically stable. Together, these show that acute physiological changes prior to discharge, alongside severity of organ dysfunction secondary to ICU illness, are important factors to consider when discharging patients. Scoring systems, including NEWS2, may have a role as supportive tools but should not be used as a sole indicator for ICU discharge. Further work using predictive modelling and scoring systems is required.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
This study is part of a wider study investigating outcomes of critical illness in intensive care (CRIT-CO). CRIT-CO is sponsored by University Hospital Southampton NHS Foundation Trust (RHM CRI 0370) and has approval from the NHS Health Research Authority (HRA, UK: IRAS 232922, 26 November 2018).
Acknowledgments
The authors would like to thank Professor Neil White and Lisa Showell for their help with data acquisition and interpretation.
References
Footnotes
X @BrownMark52728
Contributors Conceptualisation: IT, AD. Methodology: IT, MB, AS, AP. Data collection: IT, RB, OC. Data analysis: IT, AS, AP, MB, RB. Writing: IT, AS, MB, OC, AP. Supervision: AD. Neil White and Lisa Showell assisted with data acquisition and interpretation.
Overall guarantor: IT.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.