If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Predicting hospital mortality and length of stay: A prospective cohort study comparing the Intensive Care Delirium Screening Checklist versus Confusion Assessment Method for the Intensive Care Unit
Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USAMarcus Institute for Aging Research, Hebrew Senior Life, Boston, MA, USA
The objective of this study was to compare two tools, the Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method for the intensive care unit (ICU) (CAM-ICU), for their predictive validity for outcomes related to delirium, hospital mortality, and length of stay (LOS).
Methods
The prospective study conducted in six medical ICUs at a tertiary care hospital in Taiwan enrolled consecutive patients (≥20 years) without delirium at ICU admission. Delirium was screened daily using the ICDSC and CAM-ICU in random order. Arousal was assessed by the Richmond Agitation–Sedation Scale (RASS). Participants with any one positive result were classified as ICDSC- or CAM-ICU-delirium groups.
Results
Delirium incidence evaluated by the ICDSC and CAM-ICU were 69.1% (67/97) and 50.5% (49/97), respectively. Although the ICDSC identified 18 more cases as delirious, substantial concordance (κ = 0.63; p < 0.001) was found between tools. Independent of age, Acute Physiology and Chronic Health Evaluation II score, and Charlson Comorbidity Index, both ICDSC- and CAM-ICU-rated delirium significantly predicted hospital mortality (adjusted odds ratio: 4.93; 95% confidence interval [CI]:1.56 to 15.63 vs. 2.79; 95% CI: 1.12 to 6.97, respectively), and only the ICDSC significantly predicted hospital LOS with a mean of 17.59 additional days compared with the no-delirium group. Irrespective of delirium status, a sensitivity analysis of normal-to-increased arousal (RASS≥0) test results did not alter the predictive ability of ICDSC- or CAM-ICU-delirium for hospital mortality (adjusted odds ratio: 2.97; 95% CI: 1.06 to 8.37 vs. 3.82; 95% CI: 1.35 to 10.82, respectively). With reduced arousal (RASS<0), neither tool significantly predicted mortality or LOS.
Conclusions
The ICDSC identified more delirium cases and may have higher predictive validity for mortality and LOS than the CAM-ICU. However, arousal substantially affected performance. Future studies may want to consider patients’ arousal when deciding which tool to use to maximise the effects of delirium identification on patient mortality.
Screening for delirium with the intensive care delirium screening checklist (ICDSC): symptom profile and utility of individual items in the identification of delirium dependent on the level of sedation.
Postoperative delirium after cardiac surgery of elderly patients as an independent risk factor for prolonged length of stay in intensive care unit and inhospital.
Delirium can be accurately identified by widely validated screening tools such as the Intensive Care Delirium Screening Checklist (ICDSC) and the Confusion Assessment Method for the ICU (CAM-ICU),
The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies.
To encourage consistent application of effective ICU-acquired delirium screening tools, we considered that a head-to-head comparison of tools might help to guide tool selection. While both the ICDSC and CAM-ICU have been recommended by the Society of Critical Care Medicine for detecting ICU delirium,
The pain, agitation, and delirium care bundle: synergistic benefits of implementing the 2013 pain, agitation, and delirium guidelines in an integrated and interdisciplinary fashion.
it remains unclear which tool is more clinically relevant. Their ability to predict delirium-related outcomes such as mortality and LOS warrants investigation.
Outcomes in ICU settings are affected by many factors. Mortality may be more strongly influenced by altered arousal (either reduced or increased) than by delirium diagnosis alone.
Indeed, a six-study meta-analysis including 6002 hospitalised patients from medical, orthopedic wards, or emergency departments but excluding ICU patients found that altered arousal in patients with delirium was associated with higher mortality than normal arousal in patients with or without delirium.
Moreover, reduced arousal may interfere with delirium diagnosis, especially when using screening tools such as the ICDSC and the CAM-ICU. For example, reduced arousal has been suggested to overidentify delirium,
as delirium rates were reduced significantly if test results from patients on the Richmond Agitation–Sedation Scale (RASS) scores −2 to −3 were excluded.
The authors of that study noticed that a lower RASS score (−2 or deeper) tended to fulfill the delirium diagnosis, and delirium rates increased by approximately a quarter to a third.
In short, when studying the impact of delirium on mortality, reduced arousal, irrespective of delirium status, is an important confounder requiring further study.
In this prospective cohort study, we aimed to compare the predictive validity of the ICDSC versus CAM-ICU in predicting delirium-related outcomes of hospital mortality and LOS. The time spent being tested with each tool, as a proxy for their clinical utility, was also compared. Moreover, we examined whether the tools’ predictive validity and time spent in tests were affected by patients’ arousal status.
2. Materials and methods
2.1 Study design and participants
After this prospective cohort study was approved by the Institutional Review Board (IRB) and registered in the clinical trial registry (NCT04206306), we enrolled consecutive adult patients (aged ≥20 years) who expected to stay >24 h in six medical ICUs at a university-affiliated hospital in Taiwan from December 2019 to October 2020.
2.2 Sample
Patients were excluded by these criteria: (i) had pre-existing delirium; (ii) were prolonged bedridden before index hospitalisation (owing to our study aim of monitoring patients’ recovery on activities of daily living), had moderate dementia (Clinical Dementia Rating [CDR] score ≥2), had severe hearing impairment, or could not communicate; or (iii) were placed on contact and droplet precaution. All patients or their surrogates signed written informed consent forms to participate in the study. Because we screened for delirium at enrollment to ensure that all participants were not delirious, majority of our participants could sign informed consent. However, if patients were too weak at the time and preferred to let their surrogate signed the consent forms, we confirmed their consent based on the IRB protocol. The detailed study flow chart is shown in Fig. 1.
Fig. 1Study flowchart. CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist; ICU = intensive care unit; RASS = Richmond Agitation–Sedation Scale.
A standardised data entry form was developed to collect all participants’ clinical data: age (years), sex (%), education level (%; under high school; high school, and above), very mild or mild dementia (%; defined by the CDR = 0.5 or 1), Charlson Comorbidity Index (CCI) score, ICU admission diagnosis (acute respiratory failure; noncardiogenic shock; cardiac emergency; other), use of mechanical ventilation (yes/no), and the Acute Physiology and Chronic Health Evaluation (APACHE II) score (24 h within admission). Regarding study outcomes, in-hospital mortality and LOS in the hospital were abstracted from the medical record.
2.4 Delirium assessment
Research nurses screened sequential patients daily and up to 14 days of ICU stay with the ICDSC and CAM-ICU; the nurses’ screening for ICU delirium was calibrated with an experienced psychiatrist. To limit bias introduced by the same assessor's consecutive evaluation with two similar instruments, a random order was created using the REDCap data collection system and strictly followed to determine which tool was administered first at each paired assessment.
Before each delirium screening, participants’ arousal was evaluated by the RASS, with scores ranging between −5 (unarousable) and +4 (combative). For arousable participants (RASS −3 and higher), delirium was assessed by both the ICDSC and CAM-ICU. Briefly, based on Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria, the ICDSC is an eight-item checklist of delirium symptoms (consciousness, inattention, disorientation, hallucinations/delusions/psychosis, psychomotor agitation or retardation, inappropriate speech or mood, sleep/wake cycle disturbances, and symptom fluctuation); any four items presented within a 24-h time frame indicate delirium.
The CAM-ICU consists of four consecutive items ((i) acute change/fluctuation in mental status, (ii) inattention, (iii) altered level of consciousness, (iv) disorganised thinking); at least three items (1 + 2 + 3 or 1 + 2 + 4) must be presented to indicate delirium.
In this study, participants with any one positive result during the first 14 days of their ICU stay were classified accordingly into ICDSC- or CAM-ICU-delirium groups.
2.5 Outcomes: mortality, LOS, and time spent administering tools
The primary outcomes were in-hospital mortality and hospital LOS, abstracted from medical records. The secondary outcome, time spent (minutes) administering the ICDSC versus the CAM-ICU, was automatically recorded for each paired test with a built-in random tool order on the iPad data collection devices. Trained research nurses pressed the start icon and the time started to count until all tool items are completed on the iPad device. Notably, for both tools, we did not count the time research nurses spent to collect participant data in the last 24 h from the chart or from the primary nurse, given that the same assessor consecutively evaluated these two tools; only time spent in rating tool items was recorded for comparison.
2.6 Statistical analysis
Data were analysed using IBM SPSS Statistics for Windows, version 22.0 (IBM Corp, Armonk, NY), with all tests being two-tailed and p < 0.05 considered significant. The sample was described by means and standard deviations (SDs) for continuous variables, with counts and percentages for categorical variables. The degree of agreement between the ICDSC and CAM-ICU was measured using kappa (κ). Agreement was defined as moderate (0.41–0.60), substantial (0.61–0.80), or perfect (>0.80).
Mortality risk and LOS in the ICDSC- vs. CAM-ICU-delirium cohorts were estimated using multiple logistic and linear regressions models.
All models were adjusted for relevant confounders, including age, APACHE II, and CCI scores, to obtain adjusted odds ratios (aORs) and adjusted β. To test whether arousal state changed the predictive ability of delirium for mortality and LOS, we conducted sensitivity analyses to stratify normal-to-increased arousal (RASS≥0) versus reduced arousal (RASS <0) test results, irrespective of delirium status. In fact, a RASS score of ≥0 indicates either alert or aggressive symptomatology; thus, we defined RASS≥0 as “normal-to-increased arousal”, with negative RASS scores of −1 to −3 indicating drowsiness and being defined as “reduced arousal”.
2.7 Ethical considerations
The study was approved by the The National Taiwan University Hospital Research Ethics Committee (IRB # 201802070RIND) and registered in the clinical trial registry (NCT04206306). All patients or their surrogates signed written informed consent forms to participate in the study.
3. Results
3.1 Patient characteristic
Of 384 adult patients admitted to medical ICUs and prospectively screened, 97 participants were included in the analysis. Details of the study flow chart are presented in Fig. 1. As shown in Table 1, 97 participants had a mean (SD) age of 67.1 (13.9) years, 67% were male, and 55.7% (54/97) had a high-school education level or above. Only three (3.1%) had very mild or mild dementia (CDR score = 0.5 or 1). The participants’ comorbidity burden was high with a mean (SD) CCI score of 3.9 (2.3) points. The most common ICU admission diagnosis was acute respiratory failure (51/97; 52.6%), followed by noncardiogenic shock (35/97; 36.1%), cardiac emergency (7/97; 7.2%), and other (4/97; 4.2%). The mean (SD) APACHE II score at 24 h after ICU admission was 22.7 (7.2), and 66/97 (68%) of patients received mechanical ventilation. Ultimately, 35 patients died, resulting in an in-hospital mortality rate of 36.5% (35/96). The mean (SD) LOS was 32.6 (26.8) days. Notably, in 571 daily paired delirium screenings over a mean (SD) of 5.9 (4.0) days, the time spent screening with the ICDSC and CAM-ICU did not differ significantly (mean minutes [SD]: 1.1 [0.9] vs. 1.1 [0.8] for CAM-ICU; p = 0.96).
SD = standard deviation; CCI = Charlson Comorbidity Index; APACHE II = Acute Physical and Chronic Health Evaluation within 24 h of ICU admission; LOS = length of stay; CDR = Clinical Rating Scale; ICU = intensive care unit; CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist.
a Defined as CDR = 0.5 or 1.
b One participant remained hospitalised; uncounted.
3.2 ICDSC vs. CAM-ICU: concordance and predictive validity
As shown in Table 2, 67 participants were classified with delirium when assessed with the ICDSC tool, resulting in a delirium incidence of 69.1%. Conversely, only 49 participants were classified with CAM-ICU delirium, resulting in a delirium incidence of 50.5%. Although the ICDSC identified 18 more delirium cases than the CAM-ICU, concordance (κ = 0.63; p < 0.001) was substantial between the ICDSC and CAM-ICU for detecting delirium given their complete agreement on 49 delirium cases. A detailed 2 × 2 contingency table is provided in Supplemental Table 1a.
Table 2Incidence and kappa between ICDSC- and CAM-ICU-identified delirium cases.
Furthermore, as shown in Table 3, the ICDSC-delirium cohort had higher in-hospital mortality (45.5% vs. 16.7% for the no-ICDSC-delirium group; p = 0.007) and longer hospital LOS (mean days [SD]: 37.9 [29.6] vs. 20.9 [13.7] for the no-ICDSC-delirium group; p = 0.004). In the CAM-ICU group, participants had higher in-hospital mortality (47.9% vs. 25.0% for the no-delirium group; p = 0.02) but a comparable LOS (mean days [SDs]: 36.8 [28.9] vs. 28.3 [24.1] for the no-delirium group; p = 0.12). Logistic regression models revealed that belonging to the ICDSC-delirium group was associated with a 4.93-fold (aOR) higher in-hospital mortality (95% CI: 1.56–15.63; p = 0.007) than the 2.79-fold mortality risk for those in the CAM-ICU delirium group (95% CI: 1.12–6.97; p = 0.028). For LOS, linear regression analyses indicated that belonging to the ICDSC-delirium group was associated with prolonged LOS; participants in the ICDSC-delirium group stayed 17.59 days longer in the hospital (adjusted β [95% CI]: 17.59 [6.11–29.09]; p = 0.003). For the CAM-ICU delirium group, LOS was longer but not significantly so (adjusted β [95% CI]: 8.49 [−2.61–19.59]; p = 0.13).
Table 3ICDSC- and CAM-ICU-identified delirium on hospital mortality and LOS.
CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist; CCI = Charlson Comorbidity Index; APACHE II = Acute Physical and Chronic Health Evaluation; LOS = length of stay.
∗p < 0.05.
a One participant remained hospitalised; uncounted.
b All models were adjusted for age, APACHE II, and CCI scores.
As arousal may impact not only delirium diagnosis but also its predictive validity, we stratified patients’ test results by normal-to-increased arousal (RASS≥0) vs. reduced arousal (RASS<0), irrespective of delirium status, to determine whether the findings were consistent. As shown in Table 2, delirium rates varied between arousal groups. For both tools, delirium incidence was lower in the RASS≥0 subgroup (272 daily paired screenings), whereas it was higher in the RASS<0 subgroup (299 daily paired screenings). This trend was evident, especially for the ICDSC-delirium group (79.7% incidence for the reduced arousal subgroup vs. 37.9% for the normal-to-increased arousal subgroup). Similarly, concordance also varied between arousal groups. A much higher concordance (κ = 0.75; p < 0.001) was reported for RASS≥0, whereas a lower concordance (κ = 0.44) was reported for RASS <0.
The prognostic difference in mortality between arousal states also varied (Table 4). Both tools’ predictive validity for mortality diminished when stratified by reduced arousal (RASS<0). For normal-to-increased arousal, the CAM-ICU-delirium group had higher odds of mortality than the ICDSC-delirium group (aOR: 3.82 vs. 2.97 for ICDSC), although both tools predicted hospital mortality. For LOS, neither the ICDSC nor CAM-ICU predicted this outcome, regardless of arousal state. Similarly, for each tool, the time spent screening for delirium did not differ. As expected, the mean (SD) time spent screening for delirium in the RASS<0 subgroup was longer than in the RASS≥0 subgroup for both tools: 1.3 (0.9) vs. 1.0 (0.9) min for the ICDSC and 1.2 (0.8) vs. 1.0 (0.8) min for the CAM-ICU.
Table 4Stratification by the RASS: ICDSC- and CAM-ICU-identified delirium on hospital mortality and LOS.
CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist; LOS = length of stay; CI = confidence interval; RASS = Richmond Agitation–Sedation Scale; APACHE II = Acute Physical and Chronic Health Evaluation.
∗p < 0.05.
a One participant remained hospitalised; uncounted.
b All models were adjusted for age, APACHE II, and CCI scores.
The most important finding of our study is that ICU delirium identified by either screening tool, the ICDSC or CAM-ICU, was linked with increased deaths during hospitalisation. The ICDSC outperformed the CAM-ICU because it identified delirium with higher odds of mortality and predicted hospital LOS. However, the performance of both tools was affected by patient arousal.
Two findings are worth emphasising. First, as delirium incidence rates evaluated by the ICDSC and CAM-ICU were 69.1% and 50.5%, respectively, and the ICDSC identified 18 more cases of delirium, we found that a wider net was cast by the ICDSC, capturing more delirium cases than the CAM-ICU. Our result is consistent with that of three published works for comparison. Namely, a Brazilian study of 162 surgical ICU patients whose delirium rate evaluated by the ICDSC was 34.5% (n = 56) vs. 26.5% (n = 43) by the CAM-ICU.
The same research group also reported a larger cohort study involving both medical and surgical ICU patients (n = 595) whose delirium rate evaluated by the ICDSC was 30.8% (n = 183) vs. 16.1% (n = 96) by the CAM-ICU.
In another German study focusing on 174 surgical ICU patients tested in 374 paired, only a slightly higher delirium rate was reported for the ICDSC (37.4%) than for the CAM-ICU (36.1%).
Comparison of the confusion assessment method for the intensive care unit (CAM-ICU) with the Intensive Care Delirium Screening Checklist (ICDSC) for delirium in critical care patients gives high agreement rate(s).
Nevertheless, this wider net cast by the ICDSC may be welcome by certain institutions but may also bring additional diagnostic and care burden with fatigue to the nursing and physician staff as more cases were identified.
We thus focused on the tools’ relative predictive validity for important outcomes and found that the ICDSC-identified delirium had higher predictive validity for both mortality and LOS than the CAM-ICU. Participants in the ICDSC-delirium group had a 4.93-fold higher mortality risk and stayed at the hospital 17.59 days longer than those in the no-delirium group, whereas those in the CAM-ICU group were linked with a 2.79-fold lower odds of mortality. This finding is consistent with prior reports that ICU delirium increased hospital mortality in patients evaluated by either the ICDSC
Risk factors for new-onset delirium in patients with bloodstream infections: independent and quantitative effect of catheters and drainages-a four-year cohort study.
The reason for this difference requires further study, but one factor to consider is participants’ arousal state.
This leads to our second point. For both the ICDSC and CAM-ICU, reduced arousal affects performance. Although both the ICDSC and CAM-ICU can be used when patients’ RASS level was −1 to −3 (awakening to voice), their performance seemed to be less stable. Namely, agreement between tools is low (κ = 0.44 at RASS −1 to −3 compared with 0.75 at RASS≥0). Moreover, if used with patients in reduced arousal states (RASS between −1 and −3), both the ICDSC and CAM-ICU tended to identify delirium cases at a higher rate (Table 2) than with patients in normal-to-increased arousal. This trend is particularly apparent for the ICDSC, as its delirium incidence reached 79.7%, representing a 15% increased incidence from the sample mean. With this higher rate of delirium, neither ICDSC- nor CAM-ICU-identified delirium predicted hospital mortality or LOS. This loss of predictive validity is noteworthy.
Why delirium identified in the reduced arousal subgroup was not associated with mortality and LOS is an important research question. The diminished effect on mortality may be due to misclassification of delirium cases (e.g., more false positives), resulting in reduced analytic power. Moreover, when participants have RASS levels between −1 and −3, the ICDSC and CAM-ICU may measure pure sedation effects. As decreased arousal is likely to be multifactorial, combining both nonserious and serious conditions (e.g., medication/sedation effects versus serious neurologic events) may thus exert less consistent prognostic effects on mortality and/or LOS. Lastly, our study may be underpowered to detect the desired difference owing to a relatively small sample. Future studies with larger sample sizes are indicated to verify our results.
Apart from the overidentification perspective, a wider spectrum has been recommended in delirium diagnosis (i.e., more inclusive recognition of delirium).
For example, a 14-study meta-analysis (21,198 medical admission patients) found that reduced arousal (mostly defined by the Glasgow Coma Scale; only one with the RASS) on hospital admission was associated with 5.7-fold greater mortality rates.
In that study, the authors argued that as delirium and reduced arousal are closely related and both are linked with high mortality, delirium studies should include patients who are too drowsy to undergo cognitive testing or interviews.
Otherwise, the restricted spectrum (by eliminating patients with reduced arousal) may have led to underestimating the relationship between delirium and mortality.
Nevertheless, consistent with prior studies, ICU delirium evaluated by the ICDSC and CAM-ICU demonstrated substantial diagnostic agreement, and both tools could be completed in a comparable time, slightly over 1 min in our study. A more sensitive screening tool, such as the ICDSC, holds promise by casting a wider net and capturing more delirium cases than the CAM-ICU. Whether systematically using the ICDSC changes predicted outcomes requires an impact evaluation study. Moreover, given that the two tools’ agreement and predictive validity were much lower in the reduced arousal subgroup, future studies with larger samples may want to account for patients’ arousal when deciding which tool to use to maximise the effects of delirium identification on patient mortality.
4.1 Study strength and limitations
Our findings should be considered with certain limitations, including potential confounders and misclassification biases. As participants with any one positive result were classified into the delirium group, “days being assessed” was a confounder inherent in our study design; participants who stayed in the ICU longer were more likely to be classified into the delirium group. Moreover, the impact of delirium was assumed to be equal irrespective of its duration. Second, the same rater completed both measures, so their administration could not be blinded to each other. The randomisation of ordering was important but cannot overcome this limitation, which should be acknowledged. Third, delirium has no gold standard for diagnosis; thus, the “truth” of which measure better reflected delirium cannot be definitely established. Fourth, our study was limited by its small sample and power may have been limited for many of the analyses. Moreover, the study was limited by enrolling participants from a single institution with an enrolment rate of 83.3% (100 of 120 invited eligible patients). Although all eligible patients were offered enrollment, selection bias remains a limitation that will need to be addressed by replicating the findings in future studies.
5. Conclusions
Delirium, identified either by the ICDSC or CAM-ICU, was linked with hospital death in critically ill patients. A head-to-head comparison indicated that the ICDSC identified delirium with higher predictive validity for both mortality and LOS than the CAM-ICU. However, this evaluation may change based on the level of arousal. More studies are needed to consider ICU patients’ arousal when deciding which tool truly maximises the effects of delirium identification on patient mortality.
This study was supported in part by grants 107-2314-B-002-023-MY3 from the Taiwan Ministry of Science and Technology. Dr. Inouye's time was supported in part by grant no. R33AG071744 from the U.S. National Institute on Aging; Dr. Inouye also holds the Milton and Shirley F. Levy Family Chair at Hebrew Senior Life/Harvard Medical School.
Acknowledgements
The authors would like to acknowledge the help provided by Dr. Yu-Juan Xu and Mr. Chi-Hsuan Su in the study recruitment and data collection.
Appendix A. Supplementary data
The following is the Supplementary data to this article:
Screening for delirium with the intensive care delirium screening checklist (ICDSC): symptom profile and utility of individual items in the identification of delirium dependent on the level of sedation.
Postoperative delirium after cardiac surgery of elderly patients as an independent risk factor for prolonged length of stay in intensive care unit and inhospital.
The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies.
The pain, agitation, and delirium care bundle: synergistic benefits of implementing the 2013 pain, agitation, and delirium guidelines in an integrated and interdisciplinary fashion.
Comparison of the confusion assessment method for the intensive care unit (CAM-ICU) with the Intensive Care Delirium Screening Checklist (ICDSC) for delirium in critical care patients gives high agreement rate(s).
Risk factors for new-onset delirium in patients with bloodstream infections: independent and quantitative effect of catheters and drainages-a four-year cohort study.
00011-X&id=gr1.jpg){kind=link}