Advertisement

Predicting hospital mortality and length of stay: A prospective cohort study comparing the Intensive Care Delirium Screening Checklist versus Confusion Assessment Method for the Intensive Care Unit

Open AccessPublished:March 08, 2022DOI:https://doi.org/10.1016/j.aucc.2022.01.010

      Abstract

      Objective

      The objective of this study was to compare two tools, the Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method for the intensive care unit (ICU) (CAM-ICU), for their predictive validity for outcomes related to delirium, hospital mortality, and length of stay (LOS).

      Methods

      The prospective study conducted in six medical ICUs at a tertiary care hospital in Taiwan enrolled consecutive patients (≥20 years) without delirium at ICU admission. Delirium was screened daily using the ICDSC and CAM-ICU in random order. Arousal was assessed by the Richmond Agitation–Sedation Scale (RASS). Participants with any one positive result were classified as ICDSC- or CAM-ICU-delirium groups.

      Results

      Delirium incidence evaluated by the ICDSC and CAM-ICU were 69.1% (67/97) and 50.5% (49/97), respectively. Although the ICDSC identified 18 more cases as delirious, substantial concordance (κ = 0.63; p < 0.001) was found between tools. Independent of age, Acute Physiology and Chronic Health Evaluation II score, and Charlson Comorbidity Index, both ICDSC- and CAM-ICU-rated delirium significantly predicted hospital mortality (adjusted odds ratio: 4.93; 95% confidence interval [CI]:1.56 to 15.63 vs. 2.79; 95% CI: 1.12 to 6.97, respectively), and only the ICDSC significantly predicted hospital LOS with a mean of 17.59 additional days compared with the no-delirium group. Irrespective of delirium status, a sensitivity analysis of normal-to-increased arousal (RASS≥0) test results did not alter the predictive ability of ICDSC- or CAM-ICU-delirium for hospital mortality (adjusted odds ratio: 2.97; 95% CI: 1.06 to 8.37 vs. 3.82; 95% CI: 1.35 to 10.82, respectively). With reduced arousal (RASS<0), neither tool significantly predicted mortality or LOS.

      Conclusions

      The ICDSC identified more delirium cases and may have higher predictive validity for mortality and LOS than the CAM-ICU. However, arousal substantially affected performance. Future studies may want to consider patients’ arousal when deciding which tool to use to maximise the effects of delirium identification on patient mortality.

      Keywords

      1. Introduction

      Intensive care unit (ICU)–acquired delirium impacts patient outcomes, that is, higher mortality
      • Aung Thein M.Z.
      • Pereira J.V.
      • Nitchingham A.
      • Caplan G.A.
      A call to action for delirium research: meta-analysis and regression and regression of delirium association mortality.
      • Salluh J.I.F.
      • Wang H.
      • Schneider E.B.
      • Nagaraja N.
      • Yenokyan G.
      • Damluji A.
      • et al.
      Outcome of delirium in critically ill patients: systematic review and meta-analysis.
      • Abelha F.J.
      • Luís C.
      • Veiga D.
      • Fernandes V.
      • Santos P.
      • Botelho M.
      • et al.
      Outcome and quality of life in patients with postoperative delirium during an ICU stay following major surgery.
      • Ali M.A.
      • Hashmi M.
      • Ahmed W.
      • Raza S.A.
      • Khan M.F.
      • Salim B.
      Incidence and risk factors of delirium in surgical intensive care unit.
      • Pauley E.
      • Lishmanov A.
      • Schumann S.
      • Gala G.J.
      • Diepen S.
      • Katz J.N.
      Delirium is a robust predictor of morbidity and mortality among critically ill patients treated in the cardiac intensive care unit.
      • Zhang R.
      • Bai L.
      • Han X.
      • Huang S.
      • Zhou L.
      • Duan J.
      Incidence, characteristics, and outcomes of delirium in patients with noninvasive ventilation: a prospective observational study.
      and prolonged hospital length of stay (LOS).
      • Aung Thein M.Z.
      • Pereira J.V.
      • Nitchingham A.
      • Caplan G.A.
      A call to action for delirium research: meta-analysis and regression and regression of delirium association mortality.
      • Salluh J.I.F.
      • Wang H.
      • Schneider E.B.
      • Nagaraja N.
      • Yenokyan G.
      • Damluji A.
      • et al.
      Outcome of delirium in critically ill patients: systematic review and meta-analysis.
      • Abelha F.J.
      • Luís C.
      • Veiga D.
      • Fernandes V.
      • Santos P.
      • Botelho M.
      • et al.
      Outcome and quality of life in patients with postoperative delirium during an ICU stay following major surgery.
      ,
      • Zhang R.
      • Bai L.
      • Han X.
      • Huang S.
      • Zhou L.
      • Duan J.
      Incidence, characteristics, and outcomes of delirium in patients with noninvasive ventilation: a prospective observational study.
      • Dziegielewski C.
      • Skead C.
      • Canturk T.
      • Webber C.
      • Fernando S.M.
      • Thompson L.H.
      • et al.
      Delirium and associated length of stay and costs in critically.
      • Boettger S.
      • Meyer R.
      • Richter A.
      • Fernandez S.F.
      • Rudiger A.
      • Schubert M.
      • et al.
      Screening for delirium with the intensive care delirium screening checklist (ICDSC): symptom profile and utility of individual items in the identification of delirium dependent on the level of sedation.
      • Kirfel A.
      • Menzenbach J.
      • Guttenthaler V.
      • Feggeler J.
      • Mayr A.
      • Coburn M.
      • et al.
      Postoperative delirium after cardiac surgery of elderly patients as an independent risk factor for prolonged length of stay in intensive care unit and inhospital.
      Delirium can be accurately identified by widely validated screening tools such as the Intensive Care Delirium Screening Checklist (ICDSC) and the Confusion Assessment Method for the ICU (CAM-ICU),
      • Gusmao-Flores D.
      • Salluh J.I.F.
      • Chalhub R.Á.
      • Quarantini L.C.
      The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies.
      ,
      • Chen T.J.
      • Chung Y.W.
      • Chang H.C.
      • Chene P.Y.
      • Wua C.R.
      • Hsieha S.H.
      • et al.
      Diagnostic accuracy of the CAM-ICU and ICDSC in detecting intensive care unit delirium: a bivariate meta-analysis.
      but delirium in ICU settings is still poorly recognised.
      • Spronk P.E.
      • Riekerk B.
      • Hofhuis J.
      • Rommes J.H.
      Occurrence of delirium is severely underestimated in the ICU during daily care.
      Only 40% of clinicians consistently screen for ICU delirium,
      • Ely E.W.
      • Stephens R.K.
      • Jackson J.C.
      • Jackson J.C.
      • Thomason J.W.W.
      • Truman B.
      • et al.
      Current opinions regarding the importance, diagnosis, and management of delirium in the intensive care unit: a survey of 912 healthcare professional.
      and clinically effective screening tools are not consistently applied.
      • Khachaturian A.S.
      • Hayden K.M.
      • Devlin J.W.
      • Fleisher L.A.
      • Lock S.L.
      • Cunningham C.
      • et al.
      International drive to illuminate delirium: a developing public health blueprint for action.
      To encourage consistent application of effective ICU-acquired delirium screening tools, we considered that a head-to-head comparison of tools might help to guide tool selection. While both the ICDSC and CAM-ICU have been recommended by the Society of Critical Care Medicine for detecting ICU delirium,
      • Barr J.
      • Pandharipande P.P.
      The pain, agitation, and delirium care bundle: synergistic benefits of implementing the 2013 pain, agitation, and delirium guidelines in an integrated and interdisciplinary fashion.
      it remains unclear which tool is more clinically relevant. Their ability to predict delirium-related outcomes such as mortality and LOS warrants investigation.
      Outcomes in ICU settings are affected by many factors. Mortality may be more strongly influenced by altered arousal (either reduced or increased) than by delirium diagnosis alone.
      • Tieges Z.
      • Quinn T.
      • Mackenzie L.
      • Davis D.
      • Muniz-Terrera G.
      • MacLullich A.M.J.
      • et al.
      Association between components of the delirium syndrome and outcome in hospitalised adults: a systematic review and meta-analysis.
      Indeed, a six-study meta-analysis including 6002 hospitalised patients from medical, orthopedic wards, or emergency departments but excluding ICU patients found that altered arousal in patients with delirium was associated with higher mortality than normal arousal in patients with or without delirium.
      • Tieges Z.
      • Quinn T.
      • Mackenzie L.
      • Davis D.
      • Muniz-Terrera G.
      • MacLullich A.M.J.
      • et al.
      Association between components of the delirium syndrome and outcome in hospitalised adults: a systematic review and meta-analysis.
      Moreover, reduced arousal may interfere with delirium diagnosis, especially when using screening tools such as the ICDSC and the CAM-ICU. For example, reduced arousal has been suggested to overidentify delirium,
      • Gusmao-Flores D.
      • Martins J.C.S.
      • Amorin D.
      • Quarantini L.C.
      Tools for diagnosing delirium in the critically ill: is calibration needed for the less sedated patient?.
      as delirium rates were reduced significantly if test results from patients on the Richmond Agitation–Sedation Scale (RASS) scores −2 to −3 were excluded.
      • Haenggi M.
      • Blum S.
      • Brechbueh R.
      • Brunello A.
      • Jakob S.M.
      • Takala J.
      Effect of sedation level on the prevalence of delirium when assessed with CAM-ICU and ICDSC.
      The authors of that study noticed that a lower RASS score (−2 or deeper) tended to fulfill the delirium diagnosis, and delirium rates increased by approximately a quarter to a third.
      • Haenggi M.
      • Blum S.
      • Brechbueh R.
      • Brunello A.
      • Jakob S.M.
      • Takala J.
      Effect of sedation level on the prevalence of delirium when assessed with CAM-ICU and ICDSC.
      In short, when studying the impact of delirium on mortality, reduced arousal, irrespective of delirium status, is an important confounder requiring further study.
      In this prospective cohort study, we aimed to compare the predictive validity of the ICDSC versus CAM-ICU in predicting delirium-related outcomes of hospital mortality and LOS. The time spent being tested with each tool, as a proxy for their clinical utility, was also compared. Moreover, we examined whether the tools’ predictive validity and time spent in tests were affected by patients’ arousal status.

      2. Materials and methods

      2.1 Study design and participants

      After this prospective cohort study was approved by the Institutional Review Board (IRB) and registered in the clinical trial registry (NCT04206306), we enrolled consecutive adult patients (aged ≥20 years) who expected to stay >24 h in six medical ICUs at a university-affiliated hospital in Taiwan from December 2019 to October 2020.

      2.2 Sample

      Patients were excluded by these criteria: (i) had pre-existing delirium; (ii) were prolonged bedridden before index hospitalisation (owing to our study aim of monitoring patients’ recovery on activities of daily living), had moderate dementia (Clinical Dementia Rating [CDR] score ≥2), had severe hearing impairment, or could not communicate; or (iii) were placed on contact and droplet precaution. All patients or their surrogates signed written informed consent forms to participate in the study. Because we screened for delirium at enrollment to ensure that all participants were not delirious, majority of our participants could sign informed consent. However, if patients were too weak at the time and preferred to let their surrogate signed the consent forms, we confirmed their consent based on the IRB protocol. The detailed study flow chart is shown in Fig. 1.
      Fig. 1
      Fig. 1Study flowchart. CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist; ICU = intensive care unit; RASS = Richmond Agitation–Sedation Scale.

      2.3 Data collection

      A standardised data entry form was developed to collect all participants’ clinical data: age (years), sex (%), education level (%; under high school; high school, and above), very mild or mild dementia (%; defined by the CDR = 0.5 or 1), Charlson Comorbidity Index (CCI) score, ICU admission diagnosis (acute respiratory failure; noncardiogenic shock; cardiac emergency; other), use of mechanical ventilation (yes/no), and the Acute Physiology and Chronic Health Evaluation (APACHE II) score (24 h within admission). Regarding study outcomes, in-hospital mortality and LOS in the hospital were abstracted from the medical record.

      2.4 Delirium assessment

      Research nurses screened sequential patients daily and up to 14 days of ICU stay with the ICDSC and CAM-ICU; the nurses’ screening for ICU delirium was calibrated with an experienced psychiatrist. To limit bias introduced by the same assessor's consecutive evaluation with two similar instruments, a random order was created using the REDCap data collection system and strictly followed to determine which tool was administered first at each paired assessment.
      Before each delirium screening, participants’ arousal was evaluated by the RASS, with scores ranging between −5 (unarousable) and +4 (combative). For arousable participants (RASS −3 and higher), delirium was assessed by both the ICDSC and CAM-ICU. Briefly, based on Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria, the ICDSC is an eight-item checklist of delirium symptoms (consciousness, inattention, disorientation, hallucinations/delusions/psychosis, psychomotor agitation or retardation, inappropriate speech or mood, sleep/wake cycle disturbances, and symptom fluctuation); any four items presented within a 24-h time frame indicate delirium.
      • Bergeron N.
      • Dubois M.J.
      • Dumont M.
      • Dial S.
      • Skrobic Y.
      Intensive care delirium screening checklist: evaluation of a new screening tool.
      The CAM-ICU consists of four consecutive items ((i) acute change/fluctuation in mental status, (ii) inattention, (iii) altered level of consciousness, (iv) disorganised thinking); at least three items (1 + 2 + 3 or 1 + 2 + 4) must be presented to indicate delirium.
      • Ely E.W.
      • Inouye S.K.
      • Bernard G.R.
      • Gordan S.
      • Francis J.
      • May L.
      • et al.
      Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU).
      In this study, participants with any one positive result during the first 14 days of their ICU stay were classified accordingly into ICDSC- or CAM-ICU-delirium groups.

      2.5 Outcomes: mortality, LOS, and time spent administering tools

      The primary outcomes were in-hospital mortality and hospital LOS, abstracted from medical records. The secondary outcome, time spent (minutes) administering the ICDSC versus the CAM-ICU, was automatically recorded for each paired test with a built-in random tool order on the iPad data collection devices. Trained research nurses pressed the start icon and the time started to count until all tool items are completed on the iPad device. Notably, for both tools, we did not count the time research nurses spent to collect participant data in the last 24 h from the chart or from the primary nurse, given that the same assessor consecutively evaluated these two tools; only time spent in rating tool items was recorded for comparison.

      2.6 Statistical analysis

      Data were analysed using IBM SPSS Statistics for Windows, version 22.0 (IBM Corp, Armonk, NY), with all tests being two-tailed and p < 0.05 considered significant. The sample was described by means and standard deviations (SDs) for continuous variables, with counts and percentages for categorical variables. The degree of agreement between the ICDSC and CAM-ICU was measured using kappa (κ). Agreement was defined as moderate (0.41–0.60), substantial (0.61–0.80), or perfect (>0.80).
      • Landis J.R.
      • Koch G.G.
      The measurement of observer agreement for categorical data.
      Mortality risk and LOS in the ICDSC- vs. CAM-ICU-delirium cohorts were estimated using multiple logistic and linear regressions models.
      All models were adjusted for relevant confounders, including age, APACHE II, and CCI scores, to obtain adjusted odds ratios (aORs) and adjusted β. To test whether arousal state changed the predictive ability of delirium for mortality and LOS, we conducted sensitivity analyses to stratify normal-to-increased arousal (RASS≥0) versus reduced arousal (RASS <0) test results, irrespective of delirium status. In fact, a RASS score of ≥0 indicates either alert or aggressive symptomatology; thus, we defined RASS≥0 as “normal-to-increased arousal”, with negative RASS scores of −1 to −3 indicating drowsiness and being defined as “reduced arousal”.

      2.7 Ethical considerations

      The study was approved by the The National Taiwan University Hospital Research Ethics Committee (IRB # 201802070RIND) and registered in the clinical trial registry (NCT04206306). All patients or their surrogates signed written informed consent forms to participate in the study.

      3. Results

      3.1 Patient characteristic

      Of 384 adult patients admitted to medical ICUs and prospectively screened, 97 participants were included in the analysis. Details of the study flow chart are presented in Fig. 1. As shown in Table 1, 97 participants had a mean (SD) age of 67.1 (13.9) years, 67% were male, and 55.7% (54/97) had a high-school education level or above. Only three (3.1%) had very mild or mild dementia (CDR score = 0.5 or 1). The participants’ comorbidity burden was high with a mean (SD) CCI score of 3.9 (2.3) points. The most common ICU admission diagnosis was acute respiratory failure (51/97; 52.6%), followed by noncardiogenic shock (35/97; 36.1%), cardiac emergency (7/97; 7.2%), and other (4/97; 4.2%). The mean (SD) APACHE II score at 24 h after ICU admission was 22.7 (7.2), and 66/97 (68%) of patients received mechanical ventilation. Ultimately, 35 patients died, resulting in an in-hospital mortality rate of 36.5% (35/96). The mean (SD) LOS was 32.6 (26.8) days. Notably, in 571 daily paired delirium screenings over a mean (SD) of 5.9 (4.0) days, the time spent screening with the ICDSC and CAM-ICU did not differ significantly (mean minutes [SD]: 1.1 [0.9] vs. 1.1 [0.8] for CAM-ICU; p = 0.96).
      Table 1Sample characteristics and outcomes.
      CharacteristicsTotal cohort, n = 97ICDSC delirium, n = 67CAM-ICU delirium, n = 49CAM-ICU (−) but ICDSC (+), n = 18
      Demographics
       Age (yr), mean (SD)67.1 (13.9)67.6 (14.3)68.7 (12.9)64.4 (17.6)
       Male, n (%)65 (67.0)45 (67.2)33 (67.3)12 (66.7)
       Education level, n (%)
      Under high school43 (44.3)34 (50.7)28 (57.1)6 (33.3)
      High school or above54 (55.7)33 (49.3)21 (42.9)12 (66.7)
       CCI, mean (SD)3.9 (2.3)3.8 (2.1)4.0 (2.1)3.2 (2.1)
       Very mild/mild dementia
      Defined as CDR = 0.5 or 1.
      , n (%)
      3 (3.1)2 (3.0)2 (4.1)0
       ICU admission diagnosis, n (%)
      Acute respiratory failure51 (52.6)37 (55.2)29 (59.2)8 (44.4)
      Noncardiogenic shock35 (36.1)23 (34.3)16 (32.7)7 (38.9)
      Cardiac emergency7 (7.2)6 (9.0)3 (6.1)3 (16.7)
      Other4 (4.2)1 (1.5)1 (2.0)0
       APACHE II, mean (SD)22.7 (7.2)23.2 (6.8)23.8 (6.5)21.7 (7.5)
       Use of mechanical ventilator, n (%)66 (68.0)56 (83.6)44 (89.8)12 (66.7)
      Outcomes
       Hospital LOS (d), mean (SD)32.6 (26.8)
      One participant remained hospitalised; uncounted.
      37.9 (29.6)
      One participant remained hospitalised; uncounted.
      36.8 (28.9)
      One participant remained hospitalised; uncounted.
      40.6 (32.2)
       In-hospital mortality, n (%)35 (36.5)
      One participant remained hospitalised; uncounted.
      30 (45.5)
      One participant remained hospitalised; uncounted.
      23 (47.9)
      One participant remained hospitalised; uncounted.
      7 (38.9)
      SD = standard deviation; CCI = Charlson Comorbidity Index; APACHE II = Acute Physical and Chronic Health Evaluation within 24 h of ICU admission; LOS = length of stay; CDR = Clinical Rating Scale; ICU = intensive care unit; CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist.
      a Defined as CDR = 0.5 or 1.
      b One participant remained hospitalised; uncounted.

      3.2 ICDSC vs. CAM-ICU: concordance and predictive validity

      As shown in Table 2, 67 participants were classified with delirium when assessed with the ICDSC tool, resulting in a delirium incidence of 69.1%. Conversely, only 49 participants were classified with CAM-ICU delirium, resulting in a delirium incidence of 50.5%. Although the ICDSC identified 18 more delirium cases than the CAM-ICU, concordance (κ = 0.63; p < 0.001) was substantial between the ICDSC and CAM-ICU for detecting delirium given their complete agreement on 49 delirium cases. A detailed 2 × 2 contingency table is provided in Supplemental Table 1a.
      Table 2Incidence and kappa between ICDSC- and CAM-ICU-identified delirium cases.
      ICDSC delirium, n (%)CAM-ICU delirium, n (%)Cohen's kappa (κ)
      Agreement between tools was calculated; detailed 2 × 2 contingency tables are in Supplemental Table 1.
      All participants67/97 (69.1)49/97 (50.5)0.63
      Stratified by normal-to-increased arousal33/87 (37.9)27/87 (31.0)0.75
      Stratified by reduced arousal59/74 (79.7)42/74 (56.8)0.44
      CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist.
      a Agreement between tools was calculated; detailed 2 × 2 contingency tables are in Supplemental Table 1.
      Furthermore, as shown in Table 3, the ICDSC-delirium cohort had higher in-hospital mortality (45.5% vs. 16.7% for the no-ICDSC-delirium group; p = 0.007) and longer hospital LOS (mean days [SD]: 37.9 [29.6] vs. 20.9 [13.7] for the no-ICDSC-delirium group; p = 0.004). In the CAM-ICU group, participants had higher in-hospital mortality (47.9% vs. 25.0% for the no-delirium group; p = 0.02) but a comparable LOS (mean days [SDs]: 36.8 [28.9] vs. 28.3 [24.1] for the no-delirium group; p = 0.12). Logistic regression models revealed that belonging to the ICDSC-delirium group was associated with a 4.93-fold (aOR) higher in-hospital mortality (95% CI: 1.56–15.63; p = 0.007) than the 2.79-fold mortality risk for those in the CAM-ICU delirium group (95% CI: 1.12–6.97; p = 0.028). For LOS, linear regression analyses indicated that belonging to the ICDSC-delirium group was associated with prolonged LOS; participants in the ICDSC-delirium group stayed 17.59 days longer in the hospital (adjusted β [95% CI]: 17.59 [6.11–29.09]; p = 0.003). For the CAM-ICU delirium group, LOS was longer but not significantly so (adjusted β [95% CI]: 8.49 [−2.61–19.59]; p = 0.13).
      Table 3ICDSC- and CAM-ICU-identified delirium on hospital mortality and LOS.
      In-hospital mortalityDeliriumNo deliriumAdjusted odds ratio
      All models were adjusted for age, APACHE II, and CCI scores.
      95% CI
      TotalDeath(%)TotalDeath(%)
      ICDSC6730(45.5)
      One participant remained hospitalised; uncounted.
      305(16.7)4.93(1.56–15.63)∗
      CAM-ICU4923(47.9)
      One participant remained hospitalised; uncounted.
      4812(25.0)2.79(1.12–6.97)∗
      LOSDeliriumNo deliriumAdjusted difference
      All models were adjusted for age, APACHE II, and CCI scores.
      95% CI
      Mean
      One participant remained hospitalised; uncounted.
      (SD)
      One participant remained hospitalised; uncounted.
      Mean(SD)
      ICDSC37.9
      One participant remained hospitalised; uncounted.
      (29.6)20.9(13.7)17.59(6.11–29.09)∗
      CAM-ICU36.8
      One participant remained hospitalised; uncounted.
      (28.9)28.3(24.1)8.49(-2.61–19.59)
      CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist; CCI = Charlson Comorbidity Index; APACHE II = Acute Physical and Chronic Health Evaluation; LOS = length of stay.
      p < 0.05.
      a One participant remained hospitalised; uncounted.
      b All models were adjusted for age, APACHE II, and CCI scores.

      3.3 Sensitivity analysis: effect of arousal

      As arousal may impact not only delirium diagnosis but also its predictive validity, we stratified patients’ test results by normal-to-increased arousal (RASS≥0) vs. reduced arousal (RASS<0), irrespective of delirium status, to determine whether the findings were consistent. As shown in Table 2, delirium rates varied between arousal groups. For both tools, delirium incidence was lower in the RASS≥0 subgroup (272 daily paired screenings), whereas it was higher in the RASS<0 subgroup (299 daily paired screenings). This trend was evident, especially for the ICDSC-delirium group (79.7% incidence for the reduced arousal subgroup vs. 37.9% for the normal-to-increased arousal subgroup). Similarly, concordance also varied between arousal groups. A much higher concordance (κ = 0.75; p < 0.001) was reported for RASS≥0, whereas a lower concordance (κ = 0.44) was reported for RASS <0.
      The prognostic difference in mortality between arousal states also varied (Table 4). Both tools’ predictive validity for mortality diminished when stratified by reduced arousal (RASS<0). For normal-to-increased arousal, the CAM-ICU-delirium group had higher odds of mortality than the ICDSC-delirium group (aOR: 3.82 vs. 2.97 for ICDSC), although both tools predicted hospital mortality. For LOS, neither the ICDSC nor CAM-ICU predicted this outcome, regardless of arousal state. Similarly, for each tool, the time spent screening for delirium did not differ. As expected, the mean (SD) time spent screening for delirium in the RASS<0 subgroup was longer than in the RASS≥0 subgroup for both tools: 1.3 (0.9) vs. 1.0 (0.9) min for the ICDSC and 1.2 (0.8) vs. 1.0 (0.8) min for the CAM-ICU.
      Table 4Stratification by the RASS: ICDSC- and CAM-ICU-identified delirium on hospital mortality and LOS.
      Normal-to-increased arousal (RASS ≥0 subgroup)
      In-hospital mortalityDeliriumNo deliriumAdjusted odds ratio
      All models were adjusted for age, APACHE II, and CCI scores.
      95% CI
      TotalDeath(%)TotalDeath(%)
      ICDSC3315(46.9)
      One participant remained hospitalised; uncounted.
      5413(24.1)2.97(1.06–8.37)∗
      CAM-ICU2714(53.8)
      One participant remained hospitalised; uncounted.
      6014(23.3)3.82(1.35–10.82)∗
      LOSDeliriumNo deliriumAdjusted difference
      All models were adjusted for age, APACHE II, and CCI scores.
      95% CI
      Mean(SD)Mean(SD)
      ICDSC35.5
      One participant remained hospitalised; uncounted.
      (29.9)
      One participant remained hospitalised; uncounted.
      30.5(25.4)5.12(−7.84–18.07)
      CAM-ICU32.3
      One participant remained hospitalised; uncounted.
      (24.6)
      One participant remained hospitalised; uncounted.
      32.4(28.3)−1.12(−14.38–12.13)
      Reduced arousal (RASS<0 subgroup)
      In-hospital mortalityDeliriumNo deliriumAdjusted odds ratio
      All models were adjusted for age, APACHE II, and CCI scores.
      95% CI
      TotalDeath(%)TotalDeath(%)
      ICDSC5927(46.6)
      One participant remained hospitalised; uncounted.
      155(33.3)1.65(0.47–5.86)
      CAM-ICU4220(48.8)
      One participant remained hospitalised; uncounted.
      3212(37.5)1.33(0.48–3.66)
      LOSDeliriumNo deliriumAdjusted difference
      All models were adjusted for age, APACHE II, and CCI scores.
      95% CI
      Mean(SD)Mean(SD)
      ICDSC38.3
      One participant remained hospitalised; uncounted.
      (30.8)
      One participant remained hospitalised; uncounted.
      25.5(18.6)13.19(-3.74–30.13)
      CAM-ICU38.2
      One participant remained hospitalised; uncounted.
      (30.2)
      One participant remained hospitalised; uncounted.
      32.6(27.8)4.72(-9.40–18.84)
      CAM-ICU = Confusion Assessment Method for the Intensive Care Unit; ICDSC = Intensive Care Delirium Screening Checklist; LOS = length of stay; CI = confidence interval; RASS = Richmond Agitation–Sedation Scale; APACHE II = Acute Physical and Chronic Health Evaluation.
      p < 0.05.
      a One participant remained hospitalised; uncounted.
      b All models were adjusted for age, APACHE II, and CCI scores.

      4. Discussion

      The most important finding of our study is that ICU delirium identified by either screening tool, the ICDSC or CAM-ICU, was linked with increased deaths during hospitalisation. The ICDSC outperformed the CAM-ICU because it identified delirium with higher odds of mortality and predicted hospital LOS. However, the performance of both tools was affected by patient arousal.
      Two findings are worth emphasising. First, as delirium incidence rates evaluated by the ICDSC and CAM-ICU were 69.1% and 50.5%, respectively, and the ICDSC identified 18 more cases of delirium, we found that a wider net was cast by the ICDSC, capturing more delirium cases than the CAM-ICU. Our result is consistent with that of three published works for comparison. Namely, a Brazilian study of 162 surgical ICU patients whose delirium rate evaluated by the ICDSC was 34.5% (n = 56) vs. 26.5% (n = 43) by the CAM-ICU.
      • Tomasi C.D.
      • Grandi C.
      • Salluh J.
      • Soraes M.
      • Giombellia V.R.
      • Cascaesa R.
      • et al.
      Comparison of CAM-ICU and ICDSC for the detection of delirium in critically ill patients focusing on relevant clinical outcomes.
      The same research group also reported a larger cohort study involving both medical and surgical ICU patients (n = 595) whose delirium rate evaluated by the ICDSC was 30.8% (n = 183) vs. 16.1% (n = 96) by the CAM-ICU.
      • Fagundes J.A.O.
      • Tomasi C.D.
      • Giombelli V.R.
      • Alves S.C.
      • Macedo R.C.
      • Topanotti M.F.L.
      • et al.
      CAM-ICU and ICDSC agreement in medical and surgical ICU patients is influenced by disease severity.
      In another German study focusing on 174 surgical ICU patients tested in 374 paired, only a slightly higher delirium rate was reported for the ICDSC (37.4%) than for the CAM-ICU (36.1%).
      • Plaschke K.
      • von Haken R.
      • Scholz M.
      • Engelhardt R.
      • Brobeil A.
      • Martin E.
      • et al.
      Comparison of the confusion assessment method for the intensive care unit (CAM-ICU) with the Intensive Care Delirium Screening Checklist (ICDSC) for delirium in critical care patients gives high agreement rate(s).
      Nevertheless, this wider net cast by the ICDSC may be welcome by certain institutions but may also bring additional diagnostic and care burden with fatigue to the nursing and physician staff as more cases were identified.
      We thus focused on the tools’ relative predictive validity for important outcomes and found that the ICDSC-identified delirium had higher predictive validity for both mortality and LOS than the CAM-ICU. Participants in the ICDSC-delirium group had a 4.93-fold higher mortality risk and stayed at the hospital 17.59 days longer than those in the no-delirium group, whereas those in the CAM-ICU group were linked with a 2.79-fold lower odds of mortality. This finding is consistent with prior reports that ICU delirium increased hospital mortality in patients evaluated by either the ICDSC
      • Dittrich T.
      • Tschudin-Sutter S.
      • Widmer A.F.
      • Rüegg S.
      • Marsch S.
      • Sutter R.
      Risk factors for new-onset delirium in patients with bloodstream infections: independent and quantitative effect of catheters and drainages-a four-year cohort study.
      • Ouimet S.
      • Kavanagh B.P.
      • Gottfried S.B.
      • Skrobik Y.
      Incidence, risk factors and consequences of ICU delirium.
      • Yamaguchi T.
      • Tsukioka E.
      • Kishi Y.
      Outcomes after delirium in a Japanese intensive care unit.
      or CAM-ICU
      • Pauley E.
      • Lishmanov A.
      • Schumann S.
      • Gala G.J.
      • Diepen S.
      • Katz J.N.
      Delirium is a robust predictor of morbidity and mortality among critically ill patients treated in the cardiac intensive care unit.
      ,
      • Zhang R.
      • Bai L.
      • Han X.
      • Huang S.
      • Zhou L.
      • Duan J.
      Incidence, characteristics, and outcomes of delirium in patients with noninvasive ventilation: a prospective observational study.
      ,
      • van den Boogaard M.
      • Peters S.A.
      • van der Hoeven J.G.
      • Dagnelie P.C.
      • Leffers P.
      • Pickkers P.
      • et al.
      The impact of delirium on the prediction of in-hospital mortality in intensive care patients.
      ,
      • Lin S.M.
      • Huang C.D.
      • Liu C.Y.
      • Lin H.C.
      • Wang C.H.
      • Huang P.Y.
      • et al.
      Risk factors for the development of early-onset delirium and the subsequent clinical outcome in mechanically ventilated patients.
      but different from a previous finding that the CAM-ICU better predicted outcome.
      • Tomasi C.D.
      • Grandi C.
      • Salluh J.
      • Soraes M.
      • Giombellia V.R.
      • Cascaesa R.
      • et al.
      Comparison of CAM-ICU and ICDSC for the detection of delirium in critically ill patients focusing on relevant clinical outcomes.
      The reason for this difference requires further study, but one factor to consider is participants’ arousal state.
      This leads to our second point. For both the ICDSC and CAM-ICU, reduced arousal affects performance. Although both the ICDSC and CAM-ICU can be used when patients’ RASS level was −1 to −3 (awakening to voice), their performance seemed to be less stable. Namely, agreement between tools is low (κ = 0.44 at RASS −1 to −3 compared with 0.75 at RASS≥0). Moreover, if used with patients in reduced arousal states (RASS between −1 and −3), both the ICDSC and CAM-ICU tended to identify delirium cases at a higher rate (Table 2) than with patients in normal-to-increased arousal. This trend is particularly apparent for the ICDSC, as its delirium incidence reached 79.7%, representing a 15% increased incidence from the sample mean. With this higher rate of delirium, neither ICDSC- nor CAM-ICU-identified delirium predicted hospital mortality or LOS. This loss of predictive validity is noteworthy.
      Why delirium identified in the reduced arousal subgroup was not associated with mortality and LOS is an important research question. The diminished effect on mortality may be due to misclassification of delirium cases (e.g., more false positives), resulting in reduced analytic power. Moreover, when participants have RASS levels between −1 and −3, the ICDSC and CAM-ICU may measure pure sedation effects. As decreased arousal is likely to be multifactorial, combining both nonserious and serious conditions (e.g., medication/sedation effects versus serious neurologic events) may thus exert less consistent prognostic effects on mortality and/or LOS. Lastly, our study may be underpowered to detect the desired difference owing to a relatively small sample. Future studies with larger sample sizes are indicated to verify our results.
      Apart from the overidentification perspective, a wider spectrum has been recommended in delirium diagnosis (i.e., more inclusive recognition of delirium).
      European Delirium Association
      American Delirium Society. The DSM-5 criteria, level of arousal and delirium diagnosis: inclusiveness is safer.
      For example, a 14-study meta-analysis (21,198 medical admission patients) found that reduced arousal (mostly defined by the Glasgow Coma Scale; only one with the RASS) on hospital admission was associated with 5.7-fold greater mortality rates.
      • Todd A.
      • Blackley S.
      • Burton J.K.
      • Stott D.J.
      • Ely E.W.
      • Tieges Z.
      • et al.
      Reduced level of arousal and increased mortality in adult acute medical admissions: a systematic review and meta-analysis.
      In that study, the authors argued that as delirium and reduced arousal are closely related and both are linked with high mortality, delirium studies should include patients who are too drowsy to undergo cognitive testing or interviews.
      • Todd A.
      • Blackley S.
      • Burton J.K.
      • Stott D.J.
      • Ely E.W.
      • Tieges Z.
      • et al.
      Reduced level of arousal and increased mortality in adult acute medical admissions: a systematic review and meta-analysis.
      Otherwise, the restricted spectrum (by eliminating patients with reduced arousal) may have led to underestimating the relationship between delirium and mortality.
      European Delirium Association
      American Delirium Society. The DSM-5 criteria, level of arousal and delirium diagnosis: inclusiveness is safer.
      ,
      • Todd A.
      • Blackley S.
      • Burton J.K.
      • Stott D.J.
      • Ely E.W.
      • Tieges Z.
      • et al.
      Reduced level of arousal and increased mortality in adult acute medical admissions: a systematic review and meta-analysis.
      Nevertheless, consistent with prior studies, ICU delirium evaluated by the ICDSC and CAM-ICU demonstrated substantial diagnostic agreement, and both tools could be completed in a comparable time, slightly over 1 min in our study. A more sensitive screening tool, such as the ICDSC, holds promise by casting a wider net and capturing more delirium cases than the CAM-ICU. Whether systematically using the ICDSC changes predicted outcomes requires an impact evaluation study. Moreover, given that the two tools’ agreement and predictive validity were much lower in the reduced arousal subgroup, future studies with larger samples may want to account for patients’ arousal when deciding which tool to use to maximise the effects of delirium identification on patient mortality.

      4.1 Study strength and limitations

      Our findings should be considered with certain limitations, including potential confounders and misclassification biases. As participants with any one positive result were classified into the delirium group, “days being assessed” was a confounder inherent in our study design; participants who stayed in the ICU longer were more likely to be classified into the delirium group. Moreover, the impact of delirium was assumed to be equal irrespective of its duration. Second, the same rater completed both measures, so their administration could not be blinded to each other. The randomisation of ordering was important but cannot overcome this limitation, which should be acknowledged. Third, delirium has no gold standard for diagnosis; thus, the “truth” of which measure better reflected delirium cannot be definitely established. Fourth, our study was limited by its small sample and power may have been limited for many of the analyses. Moreover, the study was limited by enrolling participants from a single institution with an enrolment rate of 83.3% (100 of 120 invited eligible patients). Although all eligible patients were offered enrollment, selection bias remains a limitation that will need to be addressed by replicating the findings in future studies.

      5. Conclusions

      Delirium, identified either by the ICDSC or CAM-ICU, was linked with hospital death in critically ill patients. A head-to-head comparison indicated that the ICDSC identified delirium with higher predictive validity for both mortality and LOS than the CAM-ICU. However, this evaluation may change based on the level of arousal. More studies are needed to consider ICU patients’ arousal when deciding which tool truly maximises the effects of delirium identification on patient mortality.

      CRediT authorship contribution statement

      Hsiu-Ching Li: Investigation, Formal analysis, Writing - original draft preparation; Cheryl Chia-Hui Chen: Conceptualisation, Methodology, Investigation, Writing - original draft preparation, Writing - review & editing; Tony Yu-Chang Yeh: Methodology, Funding acquisition; Shih-Cheng Liao: Methodology; Adrian-Shengchun Hsu: Validation; Yu-Chung Wei: Formal analysis; Shiow-Ching Shun: Methodology; Shih-Chi Ku: Supervision, Project administration; Sharon K. Inouye: Methodology, Writing - review & editing.

      Funding

      This study was supported in part by grants 107-2314-B-002-023-MY3 from the Taiwan Ministry of Science and Technology . Dr. Inouye's time was supported in part by grant no. R33AG071744 from the U.S. National Institute on Aging ; Dr. Inouye also holds the Milton and Shirley F. Levy Family Chair at Hebrew Senior Life/Harvard Medical School.

      Acknowledgements

      The authors would like to acknowledge the help provided by Dr. Yu-Juan Xu and Mr. Chi-Hsuan Su in the study recruitment and data collection.

      Appendix A. Supplementary data

      The following is the Supplementary data to this article:

      References

        • Aung Thein M.Z.
        • Pereira J.V.
        • Nitchingham A.
        • Caplan G.A.
        A call to action for delirium research: meta-analysis and regression and regression of delirium association mortality.
        BMC Geriatr. 2020; 20: 325-335
        • Salluh J.I.F.
        • Wang H.
        • Schneider E.B.
        • Nagaraja N.
        • Yenokyan G.
        • Damluji A.
        • et al.
        Outcome of delirium in critically ill patients: systematic review and meta-analysis.
        BMJ. 2015; : 350
        • Abelha F.J.
        • Luís C.
        • Veiga D.
        • Fernandes V.
        • Santos P.
        • Botelho M.
        • et al.
        Outcome and quality of life in patients with postoperative delirium during an ICU stay following major surgery.
        Crit Care. 2013; 17: R257
        • Ali M.A.
        • Hashmi M.
        • Ahmed W.
        • Raza S.A.
        • Khan M.F.
        • Salim B.
        Incidence and risk factors of delirium in surgical intensive care unit.
        J Trauma Acute Care Surg. 2021; 6e000564
        • Pauley E.
        • Lishmanov A.
        • Schumann S.
        • Gala G.J.
        • Diepen S.
        • Katz J.N.
        Delirium is a robust predictor of morbidity and mortality among critically ill patients treated in the cardiac intensive care unit.
        Am Heart J. 2015; 170: 79-86
        • Zhang R.
        • Bai L.
        • Han X.
        • Huang S.
        • Zhou L.
        • Duan J.
        Incidence, characteristics, and outcomes of delirium in patients with noninvasive ventilation: a prospective observational study.
        BMC Pulm Med. 2021; 21: 157-164
        • Dziegielewski C.
        • Skead C.
        • Canturk T.
        • Webber C.
        • Fernando S.M.
        • Thompson L.H.
        • et al.
        Delirium and associated length of stay and costs in critically.
        Crit Care Res Pract. 2021; https://doi.org/10.1155/2021/6612187
        • Boettger S.
        • Meyer R.
        • Richter A.
        • Fernandez S.F.
        • Rudiger A.
        • Schubert M.
        • et al.
        Screening for delirium with the intensive care delirium screening checklist (ICDSC): symptom profile and utility of individual items in the identification of delirium dependent on the level of sedation.
        Palliat Support Care. 2019; 17: 74-81
        • Kirfel A.
        • Menzenbach J.
        • Guttenthaler V.
        • Feggeler J.
        • Mayr A.
        • Coburn M.
        • et al.
        Postoperative delirium after cardiac surgery of elderly patients as an independent risk factor for prolonged length of stay in intensive care unit and inhospital.
        Aging Clin Exp Res. 2021; 33: 3047-3056
        • Gusmao-Flores D.
        • Salluh J.I.F.
        • Chalhub R.Á.
        • Quarantini L.C.
        The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies.
        Crit Care. 2012; 16: R115
        • Chen T.J.
        • Chung Y.W.
        • Chang H.C.
        • Chene P.Y.
        • Wua C.R.
        • Hsieha S.H.
        • et al.
        Diagnostic accuracy of the CAM-ICU and ICDSC in detecting intensive care unit delirium: a bivariate meta-analysis.
        Int J Nurs Stud. 2021; 113: 103782
        • Spronk P.E.
        • Riekerk B.
        • Hofhuis J.
        • Rommes J.H.
        Occurrence of delirium is severely underestimated in the ICU during daily care.
        Intensive Care Med. 2009; 35: 1276-1280
        • Ely E.W.
        • Stephens R.K.
        • Jackson J.C.
        • Jackson J.C.
        • Thomason J.W.W.
        • Truman B.
        • et al.
        Current opinions regarding the importance, diagnosis, and management of delirium in the intensive care unit: a survey of 912 healthcare professional.
        Crit Care Med. 2004; 32: 106-112
        • Khachaturian A.S.
        • Hayden K.M.
        • Devlin J.W.
        • Fleisher L.A.
        • Lock S.L.
        • Cunningham C.
        • et al.
        International drive to illuminate delirium: a developing public health blueprint for action.
        J Alzheimer's Dis. 2020; 16: 711-725
        • Barr J.
        • Pandharipande P.P.
        The pain, agitation, and delirium care bundle: synergistic benefits of implementing the 2013 pain, agitation, and delirium guidelines in an integrated and interdisciplinary fashion.
        Crit Care Med. 2013; 41: S99-S115
        • Tieges Z.
        • Quinn T.
        • Mackenzie L.
        • Davis D.
        • Muniz-Terrera G.
        • MacLullich A.M.J.
        • et al.
        Association between components of the delirium syndrome and outcome in hospitalised adults: a systematic review and meta-analysis.
        BMC Geriatr. 2021; 21: 162-175
        • Gusmao-Flores D.
        • Martins J.C.S.
        • Amorin D.
        • Quarantini L.C.
        Tools for diagnosing delirium in the critically ill: is calibration needed for the less sedated patient?.
        Intensive Care Med. 2014; 40: 137-138
        • Haenggi M.
        • Blum S.
        • Brechbueh R.
        • Brunello A.
        • Jakob S.M.
        • Takala J.
        Effect of sedation level on the prevalence of delirium when assessed with CAM-ICU and ICDSC.
        Intensive Care Med. 2013; 39: 2171-2179
        • Bergeron N.
        • Dubois M.J.
        • Dumont M.
        • Dial S.
        • Skrobic Y.
        Intensive care delirium screening checklist: evaluation of a new screening tool.
        Intensive Care Med. 2001; 27: 859-864
        • Ely E.W.
        • Inouye S.K.
        • Bernard G.R.
        • Gordan S.
        • Francis J.
        • May L.
        • et al.
        Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU).
        JAMA. 2001; 286: 2703-2710
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; : 159-174
        • Tomasi C.D.
        • Grandi C.
        • Salluh J.
        • Soraes M.
        • Giombellia V.R.
        • Cascaesa R.
        • et al.
        Comparison of CAM-ICU and ICDSC for the detection of delirium in critically ill patients focusing on relevant clinical outcomes.
        J Crit Care. 2012; 27: 212-217
        • Fagundes J.A.O.
        • Tomasi C.D.
        • Giombelli V.R.
        • Alves S.C.
        • Macedo R.C.
        • Topanotti M.F.L.
        • et al.
        CAM-ICU and ICDSC agreement in medical and surgical ICU patients is influenced by disease severity.
        PLos One. 2012; 7e51010
        • Plaschke K.
        • von Haken R.
        • Scholz M.
        • Engelhardt R.
        • Brobeil A.
        • Martin E.
        • et al.
        Comparison of the confusion assessment method for the intensive care unit (CAM-ICU) with the Intensive Care Delirium Screening Checklist (ICDSC) for delirium in critical care patients gives high agreement rate(s).
        Intensive Care Med. 2008; 34: 431-443
        • Dittrich T.
        • Tschudin-Sutter S.
        • Widmer A.F.
        • Rüegg S.
        • Marsch S.
        • Sutter R.
        Risk factors for new-onset delirium in patients with bloodstream infections: independent and quantitative effect of catheters and drainages-a four-year cohort study.
        Ann Intensive Care. 2017; 6: 104
        • Ouimet S.
        • Kavanagh B.P.
        • Gottfried S.B.
        • Skrobik Y.
        Incidence, risk factors and consequences of ICU delirium.
        Intensive Care Med. 2007; 33: 66-73
        • Yamaguchi T.
        • Tsukioka E.
        • Kishi Y.
        Outcomes after delirium in a Japanese intensive care unit.
        Gen Hosp Psychiatry. 2014; 36: 634-636
        • van den Boogaard M.
        • Peters S.A.
        • van der Hoeven J.G.
        • Dagnelie P.C.
        • Leffers P.
        • Pickkers P.
        • et al.
        The impact of delirium on the prediction of in-hospital mortality in intensive care patients.
        Crit Care. 2010; 14: R146
        • Lin S.M.
        • Huang C.D.
        • Liu C.Y.
        • Lin H.C.
        • Wang C.H.
        • Huang P.Y.
        • et al.
        Risk factors for the development of early-onset delirium and the subsequent clinical outcome in mechanically ventilated patients.
        J Crit Care. 2008; 23: 372-379
        • European Delirium Association
        American Delirium Society. The DSM-5 criteria, level of arousal and delirium diagnosis: inclusiveness is safer.
        BMC Med. 2014; 12: 141https://doi.org/10.1186/s12916-014-0141-2
        • Todd A.
        • Blackley S.
        • Burton J.K.
        • Stott D.J.
        • Ely E.W.
        • Tieges Z.
        • et al.
        Reduced level of arousal and increased mortality in adult acute medical admissions: a systematic review and meta-analysis.
        BMC Geriatr. 2017; 17: 283