Advertisement
Research paper|Articles in Press

Intensive care unit nurses' perception of three different methods for delirium screening: A survey (DELIS-3)

Open AccessPublished:February 09, 2023DOI:https://doi.org/10.1016/j.aucc.2022.12.008

      Abstract

      Background

      Delirium is common in critically ill patients with detrimental effects in terms of increased morbidity, mortality, costs, and human suffering. Delirium detection and management depends on systematic screening for delirium, which can be challenging to implement in clinical practice.

      Objectives

      The aim of this study was to explore how nurses in the intensive care unit perceived the use of Confusion Assessment Method for the Intensive Care Unit (CAM-ICU), the Confusion Assessment Method for the Intensive Care Unit-7 (CAM-ICU-7), and Intensive Care Delirium Screening Checklist (ICDSC) for delirium screening of patients in the intensive care unit.

      Methods

      This was a cross-sectional, electronic-based survey of nurses' perceptions of delirium screening with the three different instruments for delirium screening. Nurses were asked to grade their perception of the usability of the three instruments and how well they were perceived to detect delirium and delirium symptom changes on a 1- to 6-point Likert scale. Open questions about perceived advantages and disadvantages of each instrument were analysed using the framework method.

      Results

      One hundred twenty-seven of 167 invited nurses completed the survey and rated the CAM-ICU-7 as faster and easier than the ICDSC, which was more nuanced and reflected changes in the patient's delirium better. Despite being rated as the fastest, easiest, and most used, the CAM-ICU provided less information and was considered inferior to the CAM-ICU-7 and ICDSC. Using familiar instruments made delirium screening easier, but being able to grade and nuance the delirium assessment was experienced as important for clinical practice.

      Conclusions

      Both the ICDSC and the CAM-ICU-7 were perceived well suited for detection of delirium and reflected changes in delirium intensity. The CAM-ICU was rated as fast and easy but inferior in its ability to grade and nuance the assessment of delirium. Emphasis on clinical meaningfulness and continued education in delirium screening are necessary for adherence to delirium management guidelines.

      Keywords

      1. Introduction

      Delirium is commonly occurring in the intensive care unit (ICU), and identification depends on systematically screening for detection. Especially the hypoactive form of delirium may be difficult to detect without employment of proper screening tools.
      • Kotfis K.
      • Marra A.
      • Ely E.W.
      ICU delirium - a diagnostic and therapeutic challenge in the intensive care unit.
      Systematic screening can, however, be difficult to implement in clinical practice as nurses may distrust screening instruments and prefer a subjective assessment of the patient's mental status,
      • Collet M.O.
      • Egerod I.
      • Christensen V.
      • Jensen J.
      • Thomsen T.
      Nurses' and physicians' perceptions of Confusion Assessment Method for the intensive care unit for delirium detection: focus group study.
      ,
      • Eastwood G.M.
      • Peck L.
      • Bellomo R.
      • Baldwin I.
      • Reade M.C.
      A questionnaire survey of critical care nurses' attitudes to delirium assessment before and after introduction of the CAM-ICU.
      yet nurses may be engaged in delirium detection if they find it meaningful.
      • Collet M.O.
      • Egerod I.
      • Christensen V.
      • Jensen J.
      • Thomsen T.
      Nurses' and physicians' perceptions of Confusion Assessment Method for the intensive care unit for delirium detection: focus group study.
      ,
      • Oosterhouse K.J.
      • Vincent C.
      • Foreman M.D.
      • Gruss V.A.
      • Corte C.
      • Berger B.
      Intensive care unit nurses' beliefs about delirium assessment and management.
      In this study, we explore nurses' perception of different delirium screening instruments to gain a deeper understanding of what nurses find useful in clinical practice.

      1.1 Background

      Delirium is the clinical manifestation of the pathobiological process of acute encephalopathy
      • Oldham M.A.
      • Holloway R.G.
      Delirium disorder: integrating delirium and acute encephalopathy.
      ,
      • Slooter A.J.C.
      • Otte W.M.
      • Devlin J.W.
      • Arora R.C.
      • Bleck T.P.
      • Claassen J.
      • et al.
      Updated nomenclature of delirium and acute encephalopathy: statement of ten Societies.
      and should be regarded as brain dysfunction.
      • Stollings J.L.
      • Kotfis K.
      • Chanques G.
      • Pun B.T.
      • Pandharipande P.P.
      • Ely E.W.
      Delirium in critical illness: clinical manifestations, outcomes, and management.
      ,
      • Wilson J.E.
      • Mart M.F.
      • Cunningham C.
      • Shehabi Y.
      • Girard T.D.
      • MacLullich A.M.J.
      • et al.
      Delirium.
      Delirium is characterised by acute onset, fluctuating symptoms, reduced attention, and altered cognition and is by definition triggered by an organic cause.
      • Wilson J.E.
      • Mart M.F.
      • Cunningham C.
      • Shehabi Y.
      • Girard T.D.
      • MacLullich A.M.J.
      • et al.
      Delirium.
      Delirium is associated with increased mortality and
      • Pisani M.A.
      • Kong S.Y.
      • Kasl S.V.
      • Murphy T.E.
      • Araujo K.L.
      • Van Ness P.H.
      Days of delirium are associated with 1-year mortality in an older intensive care unit population.
      increased healthcare costs
      • Stollings J.L.
      • Kotfis K.
      • Chanques G.
      • Pun B.T.
      • Pandharipande P.P.
      • Ely E.W.
      Delirium in critical illness: clinical manifestations, outcomes, and management.
      ,
      • Vasilevskis E.E.
      • Chandrasekhar R.
      • Holtze C.H.
      • Graves J.
      • Speroff T.
      • Girard T.D.
      • et al.
      The cost of ICU delirium and coma in the intensive care unit patient.
      and might be associated with poorer cognitive outcome
      • Stollings J.L.
      • Kotfis K.
      • Chanques G.
      • Pun B.T.
      • Pandharipande P.P.
      • Ely E.W.
      Delirium in critical illness: clinical manifestations, outcomes, and management.
      although this is not consistently found in the intensive care population.
      • Collet M.O.
      • Egerod I.
      • Thomsen T.
      • Wetterslev J.
      • Lange T.
      • Ebdrup B.H.
      • et al.
      Risk factors for long-term cognitive impairment in ICU survivors: a multicenter, prospective cohort study.
      Under all circumstances, delirium is a frightening experience for patients and relatives.
      • Boehm L.M.
      • Jones A.C.
      • Selim A.A.
      • Virdun C.
      • Garrard C.F.
      • Walden R.L.
      • et al.
      Delirium-related distress in the ICU: a qualitative meta-synthesis of patient and family perspectives and experiences.
      The 2018 Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU (PADIS) guideline recommends systematic and regular screening for delirium with validated tools.
      • Devlin J.W.
      • Skrobik Y.
      • Gélinas C.
      • Needham D.M.
      • Slooter A.J.C.
      • Pandharipande P.P.
      • et al.
      Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU.
      Especially hypoactive delirium is difficult to recognise in patients that may be lethargic and withdrawn
      • Kotfis K.
      • Marra A.
      • Ely E.W.
      ICU delirium - a diagnostic and therapeutic challenge in the intensive care unit.
      and consequently may be overlooked by healthcare professionals. Well-known methods for delirium detection in the ICU includes the Confusion Assessment Method for the ICU (CAM-ICU)
      • Ely E.W.
      • Margolin R.
      • Francis J.
      • May L.
      • Truman B.
      • Dittus R.
      • et al.
      Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU).
      and the Intensive Care Delirium Screening Checklist (ICDSC).
      • Bergeron N.
      • Dubois M.J.
      • Dumont M.
      • Dial S.
      • Skrobik Y.
      Intensive care delirium screening checklist: evaluation of a new screening tool.
      The CAM-ICU assesses inattention and disorganised thinking using prespecified tests,
      • Ely E.W.
      • Margolin R.
      • Francis J.
      • May L.
      • Truman B.
      • Dittus R.
      • et al.
      Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU).
      whereas the ICDSC supplements tests of inattention and disorganised thinking with clinical observations of sleep pattern, motor activity, behaviour, and hallucinations.
      • Bergeron N.
      • Dubois M.J.
      • Dumont M.
      • Dial S.
      • Skrobik Y.
      Intensive care delirium screening checklist: evaluation of a new screening tool.
      The CAM-ICU has a dichotomous outcome (delirium positive/delirium negative),
      • Ely E.W.
      • Margolin R.
      • Francis J.
      • May L.
      • Truman B.
      • Dittus R.
      • et al.
      Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU).
      whereas the ICDSC results in a score between 0 and 8: 0 denotes no delirium, 1–3 denotes subsyndromal delirium, and 4–8 denotes delirium.
      • Bergeron N.
      • Dubois M.J.
      • Dumont M.
      • Dial S.
      • Skrobik Y.
      Intensive care delirium screening checklist: evaluation of a new screening tool.
      ,
      • Ouimet S.
      • Riker R.
      • Bergeron N.
      • Cossette M.
      • Kavanagh B.
      • Skrobik Y.
      Subsyndromal delirium in the ICU: evidence for a disease spectrum.
      The Confusion Assessment Method for the Intensive Care Unit-7 (CAM-ICU-7), developed by Khan et al.
      • Khan B.A.
      • Perkins A.J.
      • Gao S.
      • Hui S.L.
      • Campbell N.L.
      • Farber M.O.
      • et al.
      The confusion assessment method for the ICU-7 delirium severity scale: a novel delirium severity instrument for use in the ICU.
      and building on the CAM-ICU,
      • Ely E.W.
      • Margolin R.
      • Francis J.
      • May L.
      • Truman B.
      • Dittus R.
      • et al.
      Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU).
      assesses the same elements as the CAM-ICU. The CAM-ICU-7 has been validated as a measure of delirium severity on a scale from 0 to 7: 0–2 denotes no delirium, 3–5 mild to moderate delirium, and 6–7 severe delirium.
      • Khan B.A.
      • Perkins A.J.
      • Gao S.
      • Hui S.L.
      • Campbell N.L.
      • Farber M.O.
      • et al.
      The confusion assessment method for the ICU-7 delirium severity scale: a novel delirium severity instrument for use in the ICU.
      Krewulak et al.
      • Krewulak K.D.
      • Rosgen B.K.
      • Ely E.W.
      • Stelfox H.T.
      • Fiest K.M.
      The CAM-ICU-7 and ICDSC as measures of delirium severity in critically ill adult patients.
      showed significant correlation between the CAM-ICU-7 and the ICDSC although the ICDSC has not been validated as a measure of delirium severity.
      Delirium screening has been shown to be pivotal for efficient delirium management
      • Kotfis K.
      • Marra A.
      • Ely E.W.
      ICU delirium - a diagnostic and therapeutic challenge in the intensive care unit.
      ; however, systematic and regular delirium screening of ICU patients was negatively affected by distrust in the reliability of screenings instruments, difficulties in using the instruments when patients were fatigued or loosing concentration, and the belief that a clinical judgement of an experienced ICU nurse was more accurate in detection of delirium.
      • Collet M.O.
      • Egerod I.
      • Thomsen T.
      • Wetterslev J.
      • Lange T.
      • Ebdrup B.H.
      • et al.
      Risk factors for long-term cognitive impairment in ICU survivors: a multicenter, prospective cohort study.
      As delirium has severe negative consequences for patients
      • Stollings J.L.
      • Kotfis K.
      • Chanques G.
      • Pun B.T.
      • Pandharipande P.P.
      • Ely E.W.
      Delirium in critical illness: clinical manifestations, outcomes, and management.
      and is difficult to detect,
      • Kotfis K.
      • Marra A.
      • Ely E.W.
      ICU delirium - a diagnostic and therapeutic challenge in the intensive care unit.
      ,
      • Pandharipande P.P.
      • Ely E.W.
      • Arora R.C.
      • Balas M.C.
      • Boustani M.A.
      • La Calle G.H.
      • et al.
      The intensive care delirium research agenda: a multinational, interprofessional perspective.
      there is a need to further explore and understand nurses' perceptions and uses of different delirium screening instruments for ICU patients.

      1.2 Aim

      The aim of this study was to explore how nurses in the ICU perceived the use of the CAM-ICU, CAM-ICU-7, and ICDSC for delirium screening of patients admitted to the ICU.

      2. Methods

      This study was an electronic, cross-sectional survey. The survey combined quantitative and qualitative questions and was issued to nurses who had completed delirium screening of ICU patients with three different instruments (the CAM-ICU, the CAM-ICU-7, and the ICDSC) as part of the overarching DELIS-3 study (ClinicalTrials.gov ID NCT04551508).

      2.1 Delirium screening instruments

      Fifteen of 17 participating ICUs had implemented the Danish version of the CAM-ICU
      • Svenningsen H.
      Confusion assessment method for the ICU (CAM-ICU).
      for delirium screening, and two ICUs used the Danish version of ICDSC.
      • Larsen L.K.
      • Frøkjaer V.G.
      • Nielsen J.S.
      • Skrobik Y.
      • Winkler Y.
      • Møller K.
      • et al.
      Delirium assessment in neuro-critically ill patients: a validation study.
      The Danish version of the ICDSC was approved by Y. Skrobik, one of the original authors of the ICDSC,
      • Bergeron N.
      • Dubois M.J.
      • Dumont M.
      • Dial S.
      • Skrobik Y.
      Intensive care delirium screening checklist: evaluation of a new screening tool.
      and had an important clarification from the original ICDSC as it took into account sedation as a confounder for assessment of consciousness and psychomotor slowing.
      • Devlin J.W.
      • Skrobik Y.
      • Gélinas C.
      • Needham D.M.
      • Slooter A.J.C.
      • Pandharipande P.P.
      • et al.
      Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU.
      ,
      • Ouimet S.
      • Kavanagh B.P.
      • Gottfried S.B.
      • Skrobik Y.
      Incidence, risk factors and consequences of ICU delirium.
      The CAM-ICU-7 had only recently been translated into Danish
      • Svenningsen H.
      • Gow N.D.
      • Nielsen A.H.
      Valideret oversættelse forbedrer redskab til deliriumvurdering hos kritisk syge.
      and was not used in any Danish ICUs to our knowledge.

      2.2 Participants and setting

      For this survey, we included a total of 176 nurses from 17 Danish ICUs (11 ICUs in regional hospitals and 6 ICUs in university hospitals) who had participated in the overarching DELIS-3 study. Participants had been recruited because of their interest in delirium in ICU patients. Nine withdrew due to maternity or sick leave or job change, leaving 168 participants. All participants were ICU nurses with more than 6 months' clinical ICU experience. Prior to completing the survey, each nurse had received training in delirium and the use of three different instruments (CAM-ICU, CAM-ICU-7, and ICDSC) for delirium screening followed by bedside training in delirium screening by an ICU nurse experienced in delirium screening. As part of the DELIS-3 study, each participating nurse was asked to screen 10 different ICU patients at least once, each time using the three different instruments. For each screening, the sequence of the screening instruments was randomised to avoid giving preferential treatment to one method. Nurses were not blinded to previous delirium screening results.

      2.3 Data collection

      The survey included questions about nurses' perceptions of (i) how time-consuming they experienced delirium screening to be, (ii) how difficult they experienced delirium screening, (iii) the perceived ability of the instruments to detect delirium, (iv) the perceived ability of the instruments to detect changes in delirium, (v) experienced advantages and disadvantages related to each instrument, and (vi) to what degree disadvantages were outweighed by the advantages. Questions 1–4 and 6 were rated on a 1- to 6-point Likert scale (1 = not at all, 6 = very much). Question 5 allowed respondents to elaborate in a short text format.
      We collected data on the participants' age, ICU work experience, perceived delirium screening competence, and habitual delirium screening method. The questionnaire was pilot tested and amended before use and is available as online supplement.
      After completing the screenings or at the end of the DELIS-3 study (whichever came first), each participant received an electronic questionnaire using REDCap electronic data capture tools. Participants were reminded to respond to the survey up to 3 times and could only respond once.

      2.4 Data analysis

      Quantitative questions were analysed using a mixed model, with three paired t-tests. In addition, we did subanalyses adjusting for age, ICU work experience, perceived delirium screening competence, and habitual delirium screening instrument. No multiple tests were made since data were not independent. For categories with less than 20 respondents, p-values were omitted since these data could not be considered normally distributed. In case of missing data, no imputation was attempted. Statistical analysis was performed in STATA.
      StataCorp
      Stata Statistical Software: Release 17.
      Responses for the qualitative question about advantages and disadvantages to the three different screening instruments were analysed using the framework method,
      • Gale N.K.
      • Heath G.
      • Cameron E.
      • Rashid S.
      • Redwood S.
      Using the framework method for the analysis of qualitative data in multi-disciplinary health research.
      beginning with familiarisation with the data and inductive coding performed in NVivo.
      QSR International Pty Ltd
      NVivo 12.
      A matrix of the instruments and codes (familiarity with screening, professional competency, monitoring delirium, confidence in screening, barriers for screening, documentation) was developed in which data were entered and condensed. Examination of the matrix allowed for the identification of differences and similarities across instruments and formulation of overarching themes.
      • Gale N.K.
      • Heath G.
      • Cameron E.
      • Rashid S.
      • Redwood S.
      Using the framework method for the analysis of qualitative data in multi-disciplinary health research.
      During the back-and-forth process of the analysis, we continually reflected on the relationship between data and possible interpretations looking for internal homogeneity and external heterogeneity between themes,
      • Braun V.
      • Clarke V.
      Using thematic analysis in psychology.
      thus ensuring rigour and trustworthiness.
      • Shenton A.K.
      Strategies for ensuring trustworthiness in qualitative research projects.

      2.5 Ethical considerations

      All participants were recruited by a nurse specialist in their own department, and all participants gave written, informed consent when responding to the electronic survey. Participants were given the opportunity to withdraw from the study after completing the survey although no one chose to. To protect participant confidentiality, all data were pseudonymised and participants' sex was not recorded as nurses of male sex were less prevalent. The study was registered with Region Central Jutland, identifier 1-16-02-546-20. Approval by the ethics committee was not required by Danish law, however, the study was conducted in accordance with the Helsinki Declaration.
      • The World Medical Association
      WMA Declaration of Helsinki - ethical principles for medical research involving human subjects.
      Data were stored in REDCap at Aarhus University and thereby protected by EU regulations and reported using the Consensus-based Checklist for Reporting of Survey Studies (CROSS) guideline.
      • Sharma A.
      • Minh Duc N.T.
      • Luu Lam Thang T.
      • Nam N.H.
      • Ng S.J.
      • Abbas K.S.
      • et al.
      A Consensus-Based Checklist for Reporting of Survey Studies (CROSS).

      3. Results

      A total of 127 of the invited 168 nurses completed the questionnaire (76%). Participants had a mean of 12.2 years (standard deviation [SD]: 8.1) working experience in the ICU and felt competent in delirium screening as indicated on a 6-point Likert scale (mean: 5.1, SD: 0.8). Most nurses (90.1%) had the CAM-ICU as their usual instrument for delirium screening, 8.7% used the ICDSC, and one nurse indicated using the Glasgow Coma Scale for delirium assessment. On average, participants had completed 7.9 delirium screenings using all three instruments (Table 1).
      Table 1Participant characteristics.
      n/NMeanSD
      Nurse age (years)127/12744.810.1
      Experience from ICU (years)127/12712.28.1
      Complete screenings by each nurse for the DELIS-3 study
      In addition, some patients were screened several times.
      127/1277.9
      Feeling of competence
      Likert scale 1–6 point: 1 = not at all. 6 = very much.
      127/1275.10.8
      Usual screening instrument (%)
       ICDSC11/1278.7
       CAM-ICU115/12790.1
       Other (GCS)1/1270.8
      CAM-ICU, Confusion Assessment Method for the Intensive Care Unit; GCS, Glasgow Coma Scale; ICU, intensive care unit; ICDSC, Intensive Care Delirium Screening Checklist; SD, standard deviation.
      a In addition, some patients were screened several times.
      b Likert scale 1–6 point: 1 = not at all. 6 = very much.
      Overall, delirium screening was perceived to be quick and easy to complete, with the ICDSC as the most time-consuming (mean: 3.31, SD: 1.22), followed by the CAM-ICU-7 and CAM-ICU as the fastest (mean: 2.42, SD: 0.99). The CAM-ICU was the easiest (mean: 1.9, SD: 0.95) followed by the CAM-ICU-7 and the ICDSC as the most difficult (mean: 2.93, SD: 1.32). The ICDSC was perceived as the most accurate tool to detect delirium (mean: 4.56, SD: 1.1.), closely followed by the CAM-ICU-7 (mean: 4.27, SD: 1.09). The ICDSC was furthermore perceived as the instrument that most accurately reflected a change in the delirium (mean: 4.55, SD: 1.11), followed by the CAM-ICU-7 (mean: 4.12, SD: 1.14) and the CAM-ICU (mean: 3.42, SD: 1.15) (Table 2).
      Table 2Absolute scores by instrument.
      n/NMeanSD
      Time-consuming
      Likert scale 1–6 point: 1 = not at all. 6 = very much.
       ICDSC127/1273.311.22
       CAM-ICU126/1272.420.99
       CAM-ICU-7126/1272.891.09
      Difficult to use
      Likert scale 1–6 point: 1 = not at all. 6 = very much.
       ICDSC126/1272.931.32
       CAM-ICU126/1271.90.95
       CAM-ICU-7126/1272.391.02
      Fit experience of whether the patient has delirium
      Likert scale 1–6 point: 1 = not at all. 6 = very much.
       ICDSC126/1274.561.1
       CAM-ICU126/1273.821.05
       CAM-ICU-7126/1274.271.09
      Reflects change in the patient's delirium
      Likert scale 1–6 point: 1 = not at all. 6 = very much.
       ICDSC40/1274.551.11
       CAM-ICU73/1273.421.15
       CAM-ICU-741/1274.121.14
      Disadvantages outweighed by advantages
      Likert scale 1–6 point: 1 = not at all. 6 = very much.
       ICDSC121/1273.951.4
       CAM-ICU121/1273.431.15
       CAM-ICU-7121/1273.611.34
      CAM-ICU, Confusion Assessment Method for the Intensive Care Unit; CAM-ICU-7, Confusion Assessment Method for the Intensive Care Unit-7; ICDSC, Intensive Care Delirium Screening Checklist; SD, standard deviation.
      a Likert scale 1–6 point: 1 = not at all. 6 = very much.
      Comparing the three instruments pairwise; the ICDSC was overall evaluated slightly better than the CAM-ICU-7, which was in turn evaluated better than the CAM-ICU (Table 3). The ICDSC was considered slightly more time-consuming (0.41, SD: 1.22, p = <0.001) and more difficult to use (0.54, SD: 1.35, p = <0.001) than the CAM-ICU-7. However, the ICDSC was perceived as more accurately reflecting the patients' delirium than CAM-ICU-7 (mean difference: 0.29, SD: 1.26, p = 0.01) and more accurately reflecting change in the patient's delirium (mean difference: 0.41, SD: 0.86, p = 0.008) than the CAM-ICU-7. Advantages of the ICDSC outweighed disadvantages slightly better than the CAM-ICU-7 (mean difference: 0.34, SD: 1.69, p = 0.029), whereas the CAM-ICU was inferior to the ICDSC and the CAM-ICU-7 (Table 3).
      Table 3Pair-wise comparison of instruments (all nurses).
      n/NMean differenceSDp-value
      Time-consuming
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs. CAM-ICU126/1270.881.32<0.001
       ICDSC vs. CAM-ICU-7126/1270.411.22<0.001
       CAM-ICU vs. CAM-ICU-7126/127−0.470.68<0.001
      Difficult to use
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs. CAM-ICU126/1271.031.49<0.001
       ICDSC vs. CAM-ICU-7126/1270.541.35<0.001
       CAM-ICU vs. CAM-ICU-7126/127−0.490.91<0.001
      Fit experience of whether the patient has delirium
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs. CAM-ICU126/1270.751.35<0.001
       ICDSC vs. CAM-ICU-7126/1270.291.260.01
       CAM-ICU vs. CAM-ICU-7126/127−0.450.94<0.001
      Reflects change in the patient's delirium
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs. CAM-ICU38/1270.951.06<0.001
       ICDSC vs. CAM-ICU-734/1270.410.860.008
       CAM-ICU vs. CAM-ICU-740/127−0.551.080.003
      Disadvantages outweighed by advantages
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs. CAM-ICU121/1270.521.760.002
       ICDSC vs. CAM-ICU-7121/1270.341.690.029
       CAM-ICU vs. CAM-ICU-7121/127−0.181.13ns
      CAM-ICU, Confusion Assessment Method for the Intensive Care Unit; CAM-ICU-7, Confusion Assessment Method for the Intensive Care Unit-7; ICDSC, Intensive Care Delirium Screening Checklist; SD, standard deviation.
      a Likert scale 1–6 point: 1 = not at all, 6 = very much.
      Dividing nurses into age groups (24–39 years, 40–50 years, 51–65 years) (Table 4) and groups of ICU experience (1–4 years, 5–11 years, >11 years) (Table S1, online supplement only) showed that among the youngest and least experienced nurses, there were no significant differences in their perceptions of the ICDSC and CAM-ICU-7 (Table 4 and S1), whereas the oldest and most experienced nurses consistently favoured the ICDSC over the CAM-ICU-7 (Table 4 and S1). Prior experience with an instrument (Table S2, online supplement only) made the instrument faster and easier to use, whereas feelings of competence did not influence perception of instruments (Table S3, online supplement only).
      Table 4Pair-wise comparison of instruments by age groups.
      Age 24–39 yearsAge 40–50 yearsAge 51–65 years
      n/NMean differenceSDp-valuen/NMean differenceSDp-valuen/NMean differenceSDp-value
      Time-consuming
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs CAM-ICU39/1270.671.20.00143/1270.951.34<0.00144/12711.4<0.001
       ICDSC vs CAM-ICU-739/1270.211.22ns43/1270.491.330.02144/1270.521.090.003
       CAM-ICU vs CAM-ICU-739/127−0.460.6<0.00143/127−0.470.7<0.00144/127−0.480.73<0.001
      Difficult to use
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs CAM-ICU39/1270.791.32<0.00143/1271.071.52<0.00144/1271.21.61<0.001
       ICDSC vs CAM-ICU-739/1270.331.32ns43/1270.741.530.00344/1270.521.170.005
       CAM-ICU vs CAM-ICU-739/127−0.460.76<0.00143/127−0.330.810.01244/127−0.681.09<0.001
      Fit experience of whether the patient has delirium
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs CAM-ICU39/1270.621.290.00543/12711.33<0.00144/1270.611.420.006
       ICDSC vs CAM-ICU-739/12701.12ns43/1270.31.12ns44/1270.551.450.017
       CAM-ICU vs CAM-ICU-739/127−0.621.090.00143/127−0.70.8<0.00144/127−0.070.82ns
      Reflects change in the patient's delirium
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs CAM-ICU17/12711.0613/1271.081.198/1270.6392
       ICDSC vs CAM-ICU-714/1270.140.6612/1270.330.788/12711.07
       CAM-ICU vs CAM-ICU-716/127−0.811.1714/127−0.640.9310/12701.05
      Disadvantages outweighed by advantages
      Likert scale 1–6 point: 1 = not at all, 6 = very much.
       ICDSC vs CAM-ICU37/1270.511.57ns41/1270.241.58ns43/1270.792.050.015
       ICDSC vs CAM-ICU-737/127−0.031.59ns41/1270.0241.59ns43/1270.951.73<0.001
       CAM-ICU vs CAM-ICU-737/127−0.541.070.00441/127−0.221.17ns43/1270.161.07ns
      CAM-ICU, Confusion Assessment Method for the Intensive Care Unit; CAM-ICU-7, Confusion Assessment Method for the Intensive Care Unit-7; ICDSC, Intensive Care Delirium Screening Checklist; SD, standard deviation.
      a Likert scale 1–6 point: 1 = not at all, 6 = very much.
      Overall, the CAM-ICU-7 was slightly faster and easier to use than the ICDSC, whereas the ICDSC better reflected the patient's symptoms and change in symptoms of delirium. Disadvantages were slightly better outweighed by advantages in the ICDSC regardless of prior experience (Fig. 1).
      Fig. 1
      Fig. 1Comparison of CAM-ICU-7 and ICDSC. ∗ The question has been reversed so that a negative score is in favour of the CAM-ICU-7. CAM-ICU, Confusion Assessment Method for the Intensive Care Unit; CAM-ICU-7, Confusion Assessment Method for the Intensive Care Unit-7; ICDSC, Intensive Care Delirium Screening Checklist.
      The qualitative data showed that using familiar instruments made delirium screening easier. Being able to grade and nuance the delirium assessment was important to clinical practice, but nurses were very particular when it came to assessments that did not accurately reflect the patients' delirium status (Table 5).
      Table 5Perceived advantages and disadvantages of delirium screening.
      AdvantagesDisadvantages
      CAM-ICU
      • Very fast and easy to repeat
      • Well known
      • Standardised screening method
      • Unnuanced
      • Does not quantify delirium intensity
      • Nurses found questions silly
      CAM-ICU-7
      • Fast and easy to repeat
      • Very alike CAM-ICU
      • Graded delirium intensity
      • Standardised screening method
      • Lacked a nuanced assessment of delirium symptoms
      • Nurses found questions silly
      • All elements had to be assessed
      ICDSC
      • Nuanced assessment of delirium symptoms
      • Graded delirium intensity
      • Prompted nurses' reflections on delirium symptoms
      • Extensive
      • Not well known
      • Relied on good documentation
      • Unclear screening procedure
      CAM-ICU, Confusion Assessment Method for the Intensive Care Unit; CAM-ICU-7, Confusion Assessment Method for the Intensive Care Unit-7; ICDSC, Intensive Care Delirium Screening Checklist.

      3.1 Using familiar delirium screening instruments and procedures

      Most nurses were familiar with the CAM-ICU, and this paved the way for the CAM-ICU-7, which has almost identical items, although the calculation of a score was new to them. Both the CAM-ICU and CAM-ICU-7 were easy to remember, complete, and repeat as needed.It's easy to do. Very simple and not time-consuming at all (perhaps because I'm used to it).” (About the CAM-ICU).
      The ICDSC was new to most nurses in the study, and at a first glance, the ICDSC felt confusing with much text describing the different elements. However, those who were accustomed to the ICDSC felt that it was easy to use as most of the needed observations were embedded in their usual clinical observations of the patients. A drawback of the ICDSC was that it required good documentation of the patient's status the previous 24 h to be able to complete all items of the ICDSC. Some nurses expressed that it could be difficult to locate these data. The CAM-ICU and the CAM-ICU-7 were less dependent on data from previous shifts; however, knowledge of the patient's habitual status made delirium screening more accurate. The expressed advantage of being familiar with the screening instrument and procedures related to locating necessary data illustrates that it may take time to know any of the instruments well.

      3.2 Being able to monitor and grade delirium

      Being able to grade the intensity of delirium and monitor delirium over time was a benefit of both the CAM-ICU-7 and ICDSC, who both produced a score that could increase or decrease over time. Nurses found this beneficial as they could initiate nonpharmacological measures to decrease or alleviate the patient's delirium symptoms. The ICDSC resulted in a score that reflected delirium across a whole shift, whereas the CAM-ICU and the CAM-ICU-7 were regarded a snapshot of the patient's delirium that could be repeated as needed and thereby monitored over time.It gives a clearer picture of the deliriousness, whether it gets better or worse and it's useful for planning interventions”. (About the CAM-ICU-7).
      Yet, some nurses were disinclined to follow the delirium screening protocol because they were concerned about troubling patients who were clearly delirious. Also, in patients who were not delirious, they found it difficult to ask the questions of unorganised thinking in the CAM-ICU and CAM-ICU-7, which they found “silly”.I find it difficult to ask the patient all those silly questions – especially when the patient is completely awake and alert”. (About the CAM-ICU and CAM-ICU-7).
      Besides grading delirium, nurses emphasised that they used their professional judgement to qualitatively assess the patients' delirium. Especially the CAM-ICU but also the CAM-ICU-7 needed to be supplemented by further assessments of delirium features such as agitation or sedation, sleep–wake cycle, and hallucinations to get the full picture of the patient's delirium. Such an assessment was provided by the ICDSC. Nurses, who were new to the ICDSC, expressed that the ICDSC opened their eyes to the subtle yet complex symptoms of delirium, and in that sense, the ICDSC was formative and increased the participating nurses' knowledge about delirium.I was surprised that patients who scored negative for delirium still had frightening dreams”. (About the ICDSC).

      3.3 Having confidence in delirium screening

      For delirium screening to be clinically meaningful, nurses needed to feel confident in the results. Across all three instruments, nurses found some patients difficult to assess, e.g., patients who were too feeble to move their hands, had difficulties hearing or seeing, were tired, were sedated, or were suffering from traumatic brain injuries. In these cases, nurses would consider a positive delirium screening as a false positive. When patients were screened as delirium negative on the CAM-ICU or CAM-ICU-7 despite suffering from hallucinations or disturbed sleep–wake cycles, nurses would consider this a false negative screening. Some nurses found patients who were unable to collaborate with delirium screening in the ICU unsuitable for delirium screening, which suggests that there may be some basic misunderstanding on how lack of attention towards screening should be interpreted.It's imperative that the patient can collaborate on screening”. (About the CAM-ICU).
      The CAM-ICU and the CAM-ICU-7 used the same approach for delirium screening every time which was considered a strength, whereas the ICDSC relied on a combination of observations of patient behaviour and questions to the patient. Taking more features into account, nurses found the ICDSC more sensitive for delirium detection and change in the patient's symptoms.That there is a dialogue with the patient about their experiences of hallucinations, nightmares and sleep. It includes the patient's own experiences”. (About the ICDSC).
      The ICDSC had more room for the nurses' subjective assessment of the patient even when based on a dialogue with the patient. This could be considered a disadvantage compared to the more standardised approaches of the CAM-ICU and the CAM-ICU-7. However, some elements of the ICDSC were open to different interpretation among nurses, who found the descriptions ambiguous and difficult to use in practice and led nurses to perceive the ICDSC as dependent on the nurse.Some things were strange like ‘inappropriate speech or mood’. What is that and how do I know it?” (About the ICDSC).

      4. Discussion

      This survey study found that nurses rated the CAM-ICU-7 as faster and easier to use than the ICDSC, which instead was more nuanced and better reflected changes in the patient's delirium. The CAM-ICU was rated as the fastest and easiest and most commonly used, but it provided less information and was therefore considered inferior to the two other instruments. In comparison, a meta-analysis
      • Chen T.J.
      • Chung Y.W.
      • Chang H.R.
      • Chen P.Y.
      • Wu C.R.
      • Hsieh S.H.
      • et al.
      Diagnostic accuracy of the CAM-ICU and ICDSC in detecting intensive care unit delirium: a bivariate meta-analysis.
      showed good predictive values of both the CAM-ICU (sensitivity: 0.84, specificity: 0.95) and the ICDSC (sensitivity: 0.83, specificity: 0.87), but present study suggests that clinical relevance of delirium screening may also be determined by direct applicability of the results in clinical practice.
      Nurses wanted to use their professional judgement when assessing patients' delirium, which fit well with the ICDSC procedure of observing the patient throughout the shift. A study by Kvande et al.
      • Kvande M.
      • Delmar C.
      • Lykkeslet E.
      • Storli S.L.
      Assessing changes in a patient's condition – perspectives of intensive care nurses.
      found that assessing changes in the ICU patient's condition required the skilled ICU nurses to be sensitive to subtle changes in the patient's unique situation. Comparing the ICDSC to the CAM-ICU-7, the ICDSC has more room for a qualitative, nuanced assessment over time, which may explain why nurses found this instrument more sensitive to delirium changes than the CAM-ICU-7. Although most nurses viewed themselves as very competent in delirium screening, the ICDSC broadened their clinical understanding of delirium. This suggests that there is an ongoing need for education on delirium management even in nurses with self-rated high competency in delirium screening. This is supported by a Dutch survey of ICU professionals by Trogrlić et al.,
      • Trogrlić Z.
      • Ista E.
      • Ponssen H.H.
      • Schoonderbeek J.F.
      • Schreiner F.
      • Verbrugge S.J.
      • et al.
      Attitudes, knowledge and practices concerning delirium: a survey among intensive care unit professionals.
      which showed that while all respondents (mainly nurses) agreed that delirium screening was useful, respondents had less confidence in their ability to reliably use a validated instrument.
      • Trogrlić Z.
      • Ista E.
      • Ponssen H.H.
      • Schoonderbeek J.F.
      • Schreiner F.
      • Verbrugge S.J.
      • et al.
      Attitudes, knowledge and practices concerning delirium: a survey among intensive care unit professionals.
      A paper by Hermes et al.
      • Hermes C.
      • Acevedo-Nuevo M.
      • Berry A.
      • Kjellgren T.
      • Negro A.
      • Massarotto P.
      Gaps in pain, agitation and delirium management in intensive care: outputs from a nurse workshop.
      discussed management of delirium and lack of adherence to evidence-based guidelines and found that educational gaps in delirium management may limit adherence to guideline-based care and thereby negatively affect patient care. Thus, a continued focus on knowledge of delirium and beliefs in the relevance of screening are necessary. Choosing an instrument that resonates with nurses' observation practices may therefore further the successful implementation of delirium screening.
      In this study, we found that across all three instruments, nurses found some patients difficult or impossible to assess for delirium. As all instruments required patients to answer questions, nurses were sometimes at loss when patients did not respond properly due to intubation, sedation, or lethargy. An American study explored inappropriate unable-to-assess responses to the CAM-ICU and found that these were associated with sedation, mechanical ventilation, night shift, higher age, and non-white population.
      • Awan O.M.
      • Buhr R.G.
      • Kamdar B.B.
      Factors influencing CAM-ICU documentation and inappropriate “unable to assess” responses.
      However, Pandharipande et al.
      • Pandharipande P.P.
      • Ely E.W.
      • Arora R.C.
      • Balas M.C.
      • Boustani M.A.
      • La Calle G.H.
      • et al.
      The intensive care delirium research agenda: a multinational, interprofessional perspective.
      refuted the unsuitability of certain patient groups for screening and argued that delirium screening is possible albeit difficult in neurologically injured patients, children, and cognitively impaired to mention some groups often relegated to unsystematised assessment only. Moreover, we found that nurses sometimes doubted the results from delirium screening talking about false positives and false negatives. While it was a strength that healthcare professionals were critical to the reliability of procedures, it also illustrated that screening results must be meaningful and applicable to the clinical context. Choosing a delirium screening instrument that allows for grading of delirium intensity may be one way to achieve more meaningful delirium screenings and thereby increase the adherence to delirium management protocols.

      4.1 Strengths and limitations

      A limitation of the study was that not all nurses had screened 10 different patients for delirium, and only a minor part had screened the same delirious patient several times using all three instruments, thereby limiting their ability to assess detection of change in delirium features. Moreover, most nurses were used to the CAM-ICU and may therefore naturally have found the CAM-ICU-7 easier and faster given that the items are nearly identical. The small number of nurses using the ICDSC may have biased our findings in favour of the CAM-ICU or the CAM-ICU-7. However, there may also be a certain novelty effect to the ICDSC that taught nurses about delirium, which would probably wear off in time. When using the screening instruments in their clinical practice, nurses were not blinded to the patients' previous screening results, and hence, expectation bias cannot be ruled out. However, this risk was minimised by randomising the order of screening instruments and the fluctuating quality of delirium well known to ICU clinicians. The three instruments used in this study were internationally well known and translated using WHO's guidelines for translating instruments.
      • Larsen L.K.
      • Frøkjaer V.G.
      • Nielsen J.S.
      • Skrobik Y.
      • Winkler Y.
      • Møller K.
      • et al.
      Delirium assessment in neuro-critically ill patients: a validation study.
      ,
      • Svenningsen H.
      • Gow N.D.
      • Nielsen A.H.
      Valideret oversættelse forbedrer redskab til deliriumvurdering hos kritisk syge.
      ,
      • Svenningsen H.
      ICU-delirium assessed by CAM-ICU and validated by interview.

      5. Conclusion

      Both the ICDSC and the CAM-ICU-7 were perceived to detect delirium and reflect changes in delirium intensity well, while the CAM-ICU was fast and easy but lacks ability to grade and nuance the assessment of delirium. Disadvantages were outweighed by advantages slightly better in the ICDSC than the CAM-ICU-7, but prior knowledge of an instrument or local preferences may determine which instrument to implement when predictive values of instruments are comparable. Emphasis on clinical meaningfulness and continued education in delirium screening are necessary for increasing adherence to delirium management guidelines.

      Funding

      The funding body did not influence the conception of the study, data collection, data analysis, conclusion or decision to publish.

      CRediT authorship contribution statement

      Nielsen, A.H.: Conceptualisation, methodology, software, formal analysis, investigation, resources, data curation, writing – original draft, review and editing, supervision, project administration.
      Svenningsen H., Larsen L.K., Collet M.O.: Conceptualisation, methodology, software, formal analysis, investigation, resources, data curation, writing – review and editing, supervision, project administration.
      Lehmkuhl L., Bekker C., Jensen J.F., Laerkner E., Nielsen T.A., Rossen B.S., Thorn L., Laursen E., Fischer S., Villumsen M., Shiv L.H., Høgh M., Rahr M.N.: Investigation, writing – review and editing, project administration.

      Conflict of interest

      Anne Højager Nielsen is funded by the Novo Nordisk Foundation, NNF 19OC0058277.
      Marie Oxenbøll Collet is funded by the Novo Nordisk Foundation, NNF21OC0072048.
      All other authors declare no conflict of interests.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Kotfis K.
        • Marra A.
        • Ely E.W.
        ICU delirium - a diagnostic and therapeutic challenge in the intensive care unit.
        Anaesthesiol Intensive Ther. 2018; 50: 160-167
        • Collet M.O.
        • Egerod I.
        • Christensen V.
        • Jensen J.
        • Thomsen T.
        Nurses' and physicians' perceptions of Confusion Assessment Method for the intensive care unit for delirium detection: focus group study.
        Nurs Crit Care. 2018; 23: 16-22
        • Eastwood G.M.
        • Peck L.
        • Bellomo R.
        • Baldwin I.
        • Reade M.C.
        A questionnaire survey of critical care nurses' attitudes to delirium assessment before and after introduction of the CAM-ICU.
        Aust Crit Care. 2012; 25: 162-169
        • Oosterhouse K.J.
        • Vincent C.
        • Foreman M.D.
        • Gruss V.A.
        • Corte C.
        • Berger B.
        Intensive care unit nurses' beliefs about delirium assessment and management.
        AACN Adv Crit Care. 2016; 27: 379-393
        • Oldham M.A.
        • Holloway R.G.
        Delirium disorder: integrating delirium and acute encephalopathy.
        Neurology. 2020; 95: 173-178
        • Slooter A.J.C.
        • Otte W.M.
        • Devlin J.W.
        • Arora R.C.
        • Bleck T.P.
        • Claassen J.
        • et al.
        Updated nomenclature of delirium and acute encephalopathy: statement of ten Societies.
        Intensive Care Med. 2020; 46: 1020-1022
        • Stollings J.L.
        • Kotfis K.
        • Chanques G.
        • Pun B.T.
        • Pandharipande P.P.
        • Ely E.W.
        Delirium in critical illness: clinical manifestations, outcomes, and management.
        Intensive Care Med. 2021; 47: 1089-1103
        • Wilson J.E.
        • Mart M.F.
        • Cunningham C.
        • Shehabi Y.
        • Girard T.D.
        • MacLullich A.M.J.
        • et al.
        Delirium.
        Nat Rev Dis Primers. 2020; 6: 90
        • Pisani M.A.
        • Kong S.Y.
        • Kasl S.V.
        • Murphy T.E.
        • Araujo K.L.
        • Van Ness P.H.
        Days of delirium are associated with 1-year mortality in an older intensive care unit population.
        Am J Respir Crit Care Med. 2009; 180: 1092-1097
        • Vasilevskis E.E.
        • Chandrasekhar R.
        • Holtze C.H.
        • Graves J.
        • Speroff T.
        • Girard T.D.
        • et al.
        The cost of ICU delirium and coma in the intensive care unit patient.
        Med Care. 2018; 56: 890-897
        • Collet M.O.
        • Egerod I.
        • Thomsen T.
        • Wetterslev J.
        • Lange T.
        • Ebdrup B.H.
        • et al.
        Risk factors for long-term cognitive impairment in ICU survivors: a multicenter, prospective cohort study.
        Acta Anaesthesiol Scand. 2021; 65: 92-99
        • Boehm L.M.
        • Jones A.C.
        • Selim A.A.
        • Virdun C.
        • Garrard C.F.
        • Walden R.L.
        • et al.
        Delirium-related distress in the ICU: a qualitative meta-synthesis of patient and family perspectives and experiences.
        Int J Nurs Stud. 2021; 122104030
        • Devlin J.W.
        • Skrobik Y.
        • Gélinas C.
        • Needham D.M.
        • Slooter A.J.C.
        • Pandharipande P.P.
        • et al.
        Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU.
        Crit Care Med. 2018; 46: e825-e873
        • Ely E.W.
        • Margolin R.
        • Francis J.
        • May L.
        • Truman B.
        • Dittus R.
        • et al.
        Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU).
        Crit Care Med. 2001; 29: 1370-1379
        • Bergeron N.
        • Dubois M.J.
        • Dumont M.
        • Dial S.
        • Skrobik Y.
        Intensive care delirium screening checklist: evaluation of a new screening tool.
        Intensive Care Med. 2001; 27: 859-864
        • Ouimet S.
        • Riker R.
        • Bergeron N.
        • Cossette M.
        • Kavanagh B.
        • Skrobik Y.
        Subsyndromal delirium in the ICU: evidence for a disease spectrum.
        Intensive Care Med. 2007; 33: 1007-1013
        • Khan B.A.
        • Perkins A.J.
        • Gao S.
        • Hui S.L.
        • Campbell N.L.
        • Farber M.O.
        • et al.
        The confusion assessment method for the ICU-7 delirium severity scale: a novel delirium severity instrument for use in the ICU.
        Crit Care Med. 2017; 45: 851-857
        • Krewulak K.D.
        • Rosgen B.K.
        • Ely E.W.
        • Stelfox H.T.
        • Fiest K.M.
        The CAM-ICU-7 and ICDSC as measures of delirium severity in critically ill adult patients.
        PLoS One. 2020; 15e0242378
        • Pandharipande P.P.
        • Ely E.W.
        • Arora R.C.
        • Balas M.C.
        • Boustani M.A.
        • La Calle G.H.
        • et al.
        The intensive care delirium research agenda: a multinational, interprofessional perspective.
        Intensive Care Med. 2017; 43: 1329-1339
        • Svenningsen H.
        Confusion assessment method for the ICU (CAM-ICU).
        (Komplet træningsmanual) Vanderbilt University Medical Center, 2010
        • Larsen L.K.
        • Frøkjaer V.G.
        • Nielsen J.S.
        • Skrobik Y.
        • Winkler Y.
        • Møller K.
        • et al.
        Delirium assessment in neuro-critically ill patients: a validation study.
        Acta Anaesthesiol Scand. 2019; 63: 352-359
        • Ouimet S.
        • Kavanagh B.P.
        • Gottfried S.B.
        • Skrobik Y.
        Incidence, risk factors and consequences of ICU delirium.
        Intensive Care Med. 2007; 33: 66-73
        • Svenningsen H.
        • Gow N.D.
        • Nielsen A.H.
        Valideret oversættelse forbedrer redskab til deliriumvurdering hos kritisk syge.
        Nord. Nurs Res/Nord. Sygeplejeforskning. 2021; 11: 277-287
        • StataCorp
        Stata Statistical Software: Release 17.
        StataCorp LLC, College Station, TX2021
        • Gale N.K.
        • Heath G.
        • Cameron E.
        • Rashid S.
        • Redwood S.
        Using the framework method for the analysis of qualitative data in multi-disciplinary health research.
        BMC Med Res Methodol. 2013; 13: 117
        • QSR International Pty Ltd
        NVivo 12.
        QSR International Pty Ltd., 2019
        • Braun V.
        • Clarke V.
        Using thematic analysis in psychology.
        Qual Res Psychol. 2006; 3: 77-101
        • Shenton A.K.
        Strategies for ensuring trustworthiness in qualitative research projects.
        Educ Inf. 2004; 22: 63-75
        • The World Medical Association
        WMA Declaration of Helsinki - ethical principles for medical research involving human subjects.
        2013
        • Sharma A.
        • Minh Duc N.T.
        • Luu Lam Thang T.
        • Nam N.H.
        • Ng S.J.
        • Abbas K.S.
        • et al.
        A Consensus-Based Checklist for Reporting of Survey Studies (CROSS).
        J Gen Intern Med. 2021; 36: 3179-3187
        • Chen T.J.
        • Chung Y.W.
        • Chang H.R.
        • Chen P.Y.
        • Wu C.R.
        • Hsieh S.H.
        • et al.
        Diagnostic accuracy of the CAM-ICU and ICDSC in detecting intensive care unit delirium: a bivariate meta-analysis.
        Int J Nurs Stud. 2021; 113103782
        • Kvande M.
        • Delmar C.
        • Lykkeslet E.
        • Storli S.L.
        Assessing changes in a patient's condition – perspectives of intensive care nurses.
        Nurs Crit Care. 2017; 22: 99-104
        • Trogrlić Z.
        • Ista E.
        • Ponssen H.H.
        • Schoonderbeek J.F.
        • Schreiner F.
        • Verbrugge S.J.
        • et al.
        Attitudes, knowledge and practices concerning delirium: a survey among intensive care unit professionals.
        Nurs Crit Care. 2017; 22: 133-140
        • Hermes C.
        • Acevedo-Nuevo M.
        • Berry A.
        • Kjellgren T.
        • Negro A.
        • Massarotto P.
        Gaps in pain, agitation and delirium management in intensive care: outputs from a nurse workshop.
        Intensive Crit Care Nurs. 2018; 48: 52-60
        • Awan O.M.
        • Buhr R.G.
        • Kamdar B.B.
        Factors influencing CAM-ICU documentation and inappropriate “unable to assess” responses.
        Am J Crit Care. 2021; 30: e99-e107
        • Svenningsen H.
        ICU-delirium assessed by CAM-ICU and validated by interview.
        CONNECT: The World of Critical Care Nursing. 2010; 7: 197-201