Ambient intelligence–based monitoring of staff and patient activity in the intensive care unit

Published:October 13, 2022DOI:


      Background: Caregiver workload in the ICU setting is difficult to numerically quantify. Ambient Intelligence utilises computer vision-guided neural networks to continuously monitor multiple datapoints in video feeds, has become increasingly efficient at automatically tracking various aspects of human movement.
      Objectives: To assess the feasibility of using Ambient Intelligence to track and quantify allpatient and caregiver activity within a bedspace over the course of an ICU admission and also to establish patient specific factors, and environmental factors such as time ofday, that might contribute to an increased workload in ICU workers.
      Methods: 5000 images were manually annotated and then used to train You Only LookOnce (YOLOv4), an open-source computer vision algorithm. Comparison of patientmotion and caregiver activity was then performed between these patients.
      Results: The algorithm was deployed on 14 patients comprising 1762800 framesof new, untrained data. There was a strong correlation between the number ofcaregivers in the room and the standardized movement of the patient (p < 0.0001) withmore caregivers associated with more movement. There was a significant difference incaregiver activity throughout the day (p < 0.05), HDU vs. ICU status (p < 0.05), delirious vs. non delirious patients (p < 0.05), and intubated vs. not intubated patients(p < 0.05). Caregiver activity was lowest between 0400 and 0800 (average .71 ± .026caregivers per hour) with statistically significant differences in activity compared to 0800-2400 (p < 0.05). Caregiver activity was highest between 1200 and 1600 (1.02 ± .031 caregivers per hour) with a statistically significant difference in activity comparedto activity from 1600 to 0800 (p < 0.05). The three most dominant predictors of workeractivity were patient motion (Standardized Dominance 78.6%), Mechanical Ventilation(Standardized Dominance 7.9%) and Delirium (Standardized Dominance 6.2%).
      Conclusion: Ambient Intelligence could potentially be used to derive a single standardized metricthat could be applied to patients to illustrate their overall workload. This could be usedto predict workflow demands for better staff deployment, monitoring of caregiver workload, and potentially as a tool to predict burnout.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Australian Critical Care
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ramírez-Elvira S.
        • Romero-Béjar J.L.
        • Suleiman-Martos N.
        • Gómez-Urquiza J.L.
        • Monsalve-Reyes C.
        • Cañadas-De la Fuente G.A.
        • et al.
        Prevalence, risk factors and burnout levels in intensive care unit nurses: a systematic review and meta-analysis.
        Int J Environ Res Publ Health. 2021; 18
        • Caruso R.
        • Annaloro C.
        • Arrigoni C.
        • Ghizzardi G.
        • Dellafiore F.
        • Magon A.
        • et al.
        Burnout and post-traumatic stress disorder in frontline nurses during the COVID-19 pandemic: a systematic literature review and meta-analysis of studies published in 2020.
        Acta Biomed. 2021; 92e2021428
        • McGrath K.
        • Matthews L.R.
        • Heard R.
        Predictors of compassion satisfaction and compassion fatigue in health care workers providing health and rehabilitation services in rural and remote locations: a scoping review.
        Aust J Rural Health. 2022; 30: 264-280
        • DeRienzo C.M.
        • Shaw R.J.
        • Meanor P.
        • Lada E.
        • Ferranti J.
        • Tanaka D.
        A discrete event simulation tool to support and predict hospital and clinic staffing.
        Health Inf J. 2017; 23: 124-133
        • Jin Z.
        • Jovaisa T.
        • Thomas B.
        • Phull M.
        Intensive care unit staffing during the periods of fluctuating bed occupancy: an alternative dynamic model.
        Intensive Crit Care Nurs. 2021; 66103063
        • Bruyneel A.
        • Tack J.
        • Droguet M.
        • Maes J.
        • Wittebole X.
        • Miranda D.R.
        • et al.
        Measuring the nursing workload in intensive care with the Nursing Activities Score (NAS): a prospective study in 16 hospitals in Belgium.
        J Crit Care. 2019; 54: 205-211
        • Ballermann M.A.
        • Shaw N.T.
        • Mayes D.C.
        • Gibney R.T.
        • Westbrook J.I.
        Validation of the Work Observation Method by Activity Timing (WOMBAT) method of conducting time-motion observations in critical care settings: an observational study.
        BMC Med Inf Decis Making. 2011; 11: 32
        • Esteva A.
        • Chou K.
        • Yeung S.
        • Naik N.
        • Madani A.
        • Mottaghi A.
        • et al.
        Deep learning-enabled medical computer vision.
        npj Digital Medicine. 2021; 4
        • Singh A.
        • Haque A.
        • Alahi A.
        • Yeung S.
        • Guo M.
        • Glassman J.R.
        • et al.
        Automatic detection of hand hygiene using computer vision technology.
        J Am Med Inf Assoc. 2020; 27: 1316-1320
        • Ma A.J.
        • Rawat N.
        • Reiter A.
        • Shrock C.
        • Zhan A.
        • Stone A.
        • et al.
        Measuring patient mobility in the ICU using a novel noninvasive sensor.
        Crit Care Med. 2017; 45: 630-636
        • Gutiérrez J.
        • Rodríguez V.
        • Martin S.
        Comprehensive review of vision-based fall detection systems.
        Sensors. 2021; 21: 947
        • Akbarian S.
        • Ghahjaverestan N.M.
        • Yadollahi A.
        • Taati B.
        Noncontact sleep monitoring with infrared video data to estimate sleep apnea severity and distinguish between positional and nonpositional sleep apnea: model development and experimental validation.
        J Med Internet Res. 2021; 23e26524
        • Kaplan R.S.
        • Porter M.E.
        How to solve the cost crisis in health care.
        Harv Bus Rev. 2011; 89 (4, 6-61 passim): 46-52
        • Bradski G.
        The OpenCV Library.
        Dr. Dobb's Journal of Software Tools., 2000
        • Alexey B.
        • Wang C.-Y.
        • Liao H.-Y.M.
        YOLOv4: optimal speed and accuracy of object detection.
        ArXiv abs/2004. 2020; 10934: n. pag.
        • Nguyen-Duc T.
        • Tay A.
        • Chen D.
        • Nguyen J.T.
        • Lyall J.
        • Freitas M.D.
        • et al.
        Estimation of clinical workload and patient Activity using deep learning and optical flow.
        IEEE Sensors Letters. 2022; 6: 1-4
        • Bernat Adell M.D.
        • Galarza Barrachina L.
        • Bisbal Andrés E.
        • Cebrián Graullera G.
        • Pagés Aznar G.
        • Morán Marmaneu M.
        • et al.
        Factors affecting sleep quality in intensive care units.
        Med Intensiva. 2021; 45: 470-476
        • Jaiswal S.J.
        • Garcia S.
        • Owens R.L.
        Sound and light levels are similarly disruptive in ICU and non-ICU wards.
        J Hosp Med. 2017; 12: 798-804
        • Martinez F.E.
        • Ho V.W.
        • Brogan G.
        • Pickering C.
        • Leung G.
        • Parrey J.
        • et al.
        Period-prevalence and risk factors for delirium in a tertiary referral ICU: a prospective cohort study.
        Acute Med. 2021; 20: 168-173
        • Rivera D.I.C.
        • Torres C.C.
        • Romero L.A.L.
        Factors associated with nursing workload in three intensive care units.
        Rev Esc Enferm USP. 2021; 55e20200272