Advertisement

Internal and external validation of machine learning–assisted prediction models for mechanical ventilation–associated severe acute kidney injury

  • Author Footnotes
    1 Equal contribution to this work as co-first authors.
    Sai Huang
    Footnotes
    1 Equal contribution to this work as co-first authors.
    Affiliations
    Department of Hematology, Fifth Medical Center of Chinese PLA General Hospital, Beijing, 100853, China

    National Clinical Research Center of Geriatric Diseases, Chinese PLA General Hospital, Beijing, 100853, China
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution to this work as co-first authors.
    Yue Teng
    Footnotes
    1 Equal contribution to this work as co-first authors.
    Affiliations
    Department of Emergency Medicine, General Hospital of Northern Theatre Command, 83 Wenhua Road, Shenyang 110016, China
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution to this work as co-first authors.
    Jiajun Du
    Footnotes
    1 Equal contribution to this work as co-first authors.
    Affiliations
    Medical Information Center, Chinese PLA General Hospital, Beijing, 100853, China
    Search for articles by this author
  • Xuan Zhou
    Affiliations
    Department of Emergency, Hainan Hospital of Chinese PLA General Hospital, Sanya, 572000, China
    Search for articles by this author
  • Feng Duan
    Correspondence
    Corresponding author.
    Affiliations
    Department of Interventional Radiology, The Fifth Medical Center, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
    Search for articles by this author
  • Cong Feng
    Correspondence
    Corresponding author. Department of Emergency, First Medical Center, General Hospital of People's Liberation Army, Beijing, 100853, China.
    Affiliations
    Department of Emergency, First Medical Center of Chinese PLA General Hospital, Beijing, 100853, China

    State Key Laboratory of Kidney Diseases, National Clinical Research Center of Kidney Diseases, General Hospital of People's Liberation Army, Beijing, 100853, China

    National Clinical Research Center of Geriatric Diseases, Chinese PLA General Hospital, Beijing, 100853, China
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution to this work as co-first authors.

      Abstract

      Background

      Currently, very few preventive or therapeutic strategies are used for mechanical ventilation (MV)-associated severe acute kidney injury (AKI).

      Objectives

      We developed clinical prediction models to detect the onset of severe AKI in the first week of intensive care unit (ICU) stay during the initiation of MV.

      Methods

      A large ICU database Medical Information Mart for Intensive Care IV (MIMIC-IV) was analysed retrospectively. Data were collected from the clinical information recorded at the time of ICU admission and during the initial 12 h of MV. Using univariate and multivariate analyses, the predictors were selected successively. For model development, two machine learning algorithms were compared. The primary goal was to predict the development of AKI stage 2 or 3 (AKI-23) and AKI stage 3 (AKI-3) in the first week of patients’ ICU stay after initial 12 h of MV. The developed models were externally validated using another multicentre ICU database (eICU Collaborative Research Database, eICU) and evaluated in various patient subpopulations.

      Results

      Models were developed using data from the development cohort (MIMIC-IV: 2008–2016; n = 3986); the random forest algorithm outperformed the logistic regression algorithm. In the internal (MIMIC-IV: 2017–2019; n = 1210) and external (eICU; n = 1494) validation cohorts, the incidences of AKI-23 were 154 (12.7%) and 119 (8.0%), respectively, with areas under the receiver operator characteristic curve of 0.78 (95% confidence interval [CI]: 0.74–0.82) and 0.80 (95% CI: 0.76–0.84); the incidences of AKI-3 were 81 (6.7%) and 67 (4.5%), with areas under the receiver operator characteristic curve of 0.81 (95% CI: 0.76–0.87) and 0.80 (95% CI: 0.73–0.86), respectively.

      Conclusions

      Models driven by machine learning and based on routine clinical data may facilitate the early prediction of MV-associated severe AKI. The validated models can be found at: https://apoet.shinyapps.io/mv_aki_2021_v2/.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Australian Critical Care
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Hoste E.A.
        • Bagshaw S.M.
        • Bellomo R.
        • Cely C.M.
        • Colman R.
        • Cruz D.N.
        • et al.
        Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study.
        Intensive Care Med. 2015; 41: 1411-1423
        • Nisula S.
        • Kaukonen K.M.
        • Vaara S.T.
        • Korhonen A.M.
        • Poukkanen M.
        • Karlsson S.
        • et al.
        Incidence, risk factors and 90-day mortality of patients with acute kidney injury in Finnish intensive care units: the FINNAKI study.
        Intensive Care Med. 2013; 39: 420-428
        • Hepokoski M.L.
        • Malhotra A.
        • Singh P.
        • Crotty Alexander L.E.
        Ventilator-Induced kidney injury: are novel biomarkers the key to prevention?.
        Nephron. 2018; 140: 90-93
        • Bellomo R.
        • Kellum J.A.
        • Ronco C.
        Acute kidney injury.
        Lancet (London, England). 2012; 380: 756-766
        • Druml W.
        • Lenz K.
        • Laggner A.N.
        Our paper 20 years later: from acute renal failure to acute kidney injury--the metamorphosis of a syndrome.
        Intensive Care Med. 2015; 41: 1941-1949
        • Lombardi R.
        • Nin N.
        • Lorente J.A.
        • Frutos-Vivar F.
        • Ferguson N.D.
        • Hurtado J.
        • et al.
        An assessment of the Acute Kidney Injury Network creatinine-based criteria in patients submitted to mechanical ventilation.
        Clin J Am Soc Nephrol : CJASN. 2011; 6: 1547-1555
        • Schetz M.
        • Schneider A.
        Focus on acute kidney injury.
        Intensive Care Med. 2017; 43: 1421-1423
        • Pickkers P.
        • Ostermann M.
        • Joannidis M.
        • Zarbock A.
        • Hoste E.
        • Bellomo R.
        • et al.
        The intensive care medicine agenda on acute kidney injury.
        Intensive Care Med. 2017; 43: 1198-1209
        • Hepokoski M.
        • Owens R.L.
        • Malhotra A.
        • Beitler J.R.
        Mechanical ventilation in acute respiratory distress syndrome at ATS 2016: the search for a patient-specific strategy.
        J Thorac Dis. 2016; 8: S550-S552
        • Darmon M.
        • Legrand M.
        • Terzi N.
        Understanding the kidney during acute respiratory failure.
        Intensive Care Med. 2017; 43: 1144-1147
        • Lombardi R.
        • Nin N.
        • Peñuelas O.
        • Ferreiro A.
        • Rios F.
        • Marin M.C.
        • et al.
        Acute kidney injury in mechanically ventilated patients: the risk factor profile depends on the timing of Aki onset.
        Shock. 2017; 48: 411-417
        • Prowle J.R.
        Measurement of AKI biomarkers in the ICU: still striving for appropriate clinical indications.
        Intensive Care Med. 2015; 41: 541-543
        • Bell M.
        • Larsson A.
        • Venge P.
        • Bellomo R.
        • Mårtensson J.
        Assessment of cell-cycle arrest biomarkers to predict early and delayed acute kidney injury.
        Dis Markers. 2015; 2015158658
        • Hodgson L.E.
        • Selby N.
        • Huang T.M.
        • Forni L.G.
        The role of risk prediction models in prevention and management of AKI.
        Semin Nephrol. 2019; 39: 421-430
        • Wilson F.P.
        Machine learning to predict acute kidney injury.
        Am J Kidney Dis Off J Natl Kidney Found. 2020; 75: 965-967
        • Hodgson L.E.
        • Sarnowski A.
        • Roderick P.J.
        • Dimitrov B.D.
        • Venn R.M.
        • Forni L.G.
        Systematic review of prognostic prediction models for acute kidney injury (AKI) in general hospital populations.
        BMJ Open. 2017; 7e016591
        • Tomašev N.
        • Glorot X.
        • Rae J.W.
        • Zielinski M.
        • Askham H.
        • Saraiva A.
        • et al.
        A clinically applicable approach to continuous prediction of future acute kidney injury.
        Nature. 2019; 572: 116-119
        • Park S.
        • Lee H.
        Acute kidney injury prediction models: current concepts and future strategies.
        Curr Opin Nephrol Hypertens. 2019; 28: 552-559
        • Flechet M.
        • Güiza F.
        • Schetz M.
        • Wouters P.
        • Vanhorebeek I.
        • Derese I.
        • et al.
        AKIpredictor, an online prognostic calculator for acute kidney injury in adult critically ill patients: development, validation and comparison to serum neutrophil gelatinase-associated lipocalin.
        Intensive Care Med. 2017; 43: 764-773
        • Zimmerman L.P.
        • Reyfman P.A.
        • Smith A.D.R.
        • Zeng Z.
        • Kho A.
        • Sanchez-Pinto L.N.
        • et al.
        Early prediction of acute kidney injury following ICU admission using a multivariate panel of physiological measurements.
        BMC Med Inf Decis Making. 2019; 19: 16
        • Leedahl D.D.
        • Frazee E.N.
        • Schramm G.E.
        • Dierkhising R.A.
        • Bergstralh E.J.
        • Chawla L.S.
        • et al.
        Derivation of urine output thresholds that identify a very high risk of AKI in patients with septic shock.
        Clin J Am Soc NephrolCJASN. 2014; 9: 1168-1174
        • Zhang Z.
        • Ho K.M.
        • Hong Y.
        Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care.
        Crit Care. 2019; 23: 112
        • Zhang Z.
        • Zhu C.
        • Mo L.
        • Hong Y.
        Effectiveness of sodium bicarbonate infusion on mortality in septic patients with metabolic acidosis.
        Intensive Care Med. 2018; 44: 1888-1895
        • Johnson A.E.
        • Pollard T.J.
        • Shen L.
        • Lehman L.W.
        • Feng M.
        • Ghassemi M.
        • et al.
        MIMIC-III, a freely accessible critical care database.
        Sci Data. 2016; 3160035
        • Johnson A.E.
        • Stone D.J.
        • Celi L.A.
        • Pollard T.J.
        The MIMIC Code Repository: enabling reproducibility in critical care research.
        J Am Med Inf Assoc JAMIA. 2018; 25: 32-39
        • Johnson A.B.L.
        • Pollard T.
        • Horng S.
        • Celi L.A.
        • Mark R.
        MIMIC-IV (ver- sion 0.4).
        2020
        • Serpa Neto A.
        • Deliberato R.O.
        • Johnson A.E.W.
        • Bos L.D.
        • Amorim P.
        • Pereira S.M.
        • et al.
        Mechanical power of ventilation is associated with mortality in critically ill patients: an analysis of patients in two observational cohorts.
        Intensive Care Med. 2018; 44: 1914-1922
        • Thomas M.E.
        • Blaine C.
        • Dawnay A.
        • Devonald M.A.
        • Ftouh S.
        • Laing C.
        • et al.
        The definition of acute kidney injury and its use in practice.
        Kidney Int. 2015; 87: 62-73
        • Malhotra R.
        • Kashani K.B.
        • Macedo E.
        • Kim J.
        • Bouchard J.
        • Wynn S.
        • et al.
        A risk prediction score for acute kidney injury in the intensive care unit.
        Nephrol Dial Transplant Off Publ Eur Dialys Transplant Assoc Eur Renal Assoc. 2017; 32: 814-822
        • Drury D.R.
        • Henry J.P.
        • Goodman J.
        The effects of continuous pressure breathing on kidney function.
        J Clin Invest. 1947; 26: 945-951
        • Husain-Syed F.
        • Slutsky A.S.
        • Ronco C.
        Lung-kidney cross-talk in the critically ill patient.
        Am J Respir Crit Care Med. 2016; 194: 402-414
        • Darmon M.
        • Ostermann M.
        • Joannidis M.
        Predictions are difficult...especially about AKI.
        Intensive Care Med. 2017; 43: 932-934
        • Sutherland S.M.
        • Chawla L.S.
        • Kane-Gill S.L.
        • Hsu R.K.
        • Kramer A.A.
        • Goldstein S.L.
        • et al.
        Utilizing electronic health records to predict acute kidney injury risk and outcomes: workgroup statements from the 15(th) ADQI Consensus Conference.
        Can J Kidney Health Dis. 2016; 3: 11
        • Abul-Husn N.S.
        • Kenny E.E.
        Personalized medicine and the power of electronic health records.
        Cell. 2019; 177: 58-69
        • Ghassemi M.
        • Wu M.
        • Hughes M.C.
        • Szolovits P.
        • Doshi-Velez F.
        Predicting intervention onset in the ICU with switching state space models.
        AMIA Jt Summits Transl Sci Proc. 2017; 2017: 82-91
        • Hodgson L.E.
        • Dimitrov B.D.
        • Roderick P.J.
        • Venn R.
        • Forni L.G.
        Predicting AKI in emergency admissions: an external validation study of the acute kidney injury prediction score (APS).
        BMJ Open. 2017; 7e013511
        • Huen S.C.
        • Parikh C.R.
        Predicting acute kidney injury after cardiac surgery: a systematic review.
        Ann Thorac Surg. 2012; 93: 337-347
        • Komorowski M.
        • Celi L.A.
        • Badawi O.
        • Gordon A.C.
        • Faisal A.A.
        The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care.
        Nat Med. 2018; 24: 1716-1720
        • Koyner J.L.
        • Adhikari R.
        • Edelson D.P.
        • Churpek M.M.
        Development of a multicenter ward-based AKI prediction model.
        Clin J Am Soc Nephrol CJASN. 2016; 11: 1935-1943
        • Mohamadlou H.
        • Lynn-Palevsky A.
        • Barton C.
        • Chettipally U.
        • Shieh L.
        • Calvert J.
        • et al.
        Prediction of acute kidney injury with a machine learning algorithm using electronic health record data.
        Can J Kidney Health Dis. 2018; 52054358118776326
        • Pozzoli S.
        • Simonini M.
        • Manunta P.
        Predicting acute kidney injury: current status and future challenges.
        J Nephrol. 2018; 31: 209-223
        • Silver S.A.
        • Shah P.M.
        • Chertow G.M.
        • Harel S.
        • Wald R.
        • Harel Z.
        Risk prediction models for contrast induced nephropathy: systematic review.
        BMJ. 2015; 351: h4395
        • Smith L.E.
        • Smith D.K.
        • Blume J.D.
        • Siew E.D.
        • Billings F.T.t.
        Latent variable modeling improves AKI risk factor identification and AKI prediction compared to traditional methods.
        BMC Nephrol. 2017; 18: 55
        • Suresh H.
        • Hunt N.
        • Johnson A.
        • Anthony Celi L.
        • Szolovits P.
        • Ghassemi M.
        Clinical intervention prediction and understanding using deep networks.
        2017
        • Wilson T.
        • Quan S.
        • Cheema K.
        • Zarnke K.
        • Quinn R.
        • de Koning L.
        • et al.
        Risk prediction models for acute kidney injury following major noncardiac surgery: systematic review.
        Nephrol Dial Transplant. 2016; 31: 231-240
        • Leite T.T.
        • Gomes C.A.M.
        • Valdivia J.M.C.
        • Libório A.B.
        Respiratory parameters and acute kidney injury in acute respiratory distress syndrome: a causal inference study.
        Ann Transl Med. 2019; 7: 742
        • Darmon M.
        • Schortgen F.
        • Leon R.
        • Moutereau S.
        • Mayaux J.
        • Di Marco F.
        • et al.
        Impact of mild hypoxemia on renal function and renal resistive index during mechanical ventilation.
        Intensive Care Med. 2009; 35: 1031-1038
        • Zz A.
        • Lin C.B.
        • Ping X.
        • Yh A.
        Predictive analytics with ensemble modeling in laparoscopic surgery: a technical note.
        Laparoscop Endoscop Robot Surg. 2022; 5: 25-34
        • Yang X.
        • Chen C.
        • Teng S.
        • Fu X.
        • Zha Y.
        • Liu H.
        • et al.
        Urinary matrix metalloproteinase-7 predicts severe AKI and poor outcomes after cardiac surgery.
        J Am Soc Nephrol JASN (J Am Soc Nephrol). 2017; 28: 3373-3382
        • Lombardi R.
        • Nin N.
        • Penuelas O.
        • Ferreiro A.
        • Rios F.
        • Marin M.C.
        • et al.
        Acute kidney injury in mechanically ventilated patients: the risk factor profile depends on the timing of Aki onset.
        Shock. 2017; 48: 411-417