Research paper|Articles in Press

Interrater reliability in assigning a lung ultrasound score

Published:November 17, 2022DOI:


      Background and purpose

      Lung ultrasound (LUS) for physiotherapists is an emerging bedside tool. The LUS score of aeration presents as a possible means of assessing and monitoring lung aeration associated with respiratory physiotherapy treatments. There are no studies to date that have assessed the interrater reliability (IRR) of physiotherapists assigning the LUS score of aeration. This study assessed the IRR of assigning the LUS score among adult, mechanically ventilated patients in an intensive care unit with a clinical suspicion of acute lobar atelectasis.


      A convenience sample of patients had an LUS performed by a physiotherapist, and images were independently reviewed by two physiotherapists. Each lung zone was assigned an LUS score between 0 and 3 (with 0 being normal aeration and 3 being complete consolidation, presence of effusion, or pneumothorax). IRR was assessed using the kappa statistic.


      A total of 1032 LUS images were obtained. Assigning of the LUS across all lung zones demonstrated substantial agreement with kappa 0.685 (95% confidence interval: 0.650, 0.720). Right (0.702 [0.653, 0.751]) and left (0.670 [0.619, 0.721]) lung zones also demonstrated substantial agreement.


      We found substantial IRR between physiotherapists in assigning the LUS score in a mechanically ventilated adult population in the intensive care unit.

      Australian New Zealand clinical trials registration number



      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Australian Critical Care
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Le Neindre A.
        • Mongodi S.
        • Philippart F.
        • Bouhemad B.
        Thoracic ultrasound: potential new tool for physiotherapists in respiratory management. A narrative review.
        J Crit Care. 2015; 31: 101-109
        • Leech M.
        • Bissett B.
        • Kot M.
        • Ntoumenopoulos G.
        Lung ultrasound for critical care physiotherapists: a narrative review: LUS for critical care physiotherapists.
        Physiother Res Int. 2015; 20: 69-76
        • Soummer A.
        • Perbet S.
        • Brisson H.
        • Arbelot C.
        • Constantin J.M.
        • Lu Q.
        • et al.
        Ultrasound assessment of lung aeration loss during a successful weaning trial predicts postextubation distress.
        Crit Care Med. 2012; 40: 2064-2072
        • Bouhemad B.
        • Brisson H.
        • Le-Guen M.
        • Arbelot C.
        • Lu Q.
        • Rouby J.-J.
        Bedside ultrasound assessment of positive end-expiratory pressure-induced lung recruitment.
        Am J Respir Crit Care Med. 2011; 183: 341-347
        • Bouhemad B.
        • Liu Z.-H.
        • Arbelot C.
        • Zhang M.
        • Ferarri F.
        • Le-Guen M.
        • et al.
        Ultrasound assessment of antibiotic-induced pulmonary reaeration in ventilator-associated pneumonia.
        Crit Care Med. 2010; 38: 84-92
        • Bouhemad B.
        • Mongodi S.
        • Via G.
        • Rouquette I.
        Ultrasound for "lung monitoring" of ventilated patients.
        Anesthesiology. 2015; 122: 437
        • Via G.
        • Storti E.
        • Gulati G.
        • Neri L.
        • Mojoli F.
        • Braschi A.
        Lung ultrasound in the ICU: from diagnostic instrument to respiratory monitoring tool.
        Minerva anestesiologica U6. 2012; 78: 1282
        • Marques A.
        • Bruton A.
        • Barney A.
        Clinically useful outcome measures for physiotherapy airway clearance techniques: a review.
        Phys Ther Rev. 2006; 11: 299-307
        • Stiller K.
        Physiotherapy in intensive care: an updated systematic review.
        Chest. 2013; 144: 825
        • Bouhemad B.
        • Zhang M.
        • Lu Q.
        • Rouby J.-J.
        Clinical review: bedside lung ultrasound in critical care practice.
        Crit Care. 2007; 11 (205-205)
        • Lichtenstein D.
        Ultrasound examination of the lungs in the intensive care unit.
        Pediatr Crit Care Med J Soc Crit Care Med World Federat Pediatr Intens Crit Care Soc. 2009; 10: 693-698
        • Hew M.
        • Heinze S.
        Chest ultrasound in practice: a review of utility in the clinical setting.
        Intern Med J. 2012; 42: 856-865
        • Lichtenstein D.
        • Goldstein I.
        • Mourgeon E.
        • Cluzel P.
        • Grenier P.
        • Rouby J.-J.
        Comparative diagnostic performances of auscultation, chest radiography, and lung ultrasonography in acute respiratory distress syndrome.
        Anesthesiology. 2004; 100: 9-15
        • Inglis A.J.
        • Nalos M.
        • Sue K.-H.
        • Hruby J.
        • Campbell D.M.
        • Braham R.M.
        • et al.
        Bedside lung ultrasound, mobile radiography and physical examination: a comparative analysis of diagnostic tools in the critically ill.
        Crit Care Resuscitation. 2016; 18: 116-124
        • Hayward S.
        • Innes S.
        • Smith M.
        Challenges and opportunities in point-of-care ultrasound: a qualitative exploration of respiratory physiotherapists’ experiences of lung ultrasound training and its adoption in critical care.
        Ultrasound. 2022; 30 (1742271X211034199): 126-133
        • Hansell L.
        • Milross M.
        • Delaney A.
        • Tian D.H.
        • Rajamani A.
        • Ntoumenopoulos G.
        Barriers and facilitators to achieving competence in lung ultrasound: a survey of physiotherapists following a lung ultrasound training course.
        Aust Crit Care. 2022; Jun 7: S1036-7314
        • Kumar A.
        • Weng Y.
        • Graglia S.
        • Chung S.
        • Duanmu Y.
        • Lalani F.
        • et al.
        Interobserver agreement of lung ultrasound findings of COVID-19.
        J Ultrasound Med. 2021; 40: 2369-2376
        • Gravel C.A.
        • Monuteaux M.C.
        • Levy J.A.
        • Miller A.F.
        • Vieira R.L.
        • Bachur R.G.
        Interrater reliability of pediatric point-of-care lung ultrasound findings.
        Am J Emerg Med. 2020; 38: 1-6
        • Brooks D.
        • Thomas J.
        Interrater reliability of auscultation of breath sounds among physical therapists.
        Phys Ther. 1995; 75: 1082-1088
        • Allingame S.
        • Williams T.
        • Jenkins S.
        • Tucker B.
        Accuracy and reliability of physiotherapists in the interpretation of tape-recorded lung sounds.
        Aust J Physiother. 1995; 41: 179-184
        • Xavier G.N.
        • Duarte A.C.M.
        • Melo-Silva C.A.
        • dos Santos C.E.V.G.
        • Amado V.M.
        Accuracy of pulmonary auscultation to detect abnormal respiratory mechanics: a cross-sectional diagnostic study.
        Med Hypotheses. 2014; 83: 733-734
        • Delrue L.
        • Gosselin R.
        • Ilsen B.
        • Van Landeghem A.
        • de Mey J.
        • Duyck P.
        Difficulties in the interpretation of chest radiography.
        Springer, 2011
        • Hansell L.
        • Milross M.
        • Delaney A.
        • Tian D.H.
        • Ntoumenopoulos G.
        Lung ultrasound has greater accuracy than conventional respiratory assessment tools for the diagnosis of pleural effusion, lung consolidation and collapse: a systematic review.
        J Physiother. 2021; 67: 41-48
        • Winkler M.H.
        • Touw H.R.
        • van de Ven P.M.
        • Twisk J.
        • Tuinman P.R.
        Diagnostic accuracy of chest radiograph, and when concomitantly studied lung ultrasound, in critically ill patients with respiratory symptoms: a systematic review and meta-analysis.
        Crit Care Med. 2018; 46: e707-e714
        • Xia Y.
        • Ying Y.
        • Wang S.
        • Li W.
        • Shen H.
        Effectiveness of lung ultrasonography for diagnosis of pneumonia in adults: a systematic review and meta-analysis.
        J Thorac Dis. 2016; 8: 2822-2831
        • Chavez M.A.
        • Shams N.
        • Ellington L.E.
        • Naithani N.
        • Gilman R.H.
        • Steinhoff M.C.
        • et al.
        Lung ultrasound for the diagnosis of pneumonia in adults: a systematic review and meta-analysis.
        Respir Res. 2014; 15: 50
        • Long L.
        • Zhao H.-T.
        • Zhang Z.-Y.
        • Wang G.-Y.
        • Zhao H.-L.
        Lung ultrasound for the diagnosis of pneumonia in adults: a meta-analysis.
        Medicine. 2017; 96
        • Volpicelli G.
        • Elbarbary M.
        • Blaivas M.
        • Lichtenstein D.A.
        • Mathis G.
        • Kirkpatrick A.W.
        • et al.
        International evidence-based recommendations for point-of-care lung ultrasound.
        Intensive Care Med. 2012; 38: 577-591
        • Miller A.
        Practical approach to lung ultrasound.
        Br J Anaesth. 2016; 16: 39-45
        • Mojoli F.
        • Bouhemad B.
        • Mongodi S.
        • Lichtenstein D.
        Lung ultrasound for critically ill patients.
        Am J Respir Crit Care Med. 2019; 199: 701-714
        • McHugh M.L.
        Interrater reliability: the kappa statistic.
        Biochem Med (Zagreb). 2012; 22: 276-282
        • McKiernan S.
        • Chiarelli P.
        • Warren-Forward H.
        Diagnostic ultrasound use in physiotherapy, emergency medicine, and anaesthesiology.
        Radiography. 2010; 16: 154-159
        • Pietersen P.I.
        • Madsen K.R.
        • Graumann O.
        • Konge L.
        • Nielsen B.U.
        • Laursen C.B.
        Lung ultrasound training: a systematic review of published literature in clinical lung ultrasound training.
        Crit Ultrasound J. 2018; 10: 23
        • Tulleken A.M.
        • Gelissen H.
        • Lust E.
        • Smits T.
        • van Galen T.
        • Girbes A.R.J.
        • et al.
        UltraNurse: teaching point-of-care ultrasound to intensive care nurses.
        Intensive Care Med. 2019; 45: 727-729
        • Hansell L.
        • Milross M.
        • Delaney A.
        • Tian D.H.
        • Rajamani A.
        • Ntoumenopoulos G.
        Barriers and facilitators to achieving competence in lung ultrasound: a survey of physiotherapists following a lung ultrasound training course.
        Aust Crit Care Off J Confederat Aust Crit Care Nurses. 2022; June 7: S1036-7314
        • Ntoumenopoulos G.
        • Parry S.M.
        • Neindre A.L.
        Impact of an intensive education programme of diagnostic lung and lower limb ultrasound on physiotherapist knowledge: a pilot study.
        Australas J Ultrasound Med. 2018; 21: 104-114
        • Le Neindre A.
        • Hansell L.
        • Wormser J.
        • Gomes Lopes A.
        • Diaz Lopez C.
        • Romanet C.
        • et al.
        Thoracic ultrasound influences physiotherapist’s clinical decision-making in respiratory management of critical care patients: a multicentre cohort study.
        Thorax. 2022; Mar 23 (thoraxjnl-2021-218217, in press)
        • Chiumello D.
        • Mongodi S.
        • Algieri I.
        • Vergani G.L.
        • Orlando A.
        • Via G.
        • et al.
        Assessment of lung aeration and recruitment by CT scan and ultrasound in acute respiratory distress syndrome patients.
        Crit Care Med. 2018; 46: 1761-1768
        • Stiller K.
        • Jenkins S.
        • Grant R.
        • Geake T.
        • Taylor J.
        • Hall B.
        Acute lobar atelectasis: a comparison of five physiotherapy regimens.
        Physiother Theory Pract. 1996; 12: 197-209