Advertisement

Statistical and clinical significance, and how to use confidence intervals to help interpret both

Published:March 29, 2010DOI:https://doi.org/10.1016/j.aucc.2010.03.001

      Summary

      Statistical significance is a statement about the likelihood of findings being due to chance. Classical significance testing, with its reliance on p values, can only provide a dichotomous result – statistically significant, or not. Limiting interpretation of research results to p values means that researchers may either overestimate or underestimate the meaning of their results. Very often the aim of clinical research is to trial an intervention with the intention that results based on a sample will generalise to the wider population. The p value on its own provides no information about the overall importance or meaning of the results to clinical practice, nor do they provide information as to what might happen in the future, or in the general population. Clinical significance is a decision based on the practical value or relevance of a particular treatment, and this may or may not involve statistical significance as an initial criterion. Confidence intervals are one way for researchers to help decide if a particular statistical result (whether significant or not) may be of relevance in practice.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Australian Critical Care
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Cohen J.
        The earth is round (p < .05).
        American Psychologist. 1994; 49: 997-1003
        • Loftus G.R.
        Psychology will be a much better science when we change the way we analyse data.
        Psychological Science. 1996; 7: 161-171
        • Hubbard R.
        • Lindsay R.M.
        Why p values are not a useful measure of evidence in statistical significance testing.
        Theory Psychology. 2008; 18: 69-88
        • Hubbard R.
        • Ryan P.A.
        The historical growth of statistical significance testing in psychology—and its future prospects.
        Educational and Psychological Measurement. 2000; 60: 661-681
        • Hubbard R.
        Alphabet soup: blurring the distinctions between p's and α’s in psychological research.
        Theory & Psychology. 2004; 14: 295-327
        • Christensen R.
        Testing Fisher, Neyman, Pearson, and Bayes.
        The American Statistician. 2005; 59: 121-126
      1. Harlow L.L. Mulaik S.A. Steiger J.H. What if there were no significance tests? Erlbaum, Mahwah, NJ1997
        • Schmidt F.L.
        Statistical significance testing and cumulative knowledge in psychology: implications for the training of researchers.
        Psychological Methods. 1996; 1: 115-129
        • Pereira S.
        • Leslie G.
        Hypothesis testing.
        Australian Critical Care. 2009; 22: 187-191
        • Jacobsen N.S.
        • Follette W.C.
        • Revenstorf D.
        Psychotherapy outcome research: methods for reporting variability and evaluating clinical significance.
        Behavior Therapy. 1984; 15: 336-352
        • LeFort S.
        The statistical versus clinical significance debate.
        Journal of Nursing Scholarship. 1993; 25: 58
        • Greenstein G.
        Clinical versus statistical significance as they relate to the efficacy of periodontal therapy.
        Journal of the American Dental Association. 2003; 134: 583-591
        • Todd K.H.
        • Funk K.G.
        • Funk J.P.
        • Bonacci R.
        Clinical significance of reported pain severity.
        Annals of Emergency Medicine. 1996; 27: 485-489
        • Sackett D.L.
        Superiority, equivalence and noninferiority trials.
        in: Haynes R.B. Sackett D.L. Guyatt G.H. Tugwell P. The principles behind the tactics of performing therapeutic trials. Clinical epidemiology: how to do clinical practice research. 3rd edition. Lippincott Williams & Wilkins, Philadelphia, PA2006: 196-206
        • Redelmeier D.A.
        • Guyat R.A.
        • Goldstein D.S.
        Assessing the minimal important difference in symptoms: a comparison of two techniques.
        Journal of Clinical Epidemiology. 1996; 49: 1215-1219
        • Wright J.G.
        The minimal important difference: who's to say what is important?.
        Journal of Clinical Epidemiology. 1996; 49: 1221-1222
      2. Busse JW, Guyatt, GH. Optimizing the use of patient data to improve outcomes for patients: narcotics for chronic noncancer pain. Expert Review of Pharmacoeconomics Outcomes Research 2009;9(2):171–9. Available at http://www.medscape.com/viewarticle/705606 [Accessed 4 February 2010].

        • Farrar J.T.
        • Portenoy R.K.
        • Berlin J.A.
        • Kinman J.L.
        • Strom B.L.
        Defining the clinically important difference in pain outcome measures.
        Pain. 2000; 88: 287-294
        • Yost K.J.
        • Eton D.T.
        Combining distribution and anchor-based approaches to determine minimally important differences: the FACIT experience.
        Evaluation and the Health Professions. 2005; 28: 172-191
        • Norman G.R.
        • Sloan J.A.
        • Wyrwich K.W.
        Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation.
        Medical Care. 2005; 41: 582-592
        • Copay A.G.
        • Subach B.R.
        • Glassman S.D.
        • Polly D.W.
        • Schuler T.C.
        Understanding the minimum clinically important difference: a review of concepts and methods.
        The Spine Journal. 2007; 7: 541-546
        • Ostelo R.
        • Deyo R.A.
        • Stratford P.
        • Waddell G.
        • Croft P.
        • Von Korff M.
        • Bouter L.M.
        • de Vet H.C.
        Interpreting change scores for pain and functional status in low back pain.
        Spine. 2008; 33: 90-94