Confidence intervals illuminate absence of evidence
http://www.100md.com
《英国医生杂志》
EDITOR—We agree with Alderson that authors should recognise that non-significant results are compatible with a range of possible findings.1 Papers in the same issue of the BMJ do not adhere to this good advice.
Koivunen et al concluded that adenoidectomy is not effective and cannot be recommended, yet the 95% confidence interval for further episodes of otitis media is compatible with an 18% absolute risk reduction.2 The clinically important difference sought was a 25% reduction.
Kariminia et al said that hands and knees exercise with pelvic rocking did not reduce the incidence of persistent occiput posterior position at birth3; the 95% confidence interval was from 1.8% reduction to 2.5% increased risk. This trial sought a risk reduction of 2.5%.
Marre et al concluded that "low dose ramipril has no effect on cardiovascular and renal outcomes"4—the 95% confidence interval was from 15% reduction to 11% increased risk. A 20% reduction was considered clinically important.
None of these non-significant trials ruled out some treatment benefit. Others may judge that a smaller benefit would be clinically useful. Even when a clinically useful effect has been ruled out, phrases such as "is not effective," "did not reduce," and "has no effect" are not justified.
Also, confidence intervals reflect only uncertainty owing to random allocation, not that owing to failure to follow the protocol, non-random loss to follow up, and so on. True uncertainty is greater, therefore, than indicated by confidence intervals.
Lastly, we cannot claim priority with the title "Absence of evidence is not evidence of absence": a paper with this title was published in 1983.5
Doug Altman, director
Cancer Research UK Medical Statistics Group, Centre for Statistics in Medicine, Oxford OX3 7LF doug.altman@cancer.org.uk
J Martin Bland, professor of health statistics
Department of Health Sciences, University of York, York YO10 5DD
Competing interests: None declared.
References
Alderson P. Absence of evidence is not evidence of absence. BMJ 2004;328: 476-7. (28 February.)
Koivunen P, Uhari M, Luotonen J, Kristo A, Raski R, Pokka T et al. Adenoidectomy versus chemoprophylaxis and placebo for recurrent acute otitis media in children aged under 2 years: randomised controlled trial. BMJ 2004;328: 487-90. (28 February.)
Kariminia A, Chamberlain ME, Keogh J, Shea A. Randomised controlled trial of effect of hands and knees posturing on incidence of occiput posterior position at birth. BMJ 2004;328: 490-3. (28 February.)
Marre M, Lievre M, Chatellier G, Mann JF, Passa P, Menard J; DIABHYCAR Study Investigators. Effects of low dose ramipril on cardiovascular and renal outcomes in patients with type 2 diabetes and raised excretion of urinary albumin: randomised, double blind, placebo controlled trial (the DIABHYCAR study). BMJ 2004;328: 495-9. (28 February.)
Hartung J, Cottrell JE, Giffin JP. Absence of evidence is not evidence of absence. Anesthesiology 1983;58: 298-300.
Koivunen et al concluded that adenoidectomy is not effective and cannot be recommended, yet the 95% confidence interval for further episodes of otitis media is compatible with an 18% absolute risk reduction.2 The clinically important difference sought was a 25% reduction.
Kariminia et al said that hands and knees exercise with pelvic rocking did not reduce the incidence of persistent occiput posterior position at birth3; the 95% confidence interval was from 1.8% reduction to 2.5% increased risk. This trial sought a risk reduction of 2.5%.
Marre et al concluded that "low dose ramipril has no effect on cardiovascular and renal outcomes"4—the 95% confidence interval was from 15% reduction to 11% increased risk. A 20% reduction was considered clinically important.
None of these non-significant trials ruled out some treatment benefit. Others may judge that a smaller benefit would be clinically useful. Even when a clinically useful effect has been ruled out, phrases such as "is not effective," "did not reduce," and "has no effect" are not justified.
Also, confidence intervals reflect only uncertainty owing to random allocation, not that owing to failure to follow the protocol, non-random loss to follow up, and so on. True uncertainty is greater, therefore, than indicated by confidence intervals.
Lastly, we cannot claim priority with the title "Absence of evidence is not evidence of absence": a paper with this title was published in 1983.5
Doug Altman, director
Cancer Research UK Medical Statistics Group, Centre for Statistics in Medicine, Oxford OX3 7LF doug.altman@cancer.org.uk
J Martin Bland, professor of health statistics
Department of Health Sciences, University of York, York YO10 5DD
Competing interests: None declared.
References
Alderson P. Absence of evidence is not evidence of absence. BMJ 2004;328: 476-7. (28 February.)
Koivunen P, Uhari M, Luotonen J, Kristo A, Raski R, Pokka T et al. Adenoidectomy versus chemoprophylaxis and placebo for recurrent acute otitis media in children aged under 2 years: randomised controlled trial. BMJ 2004;328: 487-90. (28 February.)
Kariminia A, Chamberlain ME, Keogh J, Shea A. Randomised controlled trial of effect of hands and knees posturing on incidence of occiput posterior position at birth. BMJ 2004;328: 490-3. (28 February.)
Marre M, Lievre M, Chatellier G, Mann JF, Passa P, Menard J; DIABHYCAR Study Investigators. Effects of low dose ramipril on cardiovascular and renal outcomes in patients with type 2 diabetes and raised excretion of urinary albumin: randomised, double blind, placebo controlled trial (the DIABHYCAR study). BMJ 2004;328: 495-9. (28 February.)
Hartung J, Cottrell JE, Giffin JP. Absence of evidence is not evidence of absence. Anesthesiology 1983;58: 298-300.