Inappropriate use of randomised trials to evaluate complex phenomena: case study of vaginal breech delivery
http://www.100md.com
《英国医生杂志》
1 Department of Obstetrics and Gynaecology, University of British Columbia, BC Women's Hospital, Vancouver, BC, V6H 3V5 Canada
Correspondence to: A Kotaska kotaska@bulkley.net
As randomised trials continue to ascend in the evolution of evidence based medicine, we must recognise and respect their limitations when examining complex phenomena in heterogeneous populations
Introduction
Many of the term breech trial's 121 centres were in North America, where 13% of breech presentations at term were delivered vaginally.5 The study achieved a successful vaginal delivery rate of 57% by asking those centres with vaginal birth rates under 40% in the labour group to increase the rate or withdraw from participation.6 Individual centres rates of vaginal breech delivery at baseline were not reported, but many would have tripled their vaginal delivery rate overnight.
Vaginal breech delivery is a complex procedure
Credit: MOTHER AND BABY PICTURE LIBRARY
The vaginal delivery of a breech baby involves risk. Cord prolapse and trapped fetal parts are unpredictable complications. Every practitioner knows this; and the literature, the courts, and the low baseline rate of such deliveries in North America highlight caution. Maternity units with interest and skill in delivering breech babies vaginally have achieved higher rates: 24% in the United States, 36% in Sweden, 38% in Israel, 38% in Switzerland, 39% in France, and 53% in Norway.7-12 These retrospective studies carefully selected women for a trial of labour using various safety criteria and showed lower mortality and morbidity associated with vaginal breech delivery than in the term breech trial. Few obstetrical units other than L?vset's have published vaginal delivery rates as high as the term breech trial.12
Statistical power required a high vaginal delivery rate to enhance the trial's ability to detect small differences in outcome. With this aim, the researchers encouraged practitioners to increase their vaginal breech delivery rate beyond their previous comfort level. Despite being difficult to quantify, comfort level (or "practitioner comfort level") plays a pivotal part in the safety of complex procedures. Protected from medicolegal liability by the equipoise of a randomised trial, some practitioners must have pushed their comfort levels with vaginal breech delivery. This constitutes a significant bias: one of licence. The trial protocol's liberal labour guidelines allowed 0.5 cm dilation/h and 3.5 hours for the second stage. This is considerable licence, and few obstetricians from centres with proved safety in vaginal breech delivery would find them acceptable.
A discriminating procedure
Randomisation improves the internal validity of trials by homogenising study and control populations thereby avoiding bias from differences between the two. Clinically important factors that are variable within populations are, however, homogenised as well. Their importance to the outcome is lost, and the trial loses external validity for individuals within subsets of the population. Results represent the mean outcome for all participants and are not applicable to subgroups at lower risk. Although subgroup analysis, found in the term breech trial, can potentially identify subpopulations in which a procedure may be safer, it is statistically weak. When phenomena are complex, many patient and practitioner characteristics influence outcome, rendering individual subgroups small and meaningful analysis of them difficult.
A major limit of evidence based medicine is the difficulty in applying the results of randomised trials to individual patients. For example, most would agree that a multiparous woman in advanced labour at 38 weeks with a 3000 g fetus in a frank breech presentation with flexed head and no nuchal cord represents a low risk subgroup of all breech presentations. By studying a heterogeneous group of women, the term breech trial lacks the external validity needed to guide us with the management of such women. Yet experienced obstetricians will now press for an emergency caesarean, not trusting their clinical judgment to discern low risk situations, because all women with a breech presentation have been assigned a similar risk status by a randomised controlled trial.
Multicentre trials can also lead to homogenisation of the intervention. Previous randomised trials of breech presentations were too small to detect clinically meaningful differences in outcome, so a large multicentre trial was required to improve statistical power. Yet despite the interest, altruism, and self referential experience of the practitioners, the involvement of 121 centres resulted in an average level of care. Encouraging practitioners to exceed their comfort level with vaginal breech delivery lowered that standard.
If generic levels of care had always been accepted as the ideal, none of the surgical subspecialties would have arisen. The standard of care shown by the term breech trial is not the best we can offer. Although breech deliveries commonly occur under average conditions, it does not mean that committed centres are unable to offer better than average care. Collectively we have been improving our "mean" level of care for years, and the perinatal risk associated with breech delivery has continued to drop despite stabilisation of the caesarean section rate.19
Simplified risk reduction
Randomised trials often utilise short term end points because they are easier to measure than longer term outcomes. It is also easier to show a statistical difference in a combined end point rather than a single end point. Yet combined end points can be misleading.24 In the term breech trial, the end point included perinatal mortality and various short term morbidities, including hypotonia, transient brachial plexus injury, and isolated low arterial cord pH or base excess, whose lasting significance is unclear. In countries with low perinatal mortality, this combined end point occurred in 5.7% of planned vaginal deliveries and 0.4% of women undergoing elective caesareans. Mortality was not significantly different (3 of 511 or 0.6%) in the planned vaginal delivery group compared with zero in the planned caesarean group. One of these deaths, included in the intention to treat analysis, occurred before the onset of labour in a cephalic twin weighing 1150 g, highlighting concerns about the adequacy of case selection and raising questions about the appropriateness of intention to treat analysis at all cost. Regardless, the impact of the trial's results was due primarily to an excess of short term morbidity in the planned vaginal delivery group.
Summary points
When applied to complex phenomena, randomised trials have important limitations
Vaginal breech delivery is a complex procedure that is poorly amenable to the methods of large multicentre randomised trials
One randomised controlled trial has dictated a new standard of care for vaginal breech deliveries worldwide
The use of a short term combined end point overstated any true risk of planned vaginal delivery to longer term neurodevelopmental outcome
Long term outcomes in breech babies are hard to assess epidemiologically, but were retrospectively shown to be equivalent in 1645 children, irrespective of the planned mode of delivery.25 Researchers from the term breech trial have published details on death or abnormal neurodevelopment over two years in a subgroup of 923 children from the term breech trial.26 They found a similar incidence: 2.8% in the planned vaginal delivery group and 3.1% in the elective caesarean section group. The use of a short term combined end point seems to have been misleading.
The limits of evidence based medicine
Hannah ME, Hannah WJ, Hewson SA, Hodnett ED, Saigal S, Willan AR. Planned cesarean section versus planned vaginal birth for breech presentation at term: a randomised multicentre trial. Term Breech Trial Collaborative Group. Lancet 2000;356: 1375-83.
Hauth JC, Cunningham FG. Vaginal breech delivery is still justified. Obstet Gynecol 2002;99: 1115-6.
Keirse MJ. Evidence-based childbirth only for breech babies? Birth 2002;29(1): 55-9.
Halmesmaki E. Vaginal term breech delivery—a time for reappraisal? Acta Obstet Gynecol Scand 2001;80: 187-90.
Lee KS, Khoshnood B, Sriram S, Hsieh HL, Singh J, Mittendorf R. Relationship of cesarean delivery to lower birth weight-specific neonatal mortality in singleton breech infants in the United States. Obstet Gynecol 1998;92: 769-74.
University of Toronto Maternal, Infant and Reproductive Health Research Unit. Term breech trial study protocol, 1996: 8.
Sanchez-Ramos L, Wells TL, Adair CD, Arcelin G, Kaunitz AM, Wells DS. Route of breech delivery and maternal and neonatal outcomes. Int J Gynecol Obstet 2001;73: 7-14.
Lindqvist A, Lindeberg SN, Hanson U. Perinatal mortality and route of delivery in term breech presentations. Br J Obstet Gynaecol 1997;104: 1288-91.
Schiff E, Friedman SA, Mashiach S, Hart O, Barkai G, Sibai BM. Maternal and neonatal outcome of 846 term singleton breech deliveries: seven-year experience at a single center. Am J Obstet Gynecol 1996;175: 18-23.
Irion O, Almagbaly PH, Morabia A. Planned vaginal delivery versus elective cesarean section: a study of singleton term breech presentations. Br J Obstet Gynaecol 1998;105: 710-7.
Kayem G, Goffinet F, Clement D, Hessabi M, Cabrol D. Breech presentation at term: morbidity and mortality according to the type of delivery at Port Royal Maternity hospital from 1993 through 1999. Eur J Obstet Gynecol Reprod Biol 2002;102: 137-42.
Albrechtsen S, Rasmussen S, Reigstad H, Markestad T, Irgens LM, Dalaker K. Evaluation of a protocol for selecting fetuses in breech presentation for vaginal delivery or cesarean section. Am J Obstet Gynecol 1997;177: 586-92.
Tu JV, Hannan EL, Anderson GM, Iron K, Wu K, Vranizan K, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med 1998;339: 1441-7.
Barnett HJ, Taylor DW, Eliasziw M, Fox AJ, Ferguson GG, Haynes RB, et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med 1998;339: 1415-25.
Mayer AR, Chambers SK, Graves E, Holm C, Tseng PC, Nelson BE, et al. Ovarian cancer staging: does it require a gynecologic oncologist. Gynecol Oncol 1992;47: 223-7.
Tingulstad S, Skjeldestad F, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol 2003;102: 499-505.
Varma R, Tahseen S, Lokugamage A, Kunde D. Vaginal route as the norm when planning hysterectomy for benign conditions: change in practice. Obstet Gynecol 2001;97: 613-6.
Maresh M, Metcalfe M, McPherson K, Overton C, Hall V, Hargreaves J, et al. The VALUE national hysterectomy study: description of the patients and their surgery. Br J Obstet Gynaecol 2002;109: 302-12.
Albrechtsen S, Rasmussen S, Irgens LM. Secular trends in peri- and neonatal mortality in breech presentation; Norway 1967-1994. Acta Obstet Gynecol Scand 2000;79: 508-12.
Bracht E. Zur Behandlung der Steisslage. Zentralblatt Gynaecol 1938;31: 1735-6.
Menticoglou SM. Symphysiotomy for the trapped aftercoming parts of the breech: a review of the literature and a plea for its use. Aust NZ J Obstet Gynaecol 1990;30: 1-9.
Giuliani A, Schoell W, Basver A, Tamussino K. Mode of delivery and outcome of 699 term singleton breech deliveries at a single center. Am J Obstet Gynecol 2002;187: 1694-8.
Alarab M, Regan C, O'Connell MP, Keane DP, O'Herlihy C, Foley ME. Singleton vaginal breech delivery at term: still a safe option. Obstet Gynecol 2004;103: 407-12.
Julian D. What is right and what is wrong about evidence-based medicine? J Cardiovasc Electrophysiol 2003:14(Suppl); S2-5.
Danielian P, Wang J, Hall M. Long term outcome by method of delivery of fetuses in breech presentation at term: population based follow up. BMJ 1996;312: 1451-3.
Whyte H, Hannah M, Saigal S. Outcomes of children at 2 years of age in the term breech trial. Am J Obstet Gynecol 2003;Suppl 189(6).
Tonelli MR. The philosophical limits of evidence-based medicine. Acad Med 1998;73: 1234-40.(Andrew Kotaska, senior re)
Correspondence to: A Kotaska kotaska@bulkley.net
As randomised trials continue to ascend in the evolution of evidence based medicine, we must recognise and respect their limitations when examining complex phenomena in heterogeneous populations
Introduction
Many of the term breech trial's 121 centres were in North America, where 13% of breech presentations at term were delivered vaginally.5 The study achieved a successful vaginal delivery rate of 57% by asking those centres with vaginal birth rates under 40% in the labour group to increase the rate or withdraw from participation.6 Individual centres rates of vaginal breech delivery at baseline were not reported, but many would have tripled their vaginal delivery rate overnight.
Vaginal breech delivery is a complex procedure
Credit: MOTHER AND BABY PICTURE LIBRARY
The vaginal delivery of a breech baby involves risk. Cord prolapse and trapped fetal parts are unpredictable complications. Every practitioner knows this; and the literature, the courts, and the low baseline rate of such deliveries in North America highlight caution. Maternity units with interest and skill in delivering breech babies vaginally have achieved higher rates: 24% in the United States, 36% in Sweden, 38% in Israel, 38% in Switzerland, 39% in France, and 53% in Norway.7-12 These retrospective studies carefully selected women for a trial of labour using various safety criteria and showed lower mortality and morbidity associated with vaginal breech delivery than in the term breech trial. Few obstetrical units other than L?vset's have published vaginal delivery rates as high as the term breech trial.12
Statistical power required a high vaginal delivery rate to enhance the trial's ability to detect small differences in outcome. With this aim, the researchers encouraged practitioners to increase their vaginal breech delivery rate beyond their previous comfort level. Despite being difficult to quantify, comfort level (or "practitioner comfort level") plays a pivotal part in the safety of complex procedures. Protected from medicolegal liability by the equipoise of a randomised trial, some practitioners must have pushed their comfort levels with vaginal breech delivery. This constitutes a significant bias: one of licence. The trial protocol's liberal labour guidelines allowed 0.5 cm dilation/h and 3.5 hours for the second stage. This is considerable licence, and few obstetricians from centres with proved safety in vaginal breech delivery would find them acceptable.
A discriminating procedure
Randomisation improves the internal validity of trials by homogenising study and control populations thereby avoiding bias from differences between the two. Clinically important factors that are variable within populations are, however, homogenised as well. Their importance to the outcome is lost, and the trial loses external validity for individuals within subsets of the population. Results represent the mean outcome for all participants and are not applicable to subgroups at lower risk. Although subgroup analysis, found in the term breech trial, can potentially identify subpopulations in which a procedure may be safer, it is statistically weak. When phenomena are complex, many patient and practitioner characteristics influence outcome, rendering individual subgroups small and meaningful analysis of them difficult.
A major limit of evidence based medicine is the difficulty in applying the results of randomised trials to individual patients. For example, most would agree that a multiparous woman in advanced labour at 38 weeks with a 3000 g fetus in a frank breech presentation with flexed head and no nuchal cord represents a low risk subgroup of all breech presentations. By studying a heterogeneous group of women, the term breech trial lacks the external validity needed to guide us with the management of such women. Yet experienced obstetricians will now press for an emergency caesarean, not trusting their clinical judgment to discern low risk situations, because all women with a breech presentation have been assigned a similar risk status by a randomised controlled trial.
Multicentre trials can also lead to homogenisation of the intervention. Previous randomised trials of breech presentations were too small to detect clinically meaningful differences in outcome, so a large multicentre trial was required to improve statistical power. Yet despite the interest, altruism, and self referential experience of the practitioners, the involvement of 121 centres resulted in an average level of care. Encouraging practitioners to exceed their comfort level with vaginal breech delivery lowered that standard.
If generic levels of care had always been accepted as the ideal, none of the surgical subspecialties would have arisen. The standard of care shown by the term breech trial is not the best we can offer. Although breech deliveries commonly occur under average conditions, it does not mean that committed centres are unable to offer better than average care. Collectively we have been improving our "mean" level of care for years, and the perinatal risk associated with breech delivery has continued to drop despite stabilisation of the caesarean section rate.19
Simplified risk reduction
Randomised trials often utilise short term end points because they are easier to measure than longer term outcomes. It is also easier to show a statistical difference in a combined end point rather than a single end point. Yet combined end points can be misleading.24 In the term breech trial, the end point included perinatal mortality and various short term morbidities, including hypotonia, transient brachial plexus injury, and isolated low arterial cord pH or base excess, whose lasting significance is unclear. In countries with low perinatal mortality, this combined end point occurred in 5.7% of planned vaginal deliveries and 0.4% of women undergoing elective caesareans. Mortality was not significantly different (3 of 511 or 0.6%) in the planned vaginal delivery group compared with zero in the planned caesarean group. One of these deaths, included in the intention to treat analysis, occurred before the onset of labour in a cephalic twin weighing 1150 g, highlighting concerns about the adequacy of case selection and raising questions about the appropriateness of intention to treat analysis at all cost. Regardless, the impact of the trial's results was due primarily to an excess of short term morbidity in the planned vaginal delivery group.
Summary points
When applied to complex phenomena, randomised trials have important limitations
Vaginal breech delivery is a complex procedure that is poorly amenable to the methods of large multicentre randomised trials
One randomised controlled trial has dictated a new standard of care for vaginal breech deliveries worldwide
The use of a short term combined end point overstated any true risk of planned vaginal delivery to longer term neurodevelopmental outcome
Long term outcomes in breech babies are hard to assess epidemiologically, but were retrospectively shown to be equivalent in 1645 children, irrespective of the planned mode of delivery.25 Researchers from the term breech trial have published details on death or abnormal neurodevelopment over two years in a subgroup of 923 children from the term breech trial.26 They found a similar incidence: 2.8% in the planned vaginal delivery group and 3.1% in the elective caesarean section group. The use of a short term combined end point seems to have been misleading.
The limits of evidence based medicine
Hannah ME, Hannah WJ, Hewson SA, Hodnett ED, Saigal S, Willan AR. Planned cesarean section versus planned vaginal birth for breech presentation at term: a randomised multicentre trial. Term Breech Trial Collaborative Group. Lancet 2000;356: 1375-83.
Hauth JC, Cunningham FG. Vaginal breech delivery is still justified. Obstet Gynecol 2002;99: 1115-6.
Keirse MJ. Evidence-based childbirth only for breech babies? Birth 2002;29(1): 55-9.
Halmesmaki E. Vaginal term breech delivery—a time for reappraisal? Acta Obstet Gynecol Scand 2001;80: 187-90.
Lee KS, Khoshnood B, Sriram S, Hsieh HL, Singh J, Mittendorf R. Relationship of cesarean delivery to lower birth weight-specific neonatal mortality in singleton breech infants in the United States. Obstet Gynecol 1998;92: 769-74.
University of Toronto Maternal, Infant and Reproductive Health Research Unit. Term breech trial study protocol, 1996: 8.
Sanchez-Ramos L, Wells TL, Adair CD, Arcelin G, Kaunitz AM, Wells DS. Route of breech delivery and maternal and neonatal outcomes. Int J Gynecol Obstet 2001;73: 7-14.
Lindqvist A, Lindeberg SN, Hanson U. Perinatal mortality and route of delivery in term breech presentations. Br J Obstet Gynaecol 1997;104: 1288-91.
Schiff E, Friedman SA, Mashiach S, Hart O, Barkai G, Sibai BM. Maternal and neonatal outcome of 846 term singleton breech deliveries: seven-year experience at a single center. Am J Obstet Gynecol 1996;175: 18-23.
Irion O, Almagbaly PH, Morabia A. Planned vaginal delivery versus elective cesarean section: a study of singleton term breech presentations. Br J Obstet Gynaecol 1998;105: 710-7.
Kayem G, Goffinet F, Clement D, Hessabi M, Cabrol D. Breech presentation at term: morbidity and mortality according to the type of delivery at Port Royal Maternity hospital from 1993 through 1999. Eur J Obstet Gynecol Reprod Biol 2002;102: 137-42.
Albrechtsen S, Rasmussen S, Reigstad H, Markestad T, Irgens LM, Dalaker K. Evaluation of a protocol for selecting fetuses in breech presentation for vaginal delivery or cesarean section. Am J Obstet Gynecol 1997;177: 586-92.
Tu JV, Hannan EL, Anderson GM, Iron K, Wu K, Vranizan K, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med 1998;339: 1441-7.
Barnett HJ, Taylor DW, Eliasziw M, Fox AJ, Ferguson GG, Haynes RB, et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med 1998;339: 1415-25.
Mayer AR, Chambers SK, Graves E, Holm C, Tseng PC, Nelson BE, et al. Ovarian cancer staging: does it require a gynecologic oncologist. Gynecol Oncol 1992;47: 223-7.
Tingulstad S, Skjeldestad F, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol 2003;102: 499-505.
Varma R, Tahseen S, Lokugamage A, Kunde D. Vaginal route as the norm when planning hysterectomy for benign conditions: change in practice. Obstet Gynecol 2001;97: 613-6.
Maresh M, Metcalfe M, McPherson K, Overton C, Hall V, Hargreaves J, et al. The VALUE national hysterectomy study: description of the patients and their surgery. Br J Obstet Gynaecol 2002;109: 302-12.
Albrechtsen S, Rasmussen S, Irgens LM. Secular trends in peri- and neonatal mortality in breech presentation; Norway 1967-1994. Acta Obstet Gynecol Scand 2000;79: 508-12.
Bracht E. Zur Behandlung der Steisslage. Zentralblatt Gynaecol 1938;31: 1735-6.
Menticoglou SM. Symphysiotomy for the trapped aftercoming parts of the breech: a review of the literature and a plea for its use. Aust NZ J Obstet Gynaecol 1990;30: 1-9.
Giuliani A, Schoell W, Basver A, Tamussino K. Mode of delivery and outcome of 699 term singleton breech deliveries at a single center. Am J Obstet Gynecol 2002;187: 1694-8.
Alarab M, Regan C, O'Connell MP, Keane DP, O'Herlihy C, Foley ME. Singleton vaginal breech delivery at term: still a safe option. Obstet Gynecol 2004;103: 407-12.
Julian D. What is right and what is wrong about evidence-based medicine? J Cardiovasc Electrophysiol 2003:14(Suppl); S2-5.
Danielian P, Wang J, Hall M. Long term outcome by method of delivery of fetuses in breech presentation at term: population based follow up. BMJ 1996;312: 1451-3.
Whyte H, Hannah M, Saigal S. Outcomes of children at 2 years of age in the term breech trial. Am J Obstet Gynecol 2003;Suppl 189(6).
Tonelli MR. The philosophical limits of evidence-based medicine. Acad Med 1998;73: 1234-40.(Andrew Kotaska, senior re)