Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals
http://www.100md.com
《英国医生杂志》
1 Comité de Coordination de l'Evaluation Clinique et de la Qualité en Aquitaine, H?pital Xavier Arnozan, 33604 Pessac, France
Correspondence to: P Michel philippe.michel@ccecqa.asso.fr
Abstract
Review of medical records is considered the benchmark for estimating the extent of medical injuries in hospitals.1-7 But the limitations of this retrospective method raise the issue of alternative methods, especially since the epidemiology of adverse events and medical errors is moving towards other objectives, such as assessing the impact of risk reduction programmes and studying organisational and human factors.8-10
We compared prospective and cross sectional methods for data collection with review of medical records for assessing rates of adverse events and rates of preventable adverse events in acute care hospitals in France. Although the cross sectional method is used to estimate prevalence and not incidence, we chose to study it because in France it is usually used to assess certain risks such as care related infections or adverse drug reactions.11 12
Methods
We selected 37 wards in three public and four private hospitals: medical (n = 15), surgical (n = 14), and obstetric (n = 8). Overall, 786 patients were included on the day of the cross sectional method. Eight were excluded because they were still present on day 30, precluding the review of their medical records. The three methods were therefore applied to 778 patients: 278 in medicine, 263 in surgery, and 237 in obstetrics. The adverse event rates found by the prospective and retrospective methods were similar and the point prevalence obtained by the cross sectional method was about one third lower than that of the other two methods (table). The incidence of preventable adverse events assessed prospectively was 25% higher than that assessed retrospectively.
Proportion of inpatients with at least one adverse event according to epidemiological method (n=778)
Thirteen of the 254 adverse events were excluded from the reference list and considered as false positives (12 of the adverse events identified by the cross sectional method, none by the prospective method, and one by the retrospective method). The final list comprised 241 adverse events in 174 patients (110/80 in medicine, 114/80 in surgery, and 17/14 in obstetrics). Of these 174 patients, 122 (70%) had one adverse event, 38 (22%) had two adverse events, and 14 (8%) had three adverse events (125 patients had at least one adverse event on the day of cross sectional data collection). The cross sectional method identified 39 (64%) of the medical cases, 32 (56%) of the surgical cases, and 5 (45%) of the obstetric cases, and, respectively, 18 (51%), 6 (27%), and 3 (43%) of the preventable ones (fig 2). None of the most serious adverse events were identified.
Fig 2 Venn diagrams showing number of cases identified by three methods of data collection. Cases in brackets not identified by cross sectional method as they were identified after first day of data collection
The prospective method identified 63 (79%) of the medical cases, 49 (61%) of the surgical cases, and 8 (57%) of the obstetric cases, and, respectively, 32 (74%), 12 (46%), and 4 (44%) of the preventable cases. The retrospective method identified 43 (54%) of the medical cases, 61 (76%) of the surgical cases, and 8 (57%) of the obstetric cases, and, respectively, 16 (37%), 12 (46%), and 3 (33%) of the preventable cases. The prospective method was significantly more effective than the retrospective method at identifying cases in medicine (paired 2 = 8.64, P < 0.01) and tended to be less effective at identifying cases in surgery (paired 2 = 3.24, P < 0.07). The prospective method was significantly more effective at identifying preventable cases in medicine (paired 2 = 3.0, P < 0.005) but no difference was observed in surgery. In obstetrics, effectiveness was not different but there were too few adverse events to draw conclusions.
When the medical and surgical cases were aggregated, the prospective and retrospective methods showed similar effectiveness (70% and 66% of 160 patients identified, respectively), but the prospective method was more effective at identifying preventable cases (64% and 40% of 71 patients, respectively; P < 0.02). Effectiveness was similar for adverse events occurring during hospital stay (68% and 64% of 110 patients), those responsible for hospital admissions (63% and 59% of 75 patients), and the most serious events (44% and 33% of 52 patients).
Reliability of cases identified among the 145 cases detected on the first day of study was good (global agreement 91.7%; = 0.83, 95% confidence interval 0.67 to 0.99), but agreement about preventability was low (67.8%; = 0.31, 0.05 to 0.57).
The workload for the prospective and cross sectional methods was perceived as similar, as the phase considered most time consuming for the staff was detection on the first day of data collection (average three hours in wards with 25 patients). Workload was perceived as less for the retrospective method but not negligible, especially when the information sources were multiple and the search was performed by the secretarial staff. The hospital staff constantly preferred the cross sectional and prospective methods because of their pedagogical and communicative virtues.
Discussion
Weingart SN, Wilson RM, Gibberd RW, Harrison B. Epidemiology of medical error. BMJ 2000;320: 774-7.
Wilson RM, Runciman WB, Gibberd RW, Harrisson BT, Newby L, Hamilton JD. The quality in Australian Health-Care Study. Med J Aust 1995;163: 458-71.
Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38: 261-71.
Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG. Incidence of adverse events and negligence care in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991;324: 370-6.
Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322: 517-9.
Schioler T, Lipczak H, Pedersen BL, Mogensen TS, Bech KB, Stockmarr A, et al. . Ugeskr Laeger 2001;163: 5370-8.
Davis P, Lay-Yee R, Schug S, Briant R, Scott A, Johnson S, et al. Adverse events regional feasibility study: indicative findings. NZ Med J 2001;114: 203-5.
Neale G, Woloshynowych M. Retrospective case record review: a blunt instrument that needs sharpening. Qual Safety Health Care 2003;12: 2-3.
Thomas EJ, Studdert DM, Brennan TA. The reliability of medical record review for estimating adverse event rates. Ann Intern Med 2002;136: 812-6.
Weingart SN, Davis RB, Palmer RH, Cahalane M, Hamel MB, Mukamal K, et al. Discrepancies between explicit and implicit review: physician and nurse assessments of complications and quality. Health Serv Res 2002;37: 483-98.
Pouyanne P, Haramburu F, Imbs JL, Bégaud B. Admissions to hospital caused by adverse drug reactions: a cross-sectional incidence study. French pharmacovigilance centres. BMJ 2000;320: 1036.
The French Prevalence Survey Study Group. Prevalence of nosocomial infections in France: results of the nationwide survey in 1996. J Hosp Infect 2000;46: 186-93.
Directions Régionales des Affaires Sanitaires et Sociales. STATISS 2000. Les régions fran?aises. Paris: Ministère de l'Emploi et de la Solidarité, 2000.
Brennan TA, Localio AR, Laird NM. Reliability and validity of judgments concerning adverse events and negligence. Med Care 1989;27: 1148-58.
Healey MA, Shackford SR, Osler TM, Rogers FB, Burns E. Complications in surgical patients. Arch Surg 2002;137: 611-7.
Glenister HM, Taylor LJ, Bartlett CLR, Cooke EM, Sedgwick JA, Mackintosh CA. An evaluation of surveillance methods for detecting infections in hospitals inpatients. J Hosp Infect 1993;23: 229-42.
Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL. Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. Am J Health-Syst Pharm 2002;59: 436-46.
Mantel GD, Biuchmann E, Rees H, Pattinson RC. Severe acute maternal morbidity: a pilot study of a definition for a near-miss. Br J Obstet Gynaecol 1998;105: 985-90.
Waterstone M, Bewley S, Wolfe C. Incidence and predictors of severe obstetric morbidity: case control study. BMJ 2001;322: 1089-93.
Grohskopf LA, Sinkowitz-Cochran RL, Garrett DO, Sohn AH, Levine GL, Siegel JD, et al. A national point-prevalence survey of pediatric intensive care unit-acquired infections in the United States. J Pediatr 2002;140: 432-8.
Emmerson AM, Enstone JE, Griffin M, Kelsey MC, Smyth ET. The Second National Prevalence Survey of infection in hospitals—overview of the results. J Hosp Infect 1996;32: 175-90.(Philippe Michel, medical )
Correspondence to: P Michel philippe.michel@ccecqa.asso.fr
Abstract
Review of medical records is considered the benchmark for estimating the extent of medical injuries in hospitals.1-7 But the limitations of this retrospective method raise the issue of alternative methods, especially since the epidemiology of adverse events and medical errors is moving towards other objectives, such as assessing the impact of risk reduction programmes and studying organisational and human factors.8-10
We compared prospective and cross sectional methods for data collection with review of medical records for assessing rates of adverse events and rates of preventable adverse events in acute care hospitals in France. Although the cross sectional method is used to estimate prevalence and not incidence, we chose to study it because in France it is usually used to assess certain risks such as care related infections or adverse drug reactions.11 12
Methods
We selected 37 wards in three public and four private hospitals: medical (n = 15), surgical (n = 14), and obstetric (n = 8). Overall, 786 patients were included on the day of the cross sectional method. Eight were excluded because they were still present on day 30, precluding the review of their medical records. The three methods were therefore applied to 778 patients: 278 in medicine, 263 in surgery, and 237 in obstetrics. The adverse event rates found by the prospective and retrospective methods were similar and the point prevalence obtained by the cross sectional method was about one third lower than that of the other two methods (table). The incidence of preventable adverse events assessed prospectively was 25% higher than that assessed retrospectively.
Proportion of inpatients with at least one adverse event according to epidemiological method (n=778)
Thirteen of the 254 adverse events were excluded from the reference list and considered as false positives (12 of the adverse events identified by the cross sectional method, none by the prospective method, and one by the retrospective method). The final list comprised 241 adverse events in 174 patients (110/80 in medicine, 114/80 in surgery, and 17/14 in obstetrics). Of these 174 patients, 122 (70%) had one adverse event, 38 (22%) had two adverse events, and 14 (8%) had three adverse events (125 patients had at least one adverse event on the day of cross sectional data collection). The cross sectional method identified 39 (64%) of the medical cases, 32 (56%) of the surgical cases, and 5 (45%) of the obstetric cases, and, respectively, 18 (51%), 6 (27%), and 3 (43%) of the preventable ones (fig 2). None of the most serious adverse events were identified.
Fig 2 Venn diagrams showing number of cases identified by three methods of data collection. Cases in brackets not identified by cross sectional method as they were identified after first day of data collection
The prospective method identified 63 (79%) of the medical cases, 49 (61%) of the surgical cases, and 8 (57%) of the obstetric cases, and, respectively, 32 (74%), 12 (46%), and 4 (44%) of the preventable cases. The retrospective method identified 43 (54%) of the medical cases, 61 (76%) of the surgical cases, and 8 (57%) of the obstetric cases, and, respectively, 16 (37%), 12 (46%), and 3 (33%) of the preventable cases. The prospective method was significantly more effective than the retrospective method at identifying cases in medicine (paired 2 = 8.64, P < 0.01) and tended to be less effective at identifying cases in surgery (paired 2 = 3.24, P < 0.07). The prospective method was significantly more effective at identifying preventable cases in medicine (paired 2 = 3.0, P < 0.005) but no difference was observed in surgery. In obstetrics, effectiveness was not different but there were too few adverse events to draw conclusions.
When the medical and surgical cases were aggregated, the prospective and retrospective methods showed similar effectiveness (70% and 66% of 160 patients identified, respectively), but the prospective method was more effective at identifying preventable cases (64% and 40% of 71 patients, respectively; P < 0.02). Effectiveness was similar for adverse events occurring during hospital stay (68% and 64% of 110 patients), those responsible for hospital admissions (63% and 59% of 75 patients), and the most serious events (44% and 33% of 52 patients).
Reliability of cases identified among the 145 cases detected on the first day of study was good (global agreement 91.7%; = 0.83, 95% confidence interval 0.67 to 0.99), but agreement about preventability was low (67.8%; = 0.31, 0.05 to 0.57).
The workload for the prospective and cross sectional methods was perceived as similar, as the phase considered most time consuming for the staff was detection on the first day of data collection (average three hours in wards with 25 patients). Workload was perceived as less for the retrospective method but not negligible, especially when the information sources were multiple and the search was performed by the secretarial staff. The hospital staff constantly preferred the cross sectional and prospective methods because of their pedagogical and communicative virtues.
Discussion
Weingart SN, Wilson RM, Gibberd RW, Harrison B. Epidemiology of medical error. BMJ 2000;320: 774-7.
Wilson RM, Runciman WB, Gibberd RW, Harrisson BT, Newby L, Hamilton JD. The quality in Australian Health-Care Study. Med J Aust 1995;163: 458-71.
Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38: 261-71.
Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG. Incidence of adverse events and negligence care in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991;324: 370-6.
Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322: 517-9.
Schioler T, Lipczak H, Pedersen BL, Mogensen TS, Bech KB, Stockmarr A, et al. . Ugeskr Laeger 2001;163: 5370-8.
Davis P, Lay-Yee R, Schug S, Briant R, Scott A, Johnson S, et al. Adverse events regional feasibility study: indicative findings. NZ Med J 2001;114: 203-5.
Neale G, Woloshynowych M. Retrospective case record review: a blunt instrument that needs sharpening. Qual Safety Health Care 2003;12: 2-3.
Thomas EJ, Studdert DM, Brennan TA. The reliability of medical record review for estimating adverse event rates. Ann Intern Med 2002;136: 812-6.
Weingart SN, Davis RB, Palmer RH, Cahalane M, Hamel MB, Mukamal K, et al. Discrepancies between explicit and implicit review: physician and nurse assessments of complications and quality. Health Serv Res 2002;37: 483-98.
Pouyanne P, Haramburu F, Imbs JL, Bégaud B. Admissions to hospital caused by adverse drug reactions: a cross-sectional incidence study. French pharmacovigilance centres. BMJ 2000;320: 1036.
The French Prevalence Survey Study Group. Prevalence of nosocomial infections in France: results of the nationwide survey in 1996. J Hosp Infect 2000;46: 186-93.
Directions Régionales des Affaires Sanitaires et Sociales. STATISS 2000. Les régions fran?aises. Paris: Ministère de l'Emploi et de la Solidarité, 2000.
Brennan TA, Localio AR, Laird NM. Reliability and validity of judgments concerning adverse events and negligence. Med Care 1989;27: 1148-58.
Healey MA, Shackford SR, Osler TM, Rogers FB, Burns E. Complications in surgical patients. Arch Surg 2002;137: 611-7.
Glenister HM, Taylor LJ, Bartlett CLR, Cooke EM, Sedgwick JA, Mackintosh CA. An evaluation of surveillance methods for detecting infections in hospitals inpatients. J Hosp Infect 1993;23: 229-42.
Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL. Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. Am J Health-Syst Pharm 2002;59: 436-46.
Mantel GD, Biuchmann E, Rees H, Pattinson RC. Severe acute maternal morbidity: a pilot study of a definition for a near-miss. Br J Obstet Gynaecol 1998;105: 985-90.
Waterstone M, Bewley S, Wolfe C. Incidence and predictors of severe obstetric morbidity: case control study. BMJ 2001;322: 1089-93.
Grohskopf LA, Sinkowitz-Cochran RL, Garrett DO, Sohn AH, Levine GL, Siegel JD, et al. A national point-prevalence survey of pediatric intensive care unit-acquired infections in the United States. J Pediatr 2002;140: 432-8.
Emmerson AM, Enstone JE, Griffin M, Kelsey MC, Smyth ET. The Second National Prevalence Survey of infection in hospitals—overview of the results. J Hosp Infect 1996;32: 175-90.(Philippe Michel, medical )