Star wars, NHS style
http://www.100md.com
《英国医生杂志》
1 Newcastle upon Tyne Hospitals NHS Trust, Freeman Hospital, Newcastle upon Tyne NE7 7DN, 2 School of Clinical Medical Sciences, University of Newcastle upon Tyne, Newcastle upon Tyne
Correspondence to: M Irving m.h.irving@ncl.ac.uk
NHS trusts awarded three stars are supposed to be the best performing in the country. However, problems with the 2002-3 assessment mean that this is not necessarily true
Introduction
Our review of the statistical methods adopted for the star ratings shows that inappropriate criteria were used to arrive at the judgments. We obtained data on acute trusts and indicator targets from the Commission for Health Improvement and Department of Health websites (www.chi.nhs.uk/ratings/ and www.doh.gov.uk). The outpatient indicator was a composite score calculated from the number of patients waiting longer than 26 weeks at the end of each of the first three quarters of 2002-3 plus the number of patients who were still waiting longer than 21 weeks at the end of the fourth quarter. To achieve the target, trusts had to have had no more than five breaches; more than 50 breaches constituted significant underachievement.
Use of quarter end figures
In contrast to other key targets based on percentages, the outpatient key target was an absolute test—that is, it measured the number of breaches of the target irrespective of the size and activity of the trust. The number of referrals differs greatly across trusts, which range from small specialist hospitals that receive just 64 general practitioner referrals a year to large university hospitals such as Newcastle, which receives 118 000 referrals a year. Table 2 shows that a breach of the target waiting time in up to 6% of patients could lead a trust to achieve, underachieve, or significantly underachieve this target, depending on patient workload.
Table 2 Analysis of breaches of outpatient waiting time for all NHS trusts
Clearly, it is easier for a small trust to meet the threshold of five or fewer breaches a year than it is for trusts with a large referral base. The Commission for Health Improvement used a relative test for other indicators. The rationale for choosing relative tests for some key targets and one patient focus indicator and numerical targets for other similar indicators is difficult to understand.
Six of the nine key target indicators specify absolute rather than percentage targets. Absolute targets may demand comparatively higher levels of service from larger trusts, although even those key targets measured in percentage terms may be easier to attain in small trusts. To investigate this, we considered data from the 150 trusts that had returned information for all six key target indicators concerning patient numbers.
We compared the patient population in the highest ranking trusts with that in the remaining trusts for the six key targets. For all indicators, the highest ranked trusts had smaller average patient populations (table 3).
Table 3 Relation between patient population and performance ranking. Data were analysed with the Mann-Whitney test
Discussion
Bird SM, Cox D, Farewell VT, Goldstein H, Holt T, Smith PC. Performance indicators: good, bad, and ugly. London: Royal Statistical Society, 2003.
Healthcare Commission. Indicator listings for acute trusts. http://ratings.healthcarecommission.org.uk/Indicators_2004/ (accessed 11 July 2004).(Richard M Barker, busines)
Correspondence to: M Irving m.h.irving@ncl.ac.uk
NHS trusts awarded three stars are supposed to be the best performing in the country. However, problems with the 2002-3 assessment mean that this is not necessarily true
Introduction
Our review of the statistical methods adopted for the star ratings shows that inappropriate criteria were used to arrive at the judgments. We obtained data on acute trusts and indicator targets from the Commission for Health Improvement and Department of Health websites (www.chi.nhs.uk/ratings/ and www.doh.gov.uk). The outpatient indicator was a composite score calculated from the number of patients waiting longer than 26 weeks at the end of each of the first three quarters of 2002-3 plus the number of patients who were still waiting longer than 21 weeks at the end of the fourth quarter. To achieve the target, trusts had to have had no more than five breaches; more than 50 breaches constituted significant underachievement.
Use of quarter end figures
In contrast to other key targets based on percentages, the outpatient key target was an absolute test—that is, it measured the number of breaches of the target irrespective of the size and activity of the trust. The number of referrals differs greatly across trusts, which range from small specialist hospitals that receive just 64 general practitioner referrals a year to large university hospitals such as Newcastle, which receives 118 000 referrals a year. Table 2 shows that a breach of the target waiting time in up to 6% of patients could lead a trust to achieve, underachieve, or significantly underachieve this target, depending on patient workload.
Table 2 Analysis of breaches of outpatient waiting time for all NHS trusts
Clearly, it is easier for a small trust to meet the threshold of five or fewer breaches a year than it is for trusts with a large referral base. The Commission for Health Improvement used a relative test for other indicators. The rationale for choosing relative tests for some key targets and one patient focus indicator and numerical targets for other similar indicators is difficult to understand.
Six of the nine key target indicators specify absolute rather than percentage targets. Absolute targets may demand comparatively higher levels of service from larger trusts, although even those key targets measured in percentage terms may be easier to attain in small trusts. To investigate this, we considered data from the 150 trusts that had returned information for all six key target indicators concerning patient numbers.
We compared the patient population in the highest ranking trusts with that in the remaining trusts for the six key targets. For all indicators, the highest ranked trusts had smaller average patient populations (table 3).
Table 3 Relation between patient population and performance ranking. Data were analysed with the Mann-Whitney test
Discussion
Bird SM, Cox D, Farewell VT, Goldstein H, Holt T, Smith PC. Performance indicators: good, bad, and ugly. London: Royal Statistical Society, 2003.
Healthcare Commission. Indicator listings for acute trusts. http://ratings.healthcarecommission.org.uk/Indicators_2004/ (accessed 11 July 2004).(Richard M Barker, busines)