Navigating the Terrain between Research and Practice: A Collaborative Research Network (CRN) Case Study in Diabetes Research
http://www.100md.com
《家庭医学美国版》
Abstract
Background: Clinical trials conducted in real world settings can improve the translation of research into practice. Although many experts have outlined strategies to conduct effectiveness research in primary care practices, there are few case studies describing how decisions and trade-offs influence the design and implementation of practical clinical trials.
Methods: This paper outlines the IDEALL (Improving Diabetes Efforts Across Language and Literacy) Project, a study of self-management support strategies for diabetes. We present a case study about translational diabetes research. We explore patient, clinician, and practice recruitment; which interventions to study; and decisions about the level of integration of interventions into practices.
Results: Recruitment strategies needed to take into account Institutional Review Board restrictions not allowing direct patient contact outside of the clinic. Patient diversity was increased by using a population-based database to identify eligible patients; clinic diversity was increased by including neighborhood and hospital-based clinics. An adjunctive model of care to provide interventions was used to avoid burdening clinics with integrating the interventions into clinical practice, but did not address the importance of sustainability in primary care settings.
Conclusion: Practical clinical trials involve complex strategies, decisions, and trade-offs. Researchers engaging in translational research should continue to describe how design decisions may influence the interpretation of results.
Despite many scientific advances that have increased treatment options for patients with chronic diseases, there are well-recognized barriers to providing optimal primary care. For type 2 diabetes, findings from clinical research are not readily translated into the complex realities of primary care practice. For example, results of clinical trials demonstrating that better blood pressure and glycemic control can prevent complications,1,2 and from studies demonstrating that empowerment-based communication can improve glycemic control, 3–6 have not led to widespread improvements in diabetes outcomes at the population level. This gap in quality of care often disproportionately affects those persons who are most vulnerable (often referred to as "priority populations"),7 such as the elderly, racial and ethnic minorities, non-English speakers, immigrants from outside the United States, and those with low income and low levels of education.8 To better understand this problem and identify solutions, more translational research focusing on diabetes care in primary care settings and among priority populations has been recommended.9–13
The goal of traditional clinical research has been to minimize variation in study populations and settings to increase internal validity and demonstrate efficacy, so that the results are valid and unbiased within the population studied. The goal of translational research is to determine whether research findings from homogenous populations and settings are generalizable to more diverse ones such as those encountered in "real world" practices. This generalizability is also referred to as effectiveness or external validity, and information about effectiveness enables clinicians and policymakers to make decisions regarding the design of services and estimate the public health impact of the interventions being tested. For example, in a translational study, a patient’s limited literacy or English language proficiency might be studied to determine how it influences the uptake (also known as "reach") and success of efficacious interventions, so that these interventions can be modified or tailored to meet the needs of populations they will be directed to. Similarly, at the community and practice level, barriers such as finite resources and clinicians’ competing time demands are examined to assess how these factors may limit the reach and success of interventions, so that the interventions can be best adapted to the realistic limits of practice.
The idea of practical clinical trials has been promoted as a means to improve translational research.11–14 Practical clinical trials focus on applying the principals of: (1) population-based recruitment of diverse patients; (2) testing clinically useful interventions; and (3) collecting data on diverse outcomes relevant to patients, clinicians, health care system administrators, and policymakers. To improve the evaluation of health promotion and clinical interventions, strategies for implementing practical research studies have been outlined by Tunis et al13 and also by Glasgow et al14 using the RE-AIM framework. 9,15,16 The RE-AIM framework provides a system for integrating and reporting about patient populations (reach and effectiveness of the intervention with regard to the target population), about clinicians/settings (adoption and implementation of the interventions at these levels), and across time (maintenance of the interventions).
This article presents a case study of the IDEALL (Improving Diabetes Efforts Across Language and Literacy) Project, a practical clinical trial whose purpose is to gain an understanding of how to improve diabetes outcomes among primary care patients in a safety net system in San Francisco, CA. The study examines 2 interventions (compared with usual care) to enhance diabetes self-management support among patients with poor glycemic control. Patients from the San Francisco Department of Public Health’s Community Health Network (CHNSF) primary care clinics who may face barriers to high quality diabetes care because of language and literacy barriers represent the target population for this study. In this case study, we will discuss the strategies used to increase the reach and effectiveness potential of the IDEALL Project, as well as the trade-offs encountered when applying the principles of practical clinical trials and the RE-AIM framework. By presenting this case study, we hope to demonstrate that designing and conducting practice-based, translational research often involves a complex set of strategies, decisions, and trade-offs to adapt to the ecology of the practice settings while maximizing internal and external validity.
IDEALL Project
The IDEALL Project emanated from a recent strategic initiative to improve the quality of chronic disease care in CHNSF primary care clinics. There are 9 CHNSF clinics—these clinics represent a subset of the University of California San Francisco (UCSF), Department of Family and Community Medicine’s practice-based research network, the Collaborative Research Network. The IDEALL Project is a direct outgrowth of patient’s experiences and practitioners’ attitudes and recommendations previously described in 4 studies examining health literacy, patient-physician communication, and primary care provider attitudes and behavior in the CHNSF.17–20 These studies identified that: (1) limited health literacy and limited English proficiency are common barriers but are potentially remediable determinants of the quality of chronic disease care, (2) primary care physicians consistently overestimate their diabetes patients’ literacy levels, (3) the effectiveness of physicians’ office-based communication is variable, and (4) physicians believe that adjunctive, tailored disease management systems would significantly improve diabetes care for their patients with limited health literacy.
At the start of the IDEALL Project, there was no systematic diabetes disease management system in place in the CHNSF, and one of the project’s explicit goals is to inform future plans to develop such a system. The IDEALL Project is designed to assess the reach and effectiveness of 2 diabetes self-management support interventions. One is technologically oriented and the other interpersonally oriented, each tailored to the language and literacy needs of high-risk diabetes patients. Both interventions are compared with usual primary care and to each other in a 3-arm study design (see Figure 1). Both interventions are delivered in English, Spanish, and Cantonese, the 3 most commonly spoken languages among CHNSF patients. Eligible and consenting patients with diabetes in poor control are randomly assigned to an automated telephone diabetes self-management (ATDM) program or to group medical visits (GMV) or to usual care, and are followed for 1 year after enrollment.
Patients randomized to ATDM receive weekly automated telephone calls in their native language, which include a rotating set of diabetes self-care queries (eg, questions about medication adherence, diet, exercise, smoking, etc); a number of questions regarding psychosocial aspects of diabetes care (eg, current coping, depressive symptoms, etc); and referrals for preventive services (eg, visits to the ophthalmologist, etc). Patients respond to questions via touch-tone commands; those answering out-of-range on an item (based on predetermined thresholds) receive an immediate automated narrative health education message. Patients consistently answering out-of-range receive a call back from a language concordant nurse care manager. Those randomized to GMV are invited to participate in monthly group medical visits in their native language involving 7 to 10 diabetes patients, facilitated by a bilingual primary care physician and a health educator. A clinical pharmacist is also present to review medication changes or regimens. Each intervention is delivered as an adjunct to usual care. Patients randomized to usual care continue to receive standard diabetes care. Primary outcomes of the IDEALL Project include: (1) extent of patient engagement with the IDEALL interventions; (2) changes in diabetes indices—such as diabetes-related self-efficacy and glycemic control; and (3) relative resource use.
IDEALL Implementation Strategies to Increase External Validity
Table 1 presents translational research recommendations12–14,16,21 that address the areas of recruiting diverse patient populations and comparing clinically relevant interventions in practical clinical trials. The following is a description of the strategies and rationale we used in developing the IDEALL Project and the trade-offs that were made over the course of the Project.
Translational Research Goal I: Achieving Patient Diversity across Populations and Clinic Settings
IDEALL Strategy: Minimize Exclusions
Using electronic data from the CHNSF, we developed a diabetes registry of all patients who met the following eligibility criteria: diagnosis of adult, type 2 diabetes; one or more primary care visit in the previous 12 months; English, Spanish, or Cantonese speaking; did not have limited vision or were hearing-impaired; and no diagnoses of psychotic illness or end-stage renal disease. The registry also included the most recent laboratory value for hemoglobin A1c tests and the patient’s primary care provider and clinic. The database was developed by monthly merges of CHNSF spreadsheets of clinical, appointment, and laboratory data.
The Institutional Review Board (IRB) at UCSF and at CHNSF would not allow the IDEALL Project staff to directly contact eligible patients. Instead, patients could only be enrolled through provider referral or by approaching patients at routinely scheduled clinic visits. Given this constraint, we provided each clinician with a list of their eligible diabetes patients with check-off boxes for the following exclusion criteria: patient is no longer at this clinic/is not my patient, patient does not have diabetes, patient has end-stage renal disease, patient has major psychiatric illness and other, which was left blank for the clinician to complete (eg, patient is too ill to participate). Returned forms without boxes checked indicated clinician consent. Because there were few exclusion criteria, these forms took only a few minutes to complete—all but 2 of the 118 clinicians completed them. The last eligibility criterion related to suboptimal glycemic control; a hemoglobin A1c value of 8 in the previous 12 months was derived from registry data alone.
Trade-offs
The restrictions imposed on patient recruitment by the Institutional Review Boards presented a serious barrier to extending the reach of the interventions and shaped the study design to be clinic-based rather than population-based more generally, within the CHNSF. The original study design was to directly contact patients at home, including those who have poor glycemic control and who infrequently attend primary care clinics. To reach patients who came infrequently to the clinics, we modified our design to have research assistants approach patients at their primary care and specialty appointment visits over an extended enrollment period. However, this strategy only allowed for contact with active clinic patients, and did not allow for inclusion of patients too sick to travel to the clinic or who were only seen in the emergency department or hospitalized over the study period. A few of the exclusion criteria of the IDEALL study also restricted the diversity of the patient sample. For example, patients who faced transportation barriers because of illness or care-taking responsibilities, or who frequently traveled between San Francisco and their country of origin were not eligible to be enrolled in the study—possibly representing some of the more vulnerable patients cared for in the CHNSF.
Despite the obstacle presented by the Institutional Review Boards, the merging of CHNSF electronic medical information into the IDEALL database enabled us to define our study population and pursue a population-based recruitment strategy among CHNSF patients that were active in terms of clinic attendance. The languages in the study are the 3 most commonly spoken by diabetes patients receiving care in the CHNSF,7 enabling the IDEALL Project to reach many patients in this ethnically and linguistically diverse safety net system. Although the availability of the intervention in non-English languages is a strength of the study design, there are other languages, such as Tagolog and Vietnamese, that are spoken by CHNSF diabetes patients, many of whom face communication barriers in the health care setting.
Overall, 584 patients were approached, 339 enrolled (58%), 109 were ineligible at the time of enrollment (for reasons such as no telephone, frequent travel out of the area, limited ability to speak a study language, limited vision/hearing-impaired), and 78 refused (primarily for reasons related to not wanting to leave the house if randomized to the group medical visit arm, including difficulty with transportation, serious illness, or having care-taking responsibilities). Many of these patients stated that they would probably have enrolled if they could have selected the ATDM arm. An additional 58 patients were eligible but had not been enrolled by the end of the study.
IDEALL Strategy: Recruit Patients from Diverse Settings
Because we were not able to contact patients directly, we chose a clinic-based recruitment strategy that could be used in diverse clinic settings to reach large numbers of eligible patients. Clinics were chosen using a targeted selection strategy of reviewing clinic profiles of potentially eligible patients and ranking the clinics according to the proportion of patients with matching eligibility criteria.22 The top 2 sites, based at San Francisco General Hospital, agreed to participate. However, we were hesitant to conduct the study only at hospital-based clinics located in one geographic area of the city because we believed that community clinics reflected both different patient populations and different levels of access to specialty services. The next 4 highly ranked clinics were located in neighborhoods throughout town, but 2 clinics had diabetes group visits underway or were planning them and opted out. The other 2 sites participated. Because large numbers of Cantonese-speaking patients with diabetes were seen at one of these clinics (located in the Chinatown neighborhood), enrollment of Cantonese speakers was concentrated at this clinic to simplify the clinic-based recruitment process.
Trade-offs
We were able to recruit patients from diverse clinics within the CHNSF but did not extend the reach of the interventions beyond the 4 participating clinics or beyond this clinic system. Two of the participating clinics were neighborhood health centers that had not participated in previous studies and their inclusion greatly extended the reach of the interventions to these community settings. Establishing the study at these sites, however, provided logistic challenges regarding scheduling and registration for the group medical visits that consumed considerable staff resources.
Translational Research Goal II: Selecting Interventions That Are Clinically Relevant across Settings
IDEALL Strategy: Develop Interventions That Reflect Primary Care Realities
There was a clear demand for extra care for patients managing chronic illnesses within the CHNSF and the IDEALL interventions were developed with extensive patient and clinician input. However, we did not believe that the clinics would be able to independently initiate and sustain these interventions because of the planning, training recruitment, and ongoing project management required. To avoid burdening the clinics, the IDEALL study provided the interventions as adjunctive services to those available through the clinic.
Trade-offs
Using an adjunctive model allowed us to standardize the interventions across sites and to introduce them to practices without forcing their adoption. We believed that if the interventions were successfully received by patients and providers, there would be a gradual shift in favor of integrating these interventions into clinical care across CHNSF clinics, which would enable subsequent evaluation. As much as possible, we engaged clinic staff to participate in the self-management support program, although the primary responsibility for implementation resided with IDEALL staff.
The drawbacks of an adjunctive model are important and represent the largest trade-off in implementation of the IDEALL Project. Once support for the study ends, the interventions may disappear from the minds of the clinic staff and of patients, potentially jeopardizing adoption, implementation, and maintenance efforts with respect to self-management support. In addition, with an adjunctive model of care, patients and clinicians may not be convinced that the services offered with the IDEALL Project could be provided in the "regular" context of their care, and they may not have engaged with them to the extent they would if they were offered from within the clinic itself.
IDEALL Strategy: Compare Clinically Relevant Alternatives
The 2 study interventions reflect different models of self-management support, each with different strengths and weaknesses. The ATDM is more technologically oriented and does not require patients to attend extra visits. The GMV focuses on participatory learning, interpersonal support, and the reinforcement of skill development with peers. Both are clinically relevant for the following reasons: (1) prior evidence of efficacy from controlled trials in other settings23–25; (2) their potential to overcome literacy and language barriers and promote self-management in vulnerable populations 26,27; (3) preliminary evidence in other settings suggesting significant patient engagement ("reach"); (4) their focus on changes at the organization/practice and patient levels (as opposed to changes in physician behavior)28,29; (5) their emphasis on expanding the traditional patient-physician dyadic model to a multidisciplinary, team-based model30; (6) the relatively low cost; and (7) our opinion, informed by inquiries conducted in the CHNSF and other safety net systems18,19,31 that such interventions might be welcomed by clinicians and administrators.
Trade-offs
Several research designs were considered for the IDEALL Project. A 3-arm randomized clinical trial, with randomization at the patient level, was selected for several reasons. We believed that alternative designs, such as a crossover design allowing patients in different arms of the study to later crossover to the alternative study arm would be risky, both in terms of patient retention (patients would need to stay in the study twice as long) and in terms of determining which interventions to attribute observed effects. A crossover design would not allow for a control group, which we believed essential to determining whether either intervention would provide benefit.
The randomized design was not popular with some patients from several of the clinics we approached. We observed some refusals because of the randomized nature of the study; several patients expressed interest in participation, but only if they could select which study arm. This suggests that acceptance rates in a nonrandomized study context may even be higher. Similarly, some clinics approached were unwilling or unable to participate in the randomized controlled trial, limiting the reach of the study. We also selected to randomize patients, rather than having clinicians randomized to one of the 3 groups. Because one of the objectives of the IDEALL Project was to expose participating clinicians to both interventions, randomization of patients within clinician allowed us to maximize the interventions’ reach with the clinical provider population in the CHNSF.
Conclusions
In this case study, we describe how designing and conducting practical clinical trials often involves a complex set of strategies, decisions, and trade-offs aimed at adapting to the ecology of the practice settings while maximizing internal and external validity. Researchers engaging in translational research should recognize the importance of and complexity inherent to achieving such a balance and describe how their decisions with regard to design may influence the interpretation of results. We believe the decisions and trade-offs made in the IDEALL Project situates it in an intermediate zone between pure efficacy research and evaluation of a real world, fully integrated set of interventions. It can be viewed as in-between a stage III (examining efficacy) and stage IV (examining effectiveness) clinical trial as described by Dzewaltowski, Glasgow et al.9,14 We suspect that many practice-based translational research projects are similarly situated.
We encountered several challenges in implementing the study and believe trade-offs such as those described are unavoidable. First, it is likely that Institutional Review Boards will continue to focus on patient privacy protections. Population-based recruitment strategies may need to redefine eligible patients as those who are clinically active and accessible for recruitment, whether from direct provider referrals, provider-initiated invitations or through an appointment-based strategy, as with IDEALL. Second, choosing between fully integrated or adjunctive models for delivering intensive interventions, such as those suitable for self-management support as described here for diabetes, will require careful evaluation of the setting and clinician attitudes toward the interventions. In the IDEALL experience, despite previous research within the CHNSF that supported the development of the IDEALL study interventions, it was determined that a fully integrated delivery model might be premature, might place too great a burden to the clinics and could introduce too much variation in intervention content. Finally, decisions regarding critical details of study design, such as randomization at patient versus clinician versus clinic need to be made in the context of patient, clinician, and clinic preferences. We were fortunate that sufficient clinical sites and patients were identified as willing to accept randomization, and our design did try to minimize the likelihood that an individual would be randomized to usual care by having 2 intervention arms. As such programs become more widely disseminated, there is an urgent need to develop alternative study designs to increase acceptability.
As the IDEALL Project is nearing an end, there is evidence of the potential advantage of using such an intermediate stage design. The CHNSF clinical and executive leadership is planning an expansion and diffusion of IDEALL adjunctive models for diabetes and other chronic illnesses. To fully actualize this potential and inform the next phase of implementation, the extensive data we have collected need to be analyzed with respect to patient outcomes and relative resource use at the clinic and system level. Assuming some benefit is demonstrated, we then face the welcome challenge of integrating these models into routine clinical care in the near future, as part of a health system-wide commitment to improving self-management support.
Notes
Funding: This work was supported by AHRQ grants (R21HS013544 and R21HS014864-01) and the Commonwealth Fund. DS was also supported by a National Institutes of Health Mentored Clinical Scientist Award K-23 RR16539-03. Electronic data in the CHNSF were made available through the National Center for Research Resources Grant MO1RR0083-41.
Conflict of interest: none declared.
Received for publication June 29, 2005. Revision received November 8, 2005. Accepted for publication November 9, 2005.
References
The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med 1993; 329: 977–86.
The UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 1998; 352: 837–53.
Sadur C, Moline N, Costa M, et al. Diabetes management in a health maintenance organization. Efficacy of care management using cluster visits. Diabetes Care 1999; 22: 2011–7.
Lorig K, Sobel D, Stewart A, et al. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: a randomized trial. Med Care 1999; 37: 5–14.
Piette J, Weinberger M, McPhee S. The effect of automated calls with telephone nurse follow-up on patient-centered outcomes of diabetes care: a randomized, controlled trial. Med Care 2000; 38: 218–30.
Trento M, Passera P, Tomalino M, et al. Group visits improve metabolic control in type 2 diabetes: a 2-year follow-up. Diabetes Care 2001; 24: 995–1000.
Regenstein M, Huang J, Cummings L, Lessler D, Reilly B, Schillinger D. Caring for patients with diabetes in safety net hospitals and health systems: The Commonwealth Fund; 2005. p. 11–12.
Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. National Healthcare Disparities Report. Rockville, Maryland July, 2003.
Dzewaltowski DA, Estabrooks PA, Klesges LM, Bull S, Glasgow RE. Behavior change intervention research in community settings: how generalizable are the results? Health Promot Int 2004; 19: 235–45.
Glasgow RE. Translating research to practice: lessons learned, areas for improvement, and future directions. Diabetes Care 2003; 26: 2451–6.
Murphy D, Chapel T, Clark C. Moving diabetes care from science to practice: the evolution of the National Diabetes Prevention and Control Program. Ann Intern Med 2004; 140: 978–84.
Garfield SA, Malozowski S, Chin MH, et al. Considerations for diabetes translational research in real-world settings. Diabetes Care 2003; 26: 2670–4.
Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003; 290: 1624–32.
Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care 2005; 43: 551–7.
Glasgow RE, Goldstein MG, Ockene JK, Pronk NP. Translating what we have learned into practice. Principles and hypotheses for interventions addressing multiple behaviors in primary care. Am J Prev Med 2004; 27(2 Suppl): 88–101.
Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health 2003; 93: 1261–7.
Schillinger D, Piette J, Grumbach K, et al. Closing the loop: physician communication with diabetic patients who have low health literacy. Arch Intern Med 2003; 163: 83–90.
Schillinger D, Bindman A, Wang F, Stewart A, Piette J. Functional health literacy and the quality of physician-patient communication among diabetes patients. Patient Educ Couns 2004; 52: 315–23.
Schillinger D, Grumbach K, Piette J, et al. Association of health literacy with diabetes outcomes. JAMA 2002; 288: 475–82.
Seligman HK, Wang F, Palacios JL, et al. Physician notification of their diabetes patients’ limited health literacy. A randomized, controlled trial. J Gen Intern Med 2005; 20: 1001–7.
Godwin M, Ruhland L, Casson I, et al. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol 2003; 3: 28.
Glasgow RE, Whitlock EP, Eakin EG, Lichtenstein E. A brief smoking cessation intervention for women in low-income Planned Parenthood clinics. Am J Public Health 2000; 90: 786–9.
Piette J. Interactive voice response systems in the diagnosis and management of chronic disease. Am J Managed Care 2000; 68: 817–27.
Piette J, Weinberger M. Do automated calls with nurse follow-up improve self-care and glycemic control among vulnerable patients with diabetes? Am J Med 2000; 108: 20–7.
Sadur C, Moline N. Diabetes management in a health maintenance organization. Efficacy of care management using cluster visits. Diabetes Care 1999; 22: 2011–7.
Rudd R, Comings J. Learner developed materials: an empowering product. Health Educ Q 1994; 21: 313–27.
Schillinger D. Improving the quality of chronic disease management for populations with low functional health literacy: a call to action. Disease Manag 2001; 4: 103–9.
Cabana M, Rand C, Powe N, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282: 1458–65.
Davis D, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ 1997; 157: 408–16.
Grumbach K, Bodenheimer T. Can health care teams improve primary care practice?. JAMA 2004; 291: 1246–51.
Fernandez A, Schillinger D, Grumbach K, et al. Physician language ability and cultural competence: an exploratory study of communication with Spanish-speaking patients. J Gen Intern Med 2004; 19: 167–74.(Margaret A. Handley, PhD,)
Background: Clinical trials conducted in real world settings can improve the translation of research into practice. Although many experts have outlined strategies to conduct effectiveness research in primary care practices, there are few case studies describing how decisions and trade-offs influence the design and implementation of practical clinical trials.
Methods: This paper outlines the IDEALL (Improving Diabetes Efforts Across Language and Literacy) Project, a study of self-management support strategies for diabetes. We present a case study about translational diabetes research. We explore patient, clinician, and practice recruitment; which interventions to study; and decisions about the level of integration of interventions into practices.
Results: Recruitment strategies needed to take into account Institutional Review Board restrictions not allowing direct patient contact outside of the clinic. Patient diversity was increased by using a population-based database to identify eligible patients; clinic diversity was increased by including neighborhood and hospital-based clinics. An adjunctive model of care to provide interventions was used to avoid burdening clinics with integrating the interventions into clinical practice, but did not address the importance of sustainability in primary care settings.
Conclusion: Practical clinical trials involve complex strategies, decisions, and trade-offs. Researchers engaging in translational research should continue to describe how design decisions may influence the interpretation of results.
Despite many scientific advances that have increased treatment options for patients with chronic diseases, there are well-recognized barriers to providing optimal primary care. For type 2 diabetes, findings from clinical research are not readily translated into the complex realities of primary care practice. For example, results of clinical trials demonstrating that better blood pressure and glycemic control can prevent complications,1,2 and from studies demonstrating that empowerment-based communication can improve glycemic control, 3–6 have not led to widespread improvements in diabetes outcomes at the population level. This gap in quality of care often disproportionately affects those persons who are most vulnerable (often referred to as "priority populations"),7 such as the elderly, racial and ethnic minorities, non-English speakers, immigrants from outside the United States, and those with low income and low levels of education.8 To better understand this problem and identify solutions, more translational research focusing on diabetes care in primary care settings and among priority populations has been recommended.9–13
The goal of traditional clinical research has been to minimize variation in study populations and settings to increase internal validity and demonstrate efficacy, so that the results are valid and unbiased within the population studied. The goal of translational research is to determine whether research findings from homogenous populations and settings are generalizable to more diverse ones such as those encountered in "real world" practices. This generalizability is also referred to as effectiveness or external validity, and information about effectiveness enables clinicians and policymakers to make decisions regarding the design of services and estimate the public health impact of the interventions being tested. For example, in a translational study, a patient’s limited literacy or English language proficiency might be studied to determine how it influences the uptake (also known as "reach") and success of efficacious interventions, so that these interventions can be modified or tailored to meet the needs of populations they will be directed to. Similarly, at the community and practice level, barriers such as finite resources and clinicians’ competing time demands are examined to assess how these factors may limit the reach and success of interventions, so that the interventions can be best adapted to the realistic limits of practice.
The idea of practical clinical trials has been promoted as a means to improve translational research.11–14 Practical clinical trials focus on applying the principals of: (1) population-based recruitment of diverse patients; (2) testing clinically useful interventions; and (3) collecting data on diverse outcomes relevant to patients, clinicians, health care system administrators, and policymakers. To improve the evaluation of health promotion and clinical interventions, strategies for implementing practical research studies have been outlined by Tunis et al13 and also by Glasgow et al14 using the RE-AIM framework. 9,15,16 The RE-AIM framework provides a system for integrating and reporting about patient populations (reach and effectiveness of the intervention with regard to the target population), about clinicians/settings (adoption and implementation of the interventions at these levels), and across time (maintenance of the interventions).
This article presents a case study of the IDEALL (Improving Diabetes Efforts Across Language and Literacy) Project, a practical clinical trial whose purpose is to gain an understanding of how to improve diabetes outcomes among primary care patients in a safety net system in San Francisco, CA. The study examines 2 interventions (compared with usual care) to enhance diabetes self-management support among patients with poor glycemic control. Patients from the San Francisco Department of Public Health’s Community Health Network (CHNSF) primary care clinics who may face barriers to high quality diabetes care because of language and literacy barriers represent the target population for this study. In this case study, we will discuss the strategies used to increase the reach and effectiveness potential of the IDEALL Project, as well as the trade-offs encountered when applying the principles of practical clinical trials and the RE-AIM framework. By presenting this case study, we hope to demonstrate that designing and conducting practice-based, translational research often involves a complex set of strategies, decisions, and trade-offs to adapt to the ecology of the practice settings while maximizing internal and external validity.
IDEALL Project
The IDEALL Project emanated from a recent strategic initiative to improve the quality of chronic disease care in CHNSF primary care clinics. There are 9 CHNSF clinics—these clinics represent a subset of the University of California San Francisco (UCSF), Department of Family and Community Medicine’s practice-based research network, the Collaborative Research Network. The IDEALL Project is a direct outgrowth of patient’s experiences and practitioners’ attitudes and recommendations previously described in 4 studies examining health literacy, patient-physician communication, and primary care provider attitudes and behavior in the CHNSF.17–20 These studies identified that: (1) limited health literacy and limited English proficiency are common barriers but are potentially remediable determinants of the quality of chronic disease care, (2) primary care physicians consistently overestimate their diabetes patients’ literacy levels, (3) the effectiveness of physicians’ office-based communication is variable, and (4) physicians believe that adjunctive, tailored disease management systems would significantly improve diabetes care for their patients with limited health literacy.
At the start of the IDEALL Project, there was no systematic diabetes disease management system in place in the CHNSF, and one of the project’s explicit goals is to inform future plans to develop such a system. The IDEALL Project is designed to assess the reach and effectiveness of 2 diabetes self-management support interventions. One is technologically oriented and the other interpersonally oriented, each tailored to the language and literacy needs of high-risk diabetes patients. Both interventions are compared with usual primary care and to each other in a 3-arm study design (see Figure 1). Both interventions are delivered in English, Spanish, and Cantonese, the 3 most commonly spoken languages among CHNSF patients. Eligible and consenting patients with diabetes in poor control are randomly assigned to an automated telephone diabetes self-management (ATDM) program or to group medical visits (GMV) or to usual care, and are followed for 1 year after enrollment.
Patients randomized to ATDM receive weekly automated telephone calls in their native language, which include a rotating set of diabetes self-care queries (eg, questions about medication adherence, diet, exercise, smoking, etc); a number of questions regarding psychosocial aspects of diabetes care (eg, current coping, depressive symptoms, etc); and referrals for preventive services (eg, visits to the ophthalmologist, etc). Patients respond to questions via touch-tone commands; those answering out-of-range on an item (based on predetermined thresholds) receive an immediate automated narrative health education message. Patients consistently answering out-of-range receive a call back from a language concordant nurse care manager. Those randomized to GMV are invited to participate in monthly group medical visits in their native language involving 7 to 10 diabetes patients, facilitated by a bilingual primary care physician and a health educator. A clinical pharmacist is also present to review medication changes or regimens. Each intervention is delivered as an adjunct to usual care. Patients randomized to usual care continue to receive standard diabetes care. Primary outcomes of the IDEALL Project include: (1) extent of patient engagement with the IDEALL interventions; (2) changes in diabetes indices—such as diabetes-related self-efficacy and glycemic control; and (3) relative resource use.
IDEALL Implementation Strategies to Increase External Validity
Table 1 presents translational research recommendations12–14,16,21 that address the areas of recruiting diverse patient populations and comparing clinically relevant interventions in practical clinical trials. The following is a description of the strategies and rationale we used in developing the IDEALL Project and the trade-offs that were made over the course of the Project.
Translational Research Goal I: Achieving Patient Diversity across Populations and Clinic Settings
IDEALL Strategy: Minimize Exclusions
Using electronic data from the CHNSF, we developed a diabetes registry of all patients who met the following eligibility criteria: diagnosis of adult, type 2 diabetes; one or more primary care visit in the previous 12 months; English, Spanish, or Cantonese speaking; did not have limited vision or were hearing-impaired; and no diagnoses of psychotic illness or end-stage renal disease. The registry also included the most recent laboratory value for hemoglobin A1c tests and the patient’s primary care provider and clinic. The database was developed by monthly merges of CHNSF spreadsheets of clinical, appointment, and laboratory data.
The Institutional Review Board (IRB) at UCSF and at CHNSF would not allow the IDEALL Project staff to directly contact eligible patients. Instead, patients could only be enrolled through provider referral or by approaching patients at routinely scheduled clinic visits. Given this constraint, we provided each clinician with a list of their eligible diabetes patients with check-off boxes for the following exclusion criteria: patient is no longer at this clinic/is not my patient, patient does not have diabetes, patient has end-stage renal disease, patient has major psychiatric illness and other, which was left blank for the clinician to complete (eg, patient is too ill to participate). Returned forms without boxes checked indicated clinician consent. Because there were few exclusion criteria, these forms took only a few minutes to complete—all but 2 of the 118 clinicians completed them. The last eligibility criterion related to suboptimal glycemic control; a hemoglobin A1c value of 8 in the previous 12 months was derived from registry data alone.
Trade-offs
The restrictions imposed on patient recruitment by the Institutional Review Boards presented a serious barrier to extending the reach of the interventions and shaped the study design to be clinic-based rather than population-based more generally, within the CHNSF. The original study design was to directly contact patients at home, including those who have poor glycemic control and who infrequently attend primary care clinics. To reach patients who came infrequently to the clinics, we modified our design to have research assistants approach patients at their primary care and specialty appointment visits over an extended enrollment period. However, this strategy only allowed for contact with active clinic patients, and did not allow for inclusion of patients too sick to travel to the clinic or who were only seen in the emergency department or hospitalized over the study period. A few of the exclusion criteria of the IDEALL study also restricted the diversity of the patient sample. For example, patients who faced transportation barriers because of illness or care-taking responsibilities, or who frequently traveled between San Francisco and their country of origin were not eligible to be enrolled in the study—possibly representing some of the more vulnerable patients cared for in the CHNSF.
Despite the obstacle presented by the Institutional Review Boards, the merging of CHNSF electronic medical information into the IDEALL database enabled us to define our study population and pursue a population-based recruitment strategy among CHNSF patients that were active in terms of clinic attendance. The languages in the study are the 3 most commonly spoken by diabetes patients receiving care in the CHNSF,7 enabling the IDEALL Project to reach many patients in this ethnically and linguistically diverse safety net system. Although the availability of the intervention in non-English languages is a strength of the study design, there are other languages, such as Tagolog and Vietnamese, that are spoken by CHNSF diabetes patients, many of whom face communication barriers in the health care setting.
Overall, 584 patients were approached, 339 enrolled (58%), 109 were ineligible at the time of enrollment (for reasons such as no telephone, frequent travel out of the area, limited ability to speak a study language, limited vision/hearing-impaired), and 78 refused (primarily for reasons related to not wanting to leave the house if randomized to the group medical visit arm, including difficulty with transportation, serious illness, or having care-taking responsibilities). Many of these patients stated that they would probably have enrolled if they could have selected the ATDM arm. An additional 58 patients were eligible but had not been enrolled by the end of the study.
IDEALL Strategy: Recruit Patients from Diverse Settings
Because we were not able to contact patients directly, we chose a clinic-based recruitment strategy that could be used in diverse clinic settings to reach large numbers of eligible patients. Clinics were chosen using a targeted selection strategy of reviewing clinic profiles of potentially eligible patients and ranking the clinics according to the proportion of patients with matching eligibility criteria.22 The top 2 sites, based at San Francisco General Hospital, agreed to participate. However, we were hesitant to conduct the study only at hospital-based clinics located in one geographic area of the city because we believed that community clinics reflected both different patient populations and different levels of access to specialty services. The next 4 highly ranked clinics were located in neighborhoods throughout town, but 2 clinics had diabetes group visits underway or were planning them and opted out. The other 2 sites participated. Because large numbers of Cantonese-speaking patients with diabetes were seen at one of these clinics (located in the Chinatown neighborhood), enrollment of Cantonese speakers was concentrated at this clinic to simplify the clinic-based recruitment process.
Trade-offs
We were able to recruit patients from diverse clinics within the CHNSF but did not extend the reach of the interventions beyond the 4 participating clinics or beyond this clinic system. Two of the participating clinics were neighborhood health centers that had not participated in previous studies and their inclusion greatly extended the reach of the interventions to these community settings. Establishing the study at these sites, however, provided logistic challenges regarding scheduling and registration for the group medical visits that consumed considerable staff resources.
Translational Research Goal II: Selecting Interventions That Are Clinically Relevant across Settings
IDEALL Strategy: Develop Interventions That Reflect Primary Care Realities
There was a clear demand for extra care for patients managing chronic illnesses within the CHNSF and the IDEALL interventions were developed with extensive patient and clinician input. However, we did not believe that the clinics would be able to independently initiate and sustain these interventions because of the planning, training recruitment, and ongoing project management required. To avoid burdening the clinics, the IDEALL study provided the interventions as adjunctive services to those available through the clinic.
Trade-offs
Using an adjunctive model allowed us to standardize the interventions across sites and to introduce them to practices without forcing their adoption. We believed that if the interventions were successfully received by patients and providers, there would be a gradual shift in favor of integrating these interventions into clinical care across CHNSF clinics, which would enable subsequent evaluation. As much as possible, we engaged clinic staff to participate in the self-management support program, although the primary responsibility for implementation resided with IDEALL staff.
The drawbacks of an adjunctive model are important and represent the largest trade-off in implementation of the IDEALL Project. Once support for the study ends, the interventions may disappear from the minds of the clinic staff and of patients, potentially jeopardizing adoption, implementation, and maintenance efforts with respect to self-management support. In addition, with an adjunctive model of care, patients and clinicians may not be convinced that the services offered with the IDEALL Project could be provided in the "regular" context of their care, and they may not have engaged with them to the extent they would if they were offered from within the clinic itself.
IDEALL Strategy: Compare Clinically Relevant Alternatives
The 2 study interventions reflect different models of self-management support, each with different strengths and weaknesses. The ATDM is more technologically oriented and does not require patients to attend extra visits. The GMV focuses on participatory learning, interpersonal support, and the reinforcement of skill development with peers. Both are clinically relevant for the following reasons: (1) prior evidence of efficacy from controlled trials in other settings23–25; (2) their potential to overcome literacy and language barriers and promote self-management in vulnerable populations 26,27; (3) preliminary evidence in other settings suggesting significant patient engagement ("reach"); (4) their focus on changes at the organization/practice and patient levels (as opposed to changes in physician behavior)28,29; (5) their emphasis on expanding the traditional patient-physician dyadic model to a multidisciplinary, team-based model30; (6) the relatively low cost; and (7) our opinion, informed by inquiries conducted in the CHNSF and other safety net systems18,19,31 that such interventions might be welcomed by clinicians and administrators.
Trade-offs
Several research designs were considered for the IDEALL Project. A 3-arm randomized clinical trial, with randomization at the patient level, was selected for several reasons. We believed that alternative designs, such as a crossover design allowing patients in different arms of the study to later crossover to the alternative study arm would be risky, both in terms of patient retention (patients would need to stay in the study twice as long) and in terms of determining which interventions to attribute observed effects. A crossover design would not allow for a control group, which we believed essential to determining whether either intervention would provide benefit.
The randomized design was not popular with some patients from several of the clinics we approached. We observed some refusals because of the randomized nature of the study; several patients expressed interest in participation, but only if they could select which study arm. This suggests that acceptance rates in a nonrandomized study context may even be higher. Similarly, some clinics approached were unwilling or unable to participate in the randomized controlled trial, limiting the reach of the study. We also selected to randomize patients, rather than having clinicians randomized to one of the 3 groups. Because one of the objectives of the IDEALL Project was to expose participating clinicians to both interventions, randomization of patients within clinician allowed us to maximize the interventions’ reach with the clinical provider population in the CHNSF.
Conclusions
In this case study, we describe how designing and conducting practical clinical trials often involves a complex set of strategies, decisions, and trade-offs aimed at adapting to the ecology of the practice settings while maximizing internal and external validity. Researchers engaging in translational research should recognize the importance of and complexity inherent to achieving such a balance and describe how their decisions with regard to design may influence the interpretation of results. We believe the decisions and trade-offs made in the IDEALL Project situates it in an intermediate zone between pure efficacy research and evaluation of a real world, fully integrated set of interventions. It can be viewed as in-between a stage III (examining efficacy) and stage IV (examining effectiveness) clinical trial as described by Dzewaltowski, Glasgow et al.9,14 We suspect that many practice-based translational research projects are similarly situated.
We encountered several challenges in implementing the study and believe trade-offs such as those described are unavoidable. First, it is likely that Institutional Review Boards will continue to focus on patient privacy protections. Population-based recruitment strategies may need to redefine eligible patients as those who are clinically active and accessible for recruitment, whether from direct provider referrals, provider-initiated invitations or through an appointment-based strategy, as with IDEALL. Second, choosing between fully integrated or adjunctive models for delivering intensive interventions, such as those suitable for self-management support as described here for diabetes, will require careful evaluation of the setting and clinician attitudes toward the interventions. In the IDEALL experience, despite previous research within the CHNSF that supported the development of the IDEALL study interventions, it was determined that a fully integrated delivery model might be premature, might place too great a burden to the clinics and could introduce too much variation in intervention content. Finally, decisions regarding critical details of study design, such as randomization at patient versus clinician versus clinic need to be made in the context of patient, clinician, and clinic preferences. We were fortunate that sufficient clinical sites and patients were identified as willing to accept randomization, and our design did try to minimize the likelihood that an individual would be randomized to usual care by having 2 intervention arms. As such programs become more widely disseminated, there is an urgent need to develop alternative study designs to increase acceptability.
As the IDEALL Project is nearing an end, there is evidence of the potential advantage of using such an intermediate stage design. The CHNSF clinical and executive leadership is planning an expansion and diffusion of IDEALL adjunctive models for diabetes and other chronic illnesses. To fully actualize this potential and inform the next phase of implementation, the extensive data we have collected need to be analyzed with respect to patient outcomes and relative resource use at the clinic and system level. Assuming some benefit is demonstrated, we then face the welcome challenge of integrating these models into routine clinical care in the near future, as part of a health system-wide commitment to improving self-management support.
Notes
Funding: This work was supported by AHRQ grants (R21HS013544 and R21HS014864-01) and the Commonwealth Fund. DS was also supported by a National Institutes of Health Mentored Clinical Scientist Award K-23 RR16539-03. Electronic data in the CHNSF were made available through the National Center for Research Resources Grant MO1RR0083-41.
Conflict of interest: none declared.
Received for publication June 29, 2005. Revision received November 8, 2005. Accepted for publication November 9, 2005.
References
The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med 1993; 329: 977–86.
The UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 1998; 352: 837–53.
Sadur C, Moline N, Costa M, et al. Diabetes management in a health maintenance organization. Efficacy of care management using cluster visits. Diabetes Care 1999; 22: 2011–7.
Lorig K, Sobel D, Stewart A, et al. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: a randomized trial. Med Care 1999; 37: 5–14.
Piette J, Weinberger M, McPhee S. The effect of automated calls with telephone nurse follow-up on patient-centered outcomes of diabetes care: a randomized, controlled trial. Med Care 2000; 38: 218–30.
Trento M, Passera P, Tomalino M, et al. Group visits improve metabolic control in type 2 diabetes: a 2-year follow-up. Diabetes Care 2001; 24: 995–1000.
Regenstein M, Huang J, Cummings L, Lessler D, Reilly B, Schillinger D. Caring for patients with diabetes in safety net hospitals and health systems: The Commonwealth Fund; 2005. p. 11–12.
Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. National Healthcare Disparities Report. Rockville, Maryland July, 2003.
Dzewaltowski DA, Estabrooks PA, Klesges LM, Bull S, Glasgow RE. Behavior change intervention research in community settings: how generalizable are the results? Health Promot Int 2004; 19: 235–45.
Glasgow RE. Translating research to practice: lessons learned, areas for improvement, and future directions. Diabetes Care 2003; 26: 2451–6.
Murphy D, Chapel T, Clark C. Moving diabetes care from science to practice: the evolution of the National Diabetes Prevention and Control Program. Ann Intern Med 2004; 140: 978–84.
Garfield SA, Malozowski S, Chin MH, et al. Considerations for diabetes translational research in real-world settings. Diabetes Care 2003; 26: 2670–4.
Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003; 290: 1624–32.
Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care 2005; 43: 551–7.
Glasgow RE, Goldstein MG, Ockene JK, Pronk NP. Translating what we have learned into practice. Principles and hypotheses for interventions addressing multiple behaviors in primary care. Am J Prev Med 2004; 27(2 Suppl): 88–101.
Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health 2003; 93: 1261–7.
Schillinger D, Piette J, Grumbach K, et al. Closing the loop: physician communication with diabetic patients who have low health literacy. Arch Intern Med 2003; 163: 83–90.
Schillinger D, Bindman A, Wang F, Stewart A, Piette J. Functional health literacy and the quality of physician-patient communication among diabetes patients. Patient Educ Couns 2004; 52: 315–23.
Schillinger D, Grumbach K, Piette J, et al. Association of health literacy with diabetes outcomes. JAMA 2002; 288: 475–82.
Seligman HK, Wang F, Palacios JL, et al. Physician notification of their diabetes patients’ limited health literacy. A randomized, controlled trial. J Gen Intern Med 2005; 20: 1001–7.
Godwin M, Ruhland L, Casson I, et al. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol 2003; 3: 28.
Glasgow RE, Whitlock EP, Eakin EG, Lichtenstein E. A brief smoking cessation intervention for women in low-income Planned Parenthood clinics. Am J Public Health 2000; 90: 786–9.
Piette J. Interactive voice response systems in the diagnosis and management of chronic disease. Am J Managed Care 2000; 68: 817–27.
Piette J, Weinberger M. Do automated calls with nurse follow-up improve self-care and glycemic control among vulnerable patients with diabetes? Am J Med 2000; 108: 20–7.
Sadur C, Moline N. Diabetes management in a health maintenance organization. Efficacy of care management using cluster visits. Diabetes Care 1999; 22: 2011–7.
Rudd R, Comings J. Learner developed materials: an empowering product. Health Educ Q 1994; 21: 313–27.
Schillinger D. Improving the quality of chronic disease management for populations with low functional health literacy: a call to action. Disease Manag 2001; 4: 103–9.
Cabana M, Rand C, Powe N, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282: 1458–65.
Davis D, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ 1997; 157: 408–16.
Grumbach K, Bodenheimer T. Can health care teams improve primary care practice?. JAMA 2004; 291: 1246–51.
Fernandez A, Schillinger D, Grumbach K, et al. Physician language ability and cultural competence: an exploratory study of communication with Spanish-speaking patients. J Gen Intern Med 2004; 19: 167–74.(Margaret A. Handley, PhD,)