Patient Follow-up in an Urban Resident Continuity Clinic: An Initiative to Improve Scheduling Practices

Editor's Note: The online version of this article contains a Figure: Depicting the Impact of a New Form on Scheduling Workflow in an Urban Resident Continuity Clinic, a sample appointment request form, and the study design.

Missed follow-up appointments (“no-shows”) in primary care are a well-studied problem, with demonstrated negative clinical impact,1–3 patient-level risk factors,4–7 and effective systemic interventions.8–16 By contrast, never-scheduled follow-up appointments remain poorly described. Scheduling effectiveness has been evaluated in specialty clinics,17 but rates of recommended but never-scheduled follow-up appointments in primary care have not been documented. Scheduling follow-up is particularly difficult for resident-run clinics serving disadvantaged populations. Resident schedules may not be available in advance, patients may lack stable contact information, and underresourced clinics may have unreliable scheduling systems. Unlike patient no-shows, never-scheduled follow-up appointments may not come to providers' attention. Failure to schedule follow-up appointments may therefore have serious consequences for patient health, provider continuity, and health care usage. Improvement of scheduling practices in resident continuity clinics provides a unique opportunity to improve a poorly studied and clinically important process while addressing the Accreditation Council for Graduate Medical Education (ACGME) core competencies of practice-based learning and improvement, interpersonal and communications skills, and systems-based practice.

The General Medicine Clinic of San Francisco General Hospital is a safety-net primary care clinic exclusively serving uninsured or publicly insured patients. We evaluated all patients seen by second- or third-year residents in their primary continuity clinics over a 3-month period (October to December 2009). Thirty-two internal medicine resident providers (15 second-year and 17 third-year) were included; each provider's continuity clinic occurs on one afternoon (Monday, Tuesday, or Thursday) per week.

Because open-access scheduling for residents leads to panel discontinuity, patients are scheduled using set appointments only. Follow-up appointments are scheduled using handwritten 8.5×4-inch slips with provider-specified follow-up requests (eg, “return to clinic 2–3 months”). After completing their clinical encounters, patients take these slips to the front desk for scheduling by clerical staff. If the provider's schedule is open through the requested follow-up time (generally 2 to 3 months, depending on inpatient scheduling), the patient is given a follow-up appointment immediately. If the resident's schedule is not open, the scheduler keeps the slip until the provider's schedule becomes available from the residency program (online supplementary figure 1).

Resident providers at the General Medicine Clinic participate in a quality improvement (QI) curriculum based on the American Board of Internal Medicine's Practice Improvement Modules, consisting of four 1-hour sessions per year. During these sessions, residents learn the principles of QI (eg, plan-do-study-act cycles, process diagramming, team-based problem solving) and are provided structure and support for individual or joint clinic-based QI projects. When residents discussed potential QI targets, clinic patients never receiving follow-up appointments emerged as a leading theme. After speaking with clinic leadership, one resident (D.W.D.) solicited informal input from other residents and organized a joint meeting of the clinic's medical directors, nurse manager, clerical supervisor, and members of the clerical staff. This meeting took the form of group brainstorming and consensus building, with all parties suggesting ideas for process improvement and holding veto power over proposals that posed a disproportionate burden on their constituency. Discussions emphasized feasibility for a resident-led project (including a rapid implementation/evaluation cycle and a low time/resource burden), fairness, and measurable outcomes. Consensus was reached on replacing the current blank 8.5×4-inch scheduling slip with a new appointment request form (online supplementary figure 2). This form aimed to increase providers' scheduling efficiency, reduce illegible instructions, and minimize misplacement (online supplementary figure 1), with minimal alteration to the clinic's existing workflow. The lead resident designed the draft form, which was modified by all participants before finalization.

Our primary endpoint was the proportion of patient encounters with never-scheduled follow-up, defined as no primary care follow-up appointment with a medical provider (physician or nurse practitioner) scheduled at 1 month after the latest provider-requested follow-up date. We structured our analysis to simultaneously evaluate 2 components of the intervention. To evaluate clinic-wide awareness raising, our primary analysis compared never-scheduled follow-up during the postintervention period versus the preintervention period (online supplementary figure 3). To evaluate the form itself, our secondary analysis compared 2 clinics in which the form was implemented to 1 clinic in which it was not (online supplementary figure 3). Statistical significance was determined using the Fisher exact test, Wilcoxon rank-sum test, or logistic regression as appropriate. Multivariable analyses simultaneously adjusted for all measured variables. We calculated relative risks and associated confidence intervals using log-binomial regression.18 Significance of monthly trends was evaluated with the Cuzick nonparametric test. Variance estimates were inflated to account for clustering of multiple visits by individual patients.

This study was approved by the joint Institutional Review Board of the University of California, San Francisco, and San Francisco General Hospital and was funded by internal quality-improvement funds.

Beginning in October 2009 (month 1), schedulers saved all appointment requests for analysis. On November 1, 2009, the clinic launched the program, using new scheduling request forms for Monday and Thursday clinics (new-form arm) and continuing handwritten slips on Tuesdays (blank-slip arm). All providers participated in brief interactive discussions highlighting never-scheduled appointments; Monday and Thursday providers were also oriented to the new forms.

Over 3 months, residents had 1138 encounters with 1017 patients, requesting 4-month (or closer) follow-up for 910 (798 patients). More than 80% of patients were nonwhite, and 46% were nonnative English speakers (table 1). There were no significant patient demographic differences in the preintervention versus postintervention periods, but because of resident block scheduling, the postintervention period included more third-year and categorical residents (table 1). The requested follow-up period was between 2 and 4 months for 75% of analyzed encounters.

The proportion of never-scheduled appointments was 18.8% (95% confidence interval [CI], 14.5%–23.9%) in month 1, 15.2% (11.3%–19.8%) in month 2, and 11.4% (8.1%–15.5%) in month 3. Never-scheduled follow-up was significantly higher in the preintervention than postintervention period (relative risk [RR], 1.49 for month 1 versus months 2–3; 95% CI, 1.08–2.03; P  =  .02), with no postintervention difference between the 2 arms (P  =  .93). Patients with less frequent follow-up (RR for ≥2 months in the new-form arm, 2.83; 95% CI, 1.46–5.46; P  =  .002) and unavailable request forms (RR, 1.81; 95% CI, 1.20–2.71; P  =  .004) were significantly more likely to have never-scheduled follow-up (table 2). The new-form, but not blank-slip, arm demonstrated a significant month-over-month decline in never-scheduled appointments (multivariable RR [95% CI] 0.70 per month [0.55–0.89]; P  =  .003; versus 0.84 [0.57–1.22]; P  =  .35; figure).

Among 910 patient encounters in an urban resident-run clinic, the proportion of never-scheduled appointments dropped from 18.8% to 11.4% over 3 months during a resident-facilitated quality-improvement initiative. This team-based intervention addressed core competencies in graduate medical education while also improving scheduling practices.

Given that scheduling rates improved postintervention (primary analysis), but not between new-form versus blank-slip arms (secondary analysis), we hypothesize that the quality-improvement process had greater impact than the scheduling form's content. This process involved physicians, scheduling staff, and clinic leadership in joint brainstorming and consensus building, which likely increased awareness and communication among all parties.

This resident-led project may serve as an exemplar for addressing the ACGME core competencies of practice-based learning and improvement, interpersonal and communications skills, and systems-based practice. In contrast to inpatient rotations, continuity clinics offer an opportunity to learn systems of practice, develop team-based solutions, and evaluate process-based interventions over multiple years. Residency programs should consider dedicating more time and funding to resident-led quality initiatives in continuity clinics.

This study has key limitations. First, in this nonblinded temporal analysis, the observed reduction in never-scheduled appointments may have been caused by unmeasured confounders or secular time trends, rather than the intervention itself. Second, although relevant to resident clinics serving the urban poor, our study may not generalize to other primary care settings or to patients followed less frequently than every 4 months. Third, to complete the project (conceptualization, organization, form development, ethics approval, implementation, analysis, and dissemination) in the course of a resident QI curriculum, we limited our sample size and duration of analysis. Demonstration of long-term impact, process sustainability, and small statistical effect is challenging in resident-led initiatives, and future studies should evaluate “handoffs” of QI projects between residents to evaluate these important outcomes.

Failure to schedule follow-up appointments is a potential problem of underrecognized importance in primary care clinics. We describe a model resident-led, team-based scheduling intervention, during which the proportion of patients with never-scheduled follow-up dropped from 18.8% to 11.4%. This initiative may serve as a model for addressing core competencies of practice-based learning, communication, and systems-based practice in graduate medical education through resident-led, team-based projects based in resident continuity clinics.