Fulfilling a statutory responsibility to protect the public within their jurisdictions, state medical boards have been working with the Federation of State Medical Boards (FSMB) and collaborating organizations to thoughtfully explore pathways and procedures by which Maintenance of Licensure (MOL) may be implemented for physicians in the years ahead. As a better understanding emerges of the types of continuing medical education (CME) and continuous professional development (CPD) activities physicians already engage in, and the resources that may be necessary for state boards to meaningfully implement MOL, questions have sometimes arisen about the value of these activities in contributing to quality health care and improved patient outcomes. Though MOL has not yet been formally implemented, there is a growing body of compelling evidence and rationale for the educational activities that could meet a state board's requirements for MOL. This article summarizes the recent literature on the subject, including CME and CPD, and recent policy statements of organizations and thought leaders from the house of medicine.

“When I graduated from Cornell University Medical College in 1941, I thought I was brimful of knowledge, ready to be a doctor and to bring my skills to patients. Little did I know that my learning had just begun and, even more important, that much that I had been taught would soon become obsolete if in fact it hadn't been wrong in the first place.”

—C. Everett Koop, MD (2011)1 

The quality of health care delivered in the United States has an immediate and long-term impact on the quality of life, livelihood, morbidity and mortality of its residents and on the nation's economy and national security. There are many stakeholders (e.g., patients, physicians, hospitals, health insurance plans) within our costly health care system. At the most fundamental level of health care delivery, however, what matters most is the interaction that takes place between a licensed and qualified health care provider and a patient seeking health care.

As the sole entity charged with overseeing the licensure and regulation of physicians, state medical boards have long recognized their responsibility to protect the public and promote quality health care by ensuring that only qualified individuals receive a license to practice medicine and deliver health care.2 In further fulfillment of their responsibility to protect the public, most state boards adopted continuing medical education (CME) requirements for physicians beginning in 1971 as part of the license renewal process. Several boards are now considering Maintenance of Licensure (MOL), a system of continuous professional development (CPD) that includes practice-relevant CME, to better assure the public that only those physicians who remain up-to-date in their area of practice retain that privilege.

This article reviews the origins of MOL, highlights some of the recent literature about CME and CPD, and summarizes the evidence and rationale underpinning the foundation for the FSMB's MOL efforts. It includes references to quality improvement activities of organizations across the continuum of medical education in the United States and abroad.

In the United States, the practice of medicine has long been considered a privilege granted by the public through their elected representatives. State and territorial medical boards are charged through each of their Medical Practice Acts to protect the public from the unprofessional, improper, incompetent, unlawful, fraudulent and deceptive practice of medicine. Every Medical Practice Act is built upon this premise and each state board uses rigorous criteria (e.g., graduation from medical school, training in an accredited graduate medical education program in the United States, and passage of a medical licensure examination such as the USMLE or COMLEX-USA) to assess a physician's competence and fitness to practice prior to granting initial licensure.

When an applicant for initial licensure provides evidence of successfully meeting such requirements, state medical boards — and by extension, the public — can be confident that the physician has the requisite knowledge and skills to practice medicine competently and safely. Licensure renewal, by contrast, has long been an administrative function that assumes licensees are fit to practice medicine unless adverse events, complaints or negative outcomes indicate otherwise. While the licensure renewal processes carried out by state boards are a central component of public protection, they reflect a reactive, rather than a proactive, evaluation of physicians' continued fitness to practice.

FSMB Board Chair Henry Cramblett declared in 1980, “The Federation holds firm to the premise not only that a person must show evidence of being thoroughly educated, highly qualified, and staunchly ethical in order to receive a medical license, but that these characteristics must be displayed continuously throughout that physician's active career.”3 Over the last quarter century or so there have been increasing calls from outside the state medical regulatory community in support of more robust requirements for renewal of medical licensure and more proactive evaluation of physicians' continued fitness to practice as part of the license renewal process. The nature of the physician-patient relationship has also changed, in part because of the public's awareness that there are limitations of medical knowledge and the availability of multiple, if not conflicting, treatment options and pathways.4 

In 1995, the Pew Charitable Trust recommended that states “require each licensing board to develop, implement and evaluate continuing competency requirements to assure the continuing competence of regulated health care professionals.”5 Similarly, a series of Institute of Medicine (IOM) reports beginning in 2000 addressed the importance of quality health care and patient safety, stressing the need for better regulation (including licensure) across the continuum of medical education, training and practice.6,7,8 The IOM's 2012 report, Best Care at Lower Cost, recommended an expanded commitment to the goals of a “continuously learning health care system” from leaders in health care, advising them to “incorporate basic concepts and specialized applications of continuous learning and improvement into health professions education; continuing education; and licensing…requirements.” 9 

Recent studies of the quality of health care delivered in the United States and how it may be related to clinical experience and skills lend further support for more proactive evaluation of physicians' continued fitness to practice as part of the license renewal process.

In a random sampling of adults living in 12 metropolitan areas about their health care experiences, a survey that included a review of these individuals' medical records, McGlynn and colleagues found that participants received only 55% of recommended health care, a finding that the authors said, “poses serious threats to the health of the American people.”10 In a widely quoted article from 2005 by Niteesh Choudhry and colleagues, a systematic review of the relationship between clinical experience and quality of health care found that physicians who have been in practice longer may be at risk for providing lower-quality care and that this subgroup of physicians may benefit from quality improvement interventions.11 Similarly, Epstein and colleagues found that psychiatrists in practice for 11 to 20 years had mean accuracy scores in their ability to diagnose major depression 10.5% points lower than those in practice for 0–5 years, and psychiatrists in practice for more than 20 years had scores 12.5% points lower than those in practice 0–5 years.12 

In 1994, Paul Caulford and colleagues evaluated physicians referred to the Physician Assessment Center at McMaster University in Hamilton, Ontario.13 Physicians were evaluated in seven skill areas: knowledge, communication, interviewing skills, history taking, physical exam, problem solving, management and record-keeping. The results of the study found that age, graduation year, solo versus group practice, reported CME hours and specialty certification status had significant simple correlations with competence in these areas. A multivariate analysis found that the only significant predictors of competence were age and specialty certification status.

Thus, while the conventional wisdom holds that physicians get better at the practice of medicine with age and experience, this is not unequivocally borne out by the evidence. The aforementioned studies instead demonstrate that as the rate of expansion of medical knowledge and advancements in technology continue to increase exponentially, it becomes increasingly critical that physicians actively engage in lifelong learning in their area of practice. Throughout medical school and postgraduate training, the next generation of physicians is being encouraged to embrace lifelong learning as an integral part of professionalism. Such efforts are also becoming commonplace for physicians already in practice. In either case, failure to keep pace with advances in knowledge, skills and technology has the real potential to negatively impact healthcare quality and patient safety.

In order to address the need for physicians to stay abreast of changes in the practice of medicine, most state medical boards in the 1970s and 1980s implemented mandatory continuing education requirements for license renewal. Continuing medical education was first established as a national requirement for physicians by the American Academy of General Practice (now the American Academy of Family Physicians), which has required CME for membership since 1947.14 The organization today notes on its website that “the responsibility for providing comprehensive and continuing health care to patients carries with it the responsibility to continue learning” and that “the need to keep abreast of the rapid expansion of medical knowledge necessitates CME.”

Physicians commonly spend an average of 50 hours per year in CME activities, which are required for renewal of medical licensure by nearly all of the state medical boards and are often a requirement for hospital admitting privileges and participation in many health insurance plans. Many state boards also require physicians to obtain a specified number of CME credit hours and/or content-specific CME in particular areas (e.g., Infection Control, Responsible Opioid Prescribing, End-of-Life Care). Few licensing jurisdictions, however, require licensees to take CME that is directly related to their scope or area of practice. The FSMB's MOL framework, as articulated by an MOL Implementation Group in 2011, recommends that as part of a state's adoption of MOL it require that a majority (i.e., at least half) of the required CME be in a physician's area of practice.

Early studies, including a systematic Cochrane review, examining the value of CME found that short courses and conferences have little direct impact on professional practice, though they suggested that more effective methods such as systematic practice-based interventions, interaction with opinion leaders and multifaceted activities could change physician performance.15,16 In the late 1990s, this observation was supported by studies that demonstrated evidence that “interactive CME sessions that enhanced participant activity and provided the opportunity to practice skills can effect change in professional practice and, on occasion, health care outcomes.”17 

In a more recent review of more than 130 articles related to the effectiveness of CME — selected from 68,000 citations identified in a comprehensive literature search — Marinopoulos and colleagues determined that while the quality of the evidence was “low” and firm conclusions were not possible, “CME appears to be effective at the acquisition and retention of knowledge, attitudes, skills, behaviors and clinical outcomes.”18 A promising 10-year investigation of evidence-based clinical practice in primary care, utilizing educational materials provided to physicians about lipid-lowering recommendations and an intervention group of physicians who received periodic lectures and case-based training, demonstrated substantially decreased mortality in patients with coronary heart disease among the intervention group (22%) compared with patients whose physicians only received educational materials (44%).19 

Continuing medical education, especially when it is interactive and directly related to a physician's practice, is an important part of MOL and will likely remain so in the years ahead, as long as there continues to be evidence in support of its potential and its impact on quality health care delivery and patient outcomes. Maintenance of Licensure programs will rely on efforts by educators and regulators alike to apply evidence-based approaches to new types of CME, such as simulation-based CME, that support translating knowledge into practice. A recent example of such an approach is a randomized controlled study of Internet-based CME that concluded that appropriately designed, evidence-based online CME can produce objectively measured changes in behavior as well as sustained gains in knowledge that are comparable or superior to those realized from effective, live interactive CME workshops.20 

There have been recent calls for the further reform of CME. For example, the IOM in its December 2009 report, Redesigning Continuing Education in the Health Professions, endorsed a new vision for CME based on CPD that emphasizes identification of learning needs, development of a learning plan and acquisition of lifelong learning and skills.21 This is consistent with the AMA's definition of CPD, which incorporates “the wider arena of skills and specialized education, including but not limited to cognitive knowledge, that physicians employ in the delivery of patient care.”22 

The National Institutes of Health (NIH) and the Agency for Healthcare Research and Quality (AHRQ) have now also placed greater emphasis on translating scientific knowledge into clinical practice.23 The new models of CME share some of these same basic goals, particularly behavioral change and systems redesign to improve patient outcomes.24 For example, Performance Improvement Continuing Medical Education (PI CME) was introduced about a decade ago as an attempt to address quality-improvement concerns, and the model has expanded significantly since then. In 2011, 44,275 physicians and 7,492 non-physician health professionals participated in more than 500 PI CME activities, according to the Accreditation Council for Continuing Medical Education (ACCME).25 The processes for PI CME have been defined by the American Medical Association (AMA) and the American Academy of Family Physicians (AAFP), are recognized by the ACCME, and involve three basic steps: an assessment of the physician's practice using nationally identified evidence-based performance measures (benchmarks), implementation of an intervention and reevaluation of those performance measures to gauge improvement. The AMA has since expanded its Physician Recognition Award for Category 1 Credit to include PI CME activities, Internet-based learning and point-of-care learning. As it began to introduce PI CME, the AMA also convened the Physician Consortium for Performance Improvement (PCPI) in order to enhance quality and patient safety and foster accountability. This consortium focuses on the development, testing and implementation of evidence-based performance measures for use at the point of care and aims to advance the science of clinical performance measurement and improvement.26 

For osteopathic physicians, the equivalent of PI CME is the American Osteopathic Association's (AOA) Clinical Assessment Program (CAP), first offered as a CME activity in 2005 and which has the same three-step structure as PI CME. While the FSMB's MOL framework does not mandate that physicians engage in such activities, it encourages state medical boards to include PI CME and CAP, which it mentions by name, as educational options by which a physician may be able to comply with one or more MOL components.

The interface between physician self-assessment and CME, which was part of the FSMB's MOL discussions, has also been explored recently. Most CME activities rely on the individual physician to determine gaps in his or her knowledge through a subjective self-assessment and to select the appropriate CME activities to remedy any perceived deficiencies. While physicians may find this type of self-assessment to be of value, research has shown that “physicians have a limited ability to accurately self-assess.”27 Eva and Regehr argue that “it is time to move beyond the rhetoric that self-assessment as a general, personal, unguided judgment of ability should be taught and developed as a valid basis on which to direct performance improvements.”28 Particularly relevant to the development of MOL is their conclusion that “for maintenance of competence efforts to be in any way meaningful, external feedback is essential.”29 The FSMB's MOL recommendation is consistent with these conclusions and suggests a continuous feedback loop for physicians that helps them select educational activities. Not only will physicians participating in MOL be able to engage in a process of self-assessment that is objective and externally verified, they should be able to receive important practice-related feedback and data as part of their engagement in MOL Components 2 and 3 (Assessment of Knowledge and Skills, and Performance in Practice) that better inform their selection of further educational activities.

The movement toward MOL is consistent with the increasing and ongoing efforts in recent years by organizations throughout the health care system — from those representing hospitals to specialty boards — to develop elaborate and meaningful new systems of quality measurement and improvement. While the MOL framework is also a new system, to the extent that it has not existed before, it differs from the others because it primarily seeks to recognize physicians already engaged in these new systems of quality measurement and improvement activities and to “raise the floor” of clinical competency rather than mandate an entirely new series of measurements. A brief overview of some of these new systems is provided below.

The American Board of Medical Specialties (ABMS) and all of its member specialty boards have now adopted its Maintenance of Certification (MOC) program, which requires specialty-certified physicians to provide evidence of meeting the following criteria on a continual basis in order to maintain their specialty certification status: medical licensure and professional standing, lifelong learning and self-assessment, cognitive expertise and practice performance assessment. The ABMS and its specialty boards are actively conducting research to evaluate the impact of specialty board certification and participation in MOC activities on patient care, with growing evidence of its utility in improving patient outcomes.30–34 The AOA's Bureau of Osteopathic Specialists implemented in January 2013 its Osteopathic Continuous Certification (OCC) program, which requires its specialty-certified physicians to provide evidence of meeting the following criteria on a continual basis in order to maintain specialty certification status: medical licensure, lifelong learning and CME, cognitive assessment, practice performance assessment and improvement and continuous AOA membership.

The FSMB's framework for MOL recommends to state boards that physicians already engaged in robust CPD and lifelong learning activities in their area of practice, such as the ABMS's Maintenance of Certification (MOC) program or the AOA Bureau of Osteopathic Specialists' Osteopathic Continuous Certification (OCC) program, be recognized by state boards as being “substantially in compliance” with any MOL program.35 It is important to note, however, that state boards have not required specialty certification, or participation in specialty recertification activities, for medical licensure. The MOL framework, likewise, does not recommend that state boards mandate all actively licensed physicians to obtain, or maintain, specialty certification in order to maintain their license.36 Other CPD and educational activities acceptable to state boards for MOL have been and will continue to be identified, especially for the more than 230,000 actively licensed physicians who never were — or are no longer — specialty certified or who have lifetime specialty certification.37 The National Commission on Certified Physician Assistants (NCCPA) and the American Academy of Physician Assistants (AAPA) are engaged in defining “certification maintenance” activities that all certified physician assistants (PA-C) will be required to complete every six years to maintain their certification.38 

The Accreditation Council on Graduate Medical Education (ACGME), as part of its mission to ensure and improve the quality of graduate medical education, began implementation in 1999 of its Outcomes Project. While the accreditation of its GME programs historically focused on the potential of the program to effectively educate and train its residents and fellows, the Outcomes Project focused on the actual accomplishments of the program through an assessment of its outcomes.

In 2008, the Outcomes Project was expanded into a program called Milestone Development, in which each specialty is responsible for identifying its physicians' milestones of competency development during training. By July 2013, the ACGME plans to also begin a phased implementation of the Next Accreditation System (NAS), which aims to enhance the ability of the peer-review system to prepare physicians for practice in the 21st century, to accelerate the ACGME's movement toward accreditation on the basis of educational outcomes, and to reduce the burden associated with the current structure and process-based approach.39 

The Joint Commission, which accredits hospitals, implemented new credentialing and privileging standards in 2007 and 2008 which were intended to make the credentialing and privileging process more objective and evidence-based by facilitating continuous monitoring of physicians' performance and by providing a basis for intervening when quality-of-care concerns are identified. The American Hospital Association's (AHA) Physician Leadership Forum last year released a white paper, “Lifelong Learning — Physician Competency Development,” that examines the core competencies needed to deliver coordinated, team-based, value-driven care and includes recommendations for hospitals and physician-associated organizations to develop these skills in the current and next generation of physicians. “Licensing boards,” the report said, “should also consider stronger focus on the core competencies as part of the licensing process.”40 

In 2008, the Council of Medical Specialty Societies (CMSS) released a primer, The Measurement of Health Care Performance, which summarizes recent discussions and interest in quality improvement, outcomes measures, practice measurement and the validity and integrity of physician performance. The National Committee for Quality Assurance (NCQA) has also dedicated itself to measuring, evaluating and improving the quality of health care in the United States through tools such as the Healthcare Effectiveness Data and Information Set (HEDIS) and health plan “report cards.”

The recent growth of Patient-Centered Medical Homes (PCMH) and Accountable Care Organizations (ACOs) all have at their core a commitment to continuous improvement and evidence-based outcomes, with the ultimate goal of better serving the needs of patients. In addition, the IOM's To Err Is Human report specifically challenged state medical boards to do their part in making health care safer for patients by periodically assessing providers “based on both competence and knowledge of safety practices.”41 An additional impetus for state medical and osteopathic boards to embrace change and improvement in medical regulation is the concern that if they don't, others may do so on their behalf or in their place. Medical regulation outside the bounds of state licensing authority could in turn, as one observer notes, lead to damaging effects to patients and society.42 

Internationally, the College of Physicians and Surgeons of Ontario (CPSO) evaluates the continuing competence of its licensees through its Peer Assessment Program, which it initiated in 1981. As part of the program, physicians undergo an office-based evaluation of their facilities, medical records and quality of care once every 10 years. The General Medical Council (GMC) of the United Kingdom launched Revalidation, its version of MOL, for all of its physicians in December 2012, becoming the first nation in the world to formally implement such a program. Though Revalidation's goals are similar to MOL, it includes a very different set of requirements for physicians. Because MOL has not yet been implemented in the United States, and Revalidation has only recently been initiated in the U.K., formal outcomes data are not yet available for either program. However, the FSMB, GMC and medical regulators in other nations contemplating variations of MOL have pledged to share lessons learned alongside the opportunities and challenges that are identified as best practices in licensure renewal.

Finally, efforts by public and consumer-oriented organizations and websites such as Consumers Union, Healthgrades.com and Angieslist.com to disseminate consumer-focused information about physicians also highlight a growing interest on the part of the public for information about their physicians, including the status of their credentials and the quality of care they provide. In 2007 the American Association of Retired Persons (AARP), in collaboration with Citizen Advocacy Center (CAC), conducted a survey of the residents of Virginia who were 50 years of age or older to assess their understanding and knowledge of Virginia's existing state medical licensure requirements. More than 95 percent of respondents said they believe that health care professionals should be required to show they have up-to-date knowledge and skills needed to provide quality care as a condition of retaining their medical license.43 

The adoption by the House of Delegates of the FSMB of a framework for MOL in 2010 was an important milestone that recognized emerging research in the area of physician education and professional development, as well as the cultural shift that was already occurring across the house of medicine. The adoption of a framework for MOL set into motion a desire on the part of state medical boards, articulated in a policy statement in 2004, to better support their “obligation to the public” to ensure the continuing competence of physicians as a condition of license renewal.44 

The FSMB's MOL framework recommends that state boards require physicians seeking license renewal to provide evidence of participation in a program of continuous professional development that reflects the three major components of what is known about effective lifelong learning: 1. Reflective self-assessment, 2. Assessment of knowledge and skills, and 3. Performance in practice. By design, the MOL framework does not specify the details of a continuous professional development program, instead suggesting a system that state boards may wish to consider that enables physicians in their jurisdiction to demonstrate through a selection of reasonable educational options in their area of practice that they are meaningfully engaged in these activities. The MOL framework also does not mandate a secure, high-stakes examination for compliance with any of its components, although a physician may elect such an option if desired.

While MOL is still years away from implementation by any state medical board, several state boards have been working with the FSMB and collaborating organizations to look at the operational and logistical aspects of a program that is designed to support a physician's commitment to lifelong learning and assure multiple stakeholders of the enduring value of the hard-earned license to practice medicine. As MOL advances, state boards, the FSMB and collaborating organizations will need to be mindful of what adult learning theorist Marsha Speck has described as a basic principle to be considered when professional development activities are designed: “Adults will commit to learning when the goals and objectives are considered realistic and important to them. Application in the ‘real world’ is important and relevant to the adult learner's personal and professional needs.”45 

1.
Koop
CE.
Foreword
.
In
:
Wentz
,
DK
,
ed.
Continuing Medical Education: Looking Back, Planning Ahead
.
Dartmouth College Press
:
Hanover
.
2011
.
Page
ix
.
2.
Johnson
DA
,
Chaudhry
HJ.
Medical Licensing and Discipline in America
.
Lanham, MD
:
Lexington Books
.
2012
.
3.
Cramblett
HG.
National Policies for Medical Licensure through the Federation of State Medical Boards
.
NEJM
.
303
(
23
):
1360
1363
.
December
4
,
1980
.
4.
Lueddeke
GR.
Transforming Medical Education in the 21st Century
.
London
:
Radcliffe Publishing
.
2012
.
Page
109
.
5.
Pew Health Professions Commission
.
Reforming Health Care Workforce Regulation: Policy Considerations for the 21st Century. San Francisco, California
.
1995
.
6.
Kohn
LT
,
Corrigan
J
,
Donaldson
MS.
To Err Is Human: Building a Safer Health System
.
Washington, DC
:
National Academy Press
.
2000
.
7.
Institute of Medicine
.
Committee on Quality of Health Care in America
.
Crossing the Quality Chasm: A New Health System for the 21st Century
.
Washington, DC
:
National Academy Press
.
2001
.
8.
Institute of Medicine
.
Committee on Health Professions Education Summit
.
Health Professions Education: A Bridge to Quality
.
Washington, DC
:
National Academy Press
.
2003
.
9.
Institute of Medicine
.
Best Care at Lower Cost: The Path to Continuously Learning Health Care in America
.
Washington, DC
:
National Academy Press
.
2012
.
10.
McGlynn
EA
,
Asch
SM
,
Adams
J
,
Keesey
J
,
Hicks
J
,
DeCristofaro
A
,
Kerr
EA.
The Quality of Health Care Delivered to Adults in the United States
.
NEJM
.
348
(
26
):
2635
2645
.
11.
Choudhry
NK
,
Fletcher
RH
,
Soumerai
SB.
Systematic Review: The Relationship between Clinical Experience and Quality of Health Care
.
Ann Intern Med
.
2005
.
142
:
260
273
.
12.
Epstein
SA
,
Gonzales
JJ
,
Weinfurt
K
,
Boekeloo
B
,
Yuan
N
,
Chase
G.
Are Psychiatrists' Characteristics Related to How They Care for Depression in the Mentally Ill? Results from a National Case-Vignette Survey
.
Psychosomatics
.
2001
;
42
(
6
):
482
489
.
13.
Caulford
PG
,
Lamb
SB
,
Kaigas
TB
,
Hanna
E
,
Norman
GR
,
Davis
DA.
Physician Incompetence: Specific Problems and Predictors
.
Acad Med
.
69
(
10
):
S16
S18
.
October
,
1994
.
14.
American Academy of Family Practice
.
CME Requirements for Members
. .
15.
Davis
DA
,
Thomson
MA
,
Oxman
AD
,
Haynes
RB.
Changing physician performance. A systematic review of the effect of continuing medical education strategies
.
JAMA
.
274
(
9
):
700
705
.
September
6
,
1995
.
16.
Forsetlund
L
,
Bjørndal
A
,
Rashidian
A
,
Jamtvedt
G
,
O'Brien
MA
,
Wolf
F
,
Davis
D
,
Odgaard-Jensen
J
,
Oxman
AD.
Continuing Education Meetings and Workshops: Effects on Professional Practice and Health Care Outcomes. Review
.
The Cochrane Collaboration
.
Cochrane Library 2009, Issue 2
.
Wiley
.
2009
.
17.
David
D
,
O'Brien
MAT
,
Freemantle
N
,
Wolf
FM
,
Mazmanian
P
,
Taylor-Vaisey
A.
Impact of Formal Continuing Medical Education
.
JAMA
.
282
(
9
):
867
874
.
September
1
,
1999
.
18.
Marinopoulos
SS
,
Dorman
T
,
Ratanawongsa
N
,
Wilson
LM
,
Ashar
BH
,
Magaziner
JL
,
Miller
RG
,
Thomas
PA
,
Prokopowicz
GP
,
Qayyum
R
,
Bass
EB.
Effectiveness of Continuing Medical Education
.
Evidence Report/Technology Assessment No. 149 (Prepared by the Johns Hopkins Evidence-based Practice Center, under Contract No. 290-02-0018.) AHRQ Publication No. 07-E006
.
Rockville, MD
:
Agency for Healthcare Research and Quality
.
January
2007
.
19.
Kiessling
A
,
Lewitt
M
,
Henriksson
P.
Case-Based Training of Evidence-Based Clinical Practice in Primary Care and Decreased Mortality in Patients with Coronary Heart Disease
.
Annals of Family Medicine
.
9
(
3
):
211
218
.
May/June
,
2011
.
20.
Fordis
M
,
King
JE
,
Ballantyne
CM
,
Jones
PH
,
Schneider
KH
,
Spann
SJ
,
Greenberg
SB
,
Greisinger
AJ.
Comparison of the Instructional Efficacy of Internet-Based CME with Live Interactive CME Workshops
.
JAMA
.
294
(
9
):
1043
1051
.
September
7
,
2005
.
21.
Institute of Medicine
.
Redesigning Continuing Education in the Health Professions
.
Washington, DC
:
The National Academies Press
.
2010
.
23.
U.S. Department of Health and Human Services
.
Moving Medical Innovations Forward — New Initiatives from HHS
.
Washington, DC
:
U.S. Department of Health and Human Services
.
January
2005
.
24.
Shojania
KG
,
Silver
I
,
Levinson
W.
Continuing Medical Education and Quality Improvement: A Match Made in Heaven?
Ann Internal Med
.
156
:
305
308
.
February
21
,
2012
.
25.
Krupa
C.
Demand rises for CME linked to physician quality improvement
.
Amer Med News
.
56
(
1
):
25
.
January
14
,
2013
.
27.
Davis
D
,
Mazmanian
P
,
Fordis
M
,
Van Harrison
R
,
Thorpe
K
,
Perrier
L.
Accuracy of Physician Self-Assessment Compared With Observed Measures of Competence: A Systematic Review
.
JAMA
.
296
(
9
):
1094
1102
.
September
6
,
2006
.
28.
Eva
K
,
Regehr
G.
“I'll Never Play Professional Football” and Other Fallacies of Self-Assessment
,
Journal of Continuing Education in the Health Professions
,
28
(
1
):
14
19
,
2008
.
29.
Ibid
.
31.
Hess
BJ
,
Weng
W
,
Holmboe
ES
,
Lipner
RS.
The association between physicians' cognitive skills and quality of diabetes care
.
Acad Med
.
87
(
2
):
157
63
.
2012
.
32.
Hess
BJ
,
Weng
W
,
Lynn
LA
,
Holboe
ES
,
Lipner
RS.
Setting a fair performance standard for physicians' quality of patient care
.
J Gen Intern Med
.
26
(
5
):
463
73
.
2010
.
33.
Miles
PV.
Maintenance of certification: the profession's response to physician quality
.
Ann Fam Med
.
9
(
3
):
196
7
.
2011
.
34.
Holmboe
ES
,
Meehan
TP
,
Lynn
L
,
Doyle
P
,
Sherwin
T
,
Duffy
FD.
Promoting physicians' self-assessment and quality improvement: the ABIM diabetes practice improvement module
.
J Contin Educ Health Prof
.
26
(
2
):
109
19
.
2006
.
35.
Chaudhry
H
,
Rhyne
J
,
Waters
S
,
Cain
FE
,
Talmage
L.
Maintenance of Licensure: Evolving from Framework to Implementation
.
J Med Reg
.
97
(
4
):
8
13
.
2012
.
36.
Chaudhry
H
,
Talmage
L
,
Alguire
P
,
Cain
FE
,
Waters
S
,
Rhyne
J.
Maintenance of Licensure: Supporting a Physician's Commitment to Lifelong Learning
,
Ann Internal Med
,
2012
,
Vol. 157
,
no.4
37.
Young
A
,
Chaudhry
HJ
,
Rhyne
J
,
Dugan
M.
A Census of Actively Licensed Physicians in the United States, 2010
.
J Med Reg
.
96
:
13
20
.
2010
.
38.
.
39.
Nasca
TJ
,
Philibert
I
,
Brigham
T
,
Flynn
TC.
The Next GME Accreditation System — Rationale and Benefits
.
NEJM
.
366
(
11
):
1051
1056
.
March
15
,
2012
.
40.
Combes
JR
,
Arespacochaga
E.
Lifelong Learning: Physician Competency Development
.
American Hospital Association's Physician Leadership Forum
.
Chicago, Illinois
.
June
,
2012
.
41.
Kohn
LT
,
Corrigan
J
,
Donaldson
MS.
To Err Is Human: Building a Safer Health System
.
Washington, DC
:
National Academy Press
.
2000
.
42.
Chantler
C
,
Ashton
R.
The Purpose and Limits to Professional Self-Regulation
.
JAMA
.
302
(
18
):
2032
2033
.
November
11
,
2009
.
43.
AARP
.
Strategies to Improve Health Care Quality in Virginia: Survey of Residents Age 50+
.
Washington, DC
:
AARP
.
2007
. .
44.
Chaudhry
HJ
,
Rhyne
J
,
Cain
FE
,
Young
A
,
Crane
M
,
Bush
F.
Maintenance of Licensure: Protecting the Public, Promoting Quality Health Care
.
J Med Reg
.
96
(
2
):
13
20
.
2010
.
45.
Speck
M.
Best practice in professional development for sustained educational change
.
ERS Spectrum
.
Pages
33
41
.
Spring
,
1996
.

Author notes

This article was endorsed by the Board of Directors of the Federation of State Medical Boards on February 7, 2013.