Bring up the issue of certification in almost any forum and you'll spark heated discussion. Someone will insist that getting certified doesn't make you better at your job. Someone else will respond that certification is a mark of professionalism. Others will point out that certification rarely translates into a higher salary. Still others will say it's about personal satisfaction, not the money. There's no shortage of opinions about the value of certification.
There's no shortage of opinions about the value of certification.
Since the introduction of AAMI's Benchmarking Solution (ABS) in 2009, valuable information has been accumulating about benchmarking, best practices, and performance improvement efforts in the profession of healthcare technology management. Within the ABS database is a growing amount of information that can give us some insight into the value of professional certification. Let's take a look.
The ABS Database
ABS asks respondents to provide data on the number and type of personnel in their clinical engineering programs. It also asks for the number of certified personnel, defined as those holding one or more credentials for certified clinical engineer (CCE), certified biomedical equipment technician (CBET), certified radiology equipment specialist (CRES), or certified laboratory equipment specialist (CLES). With this information, ABS calculates the percentage of certified personnel in each clinical engineering program.
This article focuses on ABS data for 2010, the most recent full year of data. A total of 139 clinical engineering programs provided data for 2010, with all but eight programs providing complete certification-related information. For the 131 programs with complete data, the percentages of certified personnel ranged from 0% to 80%, with an average of 23% (somewhat skewed by the rather large number of programs with no certified personnel).
For the purposes of this article, two subsets of data have been defined. The first subset consists of the 23 clinical engineering programs with no certified personnel. This subset is referred to as the Low Certification Group. The second subset consists of the 23 clinical engineering programs with the highest percentage of certified personnel. Clinical engineering programs in this latter subset, referred to as the High Certification Group, have an average of 55% of their personnel holding certification credentials.
The approach taken in this article is to compare the High Certification and Low Certification groups on a variety of characteristics. This simple approach sheds light on the differences in clinical engineering programs that have relatively high and relatively low levels of certification among their personnel.
Perhaps the most important performance measure in the ABS database is the cost of service ratio or COSR. (See, for example, Ted Cohen's article in the July/August 2011 issue of BI&T.) COSR is the ratio of annual maintenance costs to equipment value. Across a wide variety of industries, COSR is recognized as a concise measure of maintenance efficiency from a financial perspective.
In the 2010 ABS database, the average COSR value in the Low Certification Group is 8.1%, which is relatively high for a typical healthcare technology management program. The average COSR value in the High Certification Group is 5.0%, notably lower and more in line with expected levels. This suggests that higher levels of support for certification are associated with higher levels of performance as measured by COSR data.
In addition to data regarding clinical engineering- related certifications, the ABS database includes information regarding information technology (IT)-related certifications such as A+ and Network+ credentials. In the Low Certification Group, 1.3% of clinical engineering personnel held IT-related credentials of this type. In the High Certification Group, the number was 7.4%, more than five times as high. It appears that support for CE-related certification is accompanied by support for IT-related certification.
The ABS database also contains a wide range of qualitative (rather than quantitative) data related to “best practices” in clinical engineering management. For each clinical engineering program, ABS uses the answers to qualitative questions to measure the adoption of these best practices on a five point scale, with 1 representing a low level of adoption and 5 representing a high level of adoption.
For this article, the adoption rates for various best practices in Low Certification and High Certification groups were examined. The following paragraphs present those practices for which there was a difference in adoption rates of 0.5 points or more. This methodology highlights differences in practice that are large enough to be of interest from a managerial perspective.
For example, as might be expected, programs in the High Certification Group are more likely to have adopted (i.e., have adoption rates at least 0.5 points higher than those in the Low Certification Group) the following policies related to certification: formal recognition of certification achievements; promotion or increase in pay for certification; reimbursement for review classes; reimbursement for study materials; reimbursement for travel costs for testing; and reimbursement for the application or testing fee.1
Best practices data in ABS also suggest that clinical engineering programs in the High Certification Group interact more extensively with other groups in the organizations (when compared to clinical engineering programs in the Low Certification Group). Specifically, High Certification programs are:
More likely to be active members of the following committees: capital acquisition; facility design and renovation; IT networking or security; laser safety; nursing operations or education; performance improvement; product evaluation; quality assurance; radiation safety; risk management; safety (or environment of care); strategic planning.
More likely to routinely participate in the following safety processes for the organization: clinical alarm process reviews; environmental tours (hazard surveillance rounds); failure modes and effects analysis (FMEA) activities; patient safety committee; root cause analysis (RCA) activities; safety (or environment of care) committee.
More likely to be actively involved in planning for the following activities: clinical alarm and alarm secondary communication management; computer and network security management; integration of medical devices and IT systems; management of backup and system administrative functions in clinical equipment networks; design of networks for networked clinical systems; remote monitoring of medical device network performance; and wireless spectrum management.
Best practices data in ABS also suggest that clinical engineering programs in the High Certification Group interact more extensively with other groups in the organizations (when compared to clinical engineering programs in the Low Certification Group).
Along the same lines, clinical engineering programs in the High Certification Group are:
More likely to have clinical engineering leadership meet routinely with IT leadership for project coordination and technology planning.
More likely to have clinical engineering leadership meet routinely with clinical department leadership for project coordination and technology planning.
In terms of financial management, clinical engineering programs in the High Certification Group are more likely than those in the Low Certification Group to have accurate data regarding the acquisition costs for medical equipment in the inventory.
In terms of financial management, clinical engineering programs in the High Certification Group are more likely than those in the Low Certification Group to have accurate data regarding the acquisition costs for medical equipment in the inventory. This information is essential for calculating COSR often not readily available (see, for example, my article in the September/October 2011 issue of BI&T).
Programs in the High Certification Group are also more likely to be involved in the strategic (long range) capital budget development process, the review and approval process for requests for replacement of medical equipment, and the review and approval process for requests for new medical equipment. Moreover, programs in the High Certification Group are more likely to have their organizations require verification by clinical engineering that all conditions of purchase, including provision of a service manual, are met before authorizing full payment of invoices for new medical equipment.
The picture that emerges from the ABS database is that clinical engineering programs with higher levels of certification tend to have higher levels of adoption of a wide range of “best practices” for clinical engineering management.
Clinical engineering program managers in the High Certification Group are more likely to have the following qualifications: at least 10 years of experience in the field of clinical engineering; at least 5 years of experience in clinical engineering program supervision or management; a bachelor of science or higher degree in engineering; peer certification (CCE, CBET, CRES, CLES).
In contrast to the preceding pattern of relatively high adoption of best practices by clinical engineering programs in the High Certification Group (compared to programs in the Low Certification Group), clinical engineering programs in the High Certification Group are less likely to conduct frequent physical inventories to verify the accuracy of equipment inventories and are less likely to maintain a complete list of all critical systems in the inventory.
For sake of simplicity, this article does not include calculations of statistical significance. Instead it focuses on factors and effect sizes that are relevant to the practice of clinical engineering and healthcare technology management.
It should also be noted that, in a statistical sense, associations between various factors do not imply causality. If two factors are associated with each other (for example, level of certification and COSR), it cannot be determined statistically whether the first factor causes the second, the second factor causes the first, or both factors are caused by some third factor. It is also possible that, when making a large number of comparisons, spurious associations may appear that have no practical meaning.
However, while keeping these statistical limitations in mind, we can combine what we see in the data with our experience and professional judgment. An example often used by statisticians is the strong association between the presence of fire trucks at a building and the observation that the building is on fire. Do fire trucks cause fires or do fires cause the presence of fire trucks? Strictly speaking, statistical association cannot answer that question. But, using our experience and some logical reasoning, we can conclude for all practical purposes that fires bring fire trucks. In other words, we can sometimes determine the causal relationship between associated phenomena.
The picture that emerges from the ABS database is that clinical engineering programs with higher levels of certification tend to have higher levels of adoption of a wide range of “best practices” for clinical engineering management. They are more involved in many important activities throughout their organizations. They have more influence on processes related to equipment planning and acquisition. They have more access to data that enables comprehensive financial management. In short, at the risk of stretching beyond the statistics, we could conclude that clinical engineering programs with high levels of certification are recognized and valued throughout their organizations.
Does certification “cause” that recognition? A good case can be made that professional credentials contribute to recognition by our professional colleagues. If a member of our own family needs surgery, we'll probably look for a surgeon who is board-certified in the particular type of surgery needed. Does that certification make him or her a better surgeon? Not necessarily, but it helps us recognize and identify those surgeons who have met the requirements for certification and made the effort to acquire the credential.
Another interpretation of the data is that adoption of this set of best practices, including support for certification, is the result of an underlying factor such as an across-the-board commitment to high quality clinical engineering management. In other words, under this interpretation, good managers do good things—and they make the effort to identify what those good things (i.e., best practices) are.
The value of these management practices has a degree of confirmation in the COSR performance data, with higher levels of “best practices” adoption associated with lower COSR values. Do these best practices actually cause improved financial performance? Or does a high level of financial performance give the clinical engineering program the resources it needs to adopt best practices? Perhaps the most reasonable interpretation is that these are mutually reinforcing factors: Doing the right things improves performance and good performance lets you do the right things.
What does all this say about the value of certification? It seems clear that support for certification is an integral component of excellence in clinical engineering practice and healthcare technology management. It's something we should be doing.
1 The actual question in ABS lists these policies and asks respondents to check all that have been adopted. Scoring of the question (from 1 to 5) is based on the number of policies checked, adjusted for the total number of policies in the list. Other questions in this article that include a list of responses and allow respondents to check all that apply are handled similarly within ABS.
About the Author
Matthew F. Baretich, PE, PhD, is president of Baretich Engineering in Fort Collins, CO. He is one of the original subject matter experts for AAMI's Benchmarking Solution. E-mail: email@example.com