The purpose of this paper is to document in part the study that was conducted for a research project ASHRAE RP-1650, which investigates the training requirements for sustainable high-performance building (HPB) operations and maintenance (O&M). This paper documents the intent, procedures and findings of a portion of the research project, which includes a survey of O&M practices for high performance buildings in the industry. The purpose of the survey as reported in this paper was to confirm and refine the level of understanding of KSCs for operating HPBs as perceived by O&M personnel in the industry. The details of the survey and the results are presented in this paper. The survey was sent out to facility managers in the industry who are involved with the O&M of HPB systems and equipment. 221 facility managers were surveyed in this study.
This study utilizes a concurrent mixed-method research, which considers both qualitative and quantitative responses.
The study found that while most technicians were ‘moderately well' to ‘very well' trained for the configuration and components as well as the installation, repair and replacement of different high performance equipment and systems, they lacked essential knowledge to address issues such as: controls; tracking of variables; programming and operation of modulation devices implemented in high-performance systems and equipment. This finding implied a lack of ‘Systems Thinking' approach, which the study found to be pertinent and essential for the O&M of HPB systems and equipment.
The study is utilized to identify gaps between the training requirements for O&M of HPBs identified from the literature review of the research project and the current industry practices. The identified gaps are valuable to develop learning objectives for training programs that are aimed to train personnel responsible for the O&M of HPBs.
This paper documents the research that has been performed as part of a research project sponsored by the American Society of Heating Refrigeration and Air-conditioning Engineers (ASHRAE), ASHRAE RP-1650 (Mukhopadhyay et al. 2019). The research project investigates the training requirements for service personnel involved in the operations and maintenance (O&M) of high-performance buildings. The project was conducted in three related parts, which include: Part 1, a literature review of appropriate O&M practices for high-performance buildings (HPBs), the knowledge-skill-competencies (KSCs) that are required to optimally operate such types of buildings and the training opportunities that are currently available to service personnel; Part 2, a survey of existing practices in the industry; and Part 3, the development of learning objectives to develop appropriate training material for personnel involved in the O&M of high performance buildings. This paper documents the intent, procedures and findings of Part 2, which presents a survey of the existing O&M practices in the building industry.
A literature review was conducted to identify existing O&M practices in HPBs and the preparedness of personnel to engage in these practices. Very few publications were found for this study, which are presented below. Lewis proposed a framework to improve building operations decisions with a focus on sustainability and energy efficiency goals (Lewis 2012). The study utilized literature review of existing practices, case studies, questionnaires and insight from industry experts to formulate the framework. The study concluded that the use of this framework was helpful in making combined energy and maintenance management decisions. Some O&M practices recommended by this study included: development of maintenance plan, need to regularly calibrate sensors and meters, the use of metrics for energy and maintenance management decision making, benchmarking energy performance and maintenance training.
A study sponsored by the National Science Foundation evaluated the current conditions of buildings and trends in their O&M practices (Ehrlitch et al 2010). The study was conducted through survey, interviews and focus groups as well as literature review and observations of current practices. The study identified deficiencies with the current training requirements of O&M workforce, which include: drawbacks of current workforce education models; lack of industry practices for facility management, operation, maintenance practices and processes; and the primary focus of O&M staff being on satisfying tenants rather than energy efficient operation. The study concluded that major changes in building operations are needed to meet the sustainability goals and occupant comfort requirements in buildings. The study recommends that new processes, skill, and technologies are needed to meet the needs of HPBs, and that education and training requirements for future facility managers and building control system technicians are important to fulfill these needs.
A study by Ehrlich and Fenimore identified key elements regarding the training and certification of technicians involved in the O&M of HPBs (Ehrlich and Fenimore 2015). The study notes that although important, the training of O&M staff responsible for commercial buildings is often neglected. The study concluded that a need for O&M staff that is well trained to handle stringent requirements for energy, emissions, IEQ and sustainability is required for HPBs.
Ehrlich provides information regarding training and certification of technicians who are responsible for O&M of HPBs (Ehrlich 2016). The report concluded that there is a limited availability for formal training for technicians responsible for HPBs, there is no recognized national certificate for HPB technicians, and there is a large projected workforce shortages in this sector. The study also noted a trend towards hiring specialized contractors for conducting O&M of facilities.
The ASHRAE RP-1650 study aims to confirm the results of previous research and focuses on trends in O&M practices for high performance technologies and equipment with the intent of identifying key requirements for learning objectives that are to be used in the preparation of training programs for O&M of HPBs (Mukhopadhyay et al, 2019).
The literature review conducted by Mukhopadhyay et al. (2019) in Part 1 of this research project identified several types of mechanical systems and equipment that can be classified as high performance. The list of high performance systems that are evaluated by this survey includes: Ground source heat pump system (GSHP), variable refrigerant flow system (VRF), dedicated outdoor air system (DOAS), underfloor air distribution system (UFAD), and chilled beam system. The list of high performance mechanical equipment that are evaluated by this survey includes: Chillers (i.e., variable speed compressors, magnetic bearing, variable water flow), boilers (i.e., condensing), cooling towers (i.e., variable speed fans, reduced water consumption), heat pumps (i.e., variable speed compressors, variable water flow), heat recovery equipment, variable frequency drive (VFDs), electronically commutated motors (ECMs). The literature review identified the knowledge-skill-competencies (KSCs) required to optimally operate these systems, which prompted a fundamental shift in the approach to O&M from ‘component-based' to ‘Systems thinking'. In addition, the literature review defined HPBs to have stringent energy goals and maintenance practices. Finally, the literature review concluded that there is no shortage of high quality ‘Component-based' approach to training for O&M of HPB systems.
The survey conducted in Part 2 (a survey of existing practices in the industry) of this research project aimed to obtain, confirm and refine the level of understanding of KSCs for operating HPBs as perceived by O&M personnel in the building industry. By doing so, the survey aimed to identify gaps between the training requirements for O&M of HPBs identified from the literature review conducted in Part 1 and the current industry practices. The details of the survey and the results are presented in this paper.
The objectives of this survey were to assess the current prevailing practices in the industry for O&M of HPBs in order to:
Obtain, confirm and refine the level of understanding of knowledge-skill-competencies (KSCs) for operating HPBs as perceived by building operators in the industry
Ask questions regarding confirming and refining KSCs that have been compiled in the review of literature and training material for operation of HPB systems
Determine if KSCs are considered sufficiently valuable to warrant training investment by the sponsoring organization
This survey aims to confirm and triangulate the results of the information gathered from the literature review by gaining an insight into the experience and perspective of facility managers in the industry. The survey was designed to contain questions regarding confirming and refining the understanding of KSC's that have been compiled in literature review conducted by Mukhopadhyay et al. for the O&M of HPB systems (Mukhopadhyay et al., 2019).
SIGNIFICANCE OF THE STUDY
The targeted audience for this study included HPB facility managers, operators, educators and program developers who in their line of work need to understand and learn more about training requirements, prepare curricula, train other personnel in O&M of high performance systems and equipment. This study also provides information and curricula guidance through learning objective to prepare O&M personnel to undergo appropriate training, and appropriate knowledge for training programs for O&M of high-performance buildings' systems and equipment. The data findings for this study was focused on specific areas of HPB operation in furtherance of improving training for O&M personnel regardless of experience and education.
METHODOLOGY AND DATA COLLECTION
This study utilizes a concurrent mixed-method research design of 221 respondents, which considers both qualitative and quantitative responses (Creswell 2014). From the quantitative responses we derived both descriptive and inferential statistics. On the other hand, using qualitative responses participants provided experiential recommendations and suggestions. The survey was conducted from a training perspective with findings about specific areas of HPB operation that are likely to need improved training for operating staff. The identified areas are intended to provide the basis for development of training documents. However the development of the curriculum itself is outside the scope of this research study.
The survey was divided into a series of tasks, which included:
The survey was geared towards facility managers in the industry who are involved with the O&M of HPB systems and equipment. A nationwide survey was sent to facility managers from various organizations via different methods of dissemination such as publications in newsletters and list servers, and LinkedIn. To ensure the randomization of the sample selected for this research, the participants were not individually identified. This process of randomization ensured that a small sample size could be used to make predictions about a large population. A list of the targeted organizations, and the corresponding methods of data collection are provided Table 1.
Survey instrument preparation and piloting
The survey instrument was prepared in consultation with the subject matter experts (SMEs) and project monitoring sub-committee (PMSC). The SME and the members of the PMSCs were all involved in the facility management industry and have substantial experience with managing the O&M of commercial facilities and addressing issues associated with the operation and management of such facilities.
A pilot study was conducted and the final survey was modified accordingly. The survey was launched online with the help of Human Ecology Learning & Problem Solving (HELPS) Lab at Montana State University (MSU). A sample population for the pilot study was selected for the initial dissemination of the survey. The intention was to obtain a feedback from this set of participants in terms of the content and clarity of the survey. In addition, the participants in this group also evaluated the time spent on completing the survey. The sample population for the pilot study consisted of facility management staff at Montana State University and members of the project monitoring subcommittee. Changes were made in the final survey according to the responses and the findings from the pilot study.
The survey was sent to the intended target population, which included participants from various organizations. Various methods were used to send out the survey to the target population, which include: contacting members via list servers, posting the survey link on the news letters and social media such as LinkedIn. The survey was available for a period of one month.
The results of the Literature Review conducted by Part 1 of this study were used to inform the development of the survey questionnaire. In addition, a literature review of relevant resources was conducted to determine appropriate questions (Lewis 2012, Ehrlich et al., 2010, Ehrlich, 2016). The survey was adapted from these resources with some modifications that were relevant to the target population being considered for this study.
The survey instrument was organized into several sections. The first section inquired about the background of the participant and the O&M staff under their supervision. The second section inquired about the facilities under the participants' supervision. The third section inquired about the HPB systems and equipment at the facilities under the participants' supervision. The survey instrument was developed using a web-based survey management software (Qualtrics 2018). A flowchart outlining different sections of the survey is presented in Figure 1.
In the first section, the survey inquires about the participants' job title, role and responsibilities at the facility, years of experience, and about the type and area of facilities under the participants' supervision. The survey also inquired about the background of the O&M staff under the participants' supervision which include staff responsibilities and educational background. Finally, the survey inquired about training requirements for O&M staff to operate HPBs. The survey questions for this section are elaborated in Table 2.
In the second section of the survey, when documenting the O&M practices for HP facilities under the participants' supervision, several issues were identified as important from the literature review conducted in Task 1 of this research project. These include: energy goals, predictive and preventive maintenance practices, occupant education and training, documentation practices, supervising activities, and the use of monitoring systems such as Building Automation System (BAS) and Computer Maintenance Management System (CMMS). BAS and CMMS were identified by the literature review of this research project as key components of HPBs. Hence, the survey inquired about these systems in the HPB facilities that were under the supervision of the participant. The survey questions for this section are elaborated in Table 3.
The third section of this survey inquires about the type and number as well as the O&M of high performance systems and equipment as identified in the introductory section of this paper. This section of the survey also asks questions evaluating the training and knowledge levels of the O&M personnel tasked with the maintenance of these high performance systems and equipment. In order to evaluate the training and knowledge levels of the O&M staff operating high-performance systems and equipment several categories were identified that were associated with the O&M for these systems. The survey instrument includes questions about:
System / equipment configuration with other systems
System / equipment components
System / equipment installation, repair and replacement
System / equipment variable measurement and tracking
Programming modulation devices
Systems / equipment-level controls
The participants were asked to indicate the training and knowledge levels of the O&M personnel on a Likert scale1. Another category ‘We outsource this function' was introduced to indicate that the question was not part of the O&M activities in the facilities under the participants' supervision. The scale includes:
Not well at all
We outsource this function
Graphical tools such as bar charts and pie charts were used to evaluate data obtained from each question in the survey. In addition, correlations were performed between different variables in order to gain a better understanding of the trends projected by the results. For evaluating the training and knowledge levels, several statistical methods were used. Statistical methods implemented by this analysis included the use of mean, median and mode. One-way ANOVA tests and Dunnett's C tests were also performed to conduct the evaluation. The use of these methods was to determine any statistically significant differences between two of more independent groups of results. Analysis using the mean of data collected proved to be most effective as it provided an overall trend in the responses.
RESULTS AND ANALYSIS
A total of 221 responses to the survey were documented. The subsections below present a summary of the findings.
O&M staff and outsourcing of O&M activities
According to the results from the survey, the top three professions hired by management include electricians (830 hires), building technicians (535 hires) and HVAC-R equipment mechanics (526 hires) (Figure 2). The hiring trends indicates the primary dependence of facilities on O&M technicians of HVAC equipment and systems. This trend also indicates the populations that the proposed training programs need to target in order to address the effective O&M of HPBs. According to the results from the survey, the top three activities that are outsourced by facility management across different building types include: HVAC repair (22%), building control systems (19%) and engineering services (18%) (Figure 3). This is because higher degree of skill sets and costs are associated with hiring personnel to perform these activities.
Training and certification requirements for HPBs
A majority of participants across different building types in the survey indicated that their O&M staff attended certification programs (33%) (Figure 4). In addition, participants also indicated that some of their staff had no formal education and learnt their skills on the job (18%). The absence of formal education figured prominently for O&M personnel in K-12 schools. The results also indicated that for most building types only a fraction of O&M staff had educational training from 4 year institutions or community colleges for most building types. However, half the participants who were responsible for retail and public building indicated that their O&M staff had educational training from 4 year institutions or community colleges.
With regard to O&M of HPBs, almost half of the participants indicated that they did not require their O&M staff to undergo additional training to operate HPBs (47%) (Figure 5). However, half the participants who were responsible for retail and public building indicated requirements for additional training to operate HPBs. Training from manufacturers and vendors was the most popular (22%) (Figure 6). With regard to time period of training being encouraged, one day sessions (38%) and lunch and learns sessions were the most popular (34%) (Figure 7). The trends in responses indicates the priorities with regard to time and money allocated to training of O&M personnel. Training from manufacturers and vendors are highly focused and provide attendees with information that is specific to the product oftentimes not providing a comprehensive discussion of the basic principles and range of options that are available. When compared to community colleges, certification programs require less investment of resources and so do one-day sessions. These trends indicate a potential lack of exposure of O&M staff to the ‘Systems thinking' approach to the O&M of high performance systems and equipment. As indicated in the literature review conducted in Part 1 of the overall study ‘Systems thinking' approach is obtained on attending community college or 4 year university programs. ‘Systems thinking' approach can also be reinforced by providing the O&M staff intensive hands-on training experience with the schemes and configurations of systems and equipment that are unique to the HPB to which they are assigned.
Energy Goals and O&M Practices for HPBs
Most of the participants in the survey indicated that energy goals were set (57%) and met (66%) for HPBs at their facilities (Figure 8, Figure 9). However, a substantial number of participants indicated the lack of energy goals (30%) or were not aware if there were any energy goals set for the facility (13%). Participants also identified the three major challenges to collection of energy data, which include availability of staff time (22%), limited resources (18%), and training of personnel (14%) (Figure 10). While some of these challenges such as difficulty of working with the format of data and inadequate training of staff can be addressed with appropriate training resources, other challenges indicate priorities set by management for resource allocation.
When asked about the use of preventive maintenance standards (i.e., ASHRAE Standard-180), 52% of the participants responded that preventive maintenance standards are not used in O&M of HPB systems and equipment (Figure 11). When asked about the use of predictive maintenance practices, half of the participants responded that predictive practices are not used in their facilities (51%) (Figure 12). However, participants who indicated the use of these practices and standards in their buildings, also indicated that their O&M staff was well trained to conduct these practices.
Although majority of the participants indicated the presence of good documentation practices such as functional systems manual, updated mechanical drawings and documented sequence of operations for high performance buildings (64%), a substantial number of participants indicated the absence of these documents (29%) (Figure 13). On correlating the responses with facility type it was observed that most of the negative responses were from participants responsible for K-12 facilities. This trend confirms the lack of resources for O&M in K-12 schools, which indicates that adequate training resources may not be equally accessible to O&M personnel across different building types. The results confirmed that dwindling budgets and resources restrict access to appropriate training, tools and standard practices that are required for O&M of HPBs.
Challenges of O&M for HPBs
When asked about challenges, top three challenges that figured prominently across all participant responses included: not enough time (22%), inability to get qualified staff (21%), and need for facility managers to be more involved during design and construction process (16%) (Figure 14).
It was inferred that most participants were faced with time constraints with regards to the O&M of high performance buildings, which implies the shortage of trained staff. This implication along with other responses such as lack of involvement of facility management in preliminary stages of design and construction process and budget constraints are indicative. In addition, policies and priorities of the organizations in which the participants are employed rather than the availability of training opportunities contribute to these challenges.
KSCs and Use of Advanced Monitoring Equipment for HPBs
Participants across all building types indicated that BAS (85%) and CMMS (71%) were implemented in their facilities ((Figure 15, Figure 16) and their O&M staff ‘Moderately well' understood the working of advanced monitoring equipment and strategies, which included the use of BAS (more than 25 responses) and CMMS (more than 30 responses) (Figure 17, Figure 18). The results indicate that action was taken towards addressing a shortage of training specific to BAS, ever since it was identified by the 2010 NSF study (Ehrlich et al., 2010). However, the results indicated that additional training resources may be required to elevate the levels of KSCs of O&M staff operating BAS and CMMS to ‘Very well' or ‘Extremely well', which are levels that are desired for the optimum operation of HPBs.
KSC Levels for O&M of HPB Systems and Equipment
In general, levels of training and knowledge as reported in the survey results were indicative of similar levels of KSCs of O&M personnel in the field. Participants indicated that their O&M staff was ‘Moderately well' to ‘Very well' trained and knowledgeable in the installation, repair and replacement of high performance equipment. In addition, the participants indicated that their O&M staff was ‘Moderately well' to ‘Very well' trained and knowledgeable in understanding configuration and components of high performance equipment. Finally, the participants indicated that their O&M staff was ‘Slightly well' to ‘Moderately well' trained and knowledgeable on issues related with measurement and tracking of variables, programming modulation devices and equipment level controls. This is because the three above mentioned components require a ‘Systems thinking' approach which is currently lacking in current training programs as was concluded by the literature review conducted by the research project (Mukhopadhyay et al., 2019). However, on conducting statistical tests on the results, it was determined that for all equipment no significant differences existed between the different categories (Table 6).
Figures 19 and 20 illustrate training and knowledge assessment of O&M staff operating high performance equipment (boilers) and systems (GSHPs). The primary y-axis provides the mean of responses assessing the training and knowledge levels of their O&M staff on a Likert scale for the listed categories (0 = Outsource, 5 = Extremely well). The secondary axis provides a difference in the responses assessing training and knowledge levels.
When evaluating the difference between the mean of responses for training and knowledge levels for each category, a negative difference between training and knowledge in all the categories was observed, which indicated that O&M staff use prior knowledge (i.e., other than the training specific to the category of task) to conduct the required activities. Figures 19 presents an example of the training and knowledge assessment of O&M staff operating high performance boiler equipment.
When considering high performance systems, participants indicated that their O&M staff was ‘Slightly well' to ‘Very well' trained and knowledgeable in the categories of system/equipment configuration, system/equipment components as well as system/equipment repair and installation. The participants also indicated that their O&M staff was ‘Slightly well' to ‘Moderately well' trained and knowledgeable on issues related with measurement and tracking of variables, programming modulation devices and equipment level controls. This loosely confirms the findings from the literature review conducted by this research, which concluded that currently there is more availability of training resources for high performance equipment rather than high performance systems. The results also confirmed that the current focus of training material is ‘component-based' albeit with a strong emphasis on energy efficiency. When evaluating the difference between the mean of responses for training and knowledge levels for each category, with the exception of DOAS systems, a negative difference between training and knowledge in all the categories, which indicated that O&M staff use prior knowledge (i.e., other than the training specific to the category of task) to conduct activities. Figure 20 presents the training and knowledge assessment of O&M staff operating GSHP system.
CONCLUSIONS & RECOMMENDATIONS
The study observed a primary dependence of facilities management on the O&M technicians of HVAC equipment and systems. The study also found activities such as HVAC repair, building control systems and engineering services are currently being outsourced to independent contractors, when necessary. These results indicate a need for developing additional training programs in these areas to address effective O&M of HPBs.
When considering type and format of programs, results showed the popularity of various certification programs. In addition, short time period programs facilitated by manufacturers and vendors were the most popular means of training O&M personnel. The above mentioned results indicated the priorities set by facility management with regard to time and money allocated to the training of O&M personnel. The results also indicated a lack of exposure to ‘System Thinking' among O&M staff.
Although the survey indicated that majority of O&M staff had certifications in different aspects of O&M of HPBs as shown in Figure 4, the results assessing the knowledge and training levels of high-performance systems indicated otherwise as shown in Figure 20. Therefore, it was inferred that these certifications were not sufficient for knowledge levels that spanned across system wide issues. This inference is supported by the conclusions obtained from the review of training programs conducted by this research (Mukhopadhyay et al., 2019). This conclusion points to a lack of ‘Systems Thinking' approach that is prevalent in the industry practices for O&M of HPB.
A substantial number of participants indicated the lack of established preventive maintenance and documentation practices. In addition, participants listed the top three challenges faced by facility management for O&M of HPBs as not enough time, inability to get qualified staff, and the need for facility managers to be more involved during design and construction process. These results confirm that a lack of resources for O&M restrict access to appropriate training, tools and standard practices that are required for O&M of HPBs.
When assessing training and knowledge levels for O&M of HPB systems and equipment results indicated that the technicians were ‘Moderately well' to ‘Very well' trained for the configuration and components, as well as the installation, repair and replacement of different HPB systems and equipment However, the survey also found that technicians lacked essential knowledge to address issues such as: controls; tracking of variables; programming and operation of modulation devices implemented in HPB systems and equipment. In addition, the survey determined that technicians had moderate exposure to operating advanced monitoring systems such as BAS and CMMS. The identified tasks in which technicians had insufficient or moderate exposure correspond to ‘Systems Thinking' approach, which the study found to be pertinent and essential for the O&M of HPB systems and equipment.
This study recommends the development of training in the areas of system configuration; system and equipment level controls; programming and operation of modulation devices implemented in high-performance systems and equipment all of which emphasize the ‘Systems Thinking' approach to O&M of HPBs. In addition to the current options for certifications, the study also recommends another level of certifications that address interrelation of different systems in HPBs. Further research is recommended to investigate the impact of outsourcing of O&M activities such as HVAC repair, building control systems and engineering services, as compared to in-house development of these activities in various facilities. Finally, further research may be required to assess whether technologies such as BAS and CMMS are used to their fullest potential for O&M of HPBs.
By successfully identifying gaps in the existing practices of the HPB industry this study aims to address the bigger issues of sustainability and energy efficiency that O&M personnel need to address in order to effectively operate such buildings.
Likert developed the principle of measuring attitudes by asking people to respond to a series of statements about a topic, in terms of the extent to which they agree with them, and so tapping into the cognitive and affective components of attitudes (Likert 1932). In its final form, the Likert Scale is a five (or seven) point scale which is used to allow the individual to express how much they agree or disagree with a particular statement.