This study aims to develop and validate a strength-based instrument for assessing the academic advising needs of university students in Hong Kong using the Appreciative Advising Inventory (AAI) as a blueprint. We reviewed the content validity and cultural relevance of the AAI and developed a 37-item AAI Hong Kong Version (AAI-HK). We conducted Rasch analysis and principal component analysis. The AAI-HK has four stable and reliable factors (social competence and support; positive identity and participation; positive values; and commitment to learning and study), which represent key strengths of university students. Four AAI-HK subscales demonstrate good reliability. We found significant differences in AAI-HK scores between students who are under academic probation or not and between students who are local or nonlocal.
The strength-based approach is an emerging framework for academic advising. The Appreciative Advising Inventory (AAI) provides a useful tool for strength-based advising and enables students to conduct self-checks of their strengths and assets for study. Based on the AAI, this study aims to develop a strength-based self-assessment tool for use with university students in Hong Kong. The study reviewed the content validity and cultural relevance of AAI items and determined how to modify them for use with students in Hong Kong. We evaluated psychometric properties of the new instrument by conducting Rasch analysis, principal component analysis, and reliability analysis, then analyzed factors that may contribute to differences in groups of students, such as gender, year of study, and whether the student is on academic probation.
Academic advising aims to help students get the most from their college experience, encourages them to reach their potential, and brings their talents to their educational planning (Schreiner & Anderson, 2005). Academic advising could contribute to student success in several areas, such as increase in rates of degree completion and student retention, academic achievement, personal development, making informed educational choices, enhancing learning experiences, better life planning; and better preparation for joining the workforce (Kuh, 2008). Academic advising often aims to address the risk factors for disruptions to or dropout from study, or factors that might enhance success in university. Literature in the United States increasingly illustrates that academic advising contributes to student graduation and success (Davis, 2015; Young-Jones et al., 2013; Zarges et al., 2018). Several advisor and student characteristics are associated with success in academic advising, such as advisor accountability and empowerment, student self-efficacy, student skills, and perceived support. Trust and advisor-student relationships clearly impact the process and outcomes of academic advising (Ohrt, 2018; Soria et al., 2017; Young-Jones et al., 2013).
In Hong Kong, the shared model of academic advising is the most commonly used. Faculty members and advisors in student affairs offices jointly provide academic advising (Miller, 2012). When compared with advising in the United States, academic advising in Hong Kong universities addresses some similar, but also some very different, issues. Like in the United States, academic advising in Hong Kong focuses on academic preparedness, retention, and disengagement from study because of the need to work part-time or full-time. Unlike in the United States, it is common for Hong Kong-based academic advisors or counselors to discuss academic issues in relation to peer pressure (Chan & Chan, 2013); family support (Siu & Chang, 2011); career development (Cheung et al., 2017); academic exchange and scholarship; behavior and lifestyle; and self-determination (Shek & Cheung, 2013).
Many approaches in academic advising emphasize the management of unfavorable factors for study, student retention, or success in graduation. For a long time, developmental advising has been the dominant approach in academic advising (Burton & Wellington, 1998; Creamer & Creamer, 1994), and advising practice remains either problem or solution focused (Hutson, 2010). In the past 2 decades, however, strength-based advising has emerged as the “new lens” for academic advising (Schreiner & Anderson, 2005). This approach advocates a personal growth mindset that aims to help students realize their potential (Bloom et al., 2013). Bloom et al., (2013) developed practical guidelines for training advisors to implement the six phases of Appreciative Education (i.e., disarm, discover, dream, design, deliver, and don't settle). There is accumulating evidence that supports the effectiveness of strength-based advising (Hutson, 2010; Sanders & Hutson, 2012), and it works best by integrating it into the first-year experience to create a growth mindset in students (Soria et al., 2017; Stebleton et al., 2012). It is also important to conduct quality assessment in appreciative advising, and the AAI was developed to address this need for a quality assessment instrument (He & Hutson, 2016; Hutson & He, 2011).
We reviewed existing standardized inventories that could capture the academic advising needs of undergraduate students and found many standardized inventories for the assessment of college adaptation or university adjustment (e.g., Baker & Siryk, 1984; Clinciu & Cazan, 2014; Le et al., 2005). Conversely, inventories for the assessment of student advising needs remain scarce. In strength-based assessments, some researchers have proposed a holistic appreciative assessment of students, advisors, stakeholders, and institutional environment and support (He & Hutson, 2016). Others have suggested the use of specific tests like the StrengthsFinder 2.0 (or CliftonStrengths) or the Myers-Briggs Type Inventory, which require further training and certification for use.
The central advising office of our university conducts academic advising for students and provides in-service training on academic advising to academic personnel. In this project, the academic advising office aimed to develop a self-completed questionnaire of advising needs to enhance the current service for students. The new questionnaire formed part of the online advising system and could be used in two ways. First, it could provide a self-assessment of students' assets, strengths, and weaknesses that may affect academic success. Academic advisors could invite students to complete this online inventory before meeting with them. From the results of the inventory, students could review their own advising needs, and advisors could obtain a preliminary assessment of advising needs and identify potential areas for discussion during advising sessions. Second, the advising office could use the questionnaire to screen students who have higher advising needs and encourage them to participate in the online academic advising system.
To develop a self-completed inventory for academic advising, we sought to adopt a strength-based advising approach, include fewer than 50 items which could be completed within 10 minutes, and have an English version as English is the medium of instruction in most universities in Hong Kong. We identified the AAI as one of the few inventories that met these criteria. Based on the Appreciative Advising framework (Bloom & Hutson, 2013), the AAI is a 44-item strength-based instrument that identifies the internal and external assets of university students, and it demonstrates favorable initial psychometric evidences (Howell, 2010; Hutson & He, 2011). The subscales of internal assets include commitment to learning, positive values, social competencies, and positive identity; the subscales of external assets include support/connectedness, empowerment, boundaries and expectations, and constructive use of time. The conceptual framework and items of the AAI provide an excellent reference point for developing an advising needs questionnaire for the academic and cultural context of Hong Kong.
This is an instrument development and validation study. We obtained the permission of AAI's original author (Jennifer Bloom, PhD) to translate and modify it for use in Hong Kong. We recruited expert panels to review the content validity and cultural relevance of the AAI for use with Hong Kong university students. Based on the expert panel review results, we removed, added, and modified items to form the Appreciative Advising Inventory Hong Kong Version (AAI-HK) (see Appendix A). We then collected data from students and evaluated the internal consistency, reliability, and construct and discriminant validity of the AAI-HK. We obtained ethical approval from our Institutional Review Board to conduct the study. We then provided information on our study to all potential participants via a Research Information Sheet, and all participants (including expert panel members and survey respondents) who agreed to join the study signed a consent form.
Review of Content Validity and Cultural Relevance
We invited experienced academic advisors in Hong Kong to form two expert panels for reviewing the content validity and cultural relevance of the AAI. Members of the first panel group were recruited from full-time academic advisors of the central advising office. The seven advisors (from the central advising office) who joined the focus group for the review provided academic advising to all students in the university. All hold degrees in psychology, counseling, or social work, and had 1–4 years of work experience in the advising office. We provided these advisors with information about the AAI and guidelines for review, and then we conducted a focus group to solicit their opinions on the content validity and cultural relevance of the AAI for use in Hong Kong.
The second panel was recruited from a core group of the Community of Practice in Academic Advising. They were five faculty members who provided academic advising to students of their own faculty as part of their academic duties. All were academic personnel with 5 to more than 20 years of advising experience. Members of the second panel completed a questionnaire that required them to comment on the content and cultural relevance of each item of the AAI, and we invited them to provide comments or suggestions for changes. Based on the reviews of the AAI by the two panels, we revised the AAI to create the AAI-HK. The details of the revisions will be reported in the first part of the Results section.
Based on the reviews by the two expert panels, 32 items were retained without changes. Table 1 lists the original and revised items of the inventory and the comments or suggested changes by expert panel members. We removed 12 items and added or rewrote 11 items. The test version had 43 items. The changes to the AAI are summarized in Table 1. First, some wording of terms in some items were modified to make them more understandable to local students. For example, the word “university” is much more commonly used than “college” (item 4), and the term “personal growth” was removed from item 5 as many students may not fully understand its meaning. Second, some items were revised to make them clearer. For example, in item 9, many local students may not fully understand the meaning of “convictions” in this context. Third, some items overlapped or were very similar in meaning, so redundant items were removed (e.g., items 23, 29, and 30). Fourth, some items were regarded as belonging to subscales other than those for which they were originally designed. For example, item 14 (“I believe in myself and my abilities”) was moved from the “Social competencies” subscale to the “Positive identity” subscale. Item 47 (“I am good at planning ahead and making decisions”) was moved from the “Social competencies” subscale to the “Constructive use of time” subscale. Fifth, some items were removed as they were not relevant to academic advising or the local context (e.g., item 29, “Someone outside my family supports my educational pursuits”). Last, the items we added were mainly based on common issues discussed with students during advising (e.g., “I have built positive relationships with my friends,” “I have a healthy lifestyle,” “I communicate with people effectively”). These items were suggested by the expert panels of academic advisors or counsellors.
We recruited survey participants through email announcements and notices posted on websites and around the university, as well as referrals from academic advisors. Academic counselors of the central advising office helped to recruit participants from students under their care when these students came for advising. The students under academic probation—those with a grade point average (GPA) of less than 2.0 in their last semester—were put under academic probation. They would be asked to withdraw from study if they earned a GPA of under 2.0 in two successive semesters. These students were often required by their program to seek consultation from academic advisors. Most of these students were recruited to join the study when they met with their academic advisors.
Most of the respondents completed the AAI-HK in physical form (82.4%) while the others completed an online version (17.6%). The survey was conducted in first 2 months of the academic year (September to October 2018). Most first-year students needed to complete the physical form as many of them did not yet have access to the online survey system of the university. As an incentive to encourage participation in the study, participants could leave their email address to enter a drawing for a free tablet or book coupons. We recruited 444 students of different academic programs to complete the finalized 43-item AAI-HK. After checking the completed questionnaires for missing data, a total of 410 completed questionnaires were valid.
Study of Construct and Discriminant Validity
We investigated the following psychometric properties of the AAI-HK, including factor structure, internal consistency, test-retest reliability, and discriminant validity. We did so by comparing 38 students with disruptions in their studies or who were under academic probation with 38 students who were not under academic probation (matched by age and gender).
Study of Dimensionality by Rasch Analysis
To examine whether the four AAI-HK subscales were unidimensional, we additionally conducted a Rasch-based principal component analysis (PCA) of residuals along with goodness-of-fit analysis. If the principal component in a subscale accounts for more than 50% of the total variance and the first contrast (the largest secondary component after the principal component is removed) has an eigenvalue of less than 2.0, the unidimensionality of the AAI-HK subscales is supported (Raîche, 2005). We also used Rasch-based goodness-of-fit statistics to examine how well the items fit with the model's expectations. Infit and outfit statistics were reported by using mean square (MnSq) and standardized Z values (Zstd). Infit and outfit with MnSq < 1.4 in combination with Zstd values of < 2.0 are indicators of acceptable model fit (Chien & Bond, 2009). Items with MnSq > 1.4 and Zstd > 2 indicated misfit of item responses with expectations of the Rasch model, and the item may belong to a different construct. The misfit items would be excluded from the subscale in a stepwise manner until all retained items demonstrated acceptable fit criteria. Additionally, Rasch analysis provided item and person reliability indices for describing the reliability of the items and the participants (Bond et al., 2021). For the interpretation of item and person reliability coefficients, we followed the standard of .70–.79 indicating acceptable, .80–.89 indicating good, and ≥ 0.90 indicating excellent (Portney, 2020).
Profile of Participants
Among the 410 participants, there were more females (58.9%) than males (41.1%). Sixty percent of the participants were freshmen; the rest were in Year 2 (13.3%), Year 3 (16.5%), and Year 4 (10.3%). There were more local students (72.2%) than nonlocal students (27.8%). Local students are permanent residents of Hong Kong. They applied for university places using public examination results, and they pay local tuition fees if admitted. Nonlocal students were those from mainland China and other countries; they were not residents of Hong Kong. These students are required to pay tuition fee rates for international students. While the participants came from all disciplines, the three largest groups of participants were from science and engineering (27.5%), health care (26.7%), and business (20.8%). Around half of the students (n = 189, 46.8%) had met with their academic advisor in the previous or current semester, and 50 (12.3%) of them had met with mental health counsellors.
To compare the AAI-HK scores between probation and nonprobation students, we recruited 38 students who were under academic probation. Regarding the profile of probation students, 74.2% were male, 50% were first year students, and 86.8% were local students. When compared with nonprobation students, probation students were more likely to be male (χ2 = 15.39, p > .001), local students (χ2 = 4.46, p = .04), and admitted through the joint university admission scheme using their public examination results.
Principal Component Analysis
Based on data from 410 participants, we conducted a conventional PCA of the AAI-HK items with varimax rotation (see Table 2). The scree plot showed that a four-factor solution was preferable. We attempted three- and five-factor solutions for comparison. The four-factor solution explained 48.7% of total variance. Item 25 was removed from subsequent analyses, as its factor loading was .33 (below the standard of .40). Thus, the first factor has 11 items (items 5, 6, 7, 21, 22, 23, 26, 34, 35, 36, 43) with factor loadings ranging from 0.44 to 0.68 and is labeled “Social Competence and Support.” The two items with highest factor loadings are “I have built positive relationships with my friends” (item 26), and “I feel loved by my family” (item 34). The second factor, labeled “Positive Identity and Participation,” has 11 items (items 1, 9, 12, 14, 24, 29, 31, 33, 37, 39, 41) with factor loadings ranging from 0.46 to 0.71. The two items with the highest factor loadings are “Right now I see myself as being pretty successful” (item 33), and “I feel that I have control over many things that happen to me” (item 14). The third factor, labeled “Positive Values,” has 11 items (items 2, 4, 10, 11, 20, 27, 28, 30, 32, 38, 42) with factor loadings ranging from 0.40 to 0.71. The two items with highest factor loadings are “I am committed to being a life-long learner” (item 42) and “I have a strong desire to achieve something in my life” (item 32). The last factor has nine items (items 3, 8, 13, 15, 16, 17, 18, 19, 40) with factor loadings ranging from 0.48 to 0.68 and is labeled “Commitment to Learning and Study.” The two items with highest factor loadings are “I intend to attend all my classes” (item 16) and “It is important that I meet my professors' or teachers' expectations” (item 18).
Rasch Analysis and Reliability
Rasch analysis supports that three out of four subscales were unidimensional (see Table 3). The results of the PCA of residuals for the Positive Identity and Participation, Positive Values, and Commitment to Learning and Study subscales were all acceptable, with the total variance explained by Rasch measures ranging from 43.6% to 44.8% and the first contrast having an eigenvalue less than 2.0. However, the Social Competence and Support subscale had an eigenvalue of 2.07, and the first contrast's eigenvalue was greater than 2, signifying the potential for multidimensionality. Each domain had one to two misfit items (MnSq > 1.4 and Zstd > 2) that could be removed or reworded afterwards, including items 5, 17, 20, 28, 29 (see Table 3). With the removal of the five misfit items from the four subscales, there were increases in the percentage of the total variance explained by Rasch-derived components and decreases in the eigenvalues of the first contrast. After the removal of these items, the person and item reliabilities of all four subscales were good, ranging from .80 to .86 for person reliability and .93 to .97 for item reliability, respectively.
Factors Associated with AAI-HK scores
We conducted a Multivariate ANOVA to explore the relationship of several factors in student profiles of the AAI-HK subscale scores. We found that the AAI-HK subscale scores were only associated with whether students were under academic probation or not, or whether students were local or nonlocal. Students under probation had significantly lower scores than other students (Wilk's λ = 3.52, p = .01), while nonlocal students had significantly higher scores than local students (Wilk's λ = 2.27, p = .06). The AAI-HK scores were not associated with gender (Wilk's λ = 1.49, p = .20) or year of study (Wilk's λ = 1.25, p = .29). A history of receiving student counseling service was not associated with the AAI-HK subscales, while experience with academic advising was associated with higher scores in the Social Competence and Support (F = 10.08, p = .02) and Positive Identity and Participation subscales (F = 11.22, p = .01).
This study developed a strength-based inventory (AAI-HK) based on the AAI, which was originally developed in the United States. The original AAI conceptualized internal assets (four areas of individual values and strengths) and external assets (four areas of environmental and social support) as the keys to student success. We identified four stable factors in the AAI-HK: social competence and support (SCS); positive identity and participation (PIP); positive values (PV); and commitment to learning and study (CLS). The SCS subscale could be regarded as a kind of environmental support, whereas the other three subscales (PIP, PV, and CLS) could represent individual values and strengths. In fact, the four factor subscales do represent key domains of development in emerging adulthood. The SCS and CLS subscales are indicators of social competence and commitment to academic achievement respectively, which is widely regarded as important for setting the course for positive adult development (Barry et al., 2009; Roisman et al., 2004). Positive identity and participation, represented by PIP subscale, is a key impetus to the university life and identity construction of students (Lairio et al., 2013).
While the original AAI has two domains with four subscales in each domain, the four subscale scores in the AAI-HK offer a more effective way of highlighting the key strengths of students. The reliability and validity of these subscales were supported by the results of psychometric evaluation in this study. The subscales were also useful in delineating whether students were on academic probation, and this provides supportive evidence for discriminant validity of the AAI-HK. Academic advisors in Hong Kong could use the AAI-HK to screen students' personal strengths and social support and use the results to further explore students' needs for support and academic advising.
The AAI-HK was developed out of a review and adaptation of the AAI. Two types of suggested modifications by experts illustrated the importance of cultural and content relevance. The first is using terms that are more commonly used by locals (e.g., using “university” instead of “college”). Experts also suggested not using some terms, such as “personal growth,” as many students may not understand their meaning. The second is adding some items that reflect common concerns among students (e.g., “I feel comfortable expressing my opinions or sharing my experiences in group discussions and activities,” “I have built positive relationships with my friends,” “I have a healthy lifestyle”).
This study contributes to university student services by developing a strength-based measure for the assessment of needs for academic advising. Our psychometric evaluation of the 43-item AAI-HK includes Rasch analysis, conventional principal component analysis, reliability estimates, and known-group analysis. We found that the AAI-HK had four stable factors after we removed one item with insignificant loadings in the principal component analysis, and five items because of misfit in the Rasch analysis. The reliabilities of the total scale and the subscales are good. Students who are under academic probation have fewer personal and environmental assets, as reflected in their significantly lower scores in the AAI-HK. After the evaluation, the final 37-item AAI-HK is ready for use in screening students' needs for academic advising.
The study has several implications for practitioners and researchers. First, this study validated an instrument that is available for use in appreciative or strength-based assessment in Hong Kong. Second, the structural validity study of the AAI-HK could be summarized under three personal and one social asset factors, and this provides preliminary support to the conceptualization and assessment of strengths among university students. Further validation of the scope and definitions of strengths relevant to academic advising should be conducted. Third, the results show that probationary students had significantly fewer personal and social assets, which suggests that it may be possible to conduct appreciative assessment using the AAI-HK to identify student needs for academic advising. Further research should recruit larger samples of probationary students and examine if the AAI-HK could delineate students who need academic advising.
There are several limitations to this study. First, the study used convenient sampling of university students, and the sample was composed primarily of freshmen students. The proportion of nonlocal students in the sample was also larger than in the overall student population, as more nonlocal students were willing to visit the advising office and thus to join the study. The participants' characteristics could have potentially affected the study results. Second, both online and paper questionnaires were used, and this may have influenced the survey results. Students who visited the university advising office mostly completed the paper questionnaire, while other students mainly completed the online questionnaire. We did not find a significant difference in the study results between students who completed the paper forms and those who completed the online forms. Third, we found that it could be hard for students to complete some items of AAI-HK , such as item 30 (“My university is a caring, encouraging place”), or item 1 (“I successfully balance my academic pursuits with my personal life”) at the university orientation meetings, For item 30, students would say they do not yet know the answer, and for item 1, students may ask if this item refers to their past or current experience.
In this study, we adapted the AAI for use with university students in Hong Kong (AAI-HK) and provided preliminary evidence for its psychometric properties. In the process of developing the AAI-HK, it was important to address the issues of content validity and cultural relevance. This involved removing and revising some items and writing new items. The AAI-HK's 43 items were reduced to 37 items after misfit items in the Rasch analysis and items with low factor loadings were removed. The AAI-HK has four stable and reliable factors (i.e., social competence and support [SCS], positive identity and participation [PIP], positive values [PV], and commitment to learning and study [CLS]) which could be used for screening the strengths of students in university life. The AAI-HK subscales' scores showed a significant difference between students who were and were not under academic probation, and between local and nonlocal students. Students who received academic advising had higher scores in the SCS and PIP subscales when compared with those who did not. These findings provide support for the validity of this instrument.
We acknowledge the support of Dr. Jennifer Bloom at Florida Atlantic University, who gave us permission to translate and adapt the Appreciative Advising Inventory for use with our students; she also provided advice on this study. The authors would like to acknowledge the financial support of our university for this study; the support of core members of the Community in Practice on Academic Advising; and the counselors of the advising office, who helped in the review and development of the inventory developed in this project. We would like to thank all the students who participated in this study.
Dr. Andrew M. H. Siu is reader in occupational therapy in the Department of Health Sciences, College of Health, Medicine, & Life Sciences, at Brunel University London, Uxbridge, UK. Contact him at: firstname.lastname@example.org
Dr. C.W. Chien is associate professor in the Department of Rehabilitation Sciences at The Hong Kong Polytechnic University in Hunghom, Kowloon, Hong Kong. Contact him at: email@example.com
Ms. Rhonda Y. S. Cheung is the senior academic advising officer in Student Affairs Office at The Hong Kong Polytechnic University, Hunghom, Kowloon, Hong Kong. Contact her at: firstname.lastname@example.org
Professor Daniel T. L. Shek is chair professor of applied social sciences, associate interim vice president (Research and Innovation), vice-president (Undergraduate Programme), and Li & Fung Professor in Service Leadership Education at The Hong Kong Polytechnic University in Hunghom, Kowloon, Hong Kong. Contact him at: email@example.com