SUMMARY

Auditing fair value measurements and other complex estimates (hereafter, FVMs) has received significant attention from regulators, practitioners, and researchers. Using a survey, we gather data from audit partners with FVM expertise to gain further insights in areas that have not been fully explored in the previous literature. Specifically, we extend the literature by providing a deeper understanding of the following areas: auditors' use of different substantive approaches to test higher-risk FVMs, auditors' use of pricing services and valuation specialists, and how challenges differ when auditing financial versus nonfinancial FVMs. In doing so, our findings and analyses extend prior research and coalesce pieces of several prior studies on auditing FVMs to provide a more complete picture of current auditing practices and challenges encountered by auditors. Our study provides insights useful in reconciling seemingly inconsistent findings in previous studies and provides important implications for future research, regulation, and standard-setting.

INTRODUCTION

The subjectivity inherent in estimating future events, coupled with the potential high degree of measurement uncertainty, makes auditing fair value measurements and other complex estimates (hereafter, collectively, FVMs) challenging for auditors (Public Company Accounting Oversight Board [PCAOB] 2007a, 2007b; Bratten, Gaynor, McDaniel, Montague, and Sierra 2013; Christensen, Glover, and Wood 2012; Glover, Taylor, and Wu 2016).1 The challenging nature of auditing FVMs is consistent with repeated and frequent PCAOB reported inspection deficiencies identified for both financial and nonfinancial FVMs (e.g., PCAOB 2008a, 2010a, 2012a). The most common deficiency is failure to evaluate the reasonableness of management's underlying assumptions and methods (PCAOB 2012a; Griffith, Hammersley, and Kadous 2015a). To cope with the difficulties of auditing FVMs, auditors frequently rely on in-house valuation specialists employed by the audit firm, third-party valuation specialists engaged by the audit firm, and third-party pricing services (e.g., PCAOB 2015). However, the PCAOB has voiced additional criticisms as a result of such reliance (PCAOB 2010b, 2012a, 2015).2

The PCAOB and other international regulators are concerned with the persistence of repeated deficiencies related to FVMs, as deficiencies can negatively affect investors' perceptions of audit quality and financial statement reliability (e.g., International Forum of Independent Audit Regulators [IFIAR] 2014; PCAOB 2014b). Although auditing regulators' views and concerns are publicly known and available, practicing audit experts' views are not (e.g., PCAOB 2014b, 2015; Glover et al. 2016). The PCAOB (2014a, 2015) seeks to gain a better understanding of current practices and challenges to ensure that the Board's initiatives to improve auditor performance are moving in the appropriate directions. Thus, insights from audit experts are crucial for improving auditor performance, especially in light of the PCAOB's (2014a, 2015) current standard-setting initiatives related to FVMs and auditors' use of specialists. Using a field-based survey that provides a channel for practitioners to openly express their opinions and views (e.g., Smith-Lacroix, Durocher, and Gendron 2012; Glover et al. 2016), we gather data from auditors who have deep experience with FVMs and the processes by which the firms audit FVMs.

To provide a more complete picture of current practices and key challenges encountered when auditing complex FVMs, this study has three primary objectives. Our first objective is to obtain deeper insights into auditors' use of substantive approaches to audit FVMs given criticisms of auditors' substantive approaches (PCAOB 2013; Griffith et al. 2015a). We are particularly interested in two of the three approaches outlined in PCAOB (2002) Auditing Standard 2502, Auditing Fair Value Measurements and Disclosures (hereafter, AS 2502)—developing an independent estimate and reviewing subsequent events and transactions. Research to date has focused on the frequency of substantive approaches, who performs the procedures, and challenges encountered (Cannon and Bedard 2017; Griffith et al. 2015a), with more attention given to understanding auditors' testing of management's assumptions, valuation model, and underlying data (Griffith et al. 2015a). We extend the literature by gaining a deeper understanding of how auditors use the substantive approaches. Importantly, our study provides insights regarding auditing higher-risk FVMs that are useful in understanding seemingly inconsistent findings in previous studies (Cannon and Bedard 2017; Griffith et al. 2015a).

Our second research objective is to provide improved understanding of auditors' use of pricing services and valuation specialists (in-house and third-party) when auditing complex FVMs. Research to date has largely focused on auditors' use of in-house valuation specialists (Griffith 2016). We contribute to the literature by clearly distinguishing between auditors' use of pricing services and valuation specialists, and whether usage varies between financial and nonfinancial FVMs. We also provide insights about auditors' evaluations of the reasonableness of pricing services' and valuations specialists' assumptions and methods, as well as factors driving auditors' decisions to use pricing services versus valuation specialists. In light of the PCAOB's (2015) initiative to revise auditing standards related to use of specialists, and given the differences in the pricing services versus valuation specialists in the nature and reliability of the services provided (Harvest Investments, Ltd. 2014; Interactive Data 2014; PCAOB 2014a, 2015), gaining an understanding of how auditors use these services is important.3

Our third objective is to further explore challenges auditors encounter when auditing FVMs by distinguishing between financial versus nonfinancial FVMs and investigating how auditors respond when they encounter problems with management's valuation expertise and knowledge. Previous research is silent as to differences in auditing financial versus nonfinancial FVMs even though auditors, accounting standards, and inspection findings differentiate these types of FVMs (Ernst & Young [EY] 2015; Financial Accounting Standards Board [FASB] 2011; PCAOB 2012a).

Audit partners with significant experience and expertise in auditing FVMs participated in our study. The survey uses objective and qualitative questions. The qualitative data from open-ended questions provide rich information to help researchers, regulators, and standard-setters gain a deeper understanding of current practices and challenges encountered (e.g., Gibbins and Qu 2005; Gendron 2009; Malsch and Salterio 2016). We did not ask audit partners to focus on a specific engagement experience; instead, to increase the breadth of our study (Power and Gendron 2015), we gather audit partners' views based on their experiences with harder-to-audit FVMs (e.g., Levels 2 and 3).

Our findings provide several new insights regarding substantive approaches used when auditing higher-risk FVMs (i.e., Levels 2 and 3). When auditing complex estimates in general, Griffith et al. (2015a) find that auditors may only test management's assumptions, valuation model, and underlying data (i.e., the first approach listed in AS 2502). However, when auditing high-risk, “highly challenging” FVMs, Cannon and Bedard (2017) report that auditors frequently use multiple substantive approaches. Our findings, in context with results of these two previous studies, suggest that when auditing typical complex estimates, auditors primarily use the first approach outlined in AS 2502, but as the complexity and associated risk of the estimate increase, auditors are more likely to use a combination of substantive approaches. We also find that when auditing higher-risk FVMs, a majority of audit partners report frequently using their audit team's assumptions to form independent estimates despite PCAOB auditing standards (AS 2502) permitting auditors to use management's assumptions. Moreover, when reviewing subsequent events, we find that the nature of the security (volume, trading frequency, and market volatility) drives the length of time that is considered reasonable between the trade date and valuation date.

Regarding use of pricing services and valuation specialists (in-house and third-party), we find that auditors report more frequently obtaining estimates from multiple pricing sources than from multiple valuation specialists for both financial and nonfinancial FVMs. When auditing financial FVMs, qualitative responses indicate that key factors driving decisions to use in-house versus third-party valuation specialists include firm resources, nature of the security, and firm policies. We also find differences in reported use of in-house versus third-party specialists when developing independent estimates for financial versus nonfinancial FVMs. Notably, our results provide new insights regarding how auditors evaluate the reasonableness of assumptions used by pricing services and valuation specialists, as well as factors driving decisions to use pricing services versus valuation specialists when auditing financial FVMs.

Finally, the majority of our participants believe that the challenges auditors encounter when auditing FVMs differ between financial and nonfinancial FVMs, with heightened challenges presented by nonfinancial FVMs. The key factor increasing the audit difficulty for nonfinancial FVMs is the lack of observable market information. Moreover, a majority of audit partners report encountering problems with management's lack of valuation knowledge, and qualitative responses provide a deeper understanding of audit responses in such settings.

Collectively, our results provide implications for regulators, standard-setters, and practitioners, and suggest avenues for future research. We provide timely insights about specific areas where the PCAOB could clarify standards to improve auditors' performance with respect to audits of complex FVMs and use of pricing services and valuation specialists (see PCAOB 2014a, 2015). Importantly, this study is the first to document that nonfinancial FVMs tend to be more challenging to audit than financial FVMs. Thus, auditing nonfinancial FVMs is another area that standard-setters and researchers should consider further.

BACKGROUND AND RELATED LITERATURE

Auditing FVMs is a challenging task given the high degree of subjectivity and extreme measurement uncertainty associated with some FVMs (e.g., Christensen et al. 2012; PCAOB 2012a; Glover et al. 2016). The challenge is exacerbated when auditing standards provide limited and, at times, vague guidance. Current standards related to auditing FVMs and use of specialists provide lists of possible audit procedures with relatively little guidance (e.g., PCAOB 2001, 2002; Bell and Griffin 2012; Christensen et al. 2012; Glover et al. 2016). For instance, AS 2502 allows auditors to select from three substantive approaches: (1) testing management's assumptions, valuation model, and underlying data; (2) developing an independent estimate; and (3) reviewing subsequent events and transactions. Although inspection findings and previous research have been critical of auditors' choice of the substantive approach (PCAOB 2012b, 2013; Griffith et al. 2015a), existing standards provide no apparent ranking or preference for one approach over another or when a combination of approaches is necessary. Therefore, auditors must exercise significant professional judgment to determine their audit strategy for FVMs.

The high frequency of PCAOB reported inspection deficiencies around audits of FVMs is not surprising considering task complexity, ambiguity in existing auditing standards, and the FVMs gap (which suggests that auditors and inspectors may disagree regarding what constitutes sufficient appropriate auditing evidence) (e.g., PCAOB 2010a, 2011a; Church and Shefchik 2012; Glover et al. 2016; Griffith et al. 2015a). The frequency of reported deficiencies has increased, with valuation-related audit issues (e.g., evaluating the reasonableness of management's underlying assumptions and valuation methods) remaining one of the top deficiencies noted (Griffith et al. 2015a; Valuation Research Corporation [VRC] 2013).

Existing research suggests that in certain circumstances, auditors may lack the requisite knowledge and expertise to appropriately assess the reasonableness of management's FVMs (e.g., Smith-Lacroix et al. 2012; Bratten et al. 2013; Griffith et al. 2015a). To cope with the challenges of auditing complex FVMs, auditors often turn to pricing services and valuation specialists to evaluate complex finance and economic models (Martin, Rich, and Wilks 2006; Bratten et al. 2013; Griffith et al. 2015a; PCAOB 2015). However, extensive reliance on pricing services and valuation specialists has led to PCAOB criticisms (e.g., PCAOB 2010a, 2010b, 2012a, 2014b; Bratten et al. 2013).

Research in the area of auditing FVMs is limited (Martin et al. 2006; Bratten et al. 2013); however, a number of concurrent studies are underway.4 Both Martin et al. (2006) and Bratten et al. (2013), as well as the PCAOB (2014a, 2015), encourage research to better understand the process of auditing FVMs and to identify major challenges encountered. Concurrent studies, using semi-structured interviews and field-based surveys, are beginning to fill this void in the literature (Cannon and Bedard 2017; Griffith et al. 2015a; Griffith 2016; Glover et al. 2016). Qualitative research provides rich observations from practitioners to help researchers better understand a “new phenomenon” such as the process of auditing complex FVMs in a regulated environment (Gendron 2009), and can help move future research in more meaningful directions relevant for practitioners, regulators, and standard-setters.

Using semi-structured interviews, Griffith et al. (2015a) gather audit partners' and senior managers' views regarding the typical process of auditing complex estimates to identify areas for improvement. Their results suggest that auditors over-rely on management's assumptions, fail to reconcile conflicting evidence, and encounter coordination difficulties with firms' in-house specialists. Griffith (2016) uses semi-structured interviews of audit partners and managers to identify challenges key auditors encounter when using in-house valuation specialists. Cannon and Bedard (2017) provide survey evidence that evaluating the reasonableness of management's assumptions and methods is one of the most frequently encountered challenges auditors face when auditing “highly challenging FVMs.” They conclude that auditors have trouble proposing audit adjustments given the highly subjective nature of model assumptions and valuation methods. Finally, Glover et al. (2016) find that extreme measurement uncertainty in FVMs further exacerbates the FVMs gap and increases the task complexity, and question whether the precision expected by the PCAOB when auditing complex FVMs is truly attainable.

The three primary objectives of our study extend the previous literature by contributing to: (1) the use of multiple substantive approaches, (2) the use of different types of specialists, and (3) the challenges encountered between financial and nonfinancial FVMs. Table 1 provides key areas explored and unique contributions of the current study, and serves as a roadmap for results to be presented below.

TABLE 1

Key Areas Explored and Unique Contributions in this Studya

Key Areas Explored and Unique Contributions in this Studya
Key Areas Explored and Unique Contributions in this Studya

SURVEY DESIGN

We use a field-based survey that includes both objective and open-ended questions, similar to previous studies (e.g., Gibbins, Salterio, and Webb 2001; Nelson, Elliott, and Tarpley 2002; Cannon and Bedard 2017). The survey includes multiple sections and demographic questions.5 To ensure that our questions would be important and relevant to practitioners, regulators, and researchers, we developed preliminary questions based on recent PCAOB inspection findings, previous research, auditing standards, and results of interviews with two national-level audit partners. Moreover, the Center for Audit Quality's (CAQ) Research Advisory Board (RAB), two national-level audit partners, and one retired audit partner provided feedback regarding our initial survey questions. We refined our questions based on that feedback, as well as additional feedback from partners from each of the five participating audit firms. Last, in terms of closed-ended questions provided to participants with a list of response options (e.g., possible problems encountered in practice), those lists were developed based on existing auditing standards and discussions with audit partners to ensure that the lists covered situations typical in practice. Moreover, those lists provided an “other” option that participants could select and explain in an open-ended fashion.

DATA COLLECTION AND ANALYSIS

Data Collection

With the assistance of the CAQ's Director of Research, we distributed our survey to five firms annually inspected by the PCAOB.6 The firms included three Big 4 accounting firms and two large international network firms. We requested that the surveys be distributed to audit partners with experience auditing FVMs.7 The firms' contacts distributed surveys to potential participants with additional follow-ups to encourage participation. Surveys were mostly completed online; a few were completed via hard copy.

Coding of Open-Ended Questions

Our survey included numerous open-ended questions to improve the richness and informativeness of the data gathered. Two of the researchers independently developed coding themes and reconciled differences to arrive at one set of coding themes for each question. Subsequently, another researcher independently read the open-ended responses and further refined the coding themes to finalize the coding rubric.8 Two doctoral students, unaware of the study's objectives, independently coded responses based on the rubric. Qualitative responses range from a few words to a brief paragraph, such that responses could be coded to one or more themes. Further, we did not restrict the number of themes the doctoral students used to code each response. The two doctoral students subsequently resolved coding differences. For all open-ended questions in our survey, the average Cohen's Kappa measure is 0.63 (all p < 0.01), which is an acceptable inter-rater agreement level.9

Participants

Table 2 presents the 32 participating audit partners' demographic information.10 Mean response regarding the extent to which participants are involved in the planning, supervising, or executing of audits of significant estimates, including FVMs, is 10.28 on an 11-point scale anchored with “very little involvement” (1) and “a great deal of involvement” (11), indicating that our participants had significant experience auditing FVMs. Participants have, on average, 22.3 years of experience and practice most in the following three industries: Financial Services/Depository/Financial Institutions/Asset Management (43.8 percent), Industrial Products and Manufacturing (25.0 percent), and Consumer Products and Goods (15.6 percent). Participants' average valuation and expertise in audits of FVMs is 8.53 on an 11-point scale anchored with “very little expertise” (1) and “a great deal of expertise” (11). Nearly 19 percent of participants indicate that they specialized in a particular type of FVM, with investment securities, asset impairment, real estate, and goodwill being the most common.

TABLE 2

Participant Demographicsa,b

Participant Demographicsa,b
Participant Demographicsa,b

RESULTS

Results are generally reported in aggregate without separating among various demographic factors, such as firm size, industry, and type of FVM specialization; however, we do highlight where there are significant differences across these categorical factors.11,12 We provide implications of the results and recommendations in the “Conclusion and Implications” section of the paper.

Current Practices in Auditing Fair Value Measurements

Research Objective 1: Use of Substantive Testing Approaches

To better understand current usage of the three substantive testing approaches when the inputs used to determine the FVM are not observable (i.e., Level 2 and 3 FVMs), participants reported the frequency with which their audit teams use each of the approaches individually or in combination.13 Results in Table 3, Question 1 show that 96.9 and 84.4 percent of our participants indicate that it is common to use all three approaches or a combination of approaches for financial and nonfinancial FVMs, respectively. Griffith et al. (2015a) report that auditors frequently use only the first substantive approach outlined in AS 2502 when auditing complex estimates in general. Consistent with Cannon and Bedard (2017), we find that auditors commonly use all three approaches when auditing “highly challenging” FVMs, where market value comparisons for model inputs are not available. Thus, auditors appear more likely to deploy the second and third approaches listed in AS 2502 as the risk of the FVM increases. Findings in prior studies within the context of our study are logical in that auditors would usually begin by testing management's process, and only in situations of heightened risk and complexity employ other substantive approaches. Our study extends prior work to provide a more complete picture regarding auditors' use of the three substantive testing approaches.

TABLE 3

Current PracticesSubstantive Testing Approaches

Current PracticesSubstantive Testing Approaches
Current PracticesSubstantive Testing Approaches

According to AS 2502, auditors are permitted to develop independent estimates by using an auditor-developed model using either management's assumptions or the auditors' assumptions (PCAOB 2002, ¶40). If auditors opt to use management's assumptions, then AS 2502 requires auditors to test management's significant assumptions, the valuation model, and the underlying data as they would if using the first substantive approach. This option appears to weaken the extent to which auditors are developing a truly “independent” estimate.14 Further, extending prior research on auditors' development of an independent estimate of FVMs (Cannon and Bedard 2017; Griffith et al. 2015a), Table 3, Question 2 shows that 25.0 percent of participants indicate frequently using management's assumptions when developing independent estimates, whereas 84.4 percent of participants indicate frequently using the audit team's assumptions (Question 3).

Regarding factors that drive auditors' decisions to use management's versus the audit team's assumptions when developing an independent estimate, Table 4, Question 1 shows that the three most frequently reported factors are (1) availability of independent data (25.4 percent); (2) the reliability of management's estimates (18.6 percent); and (3) availability of verifiable data (15.3 percent). Two partners provide relevant insights:

TABLE 4

Current PracticesSubstantive Testing ApproachesOpen-Ended Responses

Current PracticesSubstantive Testing ApproachesOpen-Ended Responses
Current PracticesSubstantive Testing ApproachesOpen-Ended Responses

It depends on the type of instrument, availability of information from outside sources, firm available resources, etc. If we are not able to develop our own assumptions independent of management, we would typically perform sensitivity analyses to validate the client's valuation and [then evaluate the] materiality of any potential differences that arise from the sensitivity analysis.

Preferably, management's assumptions would not be used, as this would hinder the ability to call it an “independent” estimate. The audit team should be utilizing its own assumptions to create a truly independent estimate.

The last response reiterates the notion that when auditors form independent estimates using their own model and assumptions, the resulting independent estimate would be expected to be more objective and of higher quality than using management's assumptions.

To understand how auditors respond when their independent estimates differ from management's by an amount greater than the planned tolerable range, audit partners described their audit responses. Table 4, Question 2 indicates that the three most frequently reported audit responses (not necessarily by order in the audit process) are: (1) identify and consider the cause of the difference (28.6 percent); (2) propose an adjustment for the material audit difference (20.6 percent); and (3) evaluate the appropriateness of assumptions/methods used by management (19.0 percent). Responses from two participating audit partners provide additional insight:

While the audit response depends on the facts and circumstances, it would generally begin with developing an understanding of what assumptions are driving the difference and whether based on individual company facts and circumstances there would need to be refinements made to our independently developed assumptions. The difference between the Company's estimate and our estimate would also be evaluated for materiality, as a tolerable range around an individual financial instrument may not necessarily be material to the financial statements as a whole, and would then be accumulated [on the summary of audit differences] along with other audit differences.

Consideration of whether there is bias by management and where such bias may be elsewhere in the portfolio or broader financial statements. Also, consideration of what assumptions may have caused management's fair value to differ from the independently developed value.

Finally, we gather additional insight regarding auditors' use of reviewing subsequent events and transactions as a substantive testing approach. In untabulated results, we find that 96.8 percent (n = 30) of participants use the most recent trades to support the FVMs of recorded securities. For participants who indicated use of this substantive approach, we asked what length of time between the trade date and the valuation date is considered reasonable. Table 4, Question 3 presents the most frequently mentioned responses: (1) relatively short/a few days (44.4 percent); (2) depends on nature of the security (27.8 percent); and (3) three months or less (13.9 percent). Qualitative responses suggest that for more frequently traded securities, the length of time is shorter than those with less active markets.

Research Objective 2: Auditors' Use of Pricing Services and Valuation Specialists

In this section, we first present results on auditors' use of pricing services and valuation specialists. PCAOB inspection reports identify numerous deficiencies related to auditors' use of pricing services when auditing financial FVMs (e.g., PCAOB 2008a, 2008b, 2011a, 2012c, 2012d). PCAOB inspection findings, as well as the formation of the Board's Pricing Source Task Force, suggest that the risks of using pricing services have increased (PCAOB 2011a, 2012d). As presented in Table 5, Question 1, 76.7 percent of audit partners agree that four to five years ago, issuers' and auditors' reliance on pricing services was usually considered a relatively low-risk audit area. Further, Table 5, Question 2 indicates that 80.0 percent agree that due to the recent economic crisis and/or increased focus by PCAOB inspectors, the use of pricing services by the issuer and/or the auditor is now considered to present relatively higher audit risk. Our results raise the question of whether use of pricing services really manifests higher audit risk or merely higher “inspection risk” driven by increased regulatory activity (Glover and Prawitt 2013; Glover et al. 2016). Our results suggest that both risks have increased.

TABLE 5

Questions Regarding Auditors' Use of Third-Party Pricing Service

Questions Regarding Auditors' Use of Third-Party Pricing Service
Questions Regarding Auditors' Use of Third-Party Pricing Service

PCAOB inspection reports also cite deficiencies where auditors have used the same pricing service as management (e.g., PCAOB 2011a, 2012d). Contrary to the inspection findings, Table 5, Question 3 shows that 87.1 percent of audit partners report frequently obtaining security prices from different pricing services than those used by management, with financial industry partners reporting more frequently obtaining security prices from different pricing services than those outside the financial industry (t(18) = 2.74, p < 0.01, two-tailed).

Consistent with Cannon and Bedard (2017), results presented in Table 5, Question 4 show that 33.3 percent of our participants reported frequently being unable to obtain sufficient information on the third-parties' proprietary models or assumptions.15 Given the multiple valuation approaches that pricing services use (PCAOB 2014a), as well as instances in which auditors are unable to obtain information regarding valuation methods, to expect that auditors may need to obtain estimates from more than one pricing service is reasonable. Our results support this notion given that audit partners report that 45.2 percent of audit teams frequently obtain estimates from more than one pricing service when auditing investment securities (Table 5, Question 5), with audit partners at Big 4 firms reporting more frequently obtaining multiple estimates than partners of non-Big 4 firms in our sample (t(29) = 3.12, p < 0.01, two-tailed). Furthermore, when considering whether to obtain evidence from pricing services, the three most frequently mentioned factors, as reported in Table 6, Question 1, are: (1) nature of the securities (25.0 percent); (2) required per firm policies (18.2 percent); and (3) availability, reliability, or cost of other data (and materiality and risk, which tied for third place) (13.6 percent).16

TABLE 6

Questions Regarding Auditors' Use of Third-Party Pricing Services and Valuation Specialists (In-House and Third-Party)Open-Ended Responses

Questions Regarding Auditors' Use of Third-Party Pricing Services and Valuation Specialists (In-House and Third-Party)Open-Ended Responses
Questions Regarding Auditors' Use of Third-Party Pricing Services and Valuation Specialists (In-House and Third-Party)Open-Ended Responses

Pricing services largely provide estimates that are not customized for a particular audit firm or company, as these services typically provide consensus prices (PCAOB 2014a). However, in some instances, pricing services may also use their own valuation methodology and assumptions, based on market information and other sources, to arrive at the estimate (PCAOB 2014a). We ask questions directed at better understanding how auditors assess the reasonableness of assumptions and methods used by pricing services. Results in Table 6, Question 2 indicate that for audit areas considered to be “higher” risk, participants most frequently mentioned using the following top three approaches to assess the reasonableness of assumptions, methods, and models used by pricing services: (1) involve the firm's in-house specialists (26.7 percent); (2) gain an understanding of the pricing methodology—e.g., method, model, and assumptions (24.4 percent); and (3) comparison with auditor's independent estimate, another pricing service, or similar instrument (22.2 percent). Consistent with the previous research (Cannon and Bedard 2017; Griffith 2016), our findings indicate that audit teams rely significantly on in-house specialists to assist with evaluating the most subjective aspects of management's estimates. The following audit partners' responses elaborate on approaches used by audit teams to assess the reasonableness of assumptions used by the pricing service:

The firm has evaluated the relevance and reliability of pricing information received from certain third-party pricing services for certain securities. For those securities that the prices obtained have not been determined to be relevant and reliable, the audit team will work with the firm's valuation specialists in assessing the reasonableness of management's estimates. This can include getting more pricing data from other third-parties, looking at pricing over periods of time, evaluating the assumptions/model used by the third-party or developing an independent estimate.

[We] compare information to other related/comparable pricing providers, if available, or comparison to other clients with similar investments. We also utilize our internal valuation team as they have a broader view of such instruments and provide assistance to us if the investments are less liquid or rare.

Next, we investigate how auditors address situations where management's recorded estimate of “higher” risk investment securities differs from the estimate provided by a pricing service. We ask participants whether the audit team typically performs additional audit procedures when the difference between the issuer's estimate and the pricing service's price is within the auditor's tolerable range. Untabulated results show that only approximately 23 percent (n = 7) responded “yes.” For those who responded “yes,” the two most frequently mentioned audit responses (Table 6, Question 3) are (1) perform additional procedures to test management's assumptions, the valuation model, and the underlying data (44.4 percent); and (2) obtain an estimate from a different pricing service (22.3 percent). Conversely, when the difference between the issuer's estimate and the pricing service's estimate is outside the auditor's tolerable range, approximately 97 percent (n = 29) of participants responded “yes.” For those who responded “yes,” the two most frequently mentioned audit responses, reported in Table 6, Question 4, are to understand the cause of the difference (28.6 percent) and perform additional audit procedures (20.4 percent). These audit partner responses provide further insights:

The response varies based on type of investment. We would need to understand the differences in assumptions and methodologies used and make a determination as to which was more appropriate and/or do our own corroborative testing including seeking internal or different third party evidence.

We consider whether a bias in management judgment may exist, and to what extent. We consider whether this is pervasive and perhaps impactful to other financial statement areas. And we consider expanding the sample size (if done on sample basis) to determine if the results are representative of the broader population.

We understand and test management's assumption that results in the different value and such difference must be audited to the level that is within the tolerable range.

The third most frequently mentioned audit response is to obtain an estimate from a different pricing service (18.4 percent). In terms of how audit teams use estimates from different pricing services, the following responses provide additional insights:

The audit response is to first quantify the extent to which the security's value, or securities' values when dealing with a population, falls outside the tolerable range in order to assess the materiality of the potential misstatement. If believed to be material, our next response would be to work with both the third-party pricing service and management to hopefully identify the driver of the pricing variance. In some circumstances, we may also seek pricing from yet another third-party pricing service as a means to help resolve the pricing variance.

We typically get more than one price estimate and evaluate where the client's estimate is relative to the multiple estimates we get. If the client is always at higher or lower range of estimates we may perform additional procedures. We try to understand which assumptions are driving the differences and evaluate them.

Our participants' responses suggest that when the difference between management's estimate and the pricing service's estimate is within the auditor's tolerable range, the overwhelming majority do not perform any additional audit procedures. Conversely, when the difference is outside the auditor's tolerable range, nearly all participants report performing additional audit procedures. Our results suggest that audit teams are reacting to the increased risk of material misstatement in instances where pricing services provide estimates that differ significantly from management's estimates.

We now turn to auditors' use of valuation specialists (in-house and third-party) when auditing financial and nonfinancial FVMs. Untabulated results indicate that approximately two-thirds (one-third) of our participants reported that they use in-house (third-party) valuation specialists to support the audit work performed for financial FVMs. Moreover, approximately 87 percent (13 percent) of the audit partners indicated that they use in-house (third-party) valuation specialists to support the audit work for nonfinancial FVMs. Cannon and Bedard (2017) and Griffith (2016) find that audit teams extensively use in-house specialists when auditing FVMs in general, and our results add to the literature by demonstrating that reliance on in-house specialists is highest for nonfinancial FVMs.

We also investigate factors that drive auditors' decisions to use in-house versus third-party valuation specialists. Table 6, Question 5 reports that the three most frequently mentioned factors audit partners consider when determining which type of specialist to use are (1) audit firm's resources in terms of costs/benefits, expertise, efficiency (43.2 percent); and tied for second are (2) the nature of the security being audited (21.6 percent), and (3) firm policy requires use of in-house valuation specialist (21.6 percent). Two audit partners' responses provide further insight regarding factors driving the type of valuation specialists to use:

I would say that generally we would look to the client's third-party valuation specialist first and then, depending on complexity and materiality, have our internal [in-house] specialists review the outside valuation related to both financial and nonfinancial FVMs.

Our firm has deep valuation competencies, so it is very rare we would have to go to a third-party [specialist]. The key factors considered are experience in valuing the type of asset and reputation.

Cannon and Bedard (2017) report that audit teams frequently use in-house specialists to develop independent estimates. Our study extends their research by providing additional insights regarding whether the frequency of using in-house specialists to develop independent estimates differs between financial and nonfinancial FVMs. Table 7, Question 1 shows that for both financial and nonfinancial FVMs, auditors commonly use valuation specialists (in-house or third-party) to develop independent estimates (86.7 percent for financial instruments and 80.0 percent for nonfinancial assets/liabilities, respectively). We also find that for financial FVMs, audit partners in financial industries report more frequently using valuation specialists to develop independent estimates (t(19) = 2.95, p = 0.01, two-tailed).

TABLE 7

Questions Regarding Auditors' Use of Valuation Specialists

Questions Regarding Auditors' Use of Valuation Specialists
Questions Regarding Auditors' Use of Valuation Specialists

To further distinguish between use of in-house valuation specialists and third-party valuation specialists, results presented in Table 7, Question 2 indicate the frequency with which in-house specialists, third-party specialists, or members of the audit team are utilized to develop independent estimates. A majority of audit partners report using in-house specialists to develop independent estimates for both financial and nonfinancial FVMs; 83.4 percent and 96.7 percent, respectively. However, our results show that less than 50 percent of audit partners report frequently using audit team members and third-party specialists to develop independent estimates for both financial (46.7 percent versus 40.0 percent, respectively) and nonfinancial FVMs (46.7 percent versus 23.3 percent, respectively). Also, we find that 40.0 (23.3) percent of audit partners report using third-party specialists to develop independent estimates for financial (nonfinancial) FVMs. Consistent with Griffith (2016), our results indicate some differences between Big 4 and non-Big 4 firms in this area. Specifically, for both financial and nonfinancial FVMs, in-house valuation specialists are used more frequently to develop independent estimates for Big 4 firms than non-Big 4 firms (t(28) = 2.76, p = 0.01, two-tailed; t(28) = 3.86, p < 0.01, two-tailed, respectively). Thus, third-party valuation specialists are more frequently used by non-Big 4 firms to develop independent estimates than Big 4 firms (t(28) = 2.84, p < 0.01, two-tailed). Finally, results suggest that for nonfinancial FVMs, engagement team members in non-Big 4 firms are more likely to develop independent estimates than those of Big 4 firms (t(27) = 3.58, p < 0.01, two-tailed).

Depending on the inputs, assumptions, methods, and models used, estimates from one valuation specialist compared to another can differ. As reported in Table 7, Question 3, for both financial and nonfinancial FVMs, participants report infrequently obtaining estimates from more than one valuation specialist (16.7 percent and 6.7 percent, respectively). Moreover, for financial FVMs, we find that financial industries' audit partners report more frequently obtaining multiple estimates than nonfinancial industries' partners (t(28) = 2.44, p = 0.02, two-tailed). Results previously presented indicate that approximately 45 percent of participants frequently obtain estimates from more than one pricing service. Importantly, our findings indicate that the frequency with which auditors obtain multiple estimates differs between valuation specialists and pricing services.

We next examine how audit teams test the data and assess the reasonableness of assumptions and methods when a valuation specialist's work is used to support the audit. Table 6, Question 6 reports that the three most frequently mentioned approaches are (1) obtain support/audit evidence for data used in the model from both internal and external sources (30.2 percent); (2) reliance on the audit firm's in-house valuation specialist to evaluate the appropriateness of assumptions (24.5 percent); and (3) the auditor gains understanding of key assumptions via the in-house valuation specialists (15.1 percent). In terms of how the audit team evaluates the assumptions and relies on the in-house valuation specialist, two partners elaborate:

If it is an assumption that relates to the company's activities/operations, those would be audited by the engagement team to determine whether they were reasonable/appropriate. For items such as WACC [weighted average cost of capital], discount rates, or other direct valuation inputs, those are determined by the [in-house] valuation specialist and not separately tested by the audit team.

Our [in-house] valuation specialists are required to provide the support (articles, studies, output from Bloomberg, etc.) as part of their documentation. We will validate that information on a test basis. For those assumptions that require judgment of our [in-house] valuation specialists, we generally have two or more specialists involved in each area, both of which are required to sign the audit documentation memorandum, indicating review of these inputs by more than one qualified specialist.

These results are consistent with previous research (Cannon and Bedard 2017; Griffith 2016) and our findings regarding use of pricing services. That is, auditors extensively rely on in-house valuation specialists to test components of the estimate where significant subjectivity exists. This approach, however, potentially exposes auditors to regulators' criticisms of overreliance on valuation specialists (e.g., PCAOB 2011a, 2012d).

Finally, we investigate factors that drive auditors' decisions to use a pricing service versus a valuation specialist when auditing investment securities in inactive markets and/or more complex financial instruments. Table 6, Question 7 shows that the three most frequently reported factors that drive this decision are (1) availability of independent/reliable pricing from pricing services (26.8 percent); (2) for this category of securities, generally use in-house specialists (22.0 percent); and (3) materiality and risk of misstatements (19.5 percent). To further elaborate, two audit partners noted the following:

The decision is primarily driven by our past experience in valuing similar securities/instruments. We would typically use a pricing service by default to the extent possible and then move to a valuation specialist in instances where a pricing service was unable to value the security or where we had concerns over the quality of what a pricing service could provide based on the nature of the security and our understanding of the methodologies they employ.

For those types of securities, while we may obtain price information from a third-party pricing service, we will also use our [in-house] valuation specialists given the nature of these securities, the price obtained from the pricing service may not be relevant and reliable.

Research Objective 3: Challenges in Auditing Fair Value Measurements

In this section, we examine challenges encountered when auditing Level 2 and 3 FVMs and potential causes when problems with FVMs are identified. Several concurrent survey studies (Cannon and Bedard 2017; Griffith et al. 2015a; Griffith 2016; Glover et al. 2016), along with PCAOB (2010a, 2010b, 2011b, 2014b) inspection findings, identify challenges that contribute to deficient performance when auditing FVMs. Similar to findings in prior studies, participants report the following top three challenges when auditing complex FVM inputs (presented in Table 8, Question 1): (1) lack of verifiable and corroborative evidence (42.9 percent); (2) high degree of subjectivity involved in estimates (14.3 percent); and (3) degree of difficulty in assessing reasonableness of assumptions and benchmarks (14.3 percent).

TABLE 8

Challenges in Auditing FVMsOpen-Ended Responses

Challenges in Auditing FVMsOpen-Ended Responses
Challenges in Auditing FVMsOpen-Ended Responses

To identify the nature and frequency of problems auditors encounter when issues with FVMs are identified, we asked participants to rate the frequency of each problem given the list provided in Table 9, Question 1.17 Findings in Table 9 are consistent with findings in Cannon and Bedard (2017), and the three most frequently encountered problem areas are: (1) appropriateness of management's assumptions (90.6 percent); (2) adequacy of management's documentation (65.6 percent); and (3) sufficiency of expertise and experience of those determining the estimate (62.4 percent). Moreover, non-Big 4 partners report more frequently encountering issues associated with the appropriateness of management's method/model than Big 4 partners (t(30) = 2.77, p < 0.01, two-tailed). Further, audit partners in financial industries report more frequently encountering issues with management's assumptions than those in nonfinancial industries (t(30) = 2.17, p = 0.04, two-tailed).

TABLE 9

Challenges in Auditing FVMs

Challenges in Auditing FVMs
Challenges in Auditing FVMs

We extend the previous literature by delving deeper into auditors' responses when management lacks expertise, creating additional challenges for audit teams. As shown in Table 9, Question 2, 58.1 percent of participants report frequently encountering situations where management's knowledge and expertise of valuation processes and methodologies is relatively shallow. Our participants' responses, presented in Table 8, Question 2, indicate that the top three audit responses in such cases include (1) auditor expects that management will place greater reliance on third-party specialists (20.7 percent); (2) consider impact on internal control effectiveness (19.0 percent); and (3) auditors place greater reliance on valuation specialists and perform additional audit procedures, tied for third place at 17.2 percent.

Qualitative responses provide further insight regarding the form of additional reliance on valuation specialists. Audit partners report expecting management to seek assistance from knowledgeable third-party valuation specialists. Moreover, audit teams place greater reliance on valuation specialists (in-house and/or third-party) to assist with developing independent estimates and evaluation of models. Further, our results suggest that when management's expertise regarding valuation and methods is limited, auditors assess control risk to be higher and perform more substantive tests. To further elaborate, one audit partner stated:

If management is not knowledgeable of valuation methodologies, there would be a communication to the Board or Audit Committee of such a finding as we are required to bring such communications to those with oversight responsibilities. Our [substantive] testing would therefore be more detailed and sample sizes would increase.

Another audit partner provides additional insight on how the audit team responds when management knowledge regarding valuation methodologies is shallow:

One would expect less reliance on management's assumptions/methodologies surrounding complex financial instruments and more reliance on independent valuation methods and sources of information. Also, determination may be made that controls are not sufficient in this area to be considered effectively designed and operating, thereby requiring more substantive testing.

Last, to better understand challenges encountered when auditing FVMs, we asked whether participants believe the challenges they described are different between financial versus nonfinancial FVMs. Untabulated results indicate that roughly 63 percent (n = 20) of partners report that challenges differ between financial versus nonfinancial FVMs; the three most frequently mentioned differences, noted in Table 8, Question 3, are (1) difference in availability of market data (56.5 percent); (2) degree of auditor familiarity (13.0 percent); and (3) difference in perceptions of precision by regulators and users, and degree of consistency possible when applying processes, methods, and/or assumptions, tied for third place at 8.8 percent. Three partner responses clarify why financial and nonfinancial FVMs present different challenges:

It has been my experience, that when dealing with nonfinancial assets/liabilities you are typically having to rely on management prepared data (e.g., forecasts), where for many financial assets you are able to source some other independent data.

Financial instruments are more likely to have some observable market comparisons than non-financial instruments.

There seem to be more third party [pricing-services and] valuation [specialists] involved in valuing financial assets/liabilities. The valuation for nonfinancial assets/liabilities relies more heavily on management's assumptions.

The primary challenge audit partners report is obtaining verifiable and corroborative evidence from independent sources for nonfinancial FVMs, thus increasing challenges when auditing nonfinancial FVMs compared to financial FVMs. When such evidence is not available, auditors must focus on management's assumptions and analyses.

SUGGESTIONS FOR FUTURE RESEARCH

Our results provide several avenues for future research. Given differences in current practice noted by our study and concurrent research (e.g., Cannon and Bedard 2017; Griffith et al. 2015a), future research can investigate more fully the key factors that drive auditors' choice in substantive testing approaches when auditing FVMs. Additionally, researchers can theorize about when auditors should develop independent estimates instead of testing management's processes and whether developing independent estimates ultimately improves audit quality. Moreover, our results suggest that future research on auditing FVMs may need to consider the nature of the estimate, as well as differences in the risks and challenges encountered between financial versus nonfinancial FVMs. Our findings suggest that other fruitful research questions exist. For instance, do auditors' risk assessment techniques, control testing, or substantive testing approaches differ significantly between financial and nonfinancial FVMs? If so, how?

Our findings suggest areas where PCAOB auditing standards may require revision. Future research can investigate the likely effects of possible modifications to existing standards on auditors' performance when auditing FVMs. Future research can also investigate whether other alternatives (i.e., changes in firm methodology) or combinations of alternatives may be more effective at mitigating key challenges auditors encounter and, thus, improve audits of FVMs and reduce the frequency of repeated PCAOB inspection deficiencies.

We find that the most common response to a difference in the auditor's independent estimate and management's FVMs for higher-risk areas is proposing an audit adjustment. As previously noted, Cannon and Bedard (2017) find that the likelihood of proposing adjustments depends on whether the auditor uses a valuation specialist. Our open-ended question only asked participants to consider audit responses where an independently developed FVM estimate is outside the tolerable range for a higher-risk area. Thus, future research could pursue additional insights into the specialist's role in prompting auditors to propose adjustments.

With respect to differences between management's and pricing services' FVM estimates, we did not specifically ask audit partners to provide causes for those differences; thus, responses do not provide sufficient detail to fully understand factors driving such differences. Our survey question was designed to understand auditors' responses when such differences arise. Thus, another avenue for future research is to identify causes of such differences. Last, the additional insights we provide regarding auditors' use of various types of valuation specialists and pricing services provide questions academics can address as they investigate specific characteristics related with the use of valuation specialists that may unknowingly influence auditors' judgments and decisions (e.g., Joe et al. 2016, Joe et al. 2017).

CONCLUSION AND IMPLICATIONS

To provide a more complete picture of current practice and key challenges encountered when auditing FVMs, we surveyed audit experts with FVM expertise from several firms annually inspected by the PCAOB. The rich qualitative data we gathered from audit partners enhance regulators', standard-setters', and researchers' understanding of current practices and challenges in auditing FVMs and provide several avenues for future research. Moreover, we provide timely insights for regulators and standard-setters relating to areas specifically highlighted in the PCAOB's request for comments to better understand current practices, and we highlight areas where standards could be improved to enhance auditor performance (PCAOB 2014a, 2015). Our three primary research objectives are designed to provide insights regarding audits of FVMs in the following areas: (1) substantive audit approaches, (2) audit team use of pricing services and valuation specialists, and (3) significant challenges encountered.

With respect to Research Objective 1, our study provides a more in-depth understanding of current audit practices related to auditors' use of substantive approaches outlined in AS 2502. Our results, taken together with prior studies (Cannon and Bedard 2017; Griffith et al. 2015a), suggest that auditors are likely to use the first substantive approach (i.e., test management's assumptions and underlying data) when auditing typical or lower-risk estimates, but as audit risk and complexity increase, they are more likely to use a combination of approaches. Moreover, despite AS 2502 allowing auditors to use management's assumptions when developing an independent estimate, our results indicate that auditors typically opt instead to use the audit team's own assumptions to develop the estimate. We provide insights into auditors' responses when their independently developed estimate is outside of the tolerable range for “high-risk” FVMs. We note that the PCAOB standards currently provide little guidance regarding situations where it is, or is not, appropriate to use management's assumptions instead of developing the team's own assumptions to derive independent estimates. Our results also raise concerns as to whether using management's assumptions to develop an independent estimate is truly “independent” and has potential audit quality implications. Therefore, an area for the PCAOB to reconsider when revising existing auditing standards is to contemplate when, if at all, auditors should use management's assumptions when developing independent estimates.

With respect to Research Objective 2, our results provide several important new insights, including more clearly distinguishing between auditors' use of pricing services and valuation specialists and factors that drive this decision, as well as provide additional insights regarding differences in the use of valuation specialists for financial and nonfinancial FVMs. For instance, we find that a greater percentage of audit partners report using third-party valuation specialists to develop independent estimates for financial FVMs than for nonfinancial FVMs. Regarding the audit team's decision to use a pricing service or a valuation specialist (whether in-house or third-party) when auditing complex financial FVMs, our results indicate that reliability of the pricing service is one important factor that informs this decision. Our results highlight potential differences in reliability and risks associated with using pricing services versus valuation specialists. The valuation methods and assumptions used by third-party valuation specialists tend to go through more scrutiny than those used by pricing services and, thus, auditors should assess pricing services' reliability (Harvest Investments, Ltd. 2014). Relatedly, we also find differences in the likelihood of obtaining multiple estimates between pricing services and valuation specialists. Possible reasons why auditors are less likely to obtain multiple estimates when using valuation specialists compared to when using pricing services include: (1) easier and less costly to obtain pricing quote from another pricing service than to obtain another custom estimate from a different valuation specialist, (2) for in-house valuation specialists, his or her knowledge, skills, and ability have been evaluated and, thus, considered a trusted and reliable source of information, and (3) valuation specialists' work may be more reliable than pricing services given that they typically assume expert liability for the work they performed (Interactive Data 2014).

Our results also highlight several important areas where auditors need clearer guidance from the standard-setters regarding use of pricing services and valuation specialists (PCAOB 2014a, 2015). In one instance, we note that the PCAOB's (2015) initiatives to revise standards related to use of specialists exclude pricing services. However, inspection findings suggest that auditors' use of pricing services and third-party valuation specialists are subject to similar requirements of the PCAOB (1998) Auditing Standard 1210 (formerly AU 336), Using the Work of a Specialist, in that auditors are required to “obtain an understanding of the methods and assumptions used by the specialist” (or pricing service). Moreover, we find similarities between auditors' use of pricing services and valuation specialists, thus suggesting that the PCAOB should consider whether excluding pricing services from its revisions to the existing standards is appropriate. If auditors' use of pricing services should differ from use of valuation specialists, then auditing standards should clarify how audit approaches and procedures should differ between the two. Our results, consistent with Cannon and Bedard (2017), emphasize the frequency with which auditors are unable to obtain sufficient information on the pricing services' proprietary models or assumptions. We recommend that regulators continue to work with the pricing industry to foster cooperation in sharing information with auditors. Our findings also suggest that the PCAOB should consider providing additional guidance to address situations when pricing services will not share proprietary information.

With respect to Research Objective 3, our results shed new light on whether audit challenges differ for financial versus nonfinancial FVMs; a majority of our participants believe that challenges do differ and the key reason is the lack of observable market information for nonfinancial FVMs. Our participants also report problems with management's lack of valuation process knowledge. In such situations, auditors expect management to increase reliance on third-party valuation specialists, with the audit team evaluating the impact on internal controls, performing additional procedures, and placing more reliance on in-house valuation specialists. This is an area for which standard-setters could provide additional guidance for situations where auditors rely heavily on management's assumptions because little or limited market data are available for nonfinancial FVMs. Our results also raise concerns regarding auditability of estimates with extreme measurement uncertainty. Thus, this is an important area for regulators to consider, particularly given that Cannon and Bedard (2017) and Glover et al. (2016) report that auditors frequently encounter measurement uncertainty greater than materiality when auditing highly complex estimates.

Our results also identify areas where auditors' reported behavior differs from PCAOB inspection findings. For example, contrary to PCAOB (2012a, 2012d) inspection findings, audit partners indicate that audit teams investigate significant differences between the independent estimates developed by the audit team and those by management. In another instance, contrary to PCAOB (2011a, 2012d) inspection reports citing auditors for using management's pricing service, we find that auditors frequently obtain security prices from pricing services other than those used by management. The difference in behavior characterized by inspection findings and behavior reported by auditing experts in our study could be due to the inspection process resulting in improved auditor performance, the inspection results not being representative of auditor performance given the PCAOB's non-representative risk-based selection process, or disagreements between inspectors and auditors (see Glover et al. 2016).

This study is subject to limitations. First, we gather audit partners' views from firms annually inspected by the PCAOB; thus, our results may not reflect practices and challenges encountered in smaller audit firms. Second, our study focuses on auditors' experiences when auditing higher-risk, harder-to-audit FVMs; therefore, our results may not be representative of audit experiences for more general FVMs. Despite these limitations, this study provides important insights regarding current practices and challenges encountered when auditing high-risk FVMs.

REFERENCES

REFERENCES
Backof
,
A. G
.,
J
.
Thayer
,
and
T
.
Carpenter
.
2015
.
Auditing Complex Estimates: Management-Provided Evidence and Auditors' Consideration of Inconsistent Evidence
.
Working paper
,
University of Virginia and The University of Georgia
.
Bell
,
T. B
.,
and
J. B.
Griffin
.
2012
.
Commentary on auditing high-uncertainty fair value estimates
.
Auditing: A Journal of Practice & Theory
31
:
147
155
.10.2308/ajpt-10172
Bratten
,
B
.,
L. M
.
Gaynor
,
L
.
McDaniel
,
N. R
.
Montague
,
and
G. E
.
Sierra
.
2013
.
The audit of fair values and other estimates: The effects of underlying environmental, task, and auditor-specific factors
.
Auditing: A Journal of Practice & Theory
31
:
127
146
.
Cannon
,
N
.,
and
J. C
.
Bedard
.
2017
.
Auditing challenging fair value measurements: Evidence from the field
.
The Accounting Review
(forthcoming)
.
Christensen
,
B. E
.,
S. M
.
Glover
,
and
D. A
.
Wood
.
2012
.
Extreme estimation uncertainty in fair value estimates: Implications for audit assurance
.
Auditing: A Journal of Practice & Theory
31
(
1
):
127
146
.10.2308/ajpt-10191
Church
,
B. K
.,
and
L. B
.
Shefchik
.
2012
.
PCAOB inspections and large auditing firms
.
Auditing: A Journal of Practice & Theory
26
(
1
):
43
63
.
Earley
,
C
.,
V. B
.
Hoffman
,
and
J. R
.
Joe
.
2015
.
Are Auditors Skeptical of Management's Level 2 versus Level 3 Fair Value Classification Judgments?
Working paper
,
Providence College, University of Pittsburgh, and University of Delaware
.
Ernst & Young (EY)
.
2015
.
Financial Reporting Developments: A Comprehensive Guide to Fair Value Measurement
.
Financial Accounting Standards Board (FASB)
.
2011
.
Fair Value Measurement (Topic 820): Amendments to Achieve Common Fair Value Measurement and Disclosure Requirements in U.S. GAAP and IFRS
.
Accounting Standards Update No. 2011-04
.
Norwalk, CT
:
FASB
.
Gendron
,
Y
.
2009
.
Discussion of the audit committee oversight process: Advocating openness in accounting research
.
Contemporary Accounting Research
26
(
1
):
123
134
.10.1506/car.26.1.4
Gibbins
,
M
.,
and
S. Q
.
Qu
.
2005
.
Eliciting experts' context knowledge with theory-based experiential questionnaires
.
Behavioral Research in Accounting
17
:
71
88
.10.2308/bria.2005.17.1.71
Gibbins
,
M
.,
S
.
Salterio
,
and
A
.
Webb
.
2001
.
Evidence about auditor-client management negotiation concerning client's financial reporting
.
Journal of Accounting Research
40
(
3
):
535
564
.10.1111/1475-679X.00027
Glover
,
S. M
.,
and
D. F
.
Prawitt
.
2013
.
Enhancing Auditor Professional Skepticism. Monograph produced by the Standards Working Group of the Global Public Policy Committee
.
Glover
,
S. M
.,
M
.
Taylor
,
and
Y
.
Wu
.
2016
.
Mind the Gap: Why Do Experts Have Differences of Opinion Regarding the Sufficiency of Audit Evidence Supporting Complex Fair Value Measurements?
Working paper
,
Brigham Young University and Texas Tech University
.
Griffin
,
J. B
.
2014
.
The effects of uncertainty and disclosure on auditors' fair value materiality decisions
.
Journal of Accounting Research
52
(
5
):
1165
1193
.10.1111/1475-679X.12059
Griffith
,
E. E
.
2016
.
Auditors, Specialists, and Professional Jurisdiction in Audits of Fair Values
.
Working paper
,
University of Wisconsin–Madison
.
Griffith
,
E. E
.,
J. S
.
Hammersley
,
and
K
.
Kadous
.
2015
a
.
Audits of complex estimates as verification of management numbers: How institutional pressures shape practice
.
Contemporary Accounting Research
32
:
833
863
.10.1111/1911-3846.12104
Griffith
,
E. E
.,
J. S
.
Hammersley
,
K
.
Kadous
,
and
D
.
Young
.
2015
b
.
Auditor mindsets and audits of complex estimates
.
Journal of Accounting Research
53
(
1
):
49
78
.10.1111/1475-679X.12066
Harvest Investments, Ltd
.
2014
.
Re: Request for Public Comment—Staff Consultation Paper, Auditing Accounting Estimates and Fair Value Measurements
. (
November 3
).
Interactive Data
.
2014
.
Re: Staff Consultation Paper, Auditing Accounting Estimates and Fair Value Measurements
. (
November 3
).
International Forum of Independent Audit Regulators (IFIAR)
.
2014
.
Report on 2013 Survey of Inspection Findings
.
Joe
,
J. R
.,
S. D
.
Vandervelde
,
and
Y
.
Wu
.
2017
.
Use of High Quantification Evidence in Fair Value Audits: Do Auditors Stay in Their Comfort Zone?
Working paper
,
University of Delaware, University of South Carolina, and Texas Tech University
.
Joe
,
J. R
.,
Y
.
Wu
,
and
A
.
Zimmerman
.
2016
.
Overcoming Communication Challenges: Can Taking the Specialist's Perspective Improve Auditors' Critical Evaluation and Integration of the Specialist's Work?
Working paper
,
University of Delaware, Texas Tech University, and Case Western Reserve University
.
Landis
,
J. R
.,
and
G. G
.
Koch
.
1977
.
The measurement of observer agreement for categorical data
.
Biometrics
33
:
159
174
.10.2307/2529310
Maksymov
,
E
.,
M. W
.
Nelson
,
and
Kinney
,
W. R
. Jr.
2015
.
Planning Audits of Fair Values: Effects of Procedure Frame and Perceived Procedure Verifiability
.
Working paper
,
Arizona State University, Cornell University, and The University of Texas at Austin
.
Malsch
,
B
.,
and
S. E
.
Salterio
.
2016
.
“Doing good field research”: Assessing the quality of audit field research
.
Auditing: A Journal of Practice & Theory
35
(
1
):
1
22
.10.2308/ajpt-51170
Martin
,
R. D
.,
J. S
.
Rich
,
and
T. J
.
Wilks
.
2006
.
Auditing fair value measurement: A synthesis of relevant research
.
Accounting Horizons
20
(
3
):
287
303
.10.2308/acch.2006.20.3.287
Nelson
,
M. W
.,
J. A
.
Elliott
,
and
R. L
.
Tarpley
.
2002
.
Evidence from auditors about managers' and auditors' earnings management decisions
.
The Accounting Review
77
(
Supplement
):
175
202
.10.2308/accr.2002.77.s-1.175
Neuendorf
,
K. A
.
2002
.
The Content Analysis Guidebook
.
Thousand Oaks, CA
:
Sage Publishing
.
Power
,
M. K
.,
and
Y
.
Gendron
.
2015
.
Commentary: Qualitative research in auditing: A methodological roadmap
.
Auditing: A Journal of Practice & Theory
34
(
2
):
147
165
.10.2308/ajpt-10423
Public Company Accounting Oversight Board (PCAOB)
.
1998
.
Using the Work of a Specialist. PCAOB Interim Auditing Standard (AS) 1210
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2001
.
Auditing Accounting Estimates. PCAOB Interim Auditing Standard (AS) 2501
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2002
.
Auditing Fair Value Measurements and Disclosures. PCAOB Interim Auditing Standard (AS) 2502
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2007
a
.
Auditing Accounting Estimates and Fair Value Measurements
.
Standing Advisory Group Meeting (June 21)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2007
b
.
Matters Related to Auditing Fair Value Measurements of Financial Instruments and the Use of Specialists. Staff Audit Practice Alert No. 2. (December 10)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2008
a
.
Report on 2007 Inspection of Deloitte & Touche LLP. (May 19)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2008
b
.
Report on 2007 Inspection of KPMG LLP. (August 12)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2010
a
.
Report on Observations of PCAOB Inspectors Related to the Audit Risk Areas Affected By the Economic Crisis
.
Release No. 2010-006. (September 29)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2010
b
.
Report on 2009 Inspection of McGladrey and Pullen, LLP. (June 24)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2011
a
.
Report on 2010 Inspection of Ernst & Young, LLP. (November 30)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2011
b
.
Report on 2010 Inspection of Deloitte & Touche LLP. (December 7)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2012
a
.
Auditing the Future. (June 7)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2012
b
.
Maintaining and Applying Professional Skepticism in Audits. Staff Audit Practice Alert No. 10. (December 4)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2012
c
.
Report on 2012 Inspection of KPMG LLP. (July 30)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2012
d
.
Report on 2011 Inspection of KPMG LLP. (August 15)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2013
.
Report on 2012 Inspection of PricewaterhouseCoopers LLP. (August 20)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2014
a
.
Auditing Accounting Estimates and Fair Value Measurements
.
Staff Consultation Paper
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2014
b
.
Standing Advisory Group Meeting. (October 2)
.
Washington, DC
:
PCAOB
.
Public Company Accounting Oversight Board (PCAOB)
.
2015
.
The Auditor's Use of the Work of Specialists. Staff Consultation Paper No. 2015-01. (May 28)
.
Washington, DC
:
PCAOB
.
Pyzoha
,
J. S
.,
M. H
.
Taylor
,
and
Y
.
Wu
.
2016
.
The Effects of Tone at the Top Messaging and Specialists on Auditors' Judgments During Complex Audit Tasks
.
Working paper
,
Miami University, Case Western Reserve University, and Texas Tech University
.
Smith-Lacroix
,
J
.,
S
.
Durocher
,
and
Y
.
Gendron
.
2012
.
The erosion of jurisdiction: Auditing in a market value accounting regime
.
Critical Perspectives on Accounting
23
:
36
53
.10.1016/j.cpa.2011.09.002
Trochim
,
W. M. K
.,
and
J. P
.
Donnelly
.
2007
.
Research Methods Knowledge Base. Third edition
.
Mason, OH
:
Thomson Corporation
.
Valuation Research Corporation (VRC)
.
2013
.
Alert: PCAOB Comments Increase Auditor Scrutiny of Fair Value Measurements. (January)
.
Boston, MA
:
VRC
.
1

Consistent with the terms used in Auditing Standard (AS) 2502 (formerly AU 328), Auditing Fair Value Measurements and Disclosures, and AS 2501 (formerly AU 342), Auditing Accounting Estimates, as well as by the PCAOB (2001, 2002, 2014a), our use of the term “fair value measurements” also includes accounting estimates.

2

Pricing services generally provide uniform price information and related data for financial instruments (PCAOB 2014a). Further, FVM estimates provided by pricing services are typically not customized. Instead, “a company or auditor obtains information that is developed for, and widely available to, the public” (Interactive Data 2014, 8). On the other hand, in-house or third-party valuation specialists are typically engaged to provide customized estimates for either financial or nonfinancial FVMs (Interactive Data 2014; PCAOB 2014a, 2015). Conversations with representatives of large firms suggest that each of the largest firms has a “national pricing desk” that includes valuation specialists employed who help the engagement teams develop FVM estimates, as well as evaluate management-developed FVM estimates. The national pricing desk is not considered an in-house pricing service. Instead, when pricing information is needed by the engagement teams, the national pricing desk performs a number of functions, including gathering pricing information from third-party pricing services for the engagement teams.

3

Comment letter responses to the PCAOB's (2014a) Staff Consultation Paper, Auditing Accounting Estimates and Fair Value Measurements, from pricing services and valuation specialists highlight differences between pricing services and valuation specialists and encourage the Board to keep these differences in mind when considering changes to the existing auditing standards (e.g., Harvest Investments, Ltd. 2014; Interactive Data 2014).

4

We acknowledge a growing number of experimental studies that investigate how various environmental and task factors affect auditors' judgments and decisions when auditing FVMs (e.g., Backof, Thayer, and Carpenter 2015; Earley, Hoffman, and Joe 2015; Griffin 2014; Griffith, Hammersley, Kadous, and Young 2015b; Joe, Vandervelde, and Wu 2017; Joe, Wu, and Zimmerman 2016; Maksymov, Nelson, and Kinney 2015; Pyzoha, Taylor, and Wu 2016).

5

Results presented in the current study are part of a larger survey, including results reported in Glover et al. (2016).

6

We received approval by the Institutional Review Boards of our universities prior to distributing our survey.

7

Firm contacts mainly targeted audit partners, but the survey was also sent to some senior managers. As shown in the next section, all participant responses included in this study are from audit partners. Further, while not a unique issue to this study, we acknowledge that responses may not reflect what participants actually do in practice and, instead, reflect what appears to “sound good.” However, we believe features of our design minimize this concern. First, the highly experienced auditors we surveyed are less likely to have the incentive to provide “sound good” responses than typical social science study participants (i.e., college students), particularly because confidentiality and anonymity were assured (e.g., Gibbins et al. 2001; Nelson et al. 2002; Cannon and Bedard 2017). Second, using a survey, rather than phone or face-to-face interviews, reduces concerns that participants provided responses to simply “sound or look good” (Trochim and Donnelly 2007, 123).

8

Each question's coding themes are presented in results tables for the associated question.

9

Neuendorf (2002) suggests that Kappas between 0.40 and 0.70 are acceptable. Further, Landis and Koch's (1977) benchmarks suggest that Kappas between 0.61 and 0.80 represent “substantial” inter-rater agreement strength.

10

Participants were not required to answer each question, although nearly all did. Thus, the sample sizes are reported by questions in the tables.

11

We present t-test results when there are significant differences in mean responses across demographic factors. In instances where the variance between the two comparison groups is unequal, we present the Welch t-test (or the Satterthwaite method), along with the adjusted degrees of freedom rounded to the nearest whole number.

12

Mean valuation and audit expertise self-rated by audit partners who practice in financial industries (9.04) is significantly higher (t(30) = 2.07, p = 0.05, two-tailed) than partners who practiced in nonfinancial industries (8.11). Further, only six of our 32 participants (18.8 percent) indicate that they specialize in a particular type of FVM and find no difference between mean ratings (t(30) = 0.59, p = 0.56, two tailed) of valuation and audit expertise between those who specialize in a type of FVM (8.46) and those who do not (8.83).

13

Although not the primary focus of the current study, we asked participants whether their audit teams stratify the security portfolio by risk and type of instrument before selecting and testing a sample from the portfolio. An overwhelming percentage of our participants (over 90 percent) reported “yes,” suggesting that auditors take into account risk and potential lack of homogeneity in populations when designing their audit security portfolio testing. Glover et al. (2016) point out that PCAOB inspectors may not appropriately factor risk into their FVM inspection processes and, thus, audit partners report instances in which sampling is abandoned and 100 percent of the population is tested.

14

We thank an anonymous reviewer for this observation.

15

Recently, the PCAOB (2014a, 44) noted, “pricing sources are increasingly providing products that could provide auditors with insight as to how their prices or estimates are developed.”

16

To clarify, according to audit partners' responses, reliability of the pricing service refers to the caliber (i.e., qualifications) of that pricing service.

17

The list was developed by reviewing the previous literature and detailed discussions with audit partners, the CAQ RAB, and national office partners assigned to our project by the CAQ. Further, we provided an “other” option that allowed participants to indicate any other problems not listed. Some problems identified in the “other” category include lack of sufficient monitoring activities and management's overreliance on valuation specialists.

Author notes

We appreciate those practicing partners who responded to our survey. This paper has benefited from helpful comments from workshop participants at Case Western Reserve University and The University of Kansas Symposium. Steven M. Glover is grateful for the funding provided by the K. Fred Skousen Endowed Professorship. Mark H. Taylor appreciates the funding provided by the Andrew D. Braden Endowed Professorship. We also thank Andrea Scheetz and Aleksandra Zimmerman for assistance with the data. We thank the Center for Audit Quality for funding this project, as well as the Center's Research Advisory Board for its feedback to help refine our survey.

Competing Interests

Although funding for the research project described in this paper was provided by the Center for Audit Quality, the views expressed in this paper are those of the authors and not those of the Center for Audit Quality or the participating accounting firms.

Editor's note: Accepted by Jacqueline S. Hammersley.