Since passage of the quickly finalized Sarbanes-Oxley Act during July 2002, audit production in the U.S. has been substantially expanded by mandated internal control audits. The control audit mandate is unique to the U.S. and costly to apply, yet little is known about the conduct of control audits or the efficacy of lower-cost alternatives. This paper reflects our overall knowledge about control audit production and observation of a consistent message across public and limited non-public archival data, analytical studies, and numerous personal experiences of audit practitioners. Our primary observation is that, absent any financial misstatement, auditors find it difficult to identify material weaknesses in control design. Conversely, when auditors know about misstatements they can, and do, detect related material weaknesses and thereby identify most public companies found by mandated control audits to have ineffective controls. Thus, it appears possible to exploit this observation to identify and publicly disclose most companies with weak controls without incurring the cost of full internal control audits. We believe that U.S. markets could benefit from more transparency about the current U.S. audit production process and from informed debate about the best mechanism design for balancing the needs of all parties interested in internal control quality disclosure.
It has been more than ten years since President George W. Bush signed the Sarbanes-Oxley Act (SOX) that established the world's first mandated audits of internal control over financial reporting (ICFR) and public reports (U.S. Congress 2002a). As to the political environment in 2002, Congressman Oxley recently said, “When we passed our bill, the public was furious: You had this white-hot pressure to get something done” (Quick 2012).1 Perhaps because of the political urgency, Congress did not disclose any cost-benefit analysis or articulate how SOX Section 404 was expected to change audit production or audit cost.2
When SOX Sections 404(a), requiring management evaluation and reporting on ICFR, and 404(b), requiring an independent audit of ICFR effectiveness, were implemented in late 2004, internal control information for investors substantially increased and audit fees typically doubled (e.g., Kinney and Shepardson 2011; Raghunandan and Rama 2006). Over time, the proportion of U.S. public companies disclosing material weaknesses has declined substantially3 and so have incremental fees when companies initiate control audits, but relatively little is known about how the SOX 404(b) mandate changed production of audits mandated by U.S. securities acts.
Section 404(b) applies to all public companies trading in the U.S., but Congress subsequently exempted smaller U.S. issuers (U.S. Congress 2010) and certain growth companies (U.S. Congress 2012).4 In contrast, International Statements on Auditing do not require control audits due to audit cost, as well as lack of market demand and most national governments have not adopted the U.S. mandate approach.5 The manner in which independent audits required by 404(b) changed audit production is important because audit quality and cost-benefit discussions continue both within the U.S. and globally and also because there are alternative ways to produce internal control quality information for investors.
Extant archival research has assessed relative audit fees and material weakness disclosure rates between observable control weakness reporting alternatives within the United States (e.g., Kinney and Shepardson 2011). In this commentary, we review and build upon empirically observable U.S. regimes. We also discuss other control effectiveness disclosure possibilities based upon U.S. experiences with the three mechanisms by which companies and their auditors identify material weaknesses in internal control. In particular, we focus on experiences with known accounting mistakes, control design evaluation, and control operating effectiveness testing.
The three of us have followed 404(b) audit implementation from our perspectives as auditing educators, academic fellows at the Securities and Exchange Commission (SEC), advisor to the Public Company Accounting Oversight Board (PCAOB), standards setter as an International Auditing and Assurance Standards Board (IAASB) member, and a Big 4 audit manager applying 404(b). We have also conducted research projects; read theoretical, archival, and behavioral research of others; and had numerous control audit conversations regarding implementation with U.S. and foreign regulators, standards setters, practitioners, directors, corporate officers, and investors.
We reflect upon and synthesize our collective experiences with ICFR audits and form several observations about audit production that we believe warrant sharing in the interest of accounting and auditing educators, researchers, and students, as well as the profession and the public.6 We include education and research because it is important to understand extant audit institutions, as well as alternatives. The profession is included because we believe that, for the long term, professional services must pass a cost-benefit test even for mandated services. And, we include investors and the public interest because control audits are expensive and there is disagreement about whether the U.S. has the best approach for investors and the public at large.
The remainder of this commentary develops our ideas, but here are our main observations. As to their nature:
Audits of internal control processes are fundamentally different from audits of financial statements as to objective, value, and approach, although they share some attributes.
The three sources of control audit evidence required by PCAOB standards differ substantially in incremental costs, audit expertise required, and ability to identify material weaknesses so that:
○ Relative to design evaluation and implementation testing, auditors are effective and efficient at identifying control weaknesses that have resulted in known accounting misstatements—even if the misstatements are immaterial.
○ Absent knowledge of accounting misstatements, identification of weaknesses in control process design is difficult.
○ The appropriate scope of operating effectiveness testing remains unclear, as does when entity-level control tests can substitute for process-level control tests.
As to alternatives to mandated control audits:
No other country or auditing standards-setting body has adopted the U.S. control audit legislated mandate, even though it has been considered in multiple countries.
Some other countries have developed alternatives that partially apply the U.S. requirements and provide some control information to investors, but at less cost of production.
Regarding the public interest, it is clear that, even after ten years of experience, the public knows little about what it gets from 404(b) audits beyond that produced by implementation of 404(a) and the financial audit. To our knowledge, there are no reliable resources or comprehensive data on (1) how control audits are produced, (2) how effective audit staff and audit partners are in implementing the unique requirements of 404(b) audits, and (3) the relative effectiveness of alternatives to 404(b).7 We believe that the public interest would be served by increased transparency and independent analyses of extant audit production.
In the next section, we analyze conceptual differences between financial statement and control audits and why the differences matter. The third section integrates archival evidence and experts' comments about how control problems are identified. In the fourth section we explore alternatives to mandated 404(b) audits and we conclude in the final section.
FUNDAMENTAL DIFFERENCES BETWEEN CONTROL AND FINANCIAL AUDITS
Because control audits are about possibilities regarding a process (rather than a state or flow), they are conceptually and substantively different from financial statement audits—and so are their audit opinion implications. The financial audit says, in effect, “these particular numbers are reliable, but we express no views about the control process” and the ICFR audit says, “the control process is reliable, but we express no views about these particular numbers that it produced.” Audit risk (AR) for financial audits is the risk that undetected material misstatement exists in audited numbers. Identified material misstatements must be corrected. For ICFR audits, as defined in the United States, control audit risk (denoted CAR) is the risk that material weakness exists and is not detected by the control audit or indicated by the financial audit (Akresh 2010). In contrast to material misstatements found in the financial audit, known material weaknesses must be disclosed, but need not be corrected.
According to AS5 (PCAOB 2007a, para. A7), a material weakness exists when control deficiencies result in “a reasonable possibility that a material misstatement of the company's annual or interim financial statements will not be prevented or detected on a timely basis.” Identifying such weaknesses involves rather complex conceptual thinking. It requires considerable expertise and judgment to apply the auditing concepts of material misstatement, to estimate the likelihood of possible occurrence of misstatements of various sizes, and to evaluate how particular control activities might prevent or detect misstatements, as well as how possible misstatements might aggregate or occur jointly. Broad expertise is also needed in data processing of both routine and non-routine transactions.
How do the AR and CAR models differ, what elements are common, and what are the potential benefits from integrating the two audits at the assertion level? One common element is the “inherent risk” of material misstatement (IR): the risk that, in the (theoretical) absence of controls, material misstatement might arise for an important assertion. If IR is correctly assessed as very low for a particular assertion, then neither audit is needed because material misstatement is unlikely to arise.
A second common element is consideration of causal inferences from misstatements known to the financial auditor. The financial auditor considers implications of known misstatements for other possible misstatements, whereas the control auditor considers their implications for possible material weaknesses. Figure 1 diagrams financial audits applied in the U.S. prior to SOX and the great majority of jurisdictions globally using current International Auditing Standards (ISAs).
Financial auditing standards require that the auditor sufficiently “understand” controls to be able to plan the financial audit. When the auditor believes that the control risk of failing to prevent or detect material misstatement is maximum (CR = max) for a particular assertion, the audit requires rigorous substantive audit tests (top route in Figure 1). When control risk is believed to be less than the maximum (CR < max), the auditor may choose to test control effectiveness to justify limited substantive tests (lower route in Figure 1). In either case, the auditor plans detection risk for material misstatement (DR) conditional on IR and CR in order to achieve the target AR.8 Historically, auditors of smaller companies frequently chose the upper route due to weaker control processes that are typical for smaller companies.
Deficiencies in internal control can be identified during financial audits via three mechanisms represented by Links (a), (b), and (c) in Figure 1. Link (a), one of the two common elements in financial and control audits under AS2 and AS5, reflects deficiencies identified by root-cause analysis of financial statement adjustments/corrections detected during the financial audit or “sourcing misstatements to cause.”9 Auditors may also identify deficiencies during engagement planning procedures when obtaining an understanding of controls or during design evaluation, reflected in Link (b), or when testing operating effectiveness, reflected through Link (c).
The financial auditor's three reports in Figure 1 are an opinion about possible undetected material financial misstatement, communication of adjustment/correction of detected misstatements, and communication of any material weaknesses and/or significant deficiencies identified during the financial audit. The auditor's opinion on the financial statements is made public, but the communications about known corrections, weaknesses, and deficiencies are made only to the audit committee.
Figure 1 also shows (via dashed lines) an “enhanced control disclosure” added to the existing financial audit as a fourth possible report. Financial auditing standards could be slightly enhanced to require financial statement audit report disclosure of material weaknesses discovered through the three financial audit links (Kinney and Shepardson 2011). The enhancement would entail little incremental audit cost. Such an approach could also satisfy the European Commission's demand for disclosure of internal control deficiencies in Article 22 (EC 2011).10
As articulated by Akresh (2010), SOX creates a second audit and a second “audit risk”—control audit risk (CAR)—that arises if the auditor unknowingly fails to modify the control audit opinion when at least one undetected material weakness exists. CAR is a function of IR and three non-mutually exclusive control detection risks (CDRs) that the auditor's control and financial audit procedures fail to detect a material weakness that:
resulted in financial misstatement (whether material or not) and that the financial auditor did not identify or did not properly “source to cause” (denoted here as CDRa);11
exists in the design of controls (denoted CDRb); or
is due to ineffective operation of well-designed controls (denoted CDRc).
There are three noteworthy differences between the AR model and the CAR model. First, as to formulation, with the exception of IR, the risk factors for the financial audit risk model:
differ from factors for the control audit risk model:
Second, as noted by Akresh (2010), CAR does not have a simple formula such as AR = IR × CR × DR, which many audit firms apply for financial statement assertions. Third, the CAR formulation does not differentiate management's entity-level “controls over ICFR” intended to reduce the risk that a material weakness in ICFR might occur and go undetected by management. Thus, there is no “CR-controls” in CAR that parallels CR in AR; although, such a model could be formulated. All three differences represent gaps in conceptual model development needed to support AS5, and the third is especially relevant to support reliance on entity-level controls as a means of limiting 404(b) audit cost” (Asare et al. 2013).
Components of AR and CAR differ because the audit objectives differ. CAR addresses the likelihood that process defects exist that might allow material misstatement to occur and remain undetected, while AR addresses the likelihood that material misstatement has occurred and remains undetected. Thus, at least some of the costs of audit procedures limiting CDRa, CDRb, and CDRc will be incremental—even for processes tested by the financial auditor—and all costs will be incremental for processes not tested by the financial auditor. Figure 2 diagrams integrated financial and control audits.
All control deficiencies identified via Link (a) could be discovered with financial auditing alone whereas, at best, only a subset of the deficiencies identified via Links (b) and (c) of an integrated audit could be identified via a stand-alone financial audit. This is because U.S. control audits require design and operating effectiveness testing for all important assertions but, absent selecting a control reliance approach for a particular assertion, financial audits do not. Thus, integrated audits should yield additional control deficiencies and material weaknesses compared to financial audits with enhanced control disclosure. The critical questions are: what is the best tradeoff of additional audit costs and resulting benefits, and what do we know about this tradeoff based on a decade of experience with control audits?
MATERIAL WEAKNESS IDENTIFICATION EXPERIENCES
What can be said about auditors' experiences identifying material weaknesses by applying Links (a), (b) and (c) in Figure 2? One overarching observation is that interested parties outside of audit firms and regulatory organizations know little about how control audits are actually produced. To our knowledge, researchers have had virtually no access to audit manuals, audit work papers, or billable audit-hours data that would allow critical and independent analysis of how auditors actually conduct control audits.
As a result, our observations are based on the limited archival data available from a single study during the AS2 era (Bedard and Graham 2011), from Audit Analytics categorizations of 404(b) audit reports, and “first-person” accounts from multiple audit partners over the entire time period, as well as the Center for Audit Quality's (CAQ) “lessons learned” from SOX 404(b) implementation (CAQ 2009). Although the sources are diverse, we believe that a consistent message can be derived from them.
Link (a)—Evaluating Known Financial Misstatements
The importance of Link (a) is difficult to overstate based on U.S. experience. Analysis of archival data consistently reveals that most firms with ineffective controls12—that is, at least one material weakness—that are identified by full 404(b) audits can be identified by auditors tracing “backward” from misstatements known to the financial statement auditor applying the Link (a) guidance in PCAOB Auditing Standards (both AS2 [PCAOB 2004] and AS5 [PCAOB 2007a]). An SEC staff study also recognizes this relationship (SEC 2009).
Bedard and Graham (2011) were provided internal control audit engagement level data by several large audit firms that implemented AS2 during 2004 and 2005.13 From 76 engagements for 44 different clients with total revenue less than $1 billion, the auditors identified, or otherwise became aware of, 3,990 control problems of which 153 were material weaknesses, 461 were significant deficiencies, and 1,048 had been remediated but were not classified as to severity. The remainder (2,328) were “deficiencies” that were not reported to the audit committee. Thus, 404(b) audits are associated with identification of a large number of control problems within the 44 sample firms.
Fifteen of the 76 engagements disclosed at least one material weakness. While the authors find that only 27.5 percent of the individual material weaknesses detected during the audits were linked to a misstatement, 11 of the 15 engagements disclosing ineffective internal controls, or 73 percent, had at least one material weakness associated with a misstatement. Thus, most of the companies with ineffective internal controls—those disclosing at least one material weakness—might have been identified via Link (a) without full control audits. Regarding Links (b) and (c), of the 153 material weaknesses identified, 54.6 percent were deemed to be operating ineffectively, and 25.7 percent were associated with a control design flaw. However, the authors do not differentiate how, or through which links, the weaknesses were identified and, therefore, we cannot differentiate Links (a), (b), and (c) empirically.
It is possible that this high identification rate via Link (a) in the Bedard and Graham (2011) sample occurred early in the SOX implementation era because auditors were less familiar with Links (b) and (c). However, archival data from auditors' reports indicate material weaknesses continue to be identified via Link (a). Audit Analytics (AA) designates a company as having “material and/or numerous auditor/YE adjustments” when management or audit report descriptions in a 10-K filing indicate that at least one material weakness is linked with a misstatement identified by the financial audit, or by management in preparing for closing at year-end. Using Audit Analytics data, Besch (SEC 2009) and Kinney and Shepardson (2011) report that from about 50 percent to 80 percent of firms with material weaknesses are characterized as having “material and/or numerous auditor/YE adjustments.”14
Over the SOX decade, many audit partners commented on control audit practice experiences with Link (a). One source is a series of exploratory interviews of top technical auditing partners at the four largest global network firms, which the present author team conducted in 2006. Each of these partners had oversight, monitoring, or development responsibility for his or her firm's integrated audit methodology. The interviews were intended to learn how each firm had implemented the PCAOB control audit guidance of AS2 and the challenges they faced in that process. Each of these partners offered the view that “at least 80 percent” of individual internal control weaknesses detected in the first two years of 404(b) implementation were identified via misstatements noted during the financial audit.
In addition to the interviews, the importance of Link (a) in identifying material weaknesses was noted in May 2006 by members of a panel of Big 4 audit professionals at The University of Kansas Auditing Symposium.15 Similar views have been expressed by practitioners over subsequent years in Auditing Section Meetings and other conferences. Most recently, this same point was made during an April 2012 meeting of an IAASB group discussing how practitioners might be able to respond to the European Commission's internal control disclosure initiative (EC 2011).
It thus appears that Link (a) identifies most firms reporting at least one material weakness, leading to two questions: (1) why is this link so critical, and (2) what are the implications for balancing control ineffectiveness identification against incremental audit cost?
Link (b)—Evaluating Control Design
U.S. control audit standards require that the auditor evaluate whether controls for significant accounts and disclosures and their relevant assertions, are designed and operating effectively Links (b) and (c), respectively (AS5, PCAOB 2007a, para. 21). As to Link (b), AS5 establishes walkthroughs as the critical information-gathering procedure for evaluating design, but the guidance on how to evaluate design is also important.16 AS5 guides auditors to anticipate controls that might be missing as well as simply poorly designed by, for example, “asking himself or herself ‘what could go wrong?'” within a given significant account or disclosure (PCAOB 2007b).
Evaluating “what could go wrong” is consistent with long-standing principles of control testing (e.g., Hare 1967), but leaves auditors facing a daunting task of forming a list of all things that “could go wrong” when evaluating control design. For example, an experiment led Asare et al. (2011) to suggest that auditors have difficulties even imagining how a misstatement could occur when a deficiency has not led to any misstatements.17 Further, design completeness and effectiveness might be easy to evaluate for routine contexts, but non-routine contexts likely require considerable experience to adequately assess “what could go wrong,” the related inherent risks, and the effectiveness of controls at the process and entity level (CAQ 2009; Asare et al. 2013).
Thus, we are led to the important question: How effective are auditors at identifying design weaknesses and under what conditions? To address this question, we again turn to first-hand statements by practitioners. At a 2009 Big 4 audit firm training exercise, the presenting partner was asked how the firm identified design weaknesses. The partner said “A walkthrough is usually enough to evaluate whether ICFR design is effective.” Asked whether junior audit staff would be able to accurately assess design effectiveness merely by “walking through,” the partner said that “a partner or manager could do so, but probably not staff members.” In a similar vein, one of the “lessons learned” via the CAQ study is that auditors should consider using more experienced professionals to perform walkthroughs, especially with respect to more complex processes (CAQ 2009).
In our 2006 interviews of technical partners, we asked each how their respective firms identified material weaknesses in control design. One partner said, “Frankly, in the absence of a misstatement, whether material or not, our auditors have difficulty identifying design weaknesses.” This partner went on to add, “Even when [they] do identify a design weakness, our clients often disagree that it is material unless some actual misstatement has resulted from the design weakness.” These two statements suggest that knowledge of accounting misstatements plays a key role in Link (b), as well as Link (a). Partners interviewed at the other three firms had the same responses, in similar words, and so did the audit partner in the 2012 IAASB discussion.
Walkthroughs, experience-based judgment, and the existence of misstatements resulting from design weaknesses are major contextual factors in identifying design weaknesses. In conclusion, walkthroughs may be effective information-gathering procedures, but may be ineffective tools to identify control design flaws without evaluation by experienced auditors and without being accompanied by a resulting misstatement. The question remains: how can design weaknesses best be identified?
Link (c)—Evaluating Control Operating Effectiveness
PCAOB Auditing Standards for Link (c) are relatively straightforward in that they call for evidence that prescribed controls are being applied in practice. In concept, Link (c) is easier to apply than Link (b), but auditing standards shifted during SOX's first decade. AS2 required operational effectiveness testing of controls for all relevant assertions related to all significant accounts and disclosures and focused AS2-era audits on process-level testing rather than entity-wide controls (PCAOB 2004, para. 104).
U.S. audit firms' initial responses to operating effectiveness testing under AS2 were to continue with what they already knew how to do—test operating effectiveness of process-level controls and increase the number of processes tested. The number of process-level controls tested was in the thousands for some companies.18 Audit firms quickly hired large numbers of additional entry-level auditors to do this type of testing.
The magnitude and impact of how audit staffing for U.S. public company audits changed with 404(b) is difficult to assess and cannot be determined from publicly available data.19 We could not locate research studies or reports of audit hours pre- and post-404(b) implementation. Also, we were not able to obtain, from audit firms, comparative audit hours for even a single public issuer. In addition, one cannot look to other countries for data because such other countries have not followed the U.S. lead in mandating control audits.
Whatever the change in audit staffing, in response to high audit fees, AS5 directs the auditor to focus on “the most important matters … and eliminate[s] procedures that the Board believes are unnecessary to an effective audit of internal control” (PCAOB 2007b). Included is a top-down audit approach with emphasis on entity-level controls that “depending on how they are designed and operate, can reduce the testing of other controls related to a relevant assertion” (PCAOB 2007a). AS5 did not specify when the auditor can rely on entity-level controls and, thus, identify process-level controls that could remain untested or how to map entity-level control weaknesses to particular assertions.
The manner in which entity-level and process-level controls can be related with precision by the auditor to particular assertions, remains unspecified. The CAQ suggests an ordering of types of controls to assess in a top-down approach (CAQ 2009), but to our knowledge, there is no authoritative guidance, definitive inspections results, or best practices on how to appropriately accumulate assurance across entity-level and process-level controls. Also, we know of no auditing textbooks that discuss specific assertions vis-à-vis entity-level/process-level design and analysis as part of education for auditors. Implementation of AS5 provides auditors with a theoretical audit approach to 404(b) audits that reduces the extent of audit tests for Link (c) at the process level. But, it remains unclear how frequently Link (c) contributes to the identification of process- or assertion-level material weaknesses.
How often do auditors identify material weaknesses by detecting lack of application of well-designed controls at the entity level or process level? We do not know. If no one outside audit firms and the PCAOB knows about current audit production, then independent research is not possible and effective teaching is impaired. As an example of the need for independent research, there is some question about whether the decline in the rate of material weakness disclosures after adoption of AS5 is due to continuing reduction in actual material weakness rates or if the AS5 approach is simply less precise and detects fewer material weaknesses than did the process-level approach under AS2 (SEC 2009).
ALTERNATIVES TO MANDATED INTEGRATED AUDITS
Our consideration of production and costs of U.S. control audits raises the broader question of control audit value. Reliable financial statement information is central to many investment decisions and detection and correction of misstatements prior to public release via financial audits are important. But if the related audited financial statements are reliable, under what conditions do investors need concurrent control audit assurance and at what additional cost?20
Because the IAASB has not mandated control audits and there is debate about how best to provide control process information to investors in small and large public U.S. firms, we explore four specific alternatives that provide some control weakness information to investors at lower aggregate cost than integrated audits. These alternatives vary in levels of involvement by auditors and management. Our discussion is not intended to be exhaustive, but to encourage research that can inform the debate about alternative methods for obtaining control effectiveness information and to guide standard setters and regulators in evaluating the U.S. audit regime.
“Comply or Explain”
In contrast to legally mandated compliance, U.K. law requires firms listed on the Main Market of the London Stock Exchange to report how they comply with Listing Rules that include voluntary application of the Turnbull Guidance on internal control in the Combined Code.21 Listed firms can comply by disclosing, with or without independent auditor assurance, or by explaining why they do not comply, and let users assess the meaning of non-compliance.22
A principal advantage of comply or explain is that it allows individual firms to conduct cost-benefit analyses regarding whether, and how, to comply over time and to avoid costs when they believe net benefits would be small. It therefore encourages standards setters for internal control criteria and auditing to develop cost-effective standards. A disadvantage of comply or explain is that firms with poor internal controls may use cost-benefit concerns to avoid auditor involvement in internal control disclosure where it may be most beneficial.
Financial Audits with “Enhanced Control Disclosure”
This alternative assumes that auditors apply only financial audits in an environment that does not have management 404(a) reports or 302 disclosure control certifications. Financial audits are directed at financial misstatements and, as depicted in Figure 1, include investigation as to the cause for all misstatements (irrespective of materiality) detected by, or otherwise known to, the financial auditor. If any known misstatements are due to material weakness or if the auditor otherwise becomes aware of a weakness, the auditor's report would describe the weakness. Thus, investors would have some of the information that auditors presently discuss with the audit committee.
Because providing enhanced control weakness disclosure does not require either management or auditor assessments of control effectiveness across all processes, the incremental cost of enhanced disclosure is minimal. However, any material weaknesses that have not resulted in detected misstatements or are not otherwise detected by the financial audit would remain undetected and undisclosed.
Management Evaluation-Only and “Enhanced Control Disclosure”
This alternative retains management's control assessment under SOX 404(a) and also incorporates enhanced control disclosure.23 Management's control assessment might assist in identifying, remediating, or disclosing design and operating weaknesses prior to detecting a misstatement without more costly control audits. Under existing auditing standards (AU 550), if management makes statements in unaudited portions of Form 10-K that the auditor believes are a material misstatement of fact, such as providing a 404(a) assessment that controls are effective when the auditor believes otherwise, the issue must be resolved prior to the issuance of the auditors' opinion (PCAOB 2003; CAQ 2007). The auditor would therefore apply financial auditing procedures, investigate the cause of known misstatements, and continue with their AU 550-required consideration of management's public disclosures about internal control.
This option would provide some auditor scrutiny of management's assertions about internal control without substantial control audit cost. Due to management's greater knowledge and responsibility to evaluate all controls, management's evaluation would likely identify more material weaknesses than would only a financial audit with enhanced control disclosure. However, the identification of any incremental weaknesses would be limited to those discovered and disclosed by management, which Bedard and Graham (2011) suggest may result in under identification of weaknesses.
Entity-Level Control Audits with “Enhanced Control Disclosure”
Entity-level controls include those related to the control environment, controls over management override, risk assessment, general computer controls, monitoring controls, and controls over the period-end financial reporting process per AS5 (PCAOB 2007a). For an entity-level control audit, the auditor would test design and operating effectiveness of entity-level controls (Links (b) and (c) at the entity level), but would exclude subordinate process-level controls, and would identify the cause of all known misstatements (Link (a)). This alternative is similar to top-down audits under AS5, with the exception that under AS5 auditors are required to test process-level controls when entity-level controls do not provide a sufficiently precise mapping of assurance to a particular assertion. While this proposed alternative would not require intensive process-level testing, it might require that design testing be performed by engagement team members with sufficient experience and/or additional guidance to help less experienced team members identify entity-level design weaknesses.
The entity-level control audit would provide information regarding material weaknesses in entity-level controls and the enhanced control disclosure would provide information on process-level control failures resulting in misstatements. The benefits include reducing cost from full process-level control audits, focusing auditors on “top-down” controls, and obtaining process-level control failure information for weaknesses that resulted in current period misstatements.
In this commentary, we explore broad observations about material weaknesses identified via control audits and alternative ways that firms and their auditors can provide some control effectiveness information to investors. We caution that our observations are not a substitute for independent research, but may help focus on the need for such independent analyses.
Link (a) (sourcing known misstatements as to cause) reflects the centrality of financial audits for control audits and the possibility of leveraging financial audits to identify firms with ineffective ICFR.
Link (b) (understanding and evaluating control process design) makes clear the need for conceptual models for evaluating control design (especially entity-level controls), better standards for how to audit controls, and better understanding of how auditors develop the ability to understand and evaluate internal control design and how this ability relates to both financial statement and control audit efficiency and effectiveness.
Link (c) (evaluating control process operation) is relatively easy to apply, requires less auditor expertise—at least at the process-control level, and appears to be the main source of increased audit hours and fees, but at the entity-level, effectiveness testing is relatively unexplored in practice, standards, and research.
Our search for archival data about “auditors' experiences” applying 404(b) audits revealed very limited evidence—a single study—and it was based on AS2-era data. While we based our conclusions on the research and anecdotal evidence available, there are not enough empirical data to determine which approaches would be most cost beneficial. The frequencies of weaknesses identified via each of the analyzed mechanisms are not known, and we cannot draw any definitive conclusions about efficacy of the current auditing standard.
Overall, we believe independent research is essential for effective auditor application of the alternative presented by AS5 (for which we could find no archival studies based on audit firms' internal sources), as well as exploration of alternatives being pursued in other jurisdictions around the world and international standards. We hope that our observations will be debated, challenged, and subjected to independent research with new archival data from audit manuals, workpapers, and time budgets, as well as from data and insights via interviews, surveys, and experiments involving audit practitioners, managements, standards setters, and regulators.
We recognize that the first decade of SOX has been difficult—particularly for regulators and practitioners, as they have had to navigate uncharted waters. But after the ten-year mark, we believe that the public interest demands more transparency and analysis about how control audits are conducted, how they might be improved, and what might be better alternatives for the second decade. Investors, auditors, standard setters, academics, and auditing students could all benefit in the long term.
Senator Sarbanes also commented on the need for quick reconciliation of differences between the House and Senate versions of the proposed legislation as well as Bush administration pressure expressed as, “You've got to give us legislation so the President can sign it” (Kranacher 2008).
Congress may have been misinformed about audit production and cost in 2002. For example, the Senate Committee recommending that 404(b) be added to the SOX legislation stated, “High-quality audits typically incorporate extensive internal control testing” and “the Committee does not intend that the auditor's evaluation be the subject of a separate engagement or the basis for increased charges or fees” (U.S. Congress 2002b).
For example, the SEC staff noted that the percentage of firms disclosing material weaknesses decreased from 16 percent in 2004 to 4 percent by 2008 (SEC 2009).
Congressman Oxley recently expressed regret that SOX did not contain a “scaled-down provision” for smaller companies (Quick 2012). In this paper, we refer to control process audits as 404(b) audits, even though SOX included the requirement under Section 103(a)(2)(A)(iii). Throughout this commentary we use the terms 404(b) audits, control audits, ICFR audits, and control process audits interchangeably.
France and Japan share some U.S. control audit features, but no other country mandates full control audits. Also, the European Commission has indicated an interest in reporting control deficiencies known to the auditor from the financial audit (EC 2011).
Two recent academic literature reviews provide thorough summaries of internal control audit research. Schneider et al. (2009) provide a discussion of extant archival research about internal control disclosures made in compliance with Section 404, and for an extensive review of research on judgments and decisions faced by auditors within the ICFR auditing process, see Asare et al. (2013).
A notable exception on audit production data at the individual audit level is Bedard and Graham (2011), which is elaborated below.
Many audit firms use the multiplicative form, AR = IR × CR × DR, to determine DR = AR/(IR × CR) where AR is the target audit risk and (IR × CR) is the auditor's assessed risk of material misstatement.
According to AS5, misstatements detected in the financial audit may provide evidence of material weakness, but absence of detected misstatements does not imply that control is effective (PCAOB 2007a, para. B9).
If the auditor routinely provides the audit committee with a comprehensive list of significant deficiencies noted during the financial audit, then from an audit production cost perspective, incremental costs of publicly disclosing the list would be virtually zero.
Link (a) depends on “sourcing” known misstatements to material weakness as their cause. Because DR is directed toward detecting material misstatements (but not immaterial misstatements that might indicate existence of material weakness), the risk of failing to detect a material weakness via Link (a) is greater than the risk of failing to detect a material misstatement (i.e., CDRa > DR).
It is important to distinguish identification of firms having “ineffective ICFR” (that is, at least one material weakness) from identification of the severity or extent of the material weakness or weaknesses resulting from a full 404(b) audit. For example, a firm with ineffective controls might have a single material weakness or many such weaknesses, some of which might remain undetected. A critical policy question is whether disclosing all material weaknesses or even significant deficiencies is worth the incremental cost of detecting them, or is knowledge of “ineffective” controls sufficient for investors?
We are not aware of any other research based on access to detailed engagement level data either during the AS2 period (2003–2006) or the AS5 period (2007–present).
Besch (SEC 2009) also notes that while the percentage of “material/numerous adjustments” continued to rise after AS5 implementation, the percentage of firms reporting material weaknesses dropped.
Indirect evidence of the early importance of Link (a) also comes from a Moody's Investors Service report that only 5 percent (4 of 74) of the companies it followed, citing material weaknesses in internal control reports in 2005 (the first year of SOX 404 application), did not experience a misstatement resulting in a financial restatement (Moody's 2006).
AS5 states only that walkthroughs “ordinarily are sufficient to evaluate design effectiveness” when they include a mix of “inquiry of appropriate personnel, observation of the company's operations, and inspection of relevant documentation” (PCAOB 2007a, para. 43).
Similarly, Bedard and Graham (2011) find that, without a misstatement, auditors are less likely to classify a particular control deficiency as a material weakness.
For example, at a 2005 SEC roundtable discussion on the implementation of SOX 404(b), the vice president of Internal Audit of Lockheed Martin noted, “I think we went overboard … We have 6,000” (SEC 2005).
Staffing data for public clients only are not available, but Public Accounting Report data indicate that, across public and private clients, audit staff numbers increased about 50 percent from 2003 to 2006, while total audit partner numbers increased by less than 10 percent (Public Accounting Report 2004, 2007). Assuming public clients comprise half of total audit staffing and account for the entire staffing increase, public client audit staffs would have increased by about 100 percent, which approximates the increase in public client audit fees noted in research studies using Audit Analytics, when 404(b) was first applied.
Information about internal controls might provide warning to investors of risk of misstatements in unaudited interim financial statements, but this role has not been discussed as the objective of Section 404.
The Combined Code is a set of corporate governance principles. The Listing Rules have statutory authority and require that publicly listed companies disclose how they have complied with the Code.
In addition to the U.K., The Netherlands, Germany, Italy, and Canada have all adopted versions of the “comply or explain” framework (see Van de Poel and Vanstraelen  for an analysis of experience in The Netherlands). Other European countries have adopted “voluntary compliance with best practices.”
While we assume management makes assessments under SOX 404(a), this section could also be implemented using management disclosures under SOX 302 or some alternative mechanism. The 302 alternative imposes less assessment burden on management, but it is also less likely to yield careful analyses than is a 404(a) report based on a formal assessment by management.
We thank technical partners of four global network accounting firms for their insights offered in exploratory interviews regarding early experiences with integrated audits. We also thank Abe Akresh, Dana Hermanson, Bob Libby, Jaime Schmidt, Tom Stober, and especially Zoe-Vonna Palmrose for their insights and comments, and we acknowledge helpful suggestions of the editor and two anonymous reviewers. Funding for the research project described in this article was provided by the Center for Audit Quality.
The interviews mentioned in this paper were facilitated by the Research Advisory Board of the Center for Audit Quality. However, the views expressed in this article and its content are those of the authors alone and not those of the Center for Audit Quality.