Transfer Learning and Textual Analysis of Accounting Disclosures: Applying Big Data Methods to Small(er) Data Sets

We introduce and apply machine transfer learning methods to analyze accounting disclosures. We use the examples of the new BERT language model and sentiment analysis of quarterly earnings disclosures to demonstrate the key transfer learning concepts of: (i) pre-training on generic "Big Data", (ii) fine-tuning on small accounting datasets, and (iii) using a language model that captures context rather than stand-alone words. Overall, we show that this new approach is easy to implement, uses widely-available and low-cost computing resources, and has superior performance relative to existing textual analysis tools in accounting. We conclude with suggestions for opportunities to apply transfer learning to address important accounting research questions.