Artificial Intelligence (AI) and Data Science continue their progression towards becoming mainstream and ubiquitous. This is a very exciting time for scientists, model developers, programmers, and a lot of other technically inclined professionals. But to be honest it can be confusing and overwhelming at times. We all hear terms like “AI”, “Data Science”, “Big Data”, “Machine Learning”, “Statistical Learning”, “Data Mining”, “Deep Learning”, etc., and it’s often hard to make sense of it all even for those of us who have been writing code to implement statistical models for decades. But it seems these terms are being used among people in every field and every industry. How do remote sensing professionals use data from a satellite to create land cover maps? how do certain streaming services determine what shows or movies to recommend based on your watching habits? How did Cambridge Analytica determine the poor shmucks Donald Trump should focus on? The answers to all these questions lay in machine learning algorithms.  (If interested you can find more information on the differences or definitions of all the terms mentioned above on various discussion threads on social sites like Quora, StackExchange, LinkedIn, and KDNuggets among others.)

This article will be a little more focused on the question: how can we use machine learning in areas where statistics have traditionally been employed in credit risk?

“What is the difference between Machine Learning and Statistics?”

The difference between machine learning and statistics is a little nuanced, but at the end of the day it is all about collecting data, analyzing data, interpreting results, presenting findings, and possibly projecting future occurrences.  Modern computing has allowed us to use larger data-sets and process those data-sets faster. With better and more efficient methods to collect, clean and store data, naturally machine learning, which helps us make sense of large and sometimes unstructured data, has become more important.  Corporations are investing billions to better leverage all the data they have collected, are collecting, or plan to collect in the future. So, it follows that there is increased effort in finding more efficient methods to analyze the data.

There is a great deal of cross-fertilization of ideas between traditional statistical modeling and machine learning. Both fields often use similar theoretical approaches to build models with slightly different terminologies, so the difference is a little more subtle than some people make it out to be. For example, regressions are fundamental in both areas. Although, some academics classify techniques such as linear regressions and Logistic regressions as statistical learning and SMVs, Neural Networks, and Decision Trees as machine learning.

Statistics, the traditional approach to ‘learning from data’ has focused mainly on providing a structure around the formalization of relationships between variables in the form of mathematical equations.  Statistical Inference first defines a formal model along with the assumptions and constraints. Then these models are calibrated using the input data.

Another notable focus for Statisticians is the need for making ‘causal inferences’.  Statisticians (and for good measure, Economists and social scientists with statistical training) are not only interested in making predictions but ascribing reason to phenomena is an important part of the job.

Machine Learning on the other hand puts less emphasis on this formalization and uses the data to inform us through efficient algorithms and data structures. Machine Learning (which is a branch of Artificial Intelligence) often involves data sets that are orders of magnitude larger than those in standard statistics problems.  As such, machine learning is a set of algorithms that train on a data set or take actions to optimize some prediction. Compared to traditional statistical methods, machine learning techniques are more prone to over-fitting the data, that is, to detecting patterns that might not generalize to other data.

Supervised vs Unsupervised vs Semi-supervised

The family of machine learning algorithms can be divided into subsets of supervised, unsupervised, and semi-supervised learning algorithms.

Figure 1. Family of Machine Learning Algorithms

In supervised learning, the algorithm is supplied with training data (both the input and the correct output). Basically, we try to ‘infer’ a function from a set of labeled data. The output could be a class label (in classification or ranking) or a real number (in regression). In unsupervised learning the algorithm does not have the benefit of being provided with input values and their associated output values. Instead, the algorithm must rely on other sources of feedback to determine whether it is learning correctly. A third class of machine learning techniques, called semi-supervised learning, uses a combination of both labeled and unlabeled data for training. This approach is motivated by the fact that labeled data is often hard to obtain, whereas unlabeled data is generally not.

Parametric and Non-parametric methods

Ordinary Least Squares and Logistic Regressions are parametric models you find in both arenas. These methods are characterized by a finite set of parameters, which are usually pre-specified in the form of binding model assumptions.

Non-parametric models (such as Support Vector Machines) are generally associated with machine learning.  In Traditional statistics, which depends on sampling, formalization of equations, etc, does not lend well to non-parametric modeling.

Can we use machine learning in areas where statistics have traditionally been employed in credit risk?

The financial industry is meeting the machine learning bus head on. In the November 1, 2017  article from the Financial Stability Board titled “Artificial intelligence and machine learning in financial services Market developments and financial stability implications”, the authors do a great job in segmenting the areas in which machine learning and AI are currently being used or where significant resources are being allocated to research ways in which machine learning/AI can be leveraged:

These are: (i) customer-focused (or ‘front-office’) uses, including credit scoring, insurance, and client-facing chatbots; (ii) operations-focused (or ‘back-office’) uses, including capital optimisation, model risk management and market impact analysis; (iii) trading and portfolio management in financial markets; and (iv) uses of AI and machine learning by financial institutions for regulatory compliance (‘RegTech’) or by public authorities for supervision (‘SupTech’).

For years banks have been accumulating customer data, loan transactions data, trading data, and many other useful types of information they were either required to maintain for legal, regulatory, or other purposes. With the advent of modern computing systems and high-powered processors, financial institutions have begun finding ways to use these data to better run their business, comply to regulations, etc.

Credit Risk Modeling

To demonstrate how we may make use of machine learning algorithms in credit risk modeling, we use machine learning methods to estimate the probability of default and credit quality migration matrix for a wholesale commercial mortgage portfolio, conditioned on the macroeconomic environment. We illustrate this point with a case study where we compare the results from a traditional econometric approach with various machine learning approaches.

Correctly estimating default and credit migration rates are an essential part of pricing and risk management, which goes beyond the use for just stress-testing. Migration matrices, provide transition probabilities across rating classes and are used in virtually every credit portfolio model used by banks.  Other than stress testing, migration matrices are used for important functions such as Allowance for Loan and Lease Losses and Business planning. Based on a forecast for next year’s default rate, for example, lenders can set appropriate loan rates for short-term loans.

The matrices contain credit migration probabilities, which characterize historical changes in the financial strength of borrowers. When observed together, these migration probabilities can describe the trajectory of an entity’s credit path in a migration matrix. (For more information on estimating migration matrices please see my article “Use R to Easily Estimate Migration Matrices with RTransProb (Part 1)”.)

In instances where a bank believes economic conditions will be normal the average migration and default rates can be used project the number of defaults, credit downgrades and credit upgrades. Unfortunately, quite often the economy deviates from the norm. When the bank believes a recession or expansion is near, the average migration matrix might not be sufficient for business planning purposes.

The question therefore becomes: how do you relate an average through-the-cycle (TTC) migration matrix, or PD distribution, to macro conditions?

Credit Risk – Stress Testing

The global banking landscaped underwent significant changes following the Global Financial crisis of 2008. Banking regulators have emphasized the use of stress testing as a regulatory tool to help prevent the occurrences which led to the financial crisis. In the U.S. the Comprehensive Capital Analysis and Review (CCAR) is an annual stress test conducted by the Federal Reserve Bank with the purpose of ensuring that banks’ capital planning processes are well-defined and forward-looking.   The exercise is intended to give Banks, Regulator, and the public a picture of how banks will fare under different stress scenarios given their unique risks and to determine if they have sufficient capital to continue operations through times of economic and financial stress. As part of the CCAR, the Federal Reserve evaluates how banks manage their balance sheets, with a focus on capital adequacy, internal capital adequacy assessment processes, and plans to make capital distributions, such as dividend payments or stock repurchases.

Currently, about 30 of the largest bank holding companies in the US, with at least $50B in total consolidated assets with tier 1 material portfolios – auto, mortgage, card, and commercial meet the threshold established by the federal regulators, undergo the CCAR.

Data

All of this hypothetical data is available in the R package RTransProb (on Cran and Github). This data is for illustration purposes only.

Credit Data- The credit data used in these examples is representative of the commercial mortgage portfolio of a medium sized bank with data starting from the early 90s. As to be expected this data is not always well behaved.

Macroeconomic Data – While comparison of the results from the Credit Cycle Index approach to those from the machine learning approaches may not be straightforward, to make this as close to an “apples to apples” comparison we will use the same historical US macroeconomic variable to ‘train’ the various machine learning models (Multinomial Logistic Regression, Support Vector Machines, Linear Discriminate Analysis, and Neural Networks) and calibrate the Credit Cycle Index.

We will forecast the migration matrix using an example of baseline, adverse and severely adverse scenarios. The idea here is that as macroeconomic conditions get progressively worse they are reflected in the variables used in the model.

Since these examples are only for illustration purposes, I performed a quick and dirty feature (variable) selection operation. The feature selection exercise (Student T-test and ANOVA F-Test) produced a single variable, level of the Market Volatility Index (Adj. Rsq 0.373), which correlated the Credit Cycle Index with the economy:

Scaling – All independent variables (macroeconomic variable) have been scaled to ease convergence of the algorithms for the SVM, Neural Networks, and Linear Discriminate Analysis examples.

 

Continue Reading

Next: Machine Learning and Credit Risk (part 2) – Credit Cycle Method