Neural networks models are a flexible class of machine learning algorithms which can be used for both supervised as well as unsupervised learning and can approximate discrete or continuous functions. They are loosely modeled on the functions of human brains and attempt to allow computers to learn in manner similar to humans.

The simplest neural network structure is a single layer feed-forward neural network (or a perceptron) and consists of an *input layer*, a *hidden layer* and an *output layer* as shown in Figure 5.

- The
contains the input data which enters the model. These are the independent variables**input layer** - The
in multilayer networks is where the activation functions are applied. Note: neural networks can have many hidden layers. The diagram below displays only two layers. Deep neural networks can contain many hidden layers.**hidden layer** - The
produces the output. The output can be binary, categorical or continuous.**output layer**

**Figure 8**. Graphical illustration of an Artificial Neural Networks

The following is a quick and dirty explanation of how the neural network trains the data. First, each connection to a hidden or output node has an assigned weight. The node then applies an *activation function *to the sum of these weighted inputs and generates an output. The output values are then compared to the actual values to calculate an *error rate* or *cost function* (there are many types of cost functions). The information is then fed back into the neural network to update the weights by a process known as *back-propagation*. This process is repeated until the error rate or cost function is minimized.

Artificial Neural Networks, Convolutional Neural Networks, Recurring Neural Networks and Reinforcement Learning (all of which fall under Deep Learning) are basically extensions of the neural networks theory notably.

**Disadvantages:**

- Neural Networks are prone to over-fitting.
- While Neural Networks are flexible the learning curve is also steeper than many of the other algorithms. The responsibility for calibrating the model largely lies with the user. Decisions must be made as the number of hidden neurons, the maximum steps for the training of the neural network must be set large enough to ensure convergence, what activation function to use, number of iteration, etc.
- The training time for neural networks in usually longer than other algorithms described previously in the blog.

**Figure 9** displays the conditional matrices derived by the Neural Networks approach. It displays the forecasted migration matrices conditioned on Baseline, Adverse, and Severely Adverse economic conditions.

library("RTransProb") for (i in c(24,25,26)) { tic() data <- data histData <- histData.normz predData_ann2 <- predData_ann predData_ann2 <- subset( predData_ann2, X == i, select = c(Market.Volatility.Index..Level..normz ) ) indVars = c("Market.Volatility.Index..Level..normz" ) startDate = "1991-08-16" endDate = "2007-08-16" depVar <- c("end_rating") pct <- 1 wgt <- "mCount" ratingCat <- c("A", "B", "C", "D", "E", "F", "G") defind <- "G" ratingCat <- as.numeric(factor( ratingCat, levels = c('A', 'B', 'C', 'D', 'E', 'F', 'G'), labels = c(1, 2, 3, 4, 5, 6, 7) )) defind <- as.numeric(factor( defind, levels = c('A', 'B', 'C', 'D', 'E', 'F', 'G'), labels = c(1, 2, 3, 4, 5, 6, 7) )) method = "cohort" snapshots = 1 interval = 1 hiddenlayers = c(1) activation = "logistic" stepMax = 1e9 #increase to make sure the DNN converges calibration = "FALSE" rept = 1 ann_TM <- transForecast_ann( data, histData, predData_ann2, startDate, endDate, method, interval, snapshots, defind, depVar, indVars, ratingCat, pct, hiddenlayers, activation, stepMax, rept, calibration ) print(ann_TM) toc() }

**Continue Reading**

Previous: Machine Learning and Credit Risk (part 4) – Support vector Machines

Next: Machine Learning and Credit Risk (part 6) – Multi-class Linear Discriminate Analysis