A neural network is a computational system that creates predictions based on existing data. Let us train and test a neural network using the neuralnet library in R.
How To Construct A Neural Network?
A neural network consists of:
- Input layers: Layers that take inputs based on existing data
- Hidden layers: Layers that use backpropagation to optimise the weights of the input variables in order to improve the predictive power of the model
- Output layers: Output of predictions based on the data from the input and hidden layers
Solving classification problems with neuralnet
In this particular example, our goal is to develop a neural network to determine if a stock pays a dividend or not.
As such, we are using the neural network to solve a classification problem. By classification, we mean ones where the data is classified by categories. e.g. a fruit can be classified as an apple, banana, orange, etc.
In our dataset, we assign a value of 1 to a stock that pays a dividend. We assign a value of 0 to a stock that does not pay a dividend. The dataset for this example is available at dividendinfo.csv.
Our independent variables are as follows:
- fcfps: Free cash flow per share (in $)
- earnings_growth: Earnings growth in the past year (in %)
- de: Debt to Equity ratio
- mcap: Market Capitalization of the stock
- current_ratio: Current Ratio (or Current Assets/Current Liabilities)
We firstly set our directory and load the data into the R environment:
setwd("your directory")
mydata <- read.csv("dividendinfo.csv")
attach(mydata)
Let’s now take a look at the steps we will follow in constructing this model.
Data Normalization
One of the most important procedures when forming a neural network is data normalization. This involves adjusting the data to a common scale so as to accurately compare predicted and actual values. Failure to normalize the data will typically result in the prediction value remaining the same across all observations, regardless of the input values.
We can do this in two ways in R:
- Scale the data frame automatically using the scale function in R
- Transform the data using a max-min normalization technique
We implement both techniques below but choose to use the max-min normalization technique. Please see this useful link for further details on how to use the normalization function.
Scaled Normalization
scaleddata<-scale(mydata)
Max-Min Normalization
For this method, we invoke the following function to normalize our data:
normalize <- function(x) {
return ((x - min(x)) / (max(x) - min(x)))
}
Then, we use lapply to run the function across our existing data (we have termed the dataset loaded into R as mydata):
maxmindf <- as.data.frame(lapply(mydata, normalize))
We have now scaled our new dataset and saved it into a data frame titled maxmindf:
We base our training data (trainset) on 80% of the observations. The test data (testset) is based on the remaining 20% of observations.
# Training and Test Data trainset <- maxmindf[1:160, ] testset <- maxmindf[161:200, ]
Training a Neural Network Model using neuralnet
We now load the neuralnet library into R.
Observe that we are:
- Using neuralnet to “regress” the dependent “dividend” variable against the other independent variables
- Setting the number of hidden layers to (2,1) based on the hidden=(2,1) formula
- The linear.output variable is set to FALSE, given the impact of the independent variables on the dependent variable (dividend) is assumed to be non-linear
- The threshold is set to 0.01, meaning that if the change in error during an iteration is less than 1%, then no further optimization will be carried out by the model
Deciding on the number of hidden layers in a neural network is not an exact science. In fact, there are instances where accuracy will likely be higher without any hidden layers. Therefore, trial and error plays a significant role in this process.
One possibility is to compare how the accuracy of the predictions change as we modify the number of hidden layers.
For instance, using a (2,1) configuration ultimately yielded 92.5% classification accuracy for this example.
#Neural Network library(neuralnet) nn <- neuralnet(dividend ~ fcfps + earnings_growth + de + mcap + current_ratio, data=trainset, hidden=c(2,1), linear.output=FALSE, threshold=0.01) nn$result.matrix plot(nn)
Our neural network looks like this:
We now generate the error of the neural network model, along with the weights between the inputs, hidden layers, and outputs:
nn$result.matrix
1
error 2.027188266758
reached.threshold 0.009190064608
steps 750.000000000000
Intercept.to.1layhid1 3.287965374794
fcfps.to.1layhid1 -1.723307330428
earnings_growth.to.1layhid1 -0.076629853467
de.to.1layhid1 1.243670462201
mcap.to.1layhid1 -3.520369700429
current_ratio.to.1layhid1 -3.068677865885
Intercept.to.1layhid2 3.618803162161
fcfps.to.1layhid2 1.109150492946
earnings_growth.to.1layhid2 -11.588713924832
de.to.1layhid2 -1.526458929898
mcap.to.1layhid2 -3.769192938001
current_ratio.to.1layhid2 -4.547481937028
Intercept.to.2layhid1 2.991704593713
1layhid.1.to.2layhid1 -7.372717428050
1layhid.2.to.2layhid1 -22.367528820159
Intercept.to.dividend -5.673537382132
2layhid.1.to.dividend 17.963989719804
Testing The Accuracy Of The Model
As already mentioned, our neural network has been created using the training data. We then compare this to the test data to gauge the accuracy of the neural network forecast.
In the below:
- The “subset” function is used to eliminate the dependent variable from the test data
- The “compute” function then creates the prediction variable
- A “results” variable then compares the predicted data with the actual data
- A confusion matrix is then created with the table function to compare the number of true/false positives and negatives
#Test the resulting output
temp_test <- subset(testset, select = c("fcfps","earnings_growth", "de", "mcap", "current_ratio"))
head(temp_test)
nn.results <- compute(nn, temp_test)
results <- data.frame(actual = testset$dividend, prediction = nn.results$net.result)
The predicted results are compared to the actual results:
results
actual prediction
161 0 0.003457573932
162 1 0.999946522139
163 0 0.006824520245
...
198 0 0.005474975456
199 0 0.003427332586
200 1 0.999985252611
Confusion Matrix
Then, we round up our results using sapply and create a confusion matrix to compare the number of true/false positives and negatives:
roundedresults<-sapply(results,round,digits=0)
roundedresultsdf=data.frame(roundedresults)
attach(roundedresultsdf)
table(actual,prediction)
prediction
actual 0 1
0 17 0
1 3 20
A confusion matrix is used to determine the number of true and false positives generated by our predictions. The model generates 17 true negatives (0’s), 20 true positives (1’s), while there are 3 false negatives.
Ultimately, we yield an 92.5% (37/40) accuracy rate in determining whether a stock pays a dividend or not.
Solving regression problems using neuralnet
We have already seen how a neural network can be used to solve classification problems by attempting to group data based on its attributes. However, what if we wish to solve a regression problem using a neural network? i.e. one where the dependent variable is an interval one and can take on a wide range of values?
Let us now visit the gasoline.csv dataset. In this example, we wish to analyze the impact of the explanatory variables capacity, gasoline, and hours on the dependent variable consumption.
Essentially, we wish to determine the gasoline spend per year (in $) for a particular vehicle based on different factors.
Accordingly, our variables are as follows:
- consumption: Spend (in $) on gasoline per year for a particular vehicle
- capacity: Capacity of the vehicle’s fuel tank (in litres)
- gasoline: Average cost of gasoline per pump
- hours: Hours driven per year by owner
Data Normalization
Again, we normalize our data and split into training and test data:
# MAX-MIN NORMALIZATION
normalize <- function(x) {
return ((x - min(x)) / (max(x) - min(x)))
}
maxmindf <- as.data.frame(lapply(fullData, normalize))
# TRAINING AND TEST DATA
trainset <- maxmindf[1:32, ]
testset <- maxmindf[33:40, ]
Neural Network Output
We then run our neural network and generate our parameters:
#4. NEURAL NETWORK
library(neuralnet)
nn <- neuralnet(consumption ~ capacity + gasoline + hours,data=trainset, hidden=c(2,1), linear.output=TRUE, threshold=0.01)
nn$result.matrix
1
error 0.158611967443
reached.threshold 0.007331578682
steps 66.000000000000
Intercept.to.1layhid1 1.401987575173
capacity.to.1layhid1 1.307794013481
gasoline.to.1layhid1 -3.102267882386
hours.to.1layhid1 -3.246720660493
Intercept.to.1layhid2 -0.897276576566
capacity.to.1layhid2 -1.934594889387
gasoline.to.1layhid2 3.739470402932
hours.to.1layhid2 1.973830465259
Intercept.to.2layhid1 -1.125920206855
1layhid.1.to.2layhid1 3.175227041522
1layhid.2.to.2layhid1 -2.419360506652
Intercept.to.consumption 0.683726702522
2layhid.1.to.consumption -0.545431580477
Generated Neural Network
Here is what our neural network looks like in visual format:
Model Validation
Then, we validate (or test the accuracy of our model) by comparing the estimated gasoline spend yielded from the neural network to the actual spend as reported in the test output:
results <- data.frame(actual = testset$consumption, prediction = nn.results$net.result)
results
actual prediction
33 0.7556029883 0.6669224684
34 0.7801494130 0.6458686668
35 0.8356456777 0.6549105183
36 0.8399146211 0.6646982158
37 0.8431163287 0.6631168047
38 0.8890074707 0.6629885579
39 0.9124866596 0.6649999344
40 1.0000000000 0.6665075920
Accuracy
In the below code, we are then converting the data back to its original format, and yielding an accuracy of 90% on a mean absolute deviation basis (i.e. the average deviation between estimated and actual gasoline consumption stands at a mean of 10%). Note that we are also converting our data back into standard values given that they were previously scaled using the max-min normalization technique:
predicted=results$prediction * abs(diff(range(consumption))) + min(consumption) actual=results$actual * abs(diff(range(consumption))) + min(consumption) comparison=data.frame(predicted,actual) deviation=((actual-predicted)/actual) comparison=data.frame(predicted,actual,deviation) accuracy=1-abs(mean(deviation)) accuracy [1] 0.9017828022
You can see that we obtain 90% accuracy using a (2,1) hidden configuration. This is quite good, especially considering that our dependent variable is in the interval format. However, let’s see if we can get it higher!
What happens if we now use a (5,2) hidden configuration in our neural network? Here is the generated output:
nn <- neuralnet(consumption ~ capacity + gasoline + hours,data=trainset, hidden=c(5,2), linear.output=TRUE, threshold=0.01)
nn$result.matrix
1
error 0.049463073770
reached.threshold 0.009079608691
steps 183.000000000000
Intercept.to.1layhid1 -0.484165225327
capacity.to.1layhid1 3.271476705612
gasoline.to.1layhid1 -13.185417334090
hours.to.1layhid1 0.926588147188
Intercept.to.1layhid2 -0.931405056650
capacity.to.1layhid2 0.527977084370
gasoline.to.1layhid2 5.893120354012
hours.to.1layhid2 -0.435230849092
Intercept.to.1layhid3 0.389302962895
capacity.to.1layhid3 -1.502423111329
gasoline.to.1layhid3 -4.684748555999
hours.to.1layhid3 -6.319048800780
Intercept.to.1layhid4 -0.094490811578
capacity.to.1layhid4 -2.399916325456
gasoline.to.1layhid4 -4.115161295471
hours.to.1layhid4 5.013344559754
Intercept.to.1layhid5 0.759624731279
capacity.to.1layhid5 -0.565467044104
gasoline.to.1layhid5 -7.076912238164
hours.to.1layhid5 -6.709144936619
Intercept.to.2layhid1 0.157424617083
1layhid.1.to.2layhid1 7.364054381868
1layhid.2.to.2layhid1 -3.671237007644
1layhid.3.to.2layhid1 6.295218032535
1layhid.4.to.2layhid1 -0.303371875453
1layhid.5.to.2layhid1 12.271950628363
Intercept.to.2layhid2 0.353976458576
1layhid.1.to.2layhid2 -2.460042549015
1layhid.2.to.2layhid2 0.062791089253
1layhid.3.to.2layhid2 2.376623876363
1layhid.4.to.2layhid2 -2.385599836002
1layhid.5.to.2layhid2 5.234292659554
Intercept.to.consumption 0.921627990820
2layhid.1.to.consumption -0.524918897571
2layhid.2.to.consumption -0.669503028647
results <- data.frame(actual = testset$consumption, prediction = nn.results$net.result)
results
actual prediction
33 0.7556029883 0.6554040151
34 0.7801494130 0.7781191265
35 0.8356456777 0.7611519348
36 0.8399146211 0.7980981880
37 0.8431163287 0.8027250788
38 0.8890074707 0.8047567120
39 0.9124866596 0.7969363797
40 1.0000000000 0.7802800479
predicted=results$prediction * abs(diff(range(consumption))) + min(consumption) actual=results$actual * abs(diff(range(consumption))) + min(consumption) comparison=data.frame(predicted,actual) deviation=((actual-predicted)/actual) comparison=data.frame(predicted,actual,deviation) accuracy=1-abs(mean(deviation)) accuracy [1] 0.9577401232
We see that our accuracy rate has now increased to nearly 96%, indicating that modifying the number of hidden nodes has enhanced our model!
Conclusion
In this tutorial, you have learned how to use a neural network to solve classification problems.
Specifically, you saw how we can:
- Normalize data for meaningful analysis
- Classify data using a neural network
- Test accuracy using a confusion matrix
- Determine accuracy when the dependent variable is in interval format
Many thanks for viewing this tutorial.