Assume you are given a dataset for a large bank and you are tasked to come up with a credit risk score for each customer.You have just been briefed that you are going to work on this project and you have to come up with a prototype demonstrating how this problem could be solved.
Approach
The credit risk scoring is a very complicated process with a lot of due diligence on data, model reviews internal controls and sign offs. As a first step you could follow the steps outlined below with the accompanying code to create a straw man version of your approach. You can find full code here: Link
Data Analysis
Logistic Regression
The second step in your prototype will be to train an explainable model, such as a logistic regression model so that you can identify and explain the driving variables.
Decision Tree
The third step in your prototype will be to train a more complicated model to assess if you can improve over your explainable model through additional tuning as well.
Tuning can be done as follows:
Random Forest
The final step in your prototype will be to train using a highly robust and more black box model to assess if you can improve over your existing approaches, to see if it is worthwhile to pursue this path.
Tuning can be done as follows:
Conclusion
Depending on the problem you are trying to solve, you could pick a model that serves your case, simplest is always the better unless the complicated one is significantly better. Also note that while there may be a temptation to jump into models, most improvement in model performance come from data wrangling and creating new features for your models.
The code in github:
https://github.com/obaidpervaizgill/CreditRiskModelling
Hi Dr Gill,
Same problem that Jim McGill, the command
predCreditLogReg 0.5)
Returns the following error:
Error: unexpected numeric constant in “predCreditLogReg 0.5”
and I presume that
accuracyCreditLogReg 0.5))[1,1])
will do it as well
Sorry if I missed something.
Regards.
If you saw the code from the r-bloggers site then there are a lot of copy and paste errors. Use the code from the current page.
Yes, you are right Mr. Nguyen. Thanks for the advice. Even the code from the current page is wrong. For instance, this chunk:
tunedCreditDecTree <- train(train$default ~., data = train[,c(-17,-18)], method = "rpart", trControl = fitControlDecTree, tuneGrid = tuneGridDecTree) Returns the following: Error in `[.data.frame`(data, , all.vars(Terms), drop = FALSE) : undefined columns selected Any suggestion? Thanks again Mr. Nguyen.
Dear Tal,
Congratulations for your work with Credit Risk Scoring using R.
I tried to repeat your R code, but I can not follow after this point:
creditLogReg <- glm(train$default ~ ., data = train[,c(-17,-18)], family = "binomial" ) #removing split feature and dependent variable Error in eval(family$initialize) : valores y devem ser 0 <= y <= 1 I changed the values from yes or no from credit$default, but it didn't work. Could you help me? Best Regards, José.
Hi Dr Gill, Working through your interesting article I have come upon two problems in the second section of code (#training a model using logistic regression#). 1) I think the glm function expects the y variable to be numeric, and 2) Is there not something missing from the line “predCreditLogReg 0.5)” ? Apologies if these are misunderstandings on my part. Kind regards, JM
Dear Professor Pervaiz Gill,
I was trying to use the code reported in this post. In some cases, however, I have found errors (especially in the first part). Am I wrong somewhere, huh?