DSC 433/533 – Homework 6
Reading
“Data Mining Techniques” by Berry and Linoff (2nd edition): chapter 7 (pages 211-255).
Exercises
Hand in answers to the following questions at the beginning of the first class of week 7. The
questions are based on the German Credit Case (see separate document on the handouts page of
the course website) and the Excel dataset German.xls (available on the data page of the course
website).
1. In homework 4 you made a Standard Partition of the data into Training, Validation, and Test
samples with 500/300/200 observations in each sample. Find this file and open it in Excel
(alternatively open German.xls and re-do the partition: select all the variables to be in the
partition, use a random seed of 12345, and specify percentages of 50%, 30%, and 20%
respectively for each sample). Fit a neural network model to classify “RESPONSE.” In
particular:
Select XLMiner > Classification > Neural Network (MultiLayer Feedforward).
Step 1: Move all the variables from “CHK_ACCT” to “FOREIGN” to the “Input variables”
box and “RESPONSE” to the “Output variable” box (i.e., the only variable not used is
“OBS#”).
Step 2: Leave all options at their default values (i.e., select “Normalize input data,” set “#
hidden layers” to 1, set “# nodes” to 25, set “# epochs” to 30, set “step size for gradient
descent” to 0.1, set “weight change momentum” to 0.6, set “error tolerance” to 0.01, set
“weight decay” to 0, select “squared error” for the cost function, set “standard” for the
hidden layer sigmoid, and set “standard” for the output layer sigmoid).
Step 3:
Select “Summary report” for “Score training data.”
Select “Detailed report,” “Summary report,” and “Lift charts” for “Score validation
data.”
De-select “Summary report” for “Score test data” (we’ll be using the test data later in
the assignment).
The “classification confusion matrix” for the validation data for the neural network model is on
the “NNC_Output1” worksheet:
Classification Confusion Matrix Expected net profits
Predicted Class Predicted Class
Actual Class 1 0 Actual Class 1 0
1187 19 128050 0
063 31 0-31500 0
-3450
You can manually add a table for expected net profits using the average net profit table from
p3 of the case. For example, 187 customers offered a loan (i.e. with modeled probability of
good credit risk more than 0.5) paid it off and resulted in a profit of 150*187 = 28,050 DM.
Conversely, 63 customers offered a loan defaulted and resulted in a loss of 500*63 = 31,500
DM. The other 19+31 customers have a modeled probability of good credit risk less than 0.5
and so would not be offered a loan. Net profit is therefore 28,050 – 31,500 = –3,450 DM (i.e. a
loss).
You can change the cut-off probability from 0.5 to some other number to see whether setting a
different cut-off can lead to better profitability – in XLMiner it is the number in the blue cell
just above the “classification confusion matrix” for the validation data (cell F70?).
To turn in: Complete the following table:
