Eric Chen's Blog Credit Card Fraud Detection - Dealing with Imbalanced Data¶
It is important that credit card companies are able to recognize fraudulent credit card transactions so that customers are not charged for items that they did not purchase.
In this post, we are going to detect fraudulent transactions on credit card transaction data. This dataset can be downloaded here.
The Dataset¶
This dataset presents transactions that occurred in two days, where we have 492 frauds out of 284,807 transactions. The dataset is highly unbalanced, the positive class (frauds) account for 0.172% of all transactions.
It contains only numerical input variables which are the result of a PCA transformation. Unfortunately, due to confidentiality issues, we cannot provide the original features and more background information about the data. Features V1, V2, … V28 are the principal components obtained with PCA, the only features which have not been transformed with PCA are 'Time' and 'Amount'. Feature 'Time' contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature 'Amount' is the transaction Amount, this feature can be used for example-dependant cost-senstive learning. Feature 'Class' is the response variable and it takes value 1 in case of fraud and 0 otherwise.
Our Goal: our goal is to predict fraudulent transactions. Because in this case, the consequence of a false negative is more serious than a false positive.
Exploratory Data Analysis¶
Data Overview¶
- The data set contains total 31 rows and 284807 columns, and there are no missing values presented.
- Due to confidentiality, 28 out of 31features (V1 TO V28) are given based on PCA, this column may contain sensitive information specific to the individual customer. These are already scaled features.
- There are 3 non PCA column, 'Time', 'Amount', and 'Class', as described in the above section. we can perform EDA on them for a better understanding of the data.
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import datetime
import math
import matplotlib
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')
sns.set(style="darkgrid", palette="deep")
pd.options.display.max_columns = 50
df = pd.read_csv("creditcard.csv")
print(df.shape)
df.head()
df.columns
df.info()
df.isnull().sum()
df[['Time', 'Amount', 'Class']].describe().T
Distribution of Fraud and Non-Fraud Transactions¶
- Only 0.17% transactions are fraudulent. So, it is clearly observed that fraudulent transaction in our dataset is far less than the non-fraudulent transaction.
- We have to find a way to deal with highly imbalanced dataset.
print(f'Count of Fraud and Non Fraud Transactions: \n{df.Class.value_counts()}')
print('\n')
print(f'Percentage of Fraud and Non Fraud Transactions: \n{df.Class.value_counts(normalize=True)}')
explode = [0,0.2]
plt.figure(figsize=(6,6), dpi=60)
df['Class'].value_counts().plot(kind="pie", autopct="%1.1f%%", fontsize=20, explode=explode)
plt.title("Fraudulent and Non-Fraudulent Distribution",color='b', fontsize = 15,fontweight='bold')
plt.legend(["Non-Fraud", "Fraud"], fontsize=20)
plt.show()
Transaction Amount Distributions in Non-Fraud and Fraud Transactions¶
As we can see the maximum amount of Non-Fraud transactions is way higher than the maximum amount of Fraud transactions. You could perform an outliar removal step to remove some potential outliars. But in this post, we won't remove any of these values. Because in real world situations, legit transactions can be of any amount.
plt.figure(figsize=(8,5), dpi=80)
sns.boxplot(x='Class',y='Amount', data=df)
plt.title('Transaction Amount for Non-Fraud and Fraud Transactions', fontweight='bold')
plt.show()
Exploring Time Feature¶
- Time column represents the time gap between the first and any other transaction in second
- Here we can see last transaction happened 48 hr(approx.) after the first transaction, so we have two days data in hand
# Maximum duration of data recording
df.Time.max()/(60*60)
Distribution of transactions in two days¶
plt.figure(figsize=(10,6))
plt.title("Distribution of Transactions in two days duration", fontsize=15,fontweight='bold')
sns.distplot(df.Time/60/60, bins=48, color='c')
plt.xlabel("Time (hour)")
plt.show()
# converting Time into hours
df['Time_Hours']=df['Time']/(60*60)
plt.figure(figsize=(10,5))
plt.title('Fraudulent and Non Fraudulent Transaction Distribution in Two Days',fontsize = 20, color='k', fontweight='bold')
sns.distplot(df[df.Class==0].Time_Hours.values,color='g')
sns.distplot(df[df.Class==1].Time_Hours.values, color='r')
plt.legend(['Non-Fraud', 'Fraud'])
plt.show()
Distribution of Transactions in the First Day¶
# Ploting data in 0-24 hrs time frame
plt.figure(figsize=(10,5))
plt.title('Fraudulent and Non Fraudulent Transaction Distribution in first Day',fontsize = 20, color='k', fontweight='bold')
sns.distplot(df[df.Class==0].Time_Hours.values,color='g', bins=48)
sns.distplot(df[df.Class==1].Time_Hours.values, color='r',bins=48)
plt.xlim([0,24])
plt.legend(['Non-Fraudulent', 'Fraudulent'])
plt.show()
Distribution of Transactions in the Second Day¶
# Ploting data in 25-48 hrs time frame
plt.figure(figsize=(10,5))
plt.title('Fraudulent and Non Fraudeulent Transaction Distribution in second Day',fontsize = 20, color='k', fontweight='bold')
sns.distplot(df[df.Class==0].Time_Hours.values,color='g', bins=100)
sns.distplot(df[df.Class==1].Time_Hours.values, color='r',bins=100)
plt.xlim([25,48])
plt.legend(['Non-Fraudulent', 'Fraudulent'])
plt.show()
Scaling of Feature¶
Note:
- V1 to V28 PCA, so already in scaled form
- Distribution of Amount feature is highly positively skewed, so we will scale this feature using different method and compare the prediction of the machine learning algorithm for better accuracy for different scaling
Log Transformation of Amount¶
There are many ways you can perform to scale a feature, such as normalization, standardization, log transformation etc. In this case, we will perform log transformation, because it gives us the best scaling effect.
df['Log_Amount'] = np.log(df.Amount+0.01)
plt.figure(figsize=(15,5))
plt.subplot(121)
ax0 = sns.distplot(df[df.Class==0].Log_Amount,bins=100,color='g')
ax0 = sns.distplot(df[df.Class==1].Log_Amount,bins=100,color='r')
ax0.set_title('Non-Fraud vs Fraud Distribution', fontsize=16, fontweight='bold')
ax0.legend(['Non-Fraud', 'Fraud'])
# plt.show()
plt.subplot(122)
ax1 = sns.boxplot(x ="Class",y="Log_Amount", data=df)
ax1.set_title("Distribution After Log Transform", fontsize=16, fontweight='bold')
ax1.set_xlabel("Non-Fraud vs Fraud", fontsize=12, fontweight='bold')
ax1.set_ylabel("Amount(Log)", fontsize = 12, fontweight='bold')
plt.show()
# Creating a copy
data = df
# target
y = data.Class
# creating X
X = data.drop(columns=['Time', 'Class', 'Time_Hours', 'Amount'])
So far, we have done feature preprocessing. The next step is to deal with imbalanced data.
Deal with Imbalanced Data¶
There are multiple ways to deal with imbalanced data:
- Leave it unchanged
- Under sample the majority class
- Over sample the minority class
Normality, I would try all these methods, compare their results, and finally decide which technique to use. To make this post short and simple, we will use SMOTE over sampled data.
SMOTE Over Sampling¶
Synthetic Minority Oversampling TEchnique, or SMOTE for short. It first selects a minority class instance a at random and finds its k nearest minority class neighbors. The synthetic instance is then created by choosing one of the k nearest neighbors b at random and connecting a and b to form a line segment in the feature space. The synthetic instances are generated as a convex combination of the two chosen instances a and b
from imblearn.over_sampling import SMOTE
from collections import Counter
print('Original dataset shape %s' % Counter(y))
smote = SMOTE(random_state=42)
X_smote, y_smote = smote.fit_resample(X, y)
print('Resampled dataset shape %s' % Counter(y_smote))
plt.figure(figsize=(4,4), dpi=100)
pd.Series(y_smote).value_counts().plot(kind = 'pie', autopct='%1.1f%%', fontsize = 20)
plt.title("After SMOTE Sampling: Fraudlent and Non-Fraudlent Distribution",color='b', fontsize = 15, fontweight='bold')
plt.legend(["Non-Fraud", "Fraud"])
plt.show()
XGBoost and Random Forest on SMOTE Over Sampled Data¶
In this post, we will use two tree-based classifiers, XGBoost Classifier and Random Forest Classifier, to build our models.
Here, our goal is to predict Fraudulent traction correctly. So, our focus area is to check the accuracy percentage of fraudulent class prediction, more precisely Recall Score of Class 1.
from xgboost import XGBClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import precision_score, recall_score, f1_score, roc_auc_score
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
from sklearn.metrics import roc_curve
from sklearn.model_selection import train_test_split
# Spliting Data
X_train, X_test, y_train, y_test = train_test_split(X_smote, y_smote, test_size=0.3, random_state=0)
XGBoost on SMOTE Over Sampled Data¶
xgb = XGBClassifier()
xgb.fit(X_train, y_train)
y_pred = xgb.predict(X_test)
# Prediction Accuracy
print('\n')
print('*** Looking at Performance Measures ***')
print(f'Accuracy Score: {accuracy_score(y_pred , y_test)}')
# Confusion Matrix
cnf_matrix=confusion_matrix(y_test, y_pred)
# ROC AUC Curve
fpr, tpr, _ = roc_curve(y_test, y_pred)
auc = roc_auc_score(y_test, y_pred)
print(f'AUC Score: {auc}')
print('\n')
print(classification_report(y_test, y_pred))
print('\n')
plt.figure(figsize=(15,6))
#Ploting Confusion Matrix
plt.subplot(121)
ax = sns.heatmap(pd.DataFrame(cnf_matrix), annot = True, cmap = 'Blues', fmt = 'd')
ax.set_title("Confusion Matrix", fontsize=16, fontweight='bold')
ax.set_xlabel("Actual", fontsize=12, fontweight='bold')
ax.set_ylabel("Predicted", fontsize = 12, fontweight='bold')
# Ploting ROC AUC Curve
plt.subplot(122)
ax1 = plt.plot(fpr, tpr, label="data 1, auc="+str(auc))
plt.legend(loc=4)
plt.title('ROC_AUC Curve',fontsize=16, fontweight='bold')
plt.show()
Using default parameters of XGBoost classifier, we obtained an overall accuray of 0.98. The recall score for Fraudulent class (Class 1) is 0.97.
Random Forest Classifier on SMOTE Over Sampled Data¶
rfc = RandomForestClassifier()
rfc.fit( X_train, y_train )
# Predicting Test Data
y_pred = rfc.predict(X_test)
# Prediction Accuracy
print('\n')
print('*** Looking at Performance Measures ***')
print(f'Accuracy Score: {accuracy_score(y_pred , y_test)}')
# Confusion Matrix
cnf_matrix=confusion_matrix(y_test, y_pred)
# ROC AUC Curve
fpr, tpr, _ = roc_curve(y_test, y_pred)
auc = roc_auc_score(y_test, y_pred)
print(f'AUC Score: {auc}')
print('\n')
print(classification_report(y_test, y_pred))
print('\n')
plt.figure(figsize=(15,6))
#Ploting Confusion Matrix
plt.subplot(121)
ax = sns.heatmap(pd.DataFrame(cnf_matrix), annot = True, cmap = 'Blues', fmt = 'd')
ax.set_title("Confusion Matrix", fontsize=16, fontweight='bold')
ax.set_xlabel("Actual", fontsize=12, fontweight='bold')
ax.set_ylabel("Predicted", fontsize = 12, fontweight='bold')
# Ploting ROC AUC Curve
plt.subplot(122)
ax1 = plt.plot(fpr, tpr, label="data 1, auc="+str(auc))
plt.legend(loc=4)
plt.title('ROC_AUC Curve',fontsize=16, fontweight='bold')
plt.show()
The default parameters of Random Forest Classifier gives us 99.99% overall accuracy on test set. And the recall score for Class 1 (Fraudulent) is also close to 1.
Next Let's perform hyper-parameter tuning to see whether we can improve the performance of the XGBoost classifier.
Hyperparameter Tuning for XGBoost¶
from sklearn.model_selection import GridSearchCV
# xgboost Classifier parameters
xgb_params = {
"booster": ['gbtree','gblinear', 'dart'],
"max_depth": [3, 4, 5],
"n_estimators": [50, 75, 100],
'tree_method':['gpu_hist'],
'predictor':['gpu_predictor']
}
grid_xgb = GridSearchCV(XGBClassifier(), xgb_params,verbose = True, n_jobs=-1, cv=3)
grid_xgb.fit(X_train, y_train)
Fitting 3 folds for each of 27 candidates, totalling 81 fits
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 2 concurrent workers.
[Parallel(n_jobs=-1)]: Done 46 tasks | elapsed: 3.5min
[Parallel(n_jobs=-1)]: Done 81 out of 81 | elapsed: 15.6min finished# tree best estimator
xgb_clf = grid_xgb.best_estimator_
xgb_clf
XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
colsample_bynode=1, colsample_bytree=1, gamma=0,
learning_rate=0.1, max_delta_step=0, max_depth=5,
min_child_weight=1, missing=None, n_estimators=100, n_jobs=1,
nthread=None, objective='binary:logistic',
predictor='gpu_predictor', random_state=0, reg_alpha=0,
reg_lambda=1, scale_pos_weight=1, seed=None, silent=None,
subsample=1, tree_method='gpu_hist', verbosity=1)xgb = XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
colsample_bynode=1, colsample_bytree=1, gamma=0,
learning_rate=0.1, max_delta_step=0, max_depth=5,
min_child_weight=1, missing=None, n_estimators=100, n_jobs=1,
nthread=None, objective='binary:logistic',
predictor='gpu_predictor', random_state=0, reg_alpha=0,
reg_lambda=1, scale_pos_weight=1, seed=None, silent=None,
subsample=1, tree_method='gpu_hist', verbosity=1)
xgb.fit( X_train, y_train )
# Predicting Test Data
y_pred = xgb.predict(X_test)
# Prediction Accuracy
print('\n')
print('*** Looking at Performance Measures ***')
print(f'Accuracy Score: {accuracy_score(y_pred , y_test)}')
# Confusion Matrix
cnf_matrix=confusion_matrix(y_test, y_pred)
# ROC AUC Curve
fpr, tpr, _ = roc_curve(y_test, y_pred)
auc = roc_auc_score(y_test, y_pred)
print(f'AUC Score: {auc}')
print('\n')
print(classification_report(y_test, y_pred))
print('\n')
plt.figure(figsize=(15,6))
#Ploting Confusion Matrix
plt.subplot(121)
ax = sns.heatmap(pd.DataFrame(cnf_matrix), annot = True, cmap = 'Blues', fmt = 'd')
ax.set_title("Confusion Matrix", fontsize=16, fontweight='bold')
ax.set_xlabel("Actual", fontsize=12, fontweight='bold')
ax.set_ylabel("Predicted", fontsize = 12, fontweight='bold')
# Ploting ROC AUC Curve
plt.subplot(122)
ax1 = plt.plot(fpr, tpr, label="data 1, auc="+str(auc))
plt.legend(loc=4)
plt.title('ROC_AUC Curve',fontsize=16, fontweight='bold')
plt.show()
Summary:¶
In this post, we've build a credit card fraud detection classifier using XGBoost and Random Forest Classifer. The dataset is highly imbalanced. So we used SMOTE Over Sampling to first balance the dataset. Other methods to deal with imbalanced dataset were also briefly discussed. In practice, you should always try different methods, and select the one with the best performance.
For Random Forest Classifier, we obtained a pretty good performance using the default parameters. For XGBoost Classifier, we used a grid search to finde the best parameters, and achieved a comparative performance.
Comments
comments powered by Disqus