Classification_algorithms.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classifying loan customers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n",
    "\n",
    "Lets first load required libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-05-17 17:39:42--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 23101 (23K) [text/csv]\n",
      "Saving to: ‘loan_train.csv’\n",
      "\n",
      "100%[======================================>] 23,101      --.-K/s   in 0.1s    \n",
      "\n",
      "2020-05-17 17:39:42 (228 KB/s) - ‘loan_train.csv’ saved [23101/23101]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solving environment: done\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /opt/conda/envs/Python36\n",
      "\n",
      "  added / updated specs: \n",
      "    - seaborn\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    ca-certificates-2020.1.1   |                0         132 KB  anaconda\n",
      "    seaborn-0.10.1             |             py_0         160 KB  anaconda\n",
      "    openssl-1.1.1g             |       h7b6447c_0         3.8 MB  anaconda\n",
      "    certifi-2020.4.5.1         |           py36_0         159 KB  anaconda\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:         4.2 MB\n",
      "\n",
      "The following packages will be UPDATED:\n",
      "\n",
      "    ca-certificates: 2020.1.1-0         --> 2020.1.1-0        anaconda\n",
      "    certifi:         2020.4.5.1-py36_0  --> 2020.4.5.1-py36_0 anaconda\n",
      "    openssl:         1.1.1g-h7b6447c_0  --> 1.1.1g-h7b6447c_0 anaconda\n",
      "    seaborn:         0.9.0-pyh91ea838_1 --> 0.10.1-py_0       anaconda\n",
      "\n",
      "\n",
      "Downloading and Extracting Packages\n",
      "ca-certificates-2020 | 132 KB    | ##################################### | 100% \n",
      "seaborn-0.10.1       | 160 KB    | ##################################### | 100% \n",
      "openssl-1.1.1g       | 3.8 MB    | ##################################### | 100% \n",
      "certifi-2020.4.5.1   | 159 KB    | ##################################### | 100% \n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n"
     ]
    }
   ],
   "source": [
    "#installing seaborn might takes a few minutes\n",
    "!conda install -c anaconda seaborn -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGp1JREFUeJzt3XuYVNWZ7/HvT+gMKHil1ZYWaRFBVKbBjncMQsJD8IbjJSSOgmPGo/ESjuGImhyTE88xqIyXxCsRA4mIGhLRIYkOKkTwDtiCiEFGUVtBgUk0BlHE9/xRu3saUkB3V1XX7urf53nqqapVe6/1broWb+1Vu9ZSRGBmZpY2OxQ7ADMzs2ycoMzMLJWcoMzMLJWcoMzMLJWcoMzMLJWcoMzMLJWcoApI0l6S7pP0hqSFkp6VdGqe6h4saVY+6moNkuZKqil2HNb6SqkfSCqX9LyklyQNKmA7Hxeq7rbECapAJAmYCTwVEftHxGHAKKCySPF0LEa71r6VYD8YCrwWEQMiYl4+YrKtc4IqnCHAZxFxZ31BRLwVET8DkNRB0g2SXpS0WNL/SMoHJ2cbMyS9Jmla0smRNDwpmw/8U329knaSdE9S10uSTknKx0j6taR/B/4jl4ORNEXSHZLmJJ+Ev5K0uUzSlEbb3SFpgaSlkv7PVuoalnyKXpTE1yWX2CzVSqYfSKoGrgdGSKqV1Hlr72VJKyVdm7y2QNJASY9J+k9JFyTbdJH0RLLvkvp4s7T7vxr9+2TtUyUrInwrwA24FLhpG6+fD/wgefwPwAKgChgMfEjmE+YOwLPAsUAn4B2gNyDgQWBWsv+1wD8nj3cFlgM7AWOAOmD3rcQwD6jNcvtqlm2nAPcnbZ8CfAQcmsS4EKhOtts9ue8AzAX6J8/nAjVAN+ApYKekfDxwdbH/Xr4V5laC/WAMcGvyeKvvZWAlcGHy+CZgMdAVKAc+SMo7Ajs3qmsFoOT5x8n9MGBScqw7ALOA44r9d22tm4d9Womk28h0sM8i4stk3nj9JZ2ebLILmU73GfBCRNQl+9UCPYGPgTcj4vWk/F4ynZukrpMljUuedwJ6JI9nR8R/ZYspIpo7hv7vERGSlgDvR8SSJJalSYy1wJmSzifT+SqAfmQ6Z70jk7Knkw/EXyLzn4+1AyXSD+pt7738SHK/BOgSEX8F/ippg6Rdgb8B10o6DvgC6A7sBaxuVMew5PZS8rwLmX+fp1oYc5viBFU4S4HT6p9ExEWSupH5hAiZT0SXRMRjjXeSNBj4tFHRJv7777S1iRMFnBYRf9qiriPIdILsO0nzyHyq29K4iHg8S3l9XF9sEeMXQEdJVcA44MsR8edk6K9TllhnR8Q3txaXlZRS7AeN29vWe3mb/QU4i8wZ1WERsVHSSrL3l59ExF3biKNk+TuownkS6CTpwkZlOzZ6/BhwoaQyAEkHStppG/W9BlRJ6pU8b9wpHgMuaTRGP6ApAUbEoIioznLbVqfclp3J/EfwoaS9gK9n2eY54BhJBySx7ijpwBa2Z+lXyv0g1/fyLmSG+zZKOh7YL8s2jwH/0ui7re6S9mxGG22aE1SBRGYAeSTwFUlvSnoBmEpmnBrgbuBVYJGkV4C72MYZbURsIDOU8bvky+G3Gr18DVAGLE7quibfx9MUEfEymaGIpcA9wNNZtllDZhx/uqTFZDp531YM01pRKfeDPLyXpwE1khaQOZt6LUsb/wHcBzybDK3PIPvZXkmq/0LOzMwsVXwGZWZmqeQEZWZmqeQEZWZmqeQEZWZmqZSKBDV8+PAg89sG33wrpVvO3Dd8K9Fbk6QiQa1du7bYIZilkvuGtWepSFBmZmZbcoIyM7NU2m6CSqav/yD5ZXZ92e6SZkt6PbnfLSmXpJ9KWpFMDT+wkMGbmVnpaspksVOAW4FfNiq7AngiIiZIuiJ5Pp7M3Gu9k9sRwB3JvZlZq9m4cSN1dXVs2LCh2KG0a506daKyspKysrIW7b/dBBURT0nquUXxKWTWa4HMvFpzySSoU4BfJvNvPSdpV0kVEbGqRdGZmbVAXV0dXbt2pWfPniRzx1oriwjWrVtHXV0dVVVVLaqjpd9B7VWfdJL7+tl1u5NZTKxeXVJmZtZqNmzYwB577OHkVESS2GOPPXI6i833RRLZ3g1Zr3mXdH6yFPKCNWvW5DkMa4/2q6hAUqvc9quoKNhxuG/kh5NT8eX6N2jpgoXv1w/dSaoAPkjK64B9G21XCbyXrYKImERmKWNqamqa/MMts615e/Vq6vapbJW2Kt+rK1jd7htmGS09g3oEGJ08Hg083Kj8nORqviOBD/39k5kVW77PrptyBt2hQweqq6s55JBDOOOMM1i/fn3Daw899BCSeO21/14CauXKlRxyyCEAzJ07l1122YUBAwbQp08fjjvuOGbNmrVZ/ZMmTaJv37707duXww8/nPnz5ze8NnjwYPr06UN1dTXV1dXMmDFjs5jqbytXrszln7XgtnsGJWk6mQsiukmqA34ITAAelHQe8DZwRrL574ERwApgPXBuAWI2M2uWfJ9dN+UMunPnztTW1gJw1llnceedd3LZZZcBMH36dI499ljuv/9+fvSjH2Xdf9CgQQ1Jqba2lpEjR9K5c2eGDh3KrFmzuOuuu5g/fz7dunVj0aJFjBw5khdeeIG9994bgGnTplFTU7PVmNqC7Z5BRcQ3I6IiIsoiojIiJkfEuogYGhG9k/v/SraNiLgoInpFxKERsaDwh2Bmlm6DBg1ixYoVAHz88cc8/fTTTJ48mfvvv79J+1dXV3P11Vdz6623AnDddddxww030K1bNwAGDhzI6NGjue222wpzAEXimSTMzAro888/5w9/+AOHHnooADNnzmT48OEceOCB7L777ixatKhJ9QwcOLBhSHDp0qUcdthhm71eU1PD0qVLG56fddZZDUN569atA+CTTz5pKDv11FPzcXgF1dKLJMzMbBvqkwFkzqDOO+88IDO8N3bsWABGjRrF9OnTGThw+5PuZH5euu3XG181VwpDfE5QZmYFkC0ZrFu3jieffJJXXnkFSWzatAlJXH/99dut76WXXuKggw4CoF+/fixcuJAhQ4Y0vL5o0SL69euX34MoMg/xmZm1khkzZnDOOefw1ltvsXLlSt555x2qqqo2uwIvm8WLF3PNNddw0UUXAXD55Zczfvz4hqG72tpapkyZwne+852CH0Nr8hmUmZW8HnvvndffrvVIrpRrrunTp3PFFVdsVnbaaadx3333MX78+M3K582bx4ABA1i/fj177rknP/3pTxk6dCgAJ598Mu+++y5HH300kujatSv33nsvFQX8AXkxaHvjmq2hpqYmFizwBX+WG0mt+kPdJvSdnKcycN9omWXLljUMh1lxbeVv0aS+4SE+MzNLJScoMzNLJScoMzNLJScoMzNLJScoMzNLJScoMzNLJScoMyt5+1T2yOtyG/tU9mhSu6tXr2bUqFH06tWLfv36MWLECJYvX87SpUsZMmQIBx54IL179+aaa65p+NnClClTuPjii/+urp49e7J27drNyqZMmUJ5eflmS2i8+uqrACxfvpwRI0ZwwAEHcNBBB3HmmWfywAMPNGzXpUuXhiU5zjnnHObOncuJJ57YUPfMmTPp378/ffv25dBDD2XmzJkNr40ZM4bu3bvz6aefArB27Vp69uzZrL9JU/iHumZW8la9+w5HXP1o3up7/sfDt7tNRHDqqacyevTohlnLa2tref/99xkzZgx33HEHw4YNY/369Zx22mncfvvtDTNFNMc3vvGNhlnO623YsIETTjiBG2+8kZNOOgmAOXPmUF5e3jD90uDBg5k4cWLDfH1z585t2P/ll19m3LhxzJ49m6qqKt58802+9rWvsf/++9O/f38gs7bUPffcw4UXXtjsmJvKZ1BmZgUwZ84cysrKuOCCCxrKqqurWb58OccccwzDhg0DYMcdd+TWW29lwoQJeWv7vvvu46ijjmpITgDHH398w4KI2zNx4kSuuuoqqqqqAKiqquLKK6/khhtuaNhm7Nix3HTTTXz++ed5i3tLTlBmZgXwyiuv/N2SGJB9qYxevXrx8ccf89FHHzW7ncbDdtXV1XzyySdbbbupmrKcR48ePTj22GP51a9+1eJ2tsdDfGZmrWjLZTEa21r5tmQb4stVthizlV111VWcfPLJnHDCCXltv57PoMzMCuDggw9m4cKFWcu3nF/xjTfeoEuXLnTt2rWgbTdn/y1jzLacxwEHHEB1dTUPPvhgi9vaFicoM7MCGDJkCJ9++ik///nPG8pefPFFevfuzfz583n88ceBzMKGl156KZdffnne2v7Wt77FM888w+9+97uGskcffZQlS5Y0af9x48bxk5/8hJUrVwKwcuVKrr32Wr73ve/93bbf//73mThxYl7i3pKH+Mys5FV037dJV941p77tkcRDDz3E2LFjmTBhAp06daJnz57cfPPNPPzww1xyySVcdNFFbNq0ibPPPnuzS8unTJmy2WXdzz33HAD9+/dnhx0y5xVnnnkm/fv354EHHthsPanbb7+do48+mlmzZjF27FjGjh1LWVkZ/fv355ZbbmnS8VVXV3Pddddx0kknsXHjRsrKyrj++usbVghu7OCDD2bgwIFNXrq+OXJabkPS/wS+DQSwBDgXqADuB3YHFgFnR8Rn26rHSwpYPni5Davn5TbSoyjLbUjqDlwK1ETEIUAHYBRwHXBTRPQG/gyc19I2zMys/cr1O6iOQGdJHYEdgVXAEGBG8vpUYGSObZiZWTvU4gQVEe8CE4G3ySSmD4GFwF8iov6XW3VA92z7Szpf0gJJC9asWdPSMMxKjvtGfqRhtfD2Lte/QS5DfLsBpwBVwD7ATsDXs2yaNcKImBQRNRFRU15e3tIwzEqO+0buOnXqxLp165ykiigiWLduHZ06dWpxHblcxfdV4M2IWAMg6bfA0cCukjomZ1GVwHs5tGFm1myVlZXU1dXhM9Di6tSpE5WVLb9wKZcE9TZwpKQdgU+AocACYA5wOpkr+UYDD+fQhplZs5WVlTXMI2dtVy7fQT1P5mKIRWQuMd8BmASMBy6TtALYA5ichzjNzKydyemHuhHxQ+CHWxS/ARyeS71mZmae6sjMzFLJCcrMzFLJCcrMzFLJCcrMzFLJCcrMzFLJy21YyVCHMirfq2u1tsyssJygrGTEpo0ccfWjrdJWPtcWMrPsPMRnZmap5ARlZmap5ARlZmap5ARlZmap1CYS1H4VFUhqtdt+FRXFPmQzs3avTVzF9/bq1dTt0/I1RZqrtS5VNjOzrWsTZ1BmZtb+OEGZmVkqOUGZmVkqOUGZmVkqOUGZmVkqOUGZmVkq5ZSgJO0qaYak1yQtk3SUpN0lzZb0enK/W76CNTOz9iPXM6hbgEcjoi/wj8Ay4ArgiYjoDTyRPDczM2uWFicoSTsDxwGTASLis4j4C3AKMDXZbCowMtcgzcys/cnlDGp/YA3wC0kvSbpb0k7AXhGxCiC53zPbzpLOl7RA0oI1a9bkEIZZaXHfMMvIJUF1BAYCd0TEAOBvNGM4LyImRURNRNSUl5fnEIZZaXHfMMvIJUHVAXUR8XzyfAaZhPW+pAqA5P6D3EI0M7P2qMUJKiJWA+9I6pMUDQVeBR4BRidlo4GHc4rQzMzapVxnM78EmCbpS8AbwLlkkt6Dks4D3gbOyLENMzNrh3JKUBFRC9RkeWloLvWamZl5JgkzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0slJygzM0ulnBOUpA6SXpI0K3leJel5Sa9LekDSl3IP08zM2pt8nEF9F1jW6Pl1wE0R0Rv4M3BeHtowM7N2JqcEJakSOAG4O3kuYAgwI9lkKjAylzbMzKx9yvUM6mbgcuCL5PkewF8i4vPkeR3QPduOks6XtEDSgjVr1uQYhlnpcN8wy2hxgpJ0IvBBRCxsXJxl08i2f0RMioiaiKgpLy9vaRhmJcd9wyyjYw77HgOcLGkE0AnYmcwZ1a6SOiZnUZXAe7mHaWZm7U2Lz6Ai4sqIqIyInsAo4MmIOAuYA5yebDYaeDjnKM3MrN0pxO+gxgOXSVpB5jupyQVow8zMSlwuQ3wNImIuMDd5/AZweD7qNTOz9sszSZiZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QRXZfhUVSGq1234VFcU+ZDOzJsnLZLHWcm+vXk3dPpWt1l7le3Wt1paZWS58BmVmZqnkBGVmZqnkBGVmZqnkBGVmZqnkBGVmZqnkBGVmZqnU4gQlaV9JcyQtk7RU0neT8t0lzZb0enK/W/7CNTOz9iKXM6jPge9FxEHAkcBFkvoBVwBPRERv4InkuZmZWbO0OEFFxKqIWJQ8/iuwDOgOnAJMTTabCozMNUgzM2t/8vIdlKSewADgeWCviFgFmSQG7LmVfc6XtEDSgjVr1uQjDLOS4L5hlpFzgpLUBfgNMDYiPmrqfhExKSJqIqKmvLw81zDMSob7hllGTglKUhmZ5DQtIn6bFL8vqSJ5vQL4ILcQzcysPcrlKj4Bk4FlEXFjo5ceAUYnj0cDD7c8PDMzy1VbXTUhl9nMjwHOBpZIqk3KrgImAA9KOg94GzgjtxDNzCwXbXXVhBYnqIiYD2grLw9tab1mZmbgmSTM2rzWHL7xgpfWmrxgoVkb15rDN17w0lqTz6DMzCyVfAaVxT8AmYsUzcysWJygsvgUPGRiZlZkHuIzM7NUcoIyM7NUcoIyM7NUcoIyM7NUcoIyM7NUcoIyM7NUcoIyM7NUcoIyM7NU8g91zcxKnDqUteqkAOpQlpd6nKDMrMlaexqwHnvvzVurVrVae6UqNm3kiKsfbbX2nv/x8LzU4wRlZk3WmtOAgacCa+/8HZSZmaWSE5QVzD6VPVptIT3PPm9WejzEZwWz6t132uS4t5mlQ5tIUG31ChQzy01rXpThCzLSpyAJStJw4BagA3B3REzIpb62egVKGnkxRmtLvDZb+5b3BCWpA3Ab8DWgDnhR0iMR8Wq+27Lmc4c3y661P7x13qEDn3yxqdXaa4sKcQZ1OLAiIt4AkHQ/cArgBGVmqVWMS+j9YXHbFBH5rVA6HRgeEd9Onp8NHBERF2+x3fnA+cnTPsCftlFtN2BtXgNNDx9b29SUY1sbEc0eL25m32hqLG2Vj61t2t6xNalvFOIMKts58t9lwYiYBExqUoXSgoioyTWwNPKxtU2FPLbm9I1Cx1JsPra2KV/HVojfQdUB+zZ6Xgm8V4B2zMyshBUiQb0I9JZUJelLwCjgkQK0Y2ZmJSzvQ3wR8bmki4HHyFxmfk9ELM2x2iYPd7RBPra2KU3HlqZY8s3H1jbl5djyfpGEmZlZPnguPjMzSyUnKDMzS6XUJyhJwyX9SdIKSVcUO558kbSvpDmSlklaKum7xY4p3yR1kPSSpFnFjiWfJO0qaYak15K/31FFisN9o41y32hifWn+DiqZNmk5jaZNAr5ZCtMmSaoAKiJikaSuwEJgZCkcWz1JlwE1wM4RcWKx48kXSVOBeRFxd3Kl6o4R8ZdWjsF9ow1z32iatJ9BNUybFBGfAfXTJrV5EbEqIhYlj/8KLAO6Fzeq/JFUCZwA3F3sWPJJ0s7AccBkgIj4rLWTU8J9o41y32i6tCeo7sA7jZ7XUUJv1HqSegIDgOeLG0le3QxcDnxR7EDybH9gDfCLZIjmbkk7FSEO9422y32jidKeoJo0bVJbJqkL8BtgbER8VOx48kHSicAHEbGw2LEUQEdgIHBHRAwA/gYU4/sf9402yH2jedKeoEp62iRJZWQ64LSI+G2x48mjY4CTJa0kM/Q0RNK9xQ0pb+qAuoio/0Q/g0ynLEYc7httj/tGM6Q9QZXstEnKLDwzGVgWETcWO558iogrI6IyInqS+Zs9GRH/XOSw8iIiVgPvSOqTFA2lOEvJuG+0Qe4bzZPqJd8LNG1SWhwDnA0skVSblF0VEb8vYkzWNJcA05LE8AZwbmsH4L5hKZXXvpHqy8zNzKz9SvsQn5mZtVNOUGZmlkpOUGZmlkpOUGZmlkpOUGZmlkpOUCkk6UeSxuWxvr6SapPpR3rlq95G9c+VVJPves2ycf9oP5yg2oeRwMMRMSAi/rPYwZiljPtHSjlBpYSk7ydr+zwO9EnK/lXSi5JelvQbSTtK6irpzWQqGCTtLGmlpDJJ1ZKek7RY0kOSdpM0AhgLfDtZY+dySZcm+94k6cnk8dD6KVckDZP0rKRFkn6dzImGpMMk/VHSQkmPJcsiND6GHSRNlfR/W+0fztoF94/2yQkqBSQdRmbakwHAPwFfTl76bUR8OSL+kcySA+clyw/MJTNdP8l+v4mIjcAvgfER0R9YAvww+fX9ncBNEXE88BQwKNm3BuiSdOZjgXmSugE/AL4aEQOBBcBlyTY/A06PiMOAe4D/1+gwOgLTgOUR8YM8/vNYO+f+0X6leqqjdmQQ8FBErAeQVD+n2iHJp61dgS5kprWBzDoylwMzyUwl8q+SdgF2jYg/JttMBX6dpa2FwGHKLAT3KbCITEccBFwKHAn0A57OTInGl4BnyXxqPQSYnZR3AFY1qvcu4MGIaNwpzfLB/aOdcoJKj2xzTk0hs5Loy5LGAIMBIuJpST0lfQXoEBGvJB1w+41EbFRmJuVzgWeAxcDxQC8yn0J7AbMj4puN95N0KLA0Ira2hPMzwPGS/i0iNjQlFrNmcP9ohzzElw5PAadK6px8cjspKe8KrEqGD87aYp9fAtOBXwBExIfAnyXVD0+cDfyR7J4CxiX384ALgNrITMz4HHCMpAMAknH9A4E/AeWSjkrKyyQd3KjOycDvgV9L8gcfyyf3j3bKCSoFkuWtHwBqyayBMy956X+TWUl0NvDaFrtNA3Yj0wnrjQZukLQYqAZ+vJUm5wEVwLMR8T6wob7NiFgDjAGmJ/U8B/RNlhU/HbhO0stJrEdvcRw3khkS+ZUkv7csL9w/2i/PZt5GSTodOCUizi52LGZp4/5RGnyq2QZJ+hnwdWBEsWMxSxv3j9LhMygzM0slj4OamVkqOUGZmVkqOUGZmVkqOUGZmVkqOUGZmVkq/X+B9kdy9nlKPwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 7)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature befor One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  return self.partial_fit(X, y)\n",
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let's use the following algorithms to find the best fit for this data:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "xtrain,xtest,ytrain,ytest = train_test_split(X,y,test_size=0.2,random_state=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running KNN for multiple values of k "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.629, 0.629, 0.7, 0.714, 0.729, 0.7, 0.757, 0.686, 0.757, 0.743, 0.757]\n"
     ]
    }
   ],
   "source": [
    "accuracy = []\n",
    "\n",
    "for k in range(1,12):\n",
    "    knn = KNeighborsClassifier(n_neighbors = k,metric='minkowski').fit(xtrain,ytrain)\n",
    "    yhat = knn.predict(xtest)\n",
    "    accuracy.append(round(accuracy_score(ytest,yhat),3))\n",
    "\n",
    "print(accuracy)\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEbCAYAAADNr2OMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VNX5wPHvm419T9h3SEBkExFQIFhrFVG0ti5YW+2mtta1alu7WH8uXWxt1VbbUm1t60JxLQiiuDGgrCJhJ2FfEmCSsCWRrO/vj3sHxyEhE5i5M5O8n+eZJ5k7995z7mRy37nnnHteUVWMMcaYcCXFugLGGGMSiwUOY4wxDWKBwxhjTINY4DDGGNMgFjiMMcY0iAUOY4wxDWKBI86IyJdFxCci+0XkUxHZISKvi8jkk9jX/SISlfHWInKuu/+kkOV9RURF5JvRKLeeOqmIPHQS293vbpsSjXpFWjT/rsaEwwJHHBGR24DXgDzgO8DFQOBEeF6s6lWHc4FfcvxnqAA4G5jjdYWMMd5IiG9YTcjdwOuq+p2gZe8Bfw/9Zh+vVLUcWBLrehgTTERSgSq1O54jIiFORk1IR2BvbS+oak3wcxHpJyLPi4hfRMpFZJWIXF5fASKSIiL3ishGd7t8EXlURJqHrNdKRH4jIlvc9faKyCsi0kVE7se52gCodJt51N3uuKYqEXlWRHaLyBkislBEykQkT0S+V0v9zheRT0TkqIhsFpHvuttvr+/YatlXSxGZLSIFIjIijE1OE5H33foViMgDwQFbRJqLyB9FZK2IlLjvyWwRGRxSblcR+Zf73pa7+3pDRDqH1O23IrJNRCrcnz+rpekv8J4dFZE9IvILQMI8/lQReUhEtrtlbHefpwatE/h73eQeb4GIHHSPq2cYZVwgInPd7crc9+YuEUmuZd0bRGSl2wR7QEQWiMg5Qa/X+ZlzX6+1iS708xF0TDeLyCMikg+UA+1FJENE/iYiuW59d4nICyLSo5b9jhCR10SkyK3zJhG5133tzyKyL/i9dJe3FpEjIvLr+t67RGZXHPFlGXC9iGwF/qequbWtJCK9gKXAfuBOwA9cDbwiIl9W1VknKOM5YCrwW+Aj4DTgQaAv8FV3/2nAfGAk8GucK4h2wIVAB+BpoCdOc9oEoDqMY2sLvAA8BjwAfAv4i4hsUtX33XKH4DRxLQOmAWnAL9yya2rbaV1EpCPwBpAOnKOq28LY7HXgHzjHfKFbdg1wv/t6M6ANTvNhAU6gvxlYIiKDVTUQ9P8D9AHuAXYBXYAvAi3duqUAbwFDcN77NcA4t7yOwF3ueuk4V5x7getxTn73AL3DfBv+BVwF/ApYhNOE+HOgP/C1kHXvxfk8fBvoDDwKPA9MqqeM/sC7wJ+Ao8BonPcrA/hJYCUR+b17XM/gfOmocY+5N/BRGJ+5fWEec7CfAcuBG4Fkt3693Z/34vzfdHfr9aH7Nzzq1ncM8AGwGed/bDeQCQx39/0U8APgcmBmUJnXAq2Av59EfROHqtojTh5AFrAaUPdRCLwIXBCy3jM4H/pOIcvnA6uCnt/v/ImPPZ/o7ve6kO2udZePdJ9/231+6Qnqer+7TkrI8r7u8m8GLXvWXfaFoGXN3OObHrTsBfe4WgYt64bzj749jPdPcU7qvYENOCeNjDC2CxzLT0KW/x04ArSvY7tknGBwBLgzaHkJcNsJyvuGW152yPKfARVAZ/f5w+7z3kHrtHLfN63nmIa6Zdwfsvzn7vLhIX+vBSHr3e0u796Az6/gfBn9GXAASHKXD8T5cvGHE2wb9meuluXPBn8+go5pJSD11DkZ6OWuf3nQch9O0G95gm0/AN4NWbYSmBfue5aoD2uqiiPqXGGcgfMt72FgFc43mrdE5OdBq04G5gKHxGl6Sgn6FjtCRNrWUcRknBPRKyHbve2+nu3+vADYqye+cmmoMnWvLOBYX0gen//2PA6Yq6plQesV4HwTDtcQd/1dOIHK34BtZ4Y8nwG0xjkJAyAiV4nIUhE5CFQBpe46g4K2Ww7cIyK3i8gwEQltWpoM7MD5ph36d0jFeR/AuUJYoqo7AxuqaikwO4xjCfwtnwtZHngeeiUROphhjfvzhFc3ItLNbfrZgfPZqsQJ3u1xrlwAzsdpFp9+gl1F4zP3urpn85A6f19EckSkBOdvGHh/B7mvtwTGA88HfxZr8RTwBRHJdLc7C+f/928RPIa4ZIEjzqhqtar6VPXnqno+TlPAGuCXItLBXa0zcB3OP2nw43fu653q2H1nnOafkpDt9ods1wnYE7GDchyoZVk5ENy30i2oLsEa0kyRDfQAnlHVkgZsV1s5gec9AERkKvBfnKuZrwFjgbNwrpKCj+NqYBbwI5wryD0icl9Q/0VnnKas0L/fMvf1wN+hWy11qq2eteno/iwIWb435PWA4pDn5e7P5tTBPZ5ZwCU4weI8nPfj4ZBtA8ez+wT1jcZnLvTYEZFbcU747wBfAcbwWaAO1LcDzrnxRPUFZwTkXuAm9/n3gHzCC+wJzfo44pyq5ovI08DjOG2sy4AiYCFOP0Vt8utYXoTT7DOxnu0KCfqW7aECPvuWGqxLA/bxN5y28edEpEpVX2nAtl2ArbWUGzihTQM2q+o3Ayu4naOfOwmr6n6c9u8fiMggnP6J/8MJMH/B+Ttsw+l/qM1292cBtR97OO9HIBB0BbYELe/q/iwKYx/1GYDTp/ENVT12ZeMG2GCF7s8ewKY69hXOZy7Q/5CmqhVBy+v6olTbCKppOM1LdwXVt1/IOgdw+mCO6zD/3M5VK93/zZtF5BF334+qatWJDyPx2RVHHHE7vWsTGLUT+LY4D6eTbp2qrqjlUV77bpiH862qXR3bBQLH20DXWk4AwQJltAjv6MKyBJjiNhUATlMITrNBuFRVbwGeBGaIyJUN2Db0RD4N5+psrfu8JU7TRrBv4LST11WZTar6U5yTUeDEOA+nXb2kjr9D4ES7GBgX/LkQkVY4gxvqsyDoGIJd6/70hbGP+gT+TpWBBW4gvTZkvXdwTsQ3nmBf4Xzmdrg/g5sO2wPn1L56rVoG19f1reAnbvPUIuDrIlLf5zvwReUlnH67xt0p7rIrjviyVkTex7kE3oYzEmkKziXwzKC27vtwrjx8IvJnnG+oHXD+ofqr6rdr27mqfiAiLwIvi8gf3H3U4HQmTgF+7PazPAfcALzoDitcijOa6ELgMVXdCKx3d3uXiLwJVKvqilM8/oeAK3D6dH6P84/4C5ymmQaNqlLVO0SkGnhBRJJU9b9hbHaD2/yyHOdYv4vTuXzQfX0e8GUR+SPOiK0zgduAwOuISDucE+XzwEack9RlOH+fQF/S8zgnq3dF5FEgB6cJcQBwKfBl9+T1R5xRW2+LMwQ6MKrq0zCOf537t77f7T/5CKfP5BfAi6q6Ooz3oz4bcE7mD7vvdSXOCKTQumxx37MfikgbnOatapxmoo3u3yacz9ybwCGc+5p+ifP5+BFOcA/XPODHIvJTnM//eTifuVB34wTfxe7faDdOs/FIVb016Nj2iMhsnL7I2aq6qwF1SVyx7p23x2cPnAAxC+ef8ShOx+snOP8caSHr9sQZFrsHp1OyAGdU1deD1rmfkFEoOFeZt+OcrI7i/CPmAI/gXIkE1muN02eyI2j/L/PZiJ9knG/1+3FO6uou70vto6p213K8HwAfhCz7Es6ggHKcZqObcALpJ2G8fwo8FLLstzhXCdecYLv73W2HAu/jnJj34gyVTQp57x7CadIrwzmxnIETuJ9112mG8y10Hc4J7TBOIPpaSJnN3XI3usda7K53P0Ej1YBROM2SR92/9S9wmr00jPcj1a3vDpyT+g73eWrQOoG/13dDtj3XXX5uPWWMxPl2XoZzcn0AJ+Aq0LeWz/fqoOP9ADg73M+cu84E930qA3KBr1P3qKrv1lLfFjjNhX6c0XBvAP2ofQTaGTj9FQfdz8RGnC9Xofu8xt3+4lifQ7x6iHvgxsQlEWmNM5Z+jn7+jnpj4oKIPI/TnNpfQ27UbaysqcrEFRH5E06zSj7OzVm34zTzPB7LehkTSkTG4VxxXQ38sKkEDbDAYeJPc5zmpS44zRXLgPM1Mm3yxkTSYpzmyH/hDPFtMqypyhhjTIPYcFxjjDEN0iibqtLT07Vv376xroYxxiSUjz/+uFBVM+pbr1EGjr59+7JixaneUmCMMU2LO+dYvTxrqhKRye589ptF5Ce1vP5HcXJKrHLnyg++qaq3iLwtIhtEZL2I9PWq3sYYYz7PkysOcZK6PIlzc9duYLmIzFLVwN3HqOqdQevfinPzTcC/gYdVdb47rr/JDHszxph449UVxxicyeG2qjM52QycaRjqcg1OHopAcp8UVZ0PoKoleuKpjo0xxkSRV4GjB05+hIDd1DHzpIj0wZkC4D13URZwUEReFSel6O+klrSUxhhjvOFV4KgtR3JdN5BMA15W1UA60hScacDvxpnrvz/wzeMKELlRRFaIyAq/vyG5e4wxxjSEV4FjN8400gE9qTtnxDTcZqqgbT9xm7mqcPJCjwrdSFWnq+poVR2dkVHvaDJjjDEnyavAsRzIFJF+blL6aTizwH6Om/SmA86t/MHbdhCRQDQ4j8+m9DbGGOMxTwKHe6VwC05O7A04uSXWicgDInJp0KrXADM0aB4Ut8nqbpzcBWtwmr2aRLIUYyJl/+Gj/G9VpDOzxrdthaW8sTqfmpqmM63S/PX7mLk8+ilBPLsBUFXnAnNDlt0X8vz+Oradj5PxzhhzEv74Th4vLtvJyF7t6dOpVayr44mH52zgnQ37yM7aze+vGE7ntnWmT094n1ZU89Cc9Ty/dCejerfnijN7kpRUW9dyZNhcVcY0chVVNby5tgAAX27TGDhSUVXD4i2FDOnWlmXbipj8+ELeXre3/g0T0No9h7j4Twt5fulObsruz4s3jotq0AALHMY0eh9uLuRgWSXJScKC3ML6N2gEPt5xgNKKam4/P5M3bp1It3bNufE/H/PT19ZQVhGaNj4x1dQof12whcuf+pCy8mqe/+5Y7p1yGs1Son+3QqOcq8oY85lZOfm0a5HKBUO6MHdNARVVNaSlNO7vjL48PylJwjkDOtGmeSqv3TyeR+dvYrpvK0u2FvHEtDMY2qNdrKt50vIPfspdM3NYvLWIi4Z25VeXD6NDqzTPym/cnx5jmrijldW8vW4vFw3tyhdP60JpRTUrdx6IdbWizpfrZ1TvDrRpngpAWkoS9150Gs9/Zyxl5dVc/tSH/HXBFqoTsON8zuoCLnp8ITm7D/LIV4fz1LWjPA0aYIHDmEbtvY37Ka2oZuqI7pwzsBPJSdLo+zn8R8pZl3+Y7Kz04147Z2A68+6YyJeGdOE3b27k2qeXkH/w0xjUsuFKyqu456UcfvDCSvqmt2LObRO56qxeiES3P6M2FjiMacRm5+ST3roZ4/p3om3zVEb1bo8vr3EHjkWbnePLzqr9RuD2LdN48muj+N0Vw1m9+xAXPb6QOasLvKxig63ceYCLn1jIKyt3c+t5A3n5e2fTLz12o+MscBjTSB05Wsm7G/dzyfBuJLujbLIzM1i75zCFJeUxrl30+HIL6dgqjaHd6+7DEBGuHN2LubdNpG96K37wwkrufimHkvL46jivqq7hiXfzuPKvi6mqVmbceDZ3XTCI1OTYnrotcBjTSM1fv4+Kqhqmjuh+bNmkQc638EV5jXN0VU2NsjDPz4SB6WENSe2b3oqXv3c2t503kFdX7mbK4wvjpg9oV3EZ06Yv4Q/zc7lkeDfm3j6RMf06xrpagAUOYxqtWTn59GjfglG92x9bNrR7Ozq2Smu0/RzrCw5TWFLBpDqaqWqTmpzEDy8YxH9vOpvqGuXKvy7miXfzqKqOXdqf1z7ZzUWPL2TT3iM8dvVIHp92Bu1apMasPqEscBjTCBWXVrAor5CpI7p/rvM0KUmYMDAdX15ho5yKI9B/M7GWjvH6nNW3I2/eMZGpw7vxh/m5TJu+hF3F3qb+OfRpJbfP+IQ7/5vD4K5tmHv7RL58Rq0ZKGLKAocxjdCbawuoqlGmjuh23GvZWRkUlpSzYe/hGNQsuny5fk7r1pbObU5uepG2zVN5bNoZPD5tJJv2HuGixxfy2ie7I1zL2i3bVsyUxxfyxuoC7vpSFjNuHEevji09KbuhLHAY0wjNzslnQEYrhnRre9xr2ZnOt3FfI7uLvLS8io93HKh1GG5DXTayB3Nvn8hp3dpw539zuH3GJxz6tDICtTxeZXUNv39rE9OmLyYlWXj5e2dz6xczSYlxB/iJxG/NjDEnZe+hoyzdVnxcM1VA57bNGdy1TaPr51i8pYjKamVSZmTy8fTq2JIZN57N3Rdk8cbqAqY8vpBl24ojsu+AbYWlXPGXj/jz+5v56qiezLltImf07hDRMqLBAocxjcycNQWo8rnRVKEmZWWwYkcxpXE2/PRU+PL8tEhN5sy+kTvxJicJt5yXycvfO5uUZGHa9MX8/q1NVJ5ix7mq8t/lO7n4iYVsLyrjqWtH8bsrR9C6WWLMAmWBw5hGZlZOPqd3b8uAjNZ1rpOdlUFltbJka5GHNYsuX66fswd0isokf2f07sDc2yZyxZk9+fP7m7niLx+xrbD0pPZ1oLSC7z+3kh+/soYRPdsz746JTBl2fF9UPLPAYUwjsrOojJxdB7n0BFcbAKP7dqBFanKjaa7aUVTK9qKyY/030dCqWQqPXDGCp64dxfaiMi5+YiH/Xb6ToLxz9fpwcyGTH/fx7sZ93HvRYJ7/7li6tWsRtTpHS2JcFxljwjJ7dT4Al9QTOJqlJDOuf0cWNJLAEQiAdU0zEklThnXjjN7tuWtmDj9+ZQ3vb/Tz66+ceHba8qpqHn07l+m+rfTPaMUz15+V0LPz2hWHMY3I7Jx8RvfpQI/29X+Lzc7KYHtRGTuLvL1XIRoW5BbSs0MLz+Zv6tauBc99Zyw/nTKYdzfuY/LjPj7cXPsotc37j3D5kx8x3beVa8f2Zs6tExM6aIAFDmMajdx9R9i498gJO8WDBb6dL0jwSQ8D2f6yszI8nSk2KUm4MXsAr908ntbNUrj26aX8au4GyquqAacD/D+Lt3PxE4vYe/goT183mocvH0aLtOgnWoo2CxzGNBKzc/JJEsLuaO2f3ooe7VskfD/Hyp1Otr/sCA3DbaihPdrxxq0T+fq43kz3beXyJz9i6dYivvOvFfzif+sY278T8+6YyPlDusSkftFgfRzGNAKqyqycfM4ZkE5Gm2ZhbSMiZGdlMDsnn8rqmpjPuHqyfLl+kpOEcwZ2ilkdWqQl89CXh3FuVmd+9Mpqrp6+hLSUJH45dQjXn9036jnAvZaYnxRjzOes2XOIHUVl9Y6mCjUpK52S8ipW7oiPGWFPhi/Pz6je7WnbPPaTAJ4/pAvz7pjI988dwKxbxvOt8f0aXdAADwOHiEwWkU0isllEflLL638UkVXuI1dEDoa83lZE9ojIn72qszGJYtaqfFKThQuHdm3QducMTHeyAiZoP0dhSTlr9xyOWTNVbTq3ac6PJw9mcNfjp3tpLDwJHCKSDDwJXAQMAa4RkSHB66jqnao6UlVHAn8CXg3ZzYPAAi/qa0wiqalR3lhdwKSszg2eevtYVsAEnbcqkFfEi2G45jNeXXGMATar6lZVrQBmAJedYP1rgBcDT0TkTKAL8HZUa2lMAlq+vZi9h49y6ciGNVMFZGdmsDb/EEUJmBXQl+unY6s0hiX48NZE41Xg6AHsCnq+2112HBHpA/QD3nOfJwGPAvecqAARuVFEVojICr8/MS+7jTkZs3LyaZGazPmndT6p7bOzMlCFRXXchxCvamoUX15h2Nn+TOR4FThq+6vWdZ/+NOBlVa12n98MzFXVXXWs7+xMdbqqjlbV0RkZdtlqmobK6hreXLuX84d0oWXayQ2SHNqjHR1apibcXeQb9jq5062ZynteDcfdDfQKet4TyK9j3WnAD4Kenw1MFJGbgdZAmoiUqOpxHezGNDUfbi6kuLSCqcNPfpK85CRhQmYGC/MKUVVPb6I7FYF+mWjOT2Vq59UVx3IgU0T6iUgaTnCYFbqSiAwCOgCLA8tU9VpV7a2qfYG7gX9b0DDGMTungDbNU5g06NS+dWdnpuM/Us6GgiMRqln0+XL9DO7ahs5tTy7bnzl5ngQOVa0CbgHeAjYAM1V1nYg8ICKXBq16DTBDGzLdpDFN1NHKat5et5fJp3c95anEA809iTIst7S8ihU7iplkzVQx4dmd46o6F5gbsuy+kOf317OPZ4FnI1w1YxLSB5v8HCmvOunRVMG6BGUF/N6kARGoXXQt2epk+7P+jdiwO8eNSVCzc/JJb53G2f0jM9VGdlYGy7cnRlbABblOtr/REcz2Z8JngcOYBFRSXsW7G/cxZVg3UiI0x1R2ZuJkBfTl+hnXv2NUsv2Z+lngMCYBvbN+H0cra8KeQj0co/t2oHlqUtzPlruzqMzJ9mfNVDFjgcOYBDQ7J5/u7ZpzZu/INdU0T01mXP9O+PLi+0bAQP4QCxyxY4HDmARzsKwCX56fS0Z0j/gd09mZGWwrLGVXcfxmBfTl+unRvgX9Pcr2Z45ngcOYBDNv7V4qq7XBU6iH41hWwDhtrqqsrmHxliLPs/2Zz7PAYUyCmZWTT7/0VpzePfLTdg/IiO+sgCt3HKCkvIpJWXa3eCxZ4DAmgew/fJTFW4uYOqJ7VL5xB7ICfrSliMrqmojv/1T58gLZ/ixwxJIFDmMSyJw1BajCpSNOfm6q+gSyAn6y82D9K3vMl1sYN9n+mjILHMYkkNk5+ZzWrS0DO7eJWhnHsgLGWXNVUUk5a/MPxVW2v6bKAodpcqqqa9jiL4l1NRpsV3EZK3ceZGoUrzbAyQp4Rq/2cTdv1aLNhajaMNx4YIHDNCn7Dx/la08v5YuPLmDO6oJYV6dB3nDrO3V45EdThcrOymDNnkMUl1ZEvaxwLcj106FlKkMt21/MWeAwTcZHmwuZ8sRC1uw+RN9OLfn562vYf+RorKsVtlk5+ZzRuz29OraMelmBrIAL4+SqQ1VZmFfIhMwMki3bX8xZ4DCNXk2N8qd38/j6M0tp1yKV/90ynqevH01pRTU/fXUtiTCL/+b9R9hQcDgq927UZliPdrRvmXosWVKsbSg4gv9IuSVtihMWOEyjVlxawbeeXc6j83OZOqI7s26ZQFaXNgzs3IYfXTiIdzbs45WVe2JdzXrNyikgSeDiYdHt3whIThImDExnYZ4/LgKrz6YZiSsWOEyj9fGOA1z8xEIWbynioS8P5bGrR9Kq2WcpaL41vh9j+nbk/2avI//gpzGs6YmpKm/k5DOufydPs91lZ2Ww/0g5G/fGPitgINtfF8v2FxcscJhGR1V5ZtE2rv7bYlKShVe+fw5fH9fnuBvmkpOE3105nOoa5cevrI6Lb9a1WZd/mK2FpRGdCTccgWGvsZ5+pLS8iuXbi+1qI45Y4DCNyuGjlXz/uZU8+MZ6vjC4M2/cMpFhPesehdOnUyt+OuU0FuYV8vzSnR7WNHyzc/JJSRIuGtrV03K7tmvOoC5tYn4/x7Fsf3b/RtywwGEajXX5h5j6p0XM37CPn005jenfOJN2Leu/w/jasb2ZmJnOr+ZuYEdRqQc1DV9NjTI7J5/srAzat0zzvPzsrHRWbD9AWUXssgL6cv00T02ybH9xxAKHSXiqyovLdnL5Ux9RXlnDf28cxw3Z/cOey0lE+O1Xh5OcJNzz0mqqa+KnyerjnQfIP3TUs9FUobKzMqiorolpVkBfXiHj+neieapl+4sXFjhMQiurqOKumTnc++oaxvbryJzbJjC6b8cG76d7+xbcP/V0lm0v5p8fbotCTU/O7Jx8mqUkcf6QLjEp/6y+Hd2sgLEZlruruIxthaXWTBVnUupfxZj4tHn/EW5+fiV5+0u48/wsbjlv4CndHPaVUT2Yt24vj7y1iXMHZUR1PqhwVFXXMHdNAeef1oXWzWLzr9o8NZmx/TrFrJ8j0DFvHePxxbMrDhGZLCKbRGSziPykltf/KCKr3EeuiBx0l48UkcUisk5EVovI1V7V2cSv/63aw6V//pCikgr+/e0x3H5+5infUSwi/OryYbRKS+aHM3OoivG04ou3FlFYUhH1uanqMykrg60xygoYyPY3IMOy/cUTTwKHiCQDTwIXAUOAa0RkSPA6qnqnqo5U1ZHAn4BX3ZfKgOtU9XRgMvCYiLT3ot4m/hytrOZnr63h9hmrOL17W+bcNpGJEWzGyGjTjIcvH8bq3Yf4ywdbIrbfkzFrVT6tm6Vw7qDOMa1H4Nu+15MeVlbX8JFl+4tLXl1xjAE2q+pWVa0AZgCXnWD9a4AXAVQ1V1Xz3N/zgf2AXbc2QTuLyrjirx/x/NKd3JTdnxduGEfXdpG/IWzKsG5cOqI7j7+bx7r8QxHffzjKq6qZt24vF5zeJeadwrHKCvjJzoOW7S9OeRU4egC7gp7vdpcdR0T6AP2A92p5bQyQBhz3VVBEbhSRFSKywu+Pj4nZTOS8tW4vF/9pITuLyvj7daO5d8pppCZH7+P7wGWn06FVGnfNzKG8qjpq5dRlwSY/R45WxWw0VTAnK2A6H232NiugL9ey/cUrrwJHbdeZdY15nAa8rKqf+28VkW7Af4Bvqepxn15Vna6qo1V1dEaGXZA0FpXVNTw8Zz03/edj+nZqxZzbJvIlD0YYtW+Zxm++MoyNe4/w+Dt5US8v1OzVBXRomcr4ODlpZmdmcKS8ilW7vMsK6Mvzc0Yvy/YXj7wKHLuBXkHPewL5daw7DbeZKkBE2gJzgJ+r6pKo1NDEnYJDn3LN9CX8feE2vjGuDy9//2xPphQP+OJpXbhqdE/+umALK3ce8Kzcsooq3lm/jynDukX1qqohvM4KWFxawZo9h2w0VZzy6lO5HMgUkX4ikoYTHGaFriQig4AOwOKgZWnAa8C/VfUlj+prYmxhnp+Ln1jE+oLDPD5tJA9+eSjNUrxv6//FJUPo1q4Fd8/M4dMKb5qs3tmwn08rqz2fm+pE2rVIZWSv9p4FDmdWXhuGG688CRyqWgXcArwFbABmquo6EXlsZdx9AAAgAElEQVRARC4NWvUaYIZ+fra5q4Bs4JtBw3VHelFv473qGuWP83O57h/LSG+dxqxbJnDZyFq7wzzRpnkqj1wxnK2FpTzy1kZPypy1Kp+ubZsz5iRuZIym7MwMVnuUFdCXW0j7lqkMs2x/ccmzu4pUdS4wN2TZfSHP769lu+eA56JaORMXCkvKuWPGKhZtLuQro3rw0JeH0jIt9veojh+YzvVn9+GfH27nS0O6cM6A6PU7HCqrZEHufq4/uy9JcZbpLjsrnT++k8uizYVR7bRXVXx5fia4zWMm/sRHA6pp8pZvL+biJxayfHsxv/3qMB69ckRcBI2AH180mL6dWnLPS6spKY/ehH9vrdtLZbXGVTNVwPCe7WnfMpUFm6LbXHUs2581U8UtCxwmplSVvy3YwrTpS2iRmsxrN4/n6rN6x90NXy3TUnj0qhEUHPqUh+esj1o5s1fn06dTS4afYCr4WElOEsZ7kBXwWLY/m58qblngMDFzqKySG/79Mb9+cyMXDOnCrFsnMKR721hXq05n9unIDdn9eXHZLt7ftD/i+/cfKefDzYVMHd497gJnwKTM6GcF9OX6GdSlTVRu7jSRYYHDxMSybcVMeWIhH2zazy+nDuGpa0clxHj9O8/PIqtLa3788moOlkW2k/jNtQXUKFw6Mv6aqQImundxR2t0VVlFFSu2HyDb7haPaxY4jKcqq2t49O1NTJvupHV96Xtn863x/eL2G3ao5qnJPHrlSIpLK7h/1rqI7nvWqnwGdWlDVpfYzsp7It3atSCrS+uozVu1ZGsRFdU11r8R5yxwGM9sKyzlir8u5k/vbearo3oy57aJnNE78bK6DevZjlvOG8jrq/J5c01BRPa55+CnrNhxIK6vNgKyMzNYvi06WQF9uYU0T03irDgbimw+zwKHiTpVZebyXVz8xEK2F5by1LWj+N2VI2KWYyISfvCFgQzt0Zafvb6WwpLyU97fGznORAqXDI/tFOrhmDTIyQq4dGtxxPfty/Uztp9l+4t3FjhMVB0oreDm51fyo1dWM6Jne+bdMZEpw+L/5Fif1OQk/nDVSEqOVvHz19ae8iij2avzGdGrPX06xX/eiUBWwAUR7ufYVVzG1sJSJlkzVdyzwGGi5sPNhUx+3Mc7G/Zx70WDef67Y+nWrkWsqxUxWV3acNcFWcxbt5f/rapr6rX6bfWXsHbPYaYmwNUGBGUFjHA/x7FhuBY44p4FDhNx5VXV/HruBr7+zFJaNUvhtZvHc9OkAXF3J3QkfHdif87s04H7/reWvYeOntQ+ZucUIAKXDI///o2A7KwMtvpL2X0gclkBLdtf4rDAYSJq8/4jXP7kR/zNt5WvjenNnFsnMrQRzzeUnCQ8euUIKquVH7+yusFNVqrKrJw9jOnbMaHuW5h0bFhuYUT2V1ldw0ebi8jOSk+YEXZNWViBQ0RuExEbWG3qpKr8Z8kOLn5iEXsPH+Xp60bz8OXDaJHW+Ds5+6a34t4pg1mQ62fG8l31bxBkQ8ERtvhLE2I0VbABGa3p3q55xO7nWLXrIEfKq+xu8QQR7hXH+cB2EXlDRK4WkWbRrJRJLIUl5Xz3Xyv4xetrGdu/E/PumMj5HiRbiidfH9uH8QM78dAb69lVHH7zzaycfFKShIuGJkb/RoCTFTCDD7cUUhWBrICW7S+xhBU4VPVSoA/wJnAHsFdEnhaR7GhWzsS/9zftZ/JjC1m4uZBfTh3Cs988i85tEqfJJVKSkoRHrhiBiHD3SznU1NTfZKWqzM7JZ0JmOh1bpXlQy8jKzsrgyNHIZAX05foZ2as97VrE/+wBpgF9HKpapKpPqurZwCTgLOB9EdkuIj8TkdZRq6WJO0crq7l/1jq+9c/lbt6M8XxrfL9G2QEerh7tW3Df1CEs3VbMsx9tr3f9lTsPsufgp0xNoE7xYOMHpJMkpz79SHFpBav3HLJmqgTSoM5xEfmiiPwT+ADYB1wHfAM4A+dqxDQBGwoOc+mfF/HsR9v59vh+vP6D8QzuGr+TE3rpyjN78sXBnfntvI1s8ZeccN3ZOfmkpSRxwemJ2azXrqWTFfBU7+f4LNufNVMlinA7x38vIruBJ4CNwDBVvUBVn1fVhTiZ+86IYj1NHKipUZ5ZtI3L/vwhB8oq+de3x3Df1CF2l28QEeHXX3EGBdw1M6fO9v/qGuWN1QWcN6gzbRJgcse6ZGedelbAQLa/4T3bR7BmJprCveJoDlyuqqer6m9VdU/wi6paCYyOeO1M3Nh/+CjX/3MZD76xnuysDObdPtHu8K1D57bNefCyoazadZC/+bbWus6SrUUUlpQn3GiqUNlZGajCos0nNyxXVVmY52e8ZftLKOEGjl8Dm4MXiEgHETn2qVdVbxIyG8+9vW4vFz7mY/n2Yh6+fCh/v+5MOrW2gXUnMnVEdy4e3o3H3sllQ8Hh416fnZNPq7RkzhvcOQa1i5wRPZ0O7ZPt59i49wj7j5Qzyfo3Ekq4geN1oGfIsp7Aa5GtjoknZRVV3PvqGm78z8f06NCCN26dyLVj+9gNWmF68LKhtGuRxg9n5lBR9VmTVUVVDW+u3csFp3dN+Ga+5CRhwilkBQwEnInWv5FQwg0cg1R1TfAC9/ngyFfJxIM1uw9xyROLmLF8J9+bNIBXvz+egZ1t4FxDdGyVxq+/MowNBYf503t5x5YvzPNz6NNKpo5IrHs36pKdlc6+w+Vs2tfwrIC+PD9ZXVo3qjnMmoJwA8d+ERkYvMB9XhT5KplYqq5R/vLBFi5/6kPKKqp5/rtj+clFg0lLsdlpTsaXhnThijN78tQHW47d7zArJ5/2LVOZMLBxNM8EJiVsaHNVWUUVy7cdsGG4CSjcs8E/gFdE5BIRGSIiU4GXgafDLUhEJovIJhHZLCI/qeX1P4rIKveRKyIHg167XkTy3Mf14ZZpGib/4Kdc+/QSfjtvIxee3pV5d0zknAHWhHCq7ps6hC5tmnHXzFUcKK1g/vp9XDS0a6MJxseyAjZw3qqlW4upqK5h0iALHIkm3Ew6vwEqgd8DvYBdOEHjD+FsLCLJwJPAl4DdwHIRmaWq6wPrqOqdQevfiju8V0Q6Ar/EGbWlwMfutgfCrLsJwxur8/npq2uorlF+d8Vwrjizp/VlREjb5qk8csUIvv7MUq59eillFdVMHZHYo6lCZWdm8O8lO/i0ojrs+ckW5Pot21+CCnfKkRpV/Z2qDlbVVu7P36tquJPUjAE2q+pWVa0AZgCXnWD9a4AX3d8vBOararEbLOYDk8Ms19SjqrqGu1/K4ZYXPqF/Rmvm3j6RK0f3sqARYRMy0/nGuD6sLzhM5zbNGNuvU6yrFFHZWRlUVNWwZFv4rde+PMv2l6jCzt0pImnAICAdOHZWUdX3wti8B85VSsBuYGwd5fQB+gGB/da2bY9atrsRuBGgd+/eYVTJALy/yc/LH+/mpkn9ufuCQaQmN47mk3h075TBLN9ezOShXRvdPQtj+nWkWUoSvlw/XxhU/xDj3QfK2Oov5dqxfTyonYm0sAKHiEwAXgKaAW2Bw0AbnBN6/3B2UcuyusbuTQNeVtXqhmyrqtOB6QCjR48+tTyeTciC3P20TEvmri9Z0Ii2lmkpzLujcc4L2jw1mbH9O4XdQR7oD5lkw3ATUrhnij8Cj6hqR+CI+/NB4Kkwt9+N0zcS0BOoK9fmND5rpmrotqaBfLmFnN2/U6PpqDWxk52ZzhZ/KXsOflrvur5cP93bNWdAhg3xTkThni2ygMdDlv0GuLOWdWuzHMgUkX5uk9c0YFboSiIyCOgALA5a/BZwgXunegfgAneZOUXbC0vZWVxmOZ5NREwKc1huVXUNH24pJDsrw/rSElS4geMQThMVQIGIDME5wYf1dUFVq4BbcE74G4CZqrpORB4QkUuDVr0GmKFBt6CqajHO1c1y9/GAu8ycIl+e8w9ugcNEwsDOrekWRlbAVbsOcuRolX3uEli4neOvAlOAF4BngPdxhue+FG5BqjoXmBuy7L6Q5/fXse0/cO4lMRHky/XTq2ML+nZqGeuqmEZARMjOzGDu2gKqqmtIqaPPzJfrJ0mcfB4mMYU7HPcOVX3B/f1R4KvADbijmEziqaiqYfGWIrIzrbnARE44WQEXBLL9tUzc6eSbunoDh4gki8iW4DzjqrpIVd9swH0cJs58vOMApRXV1lxgImrCwBNnBTyW7c8+dwmt3sDhDoutxsnJYRoJX56flCThnAGN60Y0E1vtWqYyold7FuTVPv3Ios2FbrY/CxyJLNzO8ceAmSIySUQGiEj/wCOalTPR48v1M6p3h4TOPmfiU3ZmBqt3H+RALVkBfbl+2rVIZYRl+0to4QaOP+PMM/U+kIeT1Gmz+7tJMP4j5azLP2yTy5moqCsrYCDb3wTL9pfwwu0cT6rjYZPMJKBFm91huDadtYmCET3b0bZ5ynH9HJv2HWHf4XKy7W7xhGe3CzdBvtxCOrVK4/Tubetf2ZgGSklOYmJmBr6QrICBQGL9G4kv3LmqFlLH3FKq2jgn32mkamrc5oLMdJKsucBESXZWOnPWFJC7r4RBXdsAzhcWy/bXOIR7A2BowqauwHeA5yJbHRNt6wsOU1hSYc1UJqqCswIO6tqGTyuqWba9mOvG2Wy4jUFYgUNV/xW6TEReAf4JPBDpSpnoCUwzMtHamU0UdWvXgszOrfHl+bkhuz9LthVRUVVjzVSNxKn0cewBhkeqIsYbvlw/p3VrS+c2dluOia7srAyWbivm04pqfLl+mqUkMaafZftrDMLt4/h2yKKWwFeAJRGvkYmakvIqPt5xgG9P6BfrqpgmIDsrg2cWbWPptiJ8uX7G9rdsf41FuH0c3wh5Xgp8hJOnwySIxVuKqKxWJln/hvHAWDcr4Ixlu9jiL+WaMZaZs7EIt4/jC9GuiIk+X66fFqnJnNm3Q6yrYpqA5qnJjOnXkXnr9gKf5eswiS+sPg4RuU5EhocsGyEioVciJo758vycPaATzVKsucB4IxAsurVrzsDOlu2vsQi3c/xBnPziwXYBD0W2OiZadhSVsqOojOxMG01lvBMYRWXT9zcu4fZxtAUOhyw7BNhMZQnC7to1sZDZuTX3XDiIC0/vEuuqmAgK94pjPU7ypmCX46SBNQlgQW4hPTu0oF96q1hXxTQhIsIPvjCQgZ3bxLoqJoLCveL4MTBXRK4GtgADgS/ipJM1cc7J9lfIZWf0sOYCY8wpC3d23EXA6cByoBWwDBiqqh9GsW4mQlbudLP92TBcY0wEhHsDYDNgr6r+JmhZqog0U9XyqNXORIQv109yknDOQMv2Z4w5deH2ccwHzgxZdibwVmSrY6LBl+dnVO/2tLVsf8aYCAg3cAwDloYsWwaMCLcgEZksIptEZLOI/KSOda4SkfUisk5EXgha/oi7bIOIPCHWUB+2wpJy1u45bDdfGWMiJtzO8UNAF2Bv0LIuOFOP1EtEkoEncdLP7gaWi8gsVV0ftE4mcC8wXlUPiEhnd/k5wHg+m1BxETAJ+CDMujdpi/Kc9J02DNcYEynhXnG8ArwgIkNFpKWIDAP+A7wU5vZjgM2qulVVK4AZwGUh69wAPKmqBwBUdb+7XIHmQBrQDEgF9oVZbpPny/XTsVUaQ7u3i3VVjDGNRLiB42c492wsA0pwZsXdAPw8zO178Pk7z3e7y4JlAVki8qGILBGRyQCquhh4HyhwH2+p6nH3j4jIjSKyQkRW+P3+0JebpJoaxZdXyISBlu3PGBM54Q7HPaqqP8AZitsFOBsoB/LCLKe2s1ZoKtoUIBM4F7gGeFpE2ovIQOA0oCdOsDlPRI5LV6uq01V1tKqOzsiwZhmADXsPU1hSbs1UxpiICjuRk4hkALfhjKT6BBgN3B7m5ruBXkHPewL5tazzP1WtVNVtwCacQHI5sERVS1S1BHgTGBduvZsyX67bv2HzUxljIuiEgcO9V+OrIjIbJ+PfTcBrOJ3lV6lquH0cy4FMEeknImnANGBWyDqvA19wy03HabraCuwEJolIioik4nSM21QnYfDl+hnctQ2d21q2P2NM5NR3xbEP+BvOt/9xqjpEVR/EaaYKm6pWAbfgXK1sAGaq6joReUBELnVXewsoEpH1OH0a96hqEfAyzjQna4AcIEdVZzek/KaotLyKFTuKbRiuMSbi6huOuxqYAIwF8kRkW2DUU0Op6lxgbsiy+4J+V+CH7iN4nWqcKx3TAIFsf9a/YYyJtBNecajqucAA4G3gbmCv22zVCmdYrIlTvjwn299oy/ZnjImwejvHVXWHqj6oqpk4M+IWADVAjog8Eu0KmpPjy/Uzrn9Hy/ZnjIm4sEdVgTNLrqreCHQFbsWZisTEmZ1FZWwvKrNmKmNMVDQocAS493W8qKoXRbpC5tQtyLNsf8aY6DmpwGHimy/XT4/2Lehv2f6MMVFggaORqayuYfGWIrKzMizbnzEmKixwNDIrdxygpLyKSVl2t7gxJjoscDQyvrxAtj8LHMaY6LDA0cj4cgst258xJqoscDQiRSXlrM0/RHamjaYyxkSPBY5GZNHmQlRtGK4xJroscDQiC3L9dGiZytAelu3PGBM9FjgaCVVlYV4hEzIzSLZsf8aYKLLA0UhsKDiC/0i5JW0yxkSdBY5GwmfTjBhjPGKBo5EIZPvrYtn+jDFRZoGjESgtr2LF9gN2tWGM8YQFjkZgydYiKqpr7P4NY4wnLHA0Ar5cP81TkyzbnzHGExY4GgFfXiHj+neieapl+zPGRJ8FjgS3q7iMbYWl1kxljPGMBY4EtyDXhuEaY7zlWeAQkckisklENovIT+pY5yoRWS8i60TkhaDlvUXkbRHZ4L7e16t6x7tAtr8BGZbtzxjjjRQvChGRZOBJ4EvAbmC5iMxS1fVB62QC9wLjVfWAiHQO2sW/gYdVdb6ItAZqvKh3vKusruGjLUVMHdHNsv0ZYzzj1RXHGGCzqm5V1QpgBnBZyDo3AE+q6gEAVd0PICJDgBRVne8uL1HVMo/qHdc+2XmQkvIq698wxnjKq8DRA9gV9Hy3uyxYFpAlIh+KyBIRmRy0/KCIvCoin4jI79wrmM8RkRtFZIWIrPD7/VE5iHjjy7Vsf8YY73kVOGprR9GQ5ylAJnAucA3wtIi0d5dPBO4GzgL6A988bmeq01V1tKqOzshoGt/AfXl+zujVnnYtLNufMcY7XgWO3UCvoOc9gfxa1vmfqlaq6jZgE04g2Q184jZzVQGvA6M8qHNcKy6tYM2eQzaayhjjOa8Cx3IgU0T6iUgaMA2YFbLO68AXAEQkHaeJaqu7bQcRCZwhzwPW08QtzPNbtj9jTEx4EjjcK4VbgLeADcBMVV0nIg+IyKXuam8BRSKyHngfuEdVi1S1GqeZ6l0RWYPT7PV3L+odz3y5hbRvmcowy/ZnjPGYJ8NxAVR1LjA3ZNl9Qb8r8EP3EbrtfGB4tOuYKJxsf34mDEy3bH/GGM/ZneMJaOPeI+w/Um7NVMaYmLDAkYB8gWlG7P4NY0wMWOBIQL48P4O6tKFrO8v2Z4zxngWOBFNWUcXybQfIzrKb/owxsWGBI8Ecy/Zn/RvGmBixwJFgfLmFNE9N4qy+HWNdFWNME2WBI8H4cv2M7WfZ/owxsWOBI4HsKi5ja2GpNVMZY2LKAkcC8eU5w3AnWce4MSaGLHAkEF+un+7tmjMgo3Wsq2KMacIscCSIyuoaPtpcRHZWhmX7M8bElAWOBLFq10GOlFdZ/4YxJuYscCSIQLa/8ZbtzxgTYxY4EoQv189Iy/ZnjIkDFjgSQHFpBav3HLJJDY0xccECRwJYtLnQzfZnzVTGmNizwJEAfLl+2rdMZXjP9rGuijHGWOCId4Fsf+Mt258xJk5Y4Ihzm/YdYd/hciZZ/4YxJk5Y4IhzgWx/E61/wxgTJyxwxDlfbiFZXVrTrV2LWFfFGGMACxxxrayiimXbim0YrjEmrngWOERksohsEpHNIvKTOta5SkTWi8g6EXkh5LW2IrJHRP7sTY1jb+nWYsv2Z4yJOyleFCIiycCTwJeA3cByEZmlquuD1skE7gXGq+oBEekcspsHgQVe1DdeLMj10ywliTH9LNufMSZ+eHXFMQbYrKpbVbUCmAFcFrLODcCTqnoAQFX3B14QkTOBLsDbHtU3Lvjy/Iztb9n+jDHxxavA0QPYFfR8t7ssWBaQJSIfisgSEZkMICJJwKPAPScqQERuFJEVIrLC7/dHsOqxsftAGVv9pWRn2mgqY0x88Spw1HbnmoY8TwEygXOBa4CnRaQ9cDMwV1V3cQKqOl1VR6vq6IyMxO8T8OUWAjDJ+jeMMXHGkz4OnCuMXkHPewL5tayzRFUrgW0isgknkJwNTBSRm4HWQJqIlKhqrR3sjYUv10+3ds0Z2Nmy/Rlj4otXVxzLgUwR6SciacA0YFbIOq8DXwAQkXScpqutqnqtqvZW1b7A3cC/G3vQqKqu4cMthWRnWrY/Y0z88SRwqGoVcAvwFrABmKmq60TkARG51F3tLaBIRNYD7wP3qGqRF/WLN6t2HeTIUcv2Z4yJT141VaGqc4G5IcvuC/pdgR+6j7r28SzwbHRqGD98uX6SBCZYtj9jTByyO8fj0IK8QifbX0vL9meMiT8WOOLMgdIKVu8+aM1Uxpi4ZYEjznyW7c8ChzEmPlngiDO+XD/tWqQywrL9GWPilAWOOKKq+PL8TLBsf8aYOGaBI47k7ith3+Fysi1pkzEmjlngiCOBbH/Wv2GMiWcWOOKIL89PZmfL9meMiW8WOOLEpxXVLN1WbFcbxpi4Z4EjTizZVkRFlWX7M8bEP8+mHEkEB8squPKvi2NS9oGyCpqlJDHWsv0ZY+KcBY4gSUlCZpfYTWM+pm9Hy/ZnjIl7FjiCtG2eylPXnhnrahhjTFyzPg5jjDENYoHDGGNMg1jgMMYY0yAWOIwxxjSIBQ5jjDENYoHDGGNMg1jgMMYY0yAWOIwxxjSIqGqs6xBxIuIHdpzCLtKBwghVJxHKjWXZTa3cWJZtx9w0yj6Vcvuoar0T5jXKwHGqRGSFqo5uKuXGsuymVm4sy7Zjbhple1GuNVUZY4xpEAscxhhjGsQCR+2mN7FyY1l2Uys3lmXbMTeNsqNervVxGGOMaRC74jDGGNMgFjiMMcY0iAWOICLyDxHZLyJrPS63l4i8LyIbRGSdiNzuUbnNRWSZiOS45f6fF+UGlZ8sIp+IyBsel7tdRNaIyCoRWeFhue1F5GUR2ej+rc/2qNxB7rEGHodF5A6Pyr7T/WytFZEXRaS5F+W6Zd/ulrsumsdb23lDRDqKyHwRyXN/dvCw7CvdY64RkagMy7XA8XnPApNjUG4VcJeqngaMA34gIkM8KLccOE9VRwAjgckiMs6DcgNuBzZ4WF6wL6jqSI/H2T8OzFPVwcAIPDp2Vd3kHutI4EygDHgt2uWKSA/gNmC0qg4FkoFp0S7XLXsocAMwBue9vkREMqNU3LMcf974CfCuqmYC77rPvSp7LfAVwBelMi1wBFNVH1Acg3ILVHWl+/sRnBNKDw/KVVUtcZ+mug9PRkuISE/gYuBpL8qLNRFpC2QDzwCoaoWqHoxBVb4IbFHVU5lZoSFSgBYikgK0BPI9Kvc0YImqlqlqFbAAuDwaBdVx3rgM+Jf7+7+AL3tVtqpuUNVN0SgvwAJHnBGRvsAZwFKPyksWkVXAfmC+qnpSLvAY8COgxqPyginwtoh8LCI3elRmf8AP/NNtnntaRFp5VHawacCLXhSkqnuA3wM7gQLgkKq+7UXZON+6s0Wkk4i0BKYAvTwqG6CLqhaA88UQ6Oxh2VFngSOOiEhr4BXgDlU97EWZqlrtNmH0BMa4l/hRJSKXAPtV9eNol1WH8ao6CrgIp1kw24MyU4BRwF9U9QyglOg1X9RKRNKAS4GXPCqvA843735Ad6CViHzdi7JVdQPwW2A+MA/IwWkSNhFggSNOiEgqTtB4XlVf9bp8t9nkA7zp4xkPXCoi24EZwHki8pwH5QKgqvnuz/04bf1jPCh2N7A76IruZZxA4qWLgJWqus+j8s4HtqmqX1UrgVeBczwqG1V9RlVHqWo2TnNOnldlA/tEpBuA+3O/h2VHnQWOOCAigtP2vUFV/+BhuRki0t79vQXOP/rGaJerqveqak9V7YvTdPKeqnryTVREWolIm8DvwAU4zRpRpap7gV0iMshd9EVgfbTLDXENHjVTuXYC40SkpfsZ/yIeDoYQkc7uz944ncVeHvss4Hr39+uB/3lYdtSlxLoC8UREXgTOBdJFZDfwS1V9xoOixwPfANa4/Q0AP1XVuVEutxvwLxFJxvkSMVNVPR0aGwNdgNec8xgpwAuqOs+jsm8FnnebjLYC3/KoXNx2/i8BN3lVpqouFZGXgZU4zUSf4O00HK+ISCegEviBqh6IRiG1nTeA3wAzReQ7OAH0Sg/LLgb+BGQAc0RklapeGNFybcoRY4wxDWFNVcYYYxrEAocxxpgGscBhjDGmQSxwGGOMaRALHMYYYxrEAocxJ0FE+oqIunMweVluCxGZLSKHRMSTO8CNCWWBwzRJIvKWiDxQy/LLRGSv1wGhAa7AuRelk6oed2+AiNwffBe+iPRwp3F/wr0Jz5hTZoHDNFXPAt+o5WT6DZxpX+J1XqM+QG449RORPjhTa89S1dvUbtoyEWKBwzRVrwMdgYmBBe6kfJcA/3afX+zOZHtYRHaJyP117cxNDnV+0PPQb/7jROQjETkoTuKsc0+wr9NE5AN33XUicqm7/P+A+4CrRaTEvSu5rn0MwAkaL6jqj+p7M4xpiHi9HDcmqlT1UxGZCVzHZwlvrgI2qmqO+7zUfX0dMBSY707f8HpDynITGs3BuZqZhzNn0ysiMlhV/SHrpgKzgX/gzKM1AfifiIxW1V+KiAID65nbq797TE+q6q8aUldjwmFXHKYp+xdwpTwUM20AAAHJSURBVDvBIzhBIpB8B1X9QFXXqGqNqq7GmSRv0kmU83VgrqrOdfc1H1iBkyMi1DigNfAbN9nTe8AbOBMUhmso0Ar470nU1Zh6WeAwTZaqLsJJrnSZiPQHzgJeCLwuImPFyQXvF5FDwPeA9JMoqg9OgDoYeOBcSXSrZd3uwC5VDU5wtYOGZYSchXPF8p7bz2FMRFlTlWnq/o1zpTEIeDskV8ULwJ+Bi1T1qIg8Rt2BoxQnNWpA16DfdwH/UdUbwqhPPtBLRJKCgkdvIDeMbY9R1R+KSDOc4JHtZuMzJiLsisM0df/GyUNyA0HNVK42QLEbNMYAXzvBflYB00QkVURG4wybDXgOmCoiF7qpepuLyLni5F0PtRQnCP3I3de5wFSchFcNdQvwHvCuiHQ5ie2NqZUFDtOkqep24COcPoFZIS/fDDwgIkdwRjPNPMGufgEMAA4A/0dQk5eq7sJJofpTnKaxXcA91PL/p6oVOOldLwIKgaeA61S1wQm23OG3NwHLgHdE5GSa2Yw5juXjMMYY0yB2xWGMMaZBLHAYY4xpEAscxhhjGsQChzHGmAaxwGGMMaZBLHAYY4xpEAscxhhjGsQChzHGmAb5f30tGl+hrWgvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(1,12)\n",
    "plt.plot(x,accuracy)\n",
    "plt.xlabel('Value of K',fontsize =12)\n",
    "plt.ylabel('Accuracy',fontsize=12)\n",
    "plt.title('Selecting k based on accuracy',fontsize = 16)\n",
    "plt.xticks(x)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From this, it is evident that the accuracy is best at k = 7, after which we might have diminishing effects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=None, n_neighbors=7, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#hence we will select k = 7\n",
    "KNN = KNeighborsClassifier(n_neighbors = 7,metric='minkowski')\n",
    "KNN.fit(xtrain,ytrain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "dt = DecisionTreeClassifier(criterion='entropy',max_depth=12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=12,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt.fit(xtrain,ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7285714285714285"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#accuracy for the training\n",
    "accuracy_score(ytest,dt.predict(xtest))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try out all the kernel transformations to check which offers the best prediction for our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linear : 0.743\n",
      "poly : 0.757\n",
      "rbf : 0.743\n",
      "sigmoid : 0.729\n"
     ]
    }
   ],
   "source": [
    "kernel = ['linear', 'poly', 'rbf', 'sigmoid']\n",
    "\n",
    "for i,kernels in enumerate(kernel):\n",
    "    sv = SVC(kernel = kernels,gamma = 'auto')\n",
    "    sv.fit(xtrain,ytrain)\n",
    "    print(kernel[i], ':', round(accuracy_score(ytest,sv.predict(xtest)),3))\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's clear that 'poly' kernal parameter has the highest accuracy for the given data. Hence we will use poly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='poly',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SV = SVC(kernel = 'poly',gamma = 'auto')\n",
    "SV.fit(xtrain,ytrain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='warn',\n",
       "          n_jobs=None, penalty='l2', random_state=None, solver='lbfgs',\n",
       "          tol=0.0001, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LR = LogisticRegression(solver = 'lbfgs')\n",
    "LR.fit(xtrain,ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.77"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(accuracy_score(ytest,LR.predict(xtest)),2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "77% accuracy on the training set. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-05-17 17:42:25--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3642 (3.6K) [text/csv]\n",
      "Saving to: ‘loan_test.csv’\n",
      "\n",
      "100%[======================================>] 3,642       --.-K/s   in 0s      \n",
      "\n",
      "2020-05-17 17:42:26 (302 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data pre processing and feature selection for the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   50              Bechalor  female  \n",
       "1 2016-09-15   35       Master or Above    male  \n",
       "2 2016-10-09   43  High School or Below  female  \n",
       "3 2016-10-09   26               college    male  \n",
       "4 2016-09-25   29              Bechalor    male  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   50              Bechalor  female          3        0  \n",
       "1 2016-09-15   35       Master or Above    male          4        1  \n",
       "2 2016-10-09   43  High School or Below  female          5        1  \n",
       "3 2016-10-09   26               college    male          5        1  \n",
       "4 2016-09-25   29              Bechalor    male          6        1  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   50              Bechalor       1          3        0  \n",
       "1 2016-09-15   35       Master or Above       0          4        1  \n",
       "2 2016-10-09   43  High School or Below       1          5        1  \n",
       "3 2016-10-09   26               college       0          5        1  \n",
       "4 2016-09-25   29              Bechalor       0          6        1  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   50       1        0         1                     0   \n",
       "1        300      7   35       0        1         0                     0   \n",
       "2       1000     30   43       1        1         0                     1   \n",
       "3       1000     30   26       0        1         0                     0   \n",
       "4        800     15   29       0        1         1                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        1  \n",
       "4        0  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature_test = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature_test = pd.concat([Feature_test,pd.get_dummies(test_df['education'])], axis=1)\n",
    "Feature_test.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   50       1        0         1                     0   \n",
       "1        300      7   35       0        1         0                     0   \n",
       "2       1000     30   43       1        1         0                     1   \n",
       "3       1000     30   26       0        1         0                     0   \n",
       "4        800     15   29       0        1         1                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        1  \n",
       "4        0  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xtest = Feature_test\n",
    "Xtest[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'COLLECTION',\n",
       "       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n",
       "       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n",
       "       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n",
       "       'COLLECTION'], dtype=object)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ytest = test_df['loan_status'].values\n",
    "ytest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalize the test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  return self.partial_fit(X, y)\n",
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[ 0.49362588,  0.92844966,  3.05981865,  1.97714211, -1.30384048,\n",
       "         2.39791576, -0.79772404, -0.86135677],\n",
       "       [-3.56269116, -1.70427745,  0.53336288, -0.50578054,  0.76696499,\n",
       "        -0.41702883, -0.79772404, -0.86135677],\n",
       "       [ 0.49362588,  0.92844966,  1.88080596,  1.97714211,  0.76696499,\n",
       "        -0.41702883,  1.25356634, -0.86135677],\n",
       "       [ 0.49362588,  0.92844966, -0.98251057, -0.50578054,  0.76696499,\n",
       "        -0.41702883, -0.79772404,  1.16095912],\n",
       "       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.76696499,\n",
       "         2.39791576, -0.79772404, -0.86135677]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xtest = preprocessing.StandardScaler().fit(Xtest).transform(Xtest)\n",
    "Xtest[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(54, 8)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xtest.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Applying the model to the test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jaccard score : 0.69\n",
      "F1 score : 0.69\n"
     ]
    }
   ],
   "source": [
    "#knn\n",
    "yhat_test = KNN.predict(Xtest)\n",
    "knn_jaccard = jaccard_similarity_score(ytest,yhat_test)\n",
    "print(\"Jaccard score :\",round(knn_jaccard,2))\n",
    "\n",
    "knn_f1 = f1_score(ytest,yhat_test,average='micro')\n",
    "print(\"F1 score :\",round(knn_f1,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jaccard score : 0.7\n",
      "F1 score : 0.7\n"
     ]
    }
   ],
   "source": [
    "#decision tree\n",
    "yhat_test = dt.predict(Xtest)\n",
    "\n",
    "dt_jaccard = jaccard_similarity_score(ytest,yhat_test)\n",
    "print(\"Jaccard score :\",round(dt_jaccard,2))\n",
    "\n",
    "dt_f1 = f1_score(ytest,yhat_test,average='micro')\n",
    "print(\"F1 score :\",round(dt_f1,2))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jaccard score : 0.69\n",
      "F1 score : 0.69\n"
     ]
    }
   ],
   "source": [
    "#svm\n",
    "\n",
    "yhat_test  = SV.predict(Xtest)\n",
    "\n",
    "svm_jaccard = jaccard_similarity_score(ytest,yhat_test)\n",
    "print(\"Jaccard score :\",round(svm_jaccard,2))\n",
    "\n",
    "svm_f1 = f1_score(ytest,yhat_test,average='micro')\n",
    "print(\"F1 score :\",round(svm_f1,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jaccard score : 0.76\n",
      "F1 score : 0.76\n"
     ]
    }
   ],
   "source": [
    "#logistic regressin\n",
    "yhat_test = LR.predict(Xtest)\n",
    "yhat_prob = LR.predict_proba(Xtest)\n",
    "\n",
    "lr_jaccard = jaccard_similarity_score(ytest,yhat_test)\n",
    "\n",
    "print(\"Jaccard score :\",round(lr_jaccard,2))\n",
    "\n",
    "lr_f1 = f1_score(ytest,yhat_test,average='micro')\n",
    "print(\"F1 score :\",round(lr_f1,2))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert the y values to 1s and 0s for log loss function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C = [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "b,c = np.unique(ytest,return_inverse = True)\n",
    "\n",
    "print(\"C =\",c)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log loss : 0.487405594030175\n"
     ]
    }
   ],
   "source": [
    "lr_log_loss = log_loss(c,yhat_prob)\n",
    "print(\"Log loss :\",lr_log_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "Summarising the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "accuracy_scores = {'Algorithm':['KNN','Decision Tree','SVM','LogisticRegression'],'Jaccard':[round(knn_jaccard,2),round(dt_jaccard,2),round(svm_jaccard,2),round(lr_jaccard,2)],\\\n",
    "                   'F1-score':[round(knn_f1,2),round(dt_f1,2),round(svm_f1,2),round(lr_f1,2)],'LogLoss':[np.nan,np.nan,np.nan,round(lr_log_loss,3)]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Algorithm</th>\n",
       "      <th>Jaccard</th>\n",
       "      <th>F1-score</th>\n",
       "      <th>LogLoss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>KNN</td>\n",
       "      <td>0.69</td>\n",
       "      <td>0.69</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Decision Tree</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.70</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>SVM</td>\n",
       "      <td>0.69</td>\n",
       "      <td>0.69</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>LogisticRegression</td>\n",
       "      <td>0.76</td>\n",
       "      <td>0.76</td>\n",
       "      <td>0.487</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Algorithm  Jaccard  F1-score  LogLoss\n",
       "0                 KNN     0.69      0.69      NaN\n",
       "1       Decision Tree     0.70      0.70      NaN\n",
       "2                 SVM     0.69      0.69      NaN\n",
       "3  LogisticRegression     0.76      0.76    0.487"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame.from_dict(accuracy_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logistic regression has the highest accuracy score among the models. However, the logloss score shows that the model can be further improved."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}