Voting Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5d57bf20",
   "metadata": {},
   "source": [
    "# Voting Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c59b2d7e",
   "metadata": {},
   "source": [
    "#### Hi Guys, Welcome to [Tirendaz Academy](https://youtube.com/c/tirendazacademy) 😀\n",
    "#### In this notebook, I'm going to talk about voting classifier. Happy learning 🐱‍🏍 "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adaad670",
   "metadata": {},
   "source": [
    "# What is voting classifier?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ba544ee",
   "metadata": {},
   "source": [
    "The goal of Voting Classifiers is to combine different machine learning classifiers and use a majority vote to predict the class labels. This voting classifier often provides a higher accuracy than the best classifier in the ensemble."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5567fd5b",
   "metadata": {},
   "source": [
    "# Loading Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fda2f913",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7ea964e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"wdbc.data\", header = None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b351d808",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>22</th>\n",
       "      <th>23</th>\n",
       "      <th>24</th>\n",
       "      <th>25</th>\n",
       "      <th>26</th>\n",
       "      <th>27</th>\n",
       "      <th>28</th>\n",
       "      <th>29</th>\n",
       "      <th>30</th>\n",
       "      <th>31</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>842302</td>\n",
       "      <td>M</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.3001</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>...</td>\n",
       "      <td>25.38</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.1622</td>\n",
       "      <td>0.6656</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>842517</td>\n",
       "      <td>M</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.0869</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>...</td>\n",
       "      <td>24.99</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.1238</td>\n",
       "      <td>0.1866</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84300903</td>\n",
       "      <td>M</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.1974</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>...</td>\n",
       "      <td>23.57</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.1444</td>\n",
       "      <td>0.4245</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84348301</td>\n",
       "      <td>M</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.2414</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>...</td>\n",
       "      <td>14.91</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.2098</td>\n",
       "      <td>0.8663</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>84358402</td>\n",
       "      <td>M</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.1980</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>...</td>\n",
       "      <td>22.54</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.1374</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 32 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         0  1      2      3       4       5        6        7       8   \\\n",
       "0    842302  M  17.99  10.38  122.80  1001.0  0.11840  0.27760  0.3001   \n",
       "1    842517  M  20.57  17.77  132.90  1326.0  0.08474  0.07864  0.0869   \n",
       "2  84300903  M  19.69  21.25  130.00  1203.0  0.10960  0.15990  0.1974   \n",
       "3  84348301  M  11.42  20.38   77.58   386.1  0.14250  0.28390  0.2414   \n",
       "4  84358402  M  20.29  14.34  135.10  1297.0  0.10030  0.13280  0.1980   \n",
       "\n",
       "        9   ...     22     23      24      25      26      27      28      29  \\\n",
       "0  0.14710  ...  25.38  17.33  184.60  2019.0  0.1622  0.6656  0.7119  0.2654   \n",
       "1  0.07017  ...  24.99  23.41  158.80  1956.0  0.1238  0.1866  0.2416  0.1860   \n",
       "2  0.12790  ...  23.57  25.53  152.50  1709.0  0.1444  0.4245  0.4504  0.2430   \n",
       "3  0.10520  ...  14.91  26.50   98.87   567.7  0.2098  0.8663  0.6869  0.2575   \n",
       "4  0.10430  ...  22.54  16.67  152.20  1575.0  0.1374  0.2050  0.4000  0.1625   \n",
       "\n",
       "       30       31  \n",
       "0  0.4601  0.11890  \n",
       "1  0.2750  0.08902  \n",
       "2  0.3613  0.08758  \n",
       "3  0.6638  0.17300  \n",
       "4  0.2364  0.07678  \n",
       "\n",
       "[5 rows x 32 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1be5fcc6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(569, 32)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33b4637f",
   "metadata": {},
   "source": [
    "# Data Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9b9b771f",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.loc[:,2:].values\n",
    "y = df.loc[:,1].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bb019f13",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a9aece04",
   "metadata": {},
   "outputs": [],
   "source": [
    "le = LabelEncoder()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "89a97c48",
   "metadata": {},
   "outputs": [],
   "source": [
    "y = le.fit_transform(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b7835740",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "309cdf49",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y,stratify=y,random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aeb945c",
   "metadata": {},
   "source": [
    "# Building the Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cd7e71f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.ensemble import VotingClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2ebeb4fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "979a1c76",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf1 = LogisticRegression(random_state=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "clf3 = SVC(random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d9b661c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "eclf = VotingClassifier(estimators=[(\"lr\", clf1),(\"rf\", clf2),(\"svc\",clf3)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "59d01c87",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "VotingClassifier(estimators=[('lr', LogisticRegression(random_state=1)),\n",
       "                             ('rf', RandomForestClassifier(random_state=1)),\n",
       "                             ('svc', SVC(random_state=1))])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eclf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d832638a",
   "metadata": {},
   "source": [
    "# Evaluationg the Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f1d7fd92",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6fe0fc05",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9671361502347418\n"
     ]
    }
   ],
   "source": [
    "y_train_pred = eclf.predict(X_train)\n",
    "print(accuracy_score(y_train, y_train_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "0120197b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.972027972027972\n"
     ]
    }
   ],
   "source": [
    "y_test_pred = eclf.predict(X_test)\n",
    "print(accuracy_score(y_test, y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "52313c89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression:  0.965\n",
      "RandomForestClassifier:  0.951\n",
      "SVC:  0.930\n",
      "VotingClassifier:  0.972\n"
     ]
    }
   ],
   "source": [
    "for clf in (clf1, clf2, clf3, eclf):\n",
    "    clf.fit(X_train, y_train)\n",
    "    y_pred = clf.predict(X_test)\n",
    "    print(f\"{clf.__class__.__name__}: {accuracy_score(y_test, y_pred): .3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e6df886",
   "metadata": {},
   "source": [
    "Voting classifier achieved a higher accuracy than the best classifier in the ensemble."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd150b64",
   "metadata": {},
   "source": [
    "#### Don't forget to follow us on [YouTube](http://youtube.com/tirendazacademy) | [Medium](http://tirendazacademy.medium.com) | [Twitter](http://twitter.com/tirendazacademy) | [GitHub](http://github.com/tirendazacademy) | [Linkedin](https://www.linkedin.com/in/tirendaz-academy) | [Kaggle](https://www.kaggle.com/tirendazacademy) 😎"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}