update,
This commit is contained in:
louiscklaw
@@ -0,0 +1,555 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Supervised Learning\n",
|
||||
"## KNN\n",
|
||||
"> - KNN is a non-parametric learning algorithm (No assumption is made on the data) \n",
|
||||
"> - KNN can be used for classification (discrte) and regression (continuous label) \n",
|
||||
"> - All training data has to be present to determine the label of new data \n",
|
||||
"> - Sensitive to irrelavant features \n",
|
||||
"> - Sensitive to scale of data \n",
|
||||
"### Issues:\n",
|
||||
"> - Choose number of neighors *k* \n",
|
||||
"> - Choose distance metric "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Data\n",
|
||||
"### Breast Cancer\n",
|
||||
"> **Label**: Malignant or Benign \n",
|
||||
"> **30 Features**: Radius, Texture, Perimeter, Area, Smoothness, etc"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Data"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>mean radius</th>\n",
|
||||
" <th>mean texture</th>\n",
|
||||
" <th>mean perimeter</th>\n",
|
||||
" <th>mean area</th>\n",
|
||||
" <th>mean smoothness</th>\n",
|
||||
" <th>mean compactness</th>\n",
|
||||
" <th>mean concavity</th>\n",
|
||||
" <th>mean concave points</th>\n",
|
||||
" <th>mean symmetry</th>\n",
|
||||
" <th>mean fractal dimension</th>\n",
|
||||
" <th>...</th>\n",
|
||||
" <th>worst radius</th>\n",
|
||||
" <th>worst texture</th>\n",
|
||||
" <th>worst perimeter</th>\n",
|
||||
" <th>worst area</th>\n",
|
||||
" <th>worst smoothness</th>\n",
|
||||
" <th>worst compactness</th>\n",
|
||||
" <th>worst concavity</th>\n",
|
||||
" <th>worst concave points</th>\n",
|
||||
" <th>worst symmetry</th>\n",
|
||||
" <th>worst fractal dimension</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>0</th>\n",
|
||||
" <td>17.99</td>\n",
|
||||
" <td>10.38</td>\n",
|
||||
" <td>122.80</td>\n",
|
||||
" <td>1001.0</td>\n",
|
||||
" <td>0.11840</td>\n",
|
||||
" <td>0.27760</td>\n",
|
||||
" <td>0.3001</td>\n",
|
||||
" <td>0.14710</td>\n",
|
||||
" <td>0.2419</td>\n",
|
||||
" <td>0.07871</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>25.38</td>\n",
|
||||
" <td>17.33</td>\n",
|
||||
" <td>184.60</td>\n",
|
||||
" <td>2019.0</td>\n",
|
||||
" <td>0.1622</td>\n",
|
||||
" <td>0.6656</td>\n",
|
||||
" <td>0.7119</td>\n",
|
||||
" <td>0.2654</td>\n",
|
||||
" <td>0.4601</td>\n",
|
||||
" <td>0.11890</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>1</th>\n",
|
||||
" <td>20.57</td>\n",
|
||||
" <td>17.77</td>\n",
|
||||
" <td>132.90</td>\n",
|
||||
" <td>1326.0</td>\n",
|
||||
" <td>0.08474</td>\n",
|
||||
" <td>0.07864</td>\n",
|
||||
" <td>0.0869</td>\n",
|
||||
" <td>0.07017</td>\n",
|
||||
" <td>0.1812</td>\n",
|
||||
" <td>0.05667</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>24.99</td>\n",
|
||||
" <td>23.41</td>\n",
|
||||
" <td>158.80</td>\n",
|
||||
" <td>1956.0</td>\n",
|
||||
" <td>0.1238</td>\n",
|
||||
" <td>0.1866</td>\n",
|
||||
" <td>0.2416</td>\n",
|
||||
" <td>0.1860</td>\n",
|
||||
" <td>0.2750</td>\n",
|
||||
" <td>0.08902</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2</th>\n",
|
||||
" <td>19.69</td>\n",
|
||||
" <td>21.25</td>\n",
|
||||
" <td>130.00</td>\n",
|
||||
" <td>1203.0</td>\n",
|
||||
" <td>0.10960</td>\n",
|
||||
" <td>0.15990</td>\n",
|
||||
" <td>0.1974</td>\n",
|
||||
" <td>0.12790</td>\n",
|
||||
" <td>0.2069</td>\n",
|
||||
" <td>0.05999</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>23.57</td>\n",
|
||||
" <td>25.53</td>\n",
|
||||
" <td>152.50</td>\n",
|
||||
" <td>1709.0</td>\n",
|
||||
" <td>0.1444</td>\n",
|
||||
" <td>0.4245</td>\n",
|
||||
" <td>0.4504</td>\n",
|
||||
" <td>0.2430</td>\n",
|
||||
" <td>0.3613</td>\n",
|
||||
" <td>0.08758</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>3</th>\n",
|
||||
" <td>11.42</td>\n",
|
||||
" <td>20.38</td>\n",
|
||||
" <td>77.58</td>\n",
|
||||
" <td>386.1</td>\n",
|
||||
" <td>0.14250</td>\n",
|
||||
" <td>0.28390</td>\n",
|
||||
" <td>0.2414</td>\n",
|
||||
" <td>0.10520</td>\n",
|
||||
" <td>0.2597</td>\n",
|
||||
" <td>0.09744</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>14.91</td>\n",
|
||||
" <td>26.50</td>\n",
|
||||
" <td>98.87</td>\n",
|
||||
" <td>567.7</td>\n",
|
||||
" <td>0.2098</td>\n",
|
||||
" <td>0.8663</td>\n",
|
||||
" <td>0.6869</td>\n",
|
||||
" <td>0.2575</td>\n",
|
||||
" <td>0.6638</td>\n",
|
||||
" <td>0.17300</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>4</th>\n",
|
||||
" <td>20.29</td>\n",
|
||||
" <td>14.34</td>\n",
|
||||
" <td>135.10</td>\n",
|
||||
" <td>1297.0</td>\n",
|
||||
" <td>0.10030</td>\n",
|
||||
" <td>0.13280</td>\n",
|
||||
" <td>0.1980</td>\n",
|
||||
" <td>0.10430</td>\n",
|
||||
" <td>0.1809</td>\n",
|
||||
" <td>0.05883</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>22.54</td>\n",
|
||||
" <td>16.67</td>\n",
|
||||
" <td>152.20</td>\n",
|
||||
" <td>1575.0</td>\n",
|
||||
" <td>0.1374</td>\n",
|
||||
" <td>0.2050</td>\n",
|
||||
" <td>0.4000</td>\n",
|
||||
" <td>0.1625</td>\n",
|
||||
" <td>0.2364</td>\n",
|
||||
" <td>0.07678</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"<p>5 rows × 30 columns</p>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" mean radius mean texture mean perimeter mean area mean smoothness \\\n",
|
||||
"0 17.99 10.38 122.80 1001.0 0.11840 \n",
|
||||
"1 20.57 17.77 132.90 1326.0 0.08474 \n",
|
||||
"2 19.69 21.25 130.00 1203.0 0.10960 \n",
|
||||
"3 11.42 20.38 77.58 386.1 0.14250 \n",
|
||||
"4 20.29 14.34 135.10 1297.0 0.10030 \n",
|
||||
"\n",
|
||||
" mean compactness mean concavity mean concave points mean symmetry \\\n",
|
||||
"0 0.27760 0.3001 0.14710 0.2419 \n",
|
||||
"1 0.07864 0.0869 0.07017 0.1812 \n",
|
||||
"2 0.15990 0.1974 0.12790 0.2069 \n",
|
||||
"3 0.28390 0.2414 0.10520 0.2597 \n",
|
||||
"4 0.13280 0.1980 0.10430 0.1809 \n",
|
||||
"\n",
|
||||
" mean fractal dimension ... worst radius worst texture worst perimeter \\\n",
|
||||
"0 0.07871 ... 25.38 17.33 184.60 \n",
|
||||
"1 0.05667 ... 24.99 23.41 158.80 \n",
|
||||
"2 0.05999 ... 23.57 25.53 152.50 \n",
|
||||
"3 0.09744 ... 14.91 26.50 98.87 \n",
|
||||
"4 0.05883 ... 22.54 16.67 152.20 \n",
|
||||
"\n",
|
||||
" worst area worst smoothness worst compactness worst concavity \\\n",
|
||||
"0 2019.0 0.1622 0.6656 0.7119 \n",
|
||||
"1 1956.0 0.1238 0.1866 0.2416 \n",
|
||||
"2 1709.0 0.1444 0.4245 0.4504 \n",
|
||||
"3 567.7 0.2098 0.8663 0.6869 \n",
|
||||
"4 1575.0 0.1374 0.2050 0.4000 \n",
|
||||
"\n",
|
||||
" worst concave points worst symmetry worst fractal dimension \n",
|
||||
"0 0.2654 0.4601 0.11890 \n",
|
||||
"1 0.1860 0.2750 0.08902 \n",
|
||||
"2 0.2430 0.3613 0.08758 \n",
|
||||
"3 0.2575 0.6638 0.17300 \n",
|
||||
"4 0.1625 0.2364 0.07678 \n",
|
||||
"\n",
|
||||
"[5 rows x 30 columns]"
|
||||
]
|
||||
},
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"from sklearn.datasets import load_breast_cancer\n",
|
||||
"\n",
|
||||
"breast_cancer = load_breast_cancer()\n",
|
||||
"X = pd.DataFrame(breast_cancer.data, columns=breast_cancer.feature_names)\n",
|
||||
"X.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create an X with 2 features only\n",
|
||||
"\n",
|
||||
"X = X[['mean area', 'mean compactness']]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"array(['malignant', 'benign'], dtype='<U9')"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"breast_cancer.target_names"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['malignant', 'malignant', 'malignant', 'malignant', 'malignant', ..., 'malignant', 'malignant', 'malignant', 'malignant', 'benign']\n",
|
||||
"Length: 569\n",
|
||||
"Categories (2, object): ['malignant', 'benign']"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"y = pd.Categorical.from_codes(codes=breast_cancer.target, categories=breast_cancer.target_names)\n",
|
||||
"y"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>benign</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>0</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>1</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>3</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>4</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" benign\n",
|
||||
"0 0\n",
|
||||
"1 0\n",
|
||||
"2 0\n",
|
||||
"3 0\n",
|
||||
"4 0"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"y = pd.get_dummies(y, drop_first=True) \n",
|
||||
"y.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"(426, 1)\n",
|
||||
"(143, 1)\n",
|
||||
"<class 'pandas.core.frame.DataFrame'>\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"\n",
|
||||
"Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, random_state=1)\n",
|
||||
"print(ytrain.shape)\n",
|
||||
"print(ytest.shape)\n",
|
||||
"print(type(ytest))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"knn = KNeighborsClassifier(n_neighbors=5, metric='euclidean')\n",
|
||||
"knn.fit(Xtrain, ytrain.to_numpy().ravel())\n",
|
||||
"ypred = knn.predict(Xtest).reshape([143,1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>Prediction</th>\n",
|
||||
" <th>Actual</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>0</th>\n",
|
||||
" <td>1</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>1</th>\n",
|
||||
" <td>1</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2</th>\n",
|
||||
" <td>1</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>3</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>4</th>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" Prediction Actual\n",
|
||||
"0 1 1\n",
|
||||
"1 1 0\n",
|
||||
"2 1 1\n",
|
||||
"3 0 0\n",
|
||||
"4 0 0"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"pd.DataFrame(np.hstack([ypred,ytest]),columns=['Prediction','Actual']).head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Evaluation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[42 13]\n",
|
||||
" [ 9 79]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from sklearn.metrics import confusion_matrix\n",
|
||||
"print(confusion_matrix(ytest, ypred))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Reference in New Issue
Block a user