{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Ensemble Learning\n", "\n", "In this lab, we will be implementing Ensemble Learning using Scikit Learn (sklearn).\n", "\n", "\n", "## Part 1: Input Data" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Import the standard modules to be used in this lab\n", "import pandas as pd\n", "import numpy as np\n", "from matplotlib import pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
agesexcptrestbpscholfbsrestecgthalachexangoldpeakslopecathaltarget
063131452331015002.30011
137121302500118703.50021
241011302040017201.42021
356111202360117800.82021
457001203540116310.62021
\n", "
" ], "text/plain": [ " age sex cp trestbps chol fbs restecg thalach exang oldpeak slope \\\n", "0 63 1 3 145 233 1 0 150 0 2.3 0 \n", "1 37 1 2 130 250 0 1 187 0 3.5 0 \n", "2 41 0 1 130 204 0 0 172 0 1.4 2 \n", "3 56 1 1 120 236 0 1 178 0 0.8 2 \n", "4 57 0 0 120 354 0 1 163 1 0.6 2 \n", "\n", " ca thal target \n", "0 0 1 1 \n", "1 0 2 1 \n", "2 0 2 1 \n", "3 0 2 1 \n", "4 0 2 1 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_pd = pd.read_csv('heart.csv')\n", "data_pd.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that all the ranges of the features are not the same. This may cause a problem where a small change in a feature might not affect the other. So we normalize the ranges of the features to a uniform range which in this case 0 - 1." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import MinMaxScaler\n", "scaler = MinMaxScaler(feature_range=(0,1))\n", "normData = scaler.fit_transform(data_pd.iloc[:,0:-1])" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
agesexcptrestbpscholfbsrestecgthalachexangoldpeakslopecathaltarget
00.7083331.01.0000000.4811320.2442921.00.00.6030530.00.3709680.00.00.3333331
10.1666671.00.6666670.3396230.2831050.00.50.8854960.00.5645160.00.00.6666671
20.2500000.00.3333330.3396230.1780820.00.00.7709920.00.2258061.00.00.6666671
30.5625001.00.3333330.2452830.2511420.00.50.8167940.00.1290321.00.00.6666671
40.5833330.00.0000000.2452830.5205480.00.50.7022901.00.0967741.00.00.6666671
\n", "
" ], "text/plain": [ " age sex cp trestbps chol fbs restecg thalach exang \\\n", "0 0.708333 1.0 1.000000 0.481132 0.244292 1.0 0.0 0.603053 0.0 \n", "1 0.166667 1.0 0.666667 0.339623 0.283105 0.0 0.5 0.885496 0.0 \n", "2 0.250000 0.0 0.333333 0.339623 0.178082 0.0 0.0 0.770992 0.0 \n", "3 0.562500 1.0 0.333333 0.245283 0.251142 0.0 0.5 0.816794 0.0 \n", "4 0.583333 0.0 0.000000 0.245283 0.520548 0.0 0.5 0.702290 1.0 \n", "\n", " oldpeak slope ca thal target \n", "0 0.370968 0.0 0.0 0.333333 1 \n", "1 0.564516 0.0 0.0 0.666667 1 \n", "2 0.225806 1.0 0.0 0.666667 1 \n", "3 0.129032 1.0 0.0 0.666667 1 \n", "4 0.096774 1.0 0.0 0.666667 1 " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "normData_pd = pd.DataFrame(normData)\n", "normData_pd.columns = data_pd.columns[:-1]\n", "normData_pd['target'] = data_pd['target']\n", "normData_pd.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's split the dataset into training and testing sets i.e. 2/3 as training set and 1/3 as testing set." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(normData_pd.iloc[:,0:-1], \n", " normData_pd.iloc[:,-1], test_size=0.3, random_state=80)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Part 7: Exercise\n", "The lab exercise uses auto mpg datasets.

The data is technical spec of cars.\n", "Attribute Information:\n", "1. mpg (miles per gallon): continuous\n", "2. cylinders: multi-valued discrete\n", "3. displacement: continuous\n", "4. horsepower: continuous\n", "5. weight: continuous\n", "6. acceleration: continuous\n", "7. model year: multi-valued discrete\n", "8. origin: multi-valued discrete\n", "9. car name: string (unique for each instance)\n", "\n", "We would like to predict origin. Build ensemble models to perform the prediction. Use random_state=5 to split the dataset into training and test data." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
mpgcylindersdisplacementhorsepowerweightaccelerationmodel_yearorigincar_name
018.08307.0130.03504.012.0701chevrolet chevelle malibu
115.08350.0165.03693.011.5701buick skylark 320
218.08318.0150.03436.011.0701plymouth satellite
316.08304.0150.03433.012.0701amc rebel sst
417.08302.0140.03449.010.5701ford torino
\n", "
" ], "text/plain": [ " mpg cylinders displacement horsepower weight acceleration \\\n", "0 18.0 8 307.0 130.0 3504.0 12.0 \n", "1 15.0 8 350.0 165.0 3693.0 11.5 \n", "2 18.0 8 318.0 150.0 3436.0 11.0 \n", "3 16.0 8 304.0 150.0 3433.0 12.0 \n", "4 17.0 8 302.0 140.0 3449.0 10.5 \n", "\n", " model_year origin car_name \n", "0 70 1 chevrolet chevelle malibu \n", "1 70 1 buick skylark 320 \n", "2 70 1 plymouth satellite \n", "3 70 1 amc rebel sst \n", "4 70 1 ford torino " ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "auto_mpg = pd.read_csv(\"auto_mpg.csv\", delim_whitespace=True)\n", "auto_mpg.head()" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
mpgcylindersdisplacementhorsepowerweightaccelerationmodel_yearorigincar_name
00.2393621.00.6175710.4565220.5361500.2380950.010.161184
10.1595741.00.7286820.6467390.5897360.2083330.010.118421
20.2393621.00.6459950.5652170.5168700.1785710.010.759868
30.1861701.00.6098190.5652170.5160190.2380950.010.046053
40.2127661.00.6046510.5108700.5205560.1488100.010.529605
\n", "
" ], "text/plain": [ " mpg cylinders displacement horsepower weight acceleration \\\n", "0 0.239362 1.0 0.617571 0.456522 0.536150 0.238095 \n", "1 0.159574 1.0 0.728682 0.646739 0.589736 0.208333 \n", "2 0.239362 1.0 0.645995 0.565217 0.516870 0.178571 \n", "3 0.186170 1.0 0.609819 0.565217 0.516019 0.238095 \n", "4 0.212766 1.0 0.604651 0.510870 0.520556 0.148810 \n", "\n", " model_year origin car_name \n", "0 0.0 1 0.161184 \n", "1 0.0 1 0.118421 \n", "2 0.0 1 0.759868 \n", "3 0.0 1 0.046053 \n", "4 0.0 1 0.529605 " ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.preprocessing import LabelEncoder\n", "from sklearn.preprocessing import MinMaxScaler\n", "\n", "enc = LabelEncoder()\n", "enc.fit(auto_mpg['car_name'])\n", "labels = enc.transform(auto_mpg['car_name'])\n", "auto_mpg['car_name'] = labels\n", "\n", "column_names = auto_mpg.columns\n", "origin = auto_mpg['origin']\n", "scaler = MinMaxScaler()\n", "auto_mpg = scaler.fit_transform(auto_mpg)\n", "auto_mpg = pd.DataFrame(auto_mpg, columns=column_names)\n", "auto_mpg['origin'] = origin\n", "auto_mpg.head()" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0 1 2 3 4 5]\n", "[ 98.54179491 113.69086729 145.79160346 61.01898487 112.74350469\n", " 14.30828699 7.73876738]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "E:\\Programs\\Miniconda3\\envs\\myenv\\lib\\site-packages\\ipykernel_launcher.py:12: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame.\n", "Try using .loc[row_indexer,col_indexer] = value instead\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n", " if sys.path[0] == '':\n", "E:\\Programs\\Miniconda3\\envs\\myenv\\lib\\site-packages\\ipykernel_launcher.py:13: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame.\n", "Try using .loc[row_indexer,col_indexer] = value instead\n", "\n", "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n", " del sys.path[0]\n" ] } ], "source": [ "from sklearn.feature_selection import SelectKBest\n", "from sklearn.feature_selection import f_classif\n", "from sklearn.model_selection import train_test_split\n", "kBest = SelectKBest(f_classif, k=6)\n", "kBest.fit(auto_mpg.iloc[:,0:7], auto_mpg['origin']) # run the score function on the data\n", "idx = kBest.get_support(True)\n", "print(idx)\n", "scores = kBest.scores_\n", "print(scores)\n", "\n", "auto_mpg_kbest = auto_mpg.iloc[:,[0,1,2,4]]\n", "auto_mpg_kbest['origin'] = auto_mpg['origin']\n", "auto_mpg_kbest['origin'] = auto_mpg_kbest['origin'].astype(int)\n", "auto_mpg_kbest.head()\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(auto_mpg_kbest.iloc[:,0:-1], \n", " auto_mpg_kbest.iloc[:,-1], test_size=0.3, random_state=5, shuffle=False)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 3, 2], dtype=int64)" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "auto_mpg['origin'].unique()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "from sklearn.tree import DecisionTreeClassifier\n", "from sklearn.svm import SVC\n", "from sklearn.neighbors import KNeighborsClassifier\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.naive_bayes import GaussianNB\n", "\n", "from sklearn.metrics import confusion_matrix, classification_report, accuracy_score\n", "from matplotlib import pyplot as plt" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[52 9 2]\n", " [ 7 7 6]\n", " [ 2 13 22]]\n", " precision recall f1-score support\n", "\n", " 1 0.85 0.83 0.84 63\n", " 2 0.24 0.35 0.29 20\n", " 3 0.73 0.59 0.66 37\n", "\n", " accuracy 0.68 120\n", " macro avg 0.61 0.59 0.59 120\n", "weighted avg 0.71 0.68 0.69 120\n", "\n", "0.675\n" ] } ], "source": [ "dt = DecisionTreeClassifier(criterion='gini', max_depth=6, random_state=0)\n", "dt.fit(X_train, y_train)\n", "y_pred_dt = dt.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_dt))\n", "print(classification_report(y_test, y_pred_dt))\n", "print(accuracy_score(y_test, y_pred_dt))" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[58 3 2]\n", " [14 3 3]\n", " [16 4 17]]\n", " precision recall f1-score support\n", "\n", " 1 0.66 0.92 0.77 63\n", " 2 0.30 0.15 0.20 20\n", " 3 0.77 0.46 0.58 37\n", "\n", " accuracy 0.65 120\n", " macro avg 0.58 0.51 0.51 120\n", "weighted avg 0.63 0.65 0.61 120\n", "\n", "0.65\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "E:\\Programs\\Miniconda3\\envs\\myenv\\lib\\site-packages\\sklearn\\svm\\_base.py:231: ConvergenceWarning: Solver terminated early (max_iter=500). Consider pre-processing your data with StandardScaler or MinMaxScaler.\n", " % self.max_iter, ConvergenceWarning)\n" ] } ], "source": [ "sv = SVC(gamma='scale', degree=7, probability=True, kernel='poly', max_iter=500)\n", "sv.fit(X_train, y_train)\n", "y_pred_sv = sv.predict(X_test)\n", "print(confusion_matrix(y_test, y_pred_sv))\n", "print(classification_report(y_test, y_pred_sv))\n", "print(accuracy_score(y_test, y_pred_sv))" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(array([10], dtype=int64),)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD4CAYAAADrRI2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXycVb348c93ZrLvabY2ky5J17RNCpRFWQSkZRELblfcLv68yuUiV72i/kAWFcTrvv3EBcGreEVEFCiI0IKsQoGwNGmTBpJ0m6RZm32fzPn9Mc9MJ8kkmbRJJpn5vl+vvJrnzPM8c54+7XfOnPN9zhFjDEoppaKHLdwVUEopNbc08CulVJTRwK+UUlFGA79SSkUZDfxKKRVlHOGuwFhZWVlm+fLl4a6GUkotKK+99lqrMSY7lH3nXeBfvnw5ZWVl4a6GUkotKCJyMNR9tatHKaWijAZ+pZSKMhr4lVIqymjgV0qpKKOBXymloowGfqWUijIa+JVSKspo4J8hrx9qZ1ddW7iroZRSU9LAPwOO9g7xqd++ylceKA93VZRSakoa+GfAtx6roqNvmENH+2jvHQp3dZRSalIhBX4RuUhEqkWkRkSun2CffxGRShHZKyL3BpSPiMib1s/2mar4fLGrro0HXnNx6vIMAMrrO8NcI6WUmtyUgV9E7MAdwMVAMfARESkes88q4AbgTGPMeuALAS/3G2M2WT/bZq7q4Tfk9nDTQ3twZiRwx0dPBqD8cEeYa6WUUpMLpcV/GlBjjKkzxgwB9wGXjdnnM8Adxph2AGNM88xWc3769fN11DT3cNtlG8hJjacwO2nSFv/wiIfHKo7g8eg6x0qp8Akl8OcDhwO2XVZZoNXAahH5p4jsEpGLAl6LF5Eyq/zyYG8gIldZ+5S1tLRM6wLC5VBbHz996m0u3pDHeWtzACjJT6PcNXGL/6+vu7jmD6/zj31R8bmolJqnQgn8EqRsbJPVAawCzgU+AtwlIunWa0uNMZuBjwI/FpGicScz5k5jzGZjzObs7JCmkw4rYww3P7wHh0342nvX+8tLnOk0dQ3S1DUQ9Lgde5u8f1Y2zkk9lVIqmFACvwsoCNh2Ag1B9nnYGDNsjNkPVOP9IMAY02D9WQc8A5x0gnUOu8cqGnn2rRau27qGvLR4f3lpQRoAu4P08/cNuXmhphWAp6qaGdHuHqVUmIQS+F8FVonIChGJBa4AxmbnPAScByAiWXi7fupEJENE4gLKzwQqZ6ry4dA9MMw3HtnL+iWp/Os7lo16rXhxGnabUO4a38//3FutDLo9fPT0pbT1DvHGofa5qrJSSo0yZeA3xriBa4EngCrgfmPMXhG5VUR8WTpPAG0iUgk8DXzZGNMGrAPKRGS3Vf5tY8y8D/wej+GFt1uDtsp/sOMtWnoG+db7NuKwj/7rS4i1szo3JegA787KJlLjHXx56xpi7MLOyqZp16u5e4DKhq5pH6eUUoFCyuM3xjxmjFltjCkyxtxuld1ijNlu/W6MMV80xhQbYzYaY+6zyl+0tkutP++evUuZObvq2vj43S/zmxf2jyqvcHVyz0sH+MQZyygtSA96bKnTO8BrzLEPDfeIh3/sa+L8tTlkJMVyRuEidlQ2jdonFF97eC8fvvMlBt0j074mpZTy0Sd3g9jf1gvAD3e+RX1HPwAjHsNXH6xgUXIcX7pwzYTHbnSm0dE3zOGj/f6ysoPttPcNs3V9HgBbi3PZ39pLbUtPyHUaGB7hmeoWugfc7Ko7ejyXpZRSgAb+oFzt/Ths3mSmr2/fC8DvXzpARX0nt1xaTGp8zITHljq93wR2B6R17qxsItZu45zV3oylC4pzAdgxje6ef9a00j88Yp1Ps4KUUsdPA38Q9e39LElP4PMXrGJnZRO/33WQ7+94i3NWZ3NpyeJJj12Tl0Ksw+bP5zfGsLOyiXeuXERynAOAxWkJbMxPm1Y//87KJpLjHLx7bQ47K5v0ITCl1HFzhLsCs6W2pYeclDhSJmmdT8TV3oczI4F/O2sFD71Rz80P7SHOYeO2y9YjEuyxhmNi7DaKF6fyz5o2nqxsorVnkENH+/j3dxWO2m9LcS4/evItHtndQEKMfdRrInB64bEPihGP4cmqJs5dk835a3N4al8zFfWdE44zKKXUZCIy8O9v7eXinzzPB0528t/v3zjt413t/Zy7JpsYu43b37eBD/9qF5+/YBXLFiWFdPxpKzK587k6Pn1PGQAxdmHLutxR+1y8IY8fPfkW//nHN4Ke46Sl6fzl6ndiswlvHm6ntWeILcW5vGt1NnabNytIA79S6nhEXOA3xnDzQ3sYcnvYWdnINy/fgN02eSs90KB7hObuQfLTEwE4ZVkmr954ARlJsSGf47qtq9lWugRf0k56Ygw5qfGj9lmVm8LT151L94B73PEv72/jm3+r4t5XDvHxM5axo7IJh004d00OaQkxnLo8g52VTZMOMiul1EQiLvBv393ACzWtvLNoES/WtvHm4XZOWZYZ8vENHd7pFpwZCf6y6QR9gDiHnQ35aVPutzwr+DeIDfmpPFXVzHce38eF6/PYWdnEGYWLSEvwdlttKc7jtkcrOdjWG/K3EKWU8omowd3O/mFue7SKEmcaP//YyThsMq3MGfD278PowD/XRIRvvm8Dg8MePvuH16lr6WXr+mNdRVutrKDjeQhMKaUiKvB/74l9HO31PlWbnhjLO4oWjQuOPYNu2noGJzxHfbs3/z4/jIEfoCg7mavfVcgrB7w5+xcEjBEUZCayNi9l2h9qJ8r3oaiUWtgiJvDXtfTwh5cPceU7l/u7WbYU51LX0ktNs/dBqRGP4RN3v8zH735lwvO42vux24S8MX3y4XDNeStZkZVEaUE6S9JHfxBduD6PsgNH2dc4N1M4lLs6OOs7T1N2QB8eU2qhi5jAX5idzG+uPJXrth4b8PS1kn2t/ntfPsgbhzqoOtJF18Bw0PO42vtYnBY/bh6ecIiPsfPA1e/g7is3j3vtk+9cTnpiLDc+uGdOcvpf2e8N+NVN3bP+Xkqp2RX+6DaDzlub4899B1iSnsCG/FR2VjbS3D3Adx+vJis5DoA9QWbQBG+LPz89vN08gRYlx/nrHCgjKZavXrKO1w6286eyw0GOnFm7rb8vX1eYUmrhiqjAH8yWdXm8cbiD6+7fzaDbw68+cQpwLJCNVd/RjzMjcS6reNw+cHI+p6/I5Nt/30frJOMWM8H3JLJLA79SC17kB/7iXIyB599u5ZrzijhlWQYFmQlU1I9fLGXI7aGxayCsGT3TISLc/r4N9A25+cYjlbzd1M3bTd0cauub9syfk+nsG+Zgm3dgVwd4lVr4Ii6Pf6x1i1NYmpmI3SZc/S7vqo8lznTePDQ+8B/p7MeY8Gf0TMfKnBT+/ZwifvZ0DY/sPrYw2j2fOs0/KdyJKrc+JPPTE/yzlSqlFq6ID/wiwj2fOo24GBvx1pw4pc40/lZ+hLaeQRYF9J/7+q8XSovf57+2rOaUZRn0DrnxGPivP73JK/uPzlzgt7rFLtqQx90v7GfQPUKcwz7FUUqp+SriAz+Mf0J2Y753jptyVyfnrc3xl/v6rwsWSB+/j90mo67jF8/UjpoW+kTtPtxBYVYSxYtTAe/TzSsmeOpYKTX/RXwffzAbnWmIMC44utr7sAmjFlBfiEry06io75yxfv5yVycbnWn+LjDt51dqYYvKwJ8c56AoO3ncouiujn7yUuOJmQc5/CeipMC7CtihoxMHaI/HhJT/39w1QGPXACXOdH8X2NiUziG3Z0YHk5VSs2thR7gTUBJkbVxX+8JJ5ZyMbxWwsR9sga76/Wt88f43pzyXL+211JlGXmo8dpuMSuk0xnD+D57htkerTrDWSqm5ErWBv9SZTmvPEEc6B/xl9e39C25gN5ixq4CN5R7x8M+aVp5/u3XKlnqFqwO7TVi/JA2H3UZeavyorp4DbX242vv5nxf388ah9hm9DqXU7IjawF/i9M7n4wuO7hFvDv9CSuWciG8VsIkeUqtp6aF/eIS23qEp0zN3uzpZlZNMQqw3i8eZMTql0/f3lxBj58YH9+Ae8czQVSilZktIgV9ELhKRahGpEZHrJ9jnX0SkUkT2isi9AeVXisjb1s+VM1XxE7VucSoOm/iD45HOAUY8JiJa/ODtmtlT38lIkH788sPHPhAm6w4yxlDu6vB3HQE4MxJHdfWUuzqJc9j4zgdKqDzSxW9fPDAzF6CUmjVTBn4RsQN3ABcDxcBHRKR4zD6rgBuAM40x64EvWOWZwNeA04HTgK+JSMaMXsFxio+xsyYvhd2HOxh0j3CgrRcgIvr4ATY60+kbGqG2pWfca7tdHSTHOYi124KmfQ65PQy6R9jf2kt73zAbnccWlcnPSKCxa4Aht7dlX+7qYP2SVC4tWcz5a3P44c63aJjmQ17uEe/7DbpHGNZvDErNulDy+E8DaowxdQAich9wGVAZsM9ngDuMMe0Axphmq/xCYKcx5qh17E7gIuCPM1P9E1NakM69Lx9izU2P+8sWWg7/REr9XVmdrM5NGfVauauTjflp9A65R7X+Af5310FuemjPqLJNBYEt/gSMgcbOAZakx7OnvosPn1qAiPCNbevZ8qNn+f6Oan74L5tCquee+k7e//MXGbICvt0m/Pb/nMrZq2bm4TOl1HihBP58IHD6RxfeFnyg1QAi8k/ADnzdGPP4BMfmj30DEbkKuApg6dKlodb9hF1zbhEFGYl4rAHO7JQ4CjIjo6unMDuZpFg75a4OPniK018+6B5hX2MXnzprBb2Dbh5+owGPx2Cz1iX+y+suli9K5EObCwDITIpl/ZJU//HOgFz+vmE3/cMj/vGSgsxELivN57GKIwy5PcQ6pu5JfGR3AwbDl631g3+wo5pXD7Rr4FdqFoUS+IOtVD6249gBrALOBZzA8yKyIcRjMcbcCdwJsHnz5jlLCHdmJPIf5xbN1dvNKbtN2JCfNm6Ad9+RboZHDKXOdHoG3fzvrkPsb+ulKDuZ5q4B3jzcwRcvWM1nz1sZ9LxOaxF6V3u/v6+/JGAMYEtxLn8qO8yuurYpp4wwxrDDWk/Y9373lx0O2j2llJo5oQzuuoCCgG0n0BBkn4eNMcPGmP1ANd4PglCOVbOktCCdqoYuf388HMvCKXGmjctserKqGWNgS8D6vmPlpcVjE+/DbuX1HaTEOSgMmL7hrFVZJMTYQ1oPuLalh/2tvf41hMG75GRdS+/0LlQpNS2hBP5XgVUiskJEYoErgO1j9nkIOA9ARLLwdv3UAU8AW0UkwxrU3WqVqTlQ4kxjaMTDWwGrZu12dbIoKZb89ARWZieTEGNnt9XPv7OykYLMBNaMGRMIFOs4lstf7upkQ36av5sIvIPm56zOYmdl05TPCPjWDL4gIPAXZiVR19IzJ6uKKRWtpgz8xhg3cC3egF0F3G+M2Ssit4rINmu3J4A2EakEnga+bIxpswZ1b8P74fEqcKtvoFfNPl8a5ou1rf6yCmveHRHBYbexIT+VclcHvYNu/lnbxtbiPESC9dAd48xIZH9rL1VHuigpSBv3+pbiPBq7BqioP9bNdM9LB/jQL1+kb8jtL9tZ2cTG/DQWpx0bVynKSWbQ7dHpn5WaRSHl8RtjHjPGrDbGFBljbrfKbjHGbLd+N8aYLxpjio0xG40x9wUc+xtjzErr539m5zJUMM6MBN5RuIif/aOGlu5B+obcvN3cPapPvsSZzt6GLv6xr5kht4ctxRN38/jkZyTw5uEO/1jBWOevzcEmx9Y63t/ayzf/VsWrB9r56VM1AP7xhLHv5+s2qmvV7h6lZkvUPrkbDUSE2y7fQP/wCN96rIo99V14zLFUT/B2Bw26Pfz8mVrSE2PYvGzqxyx8KZ0AG/PHt/gzk2LZvDzT391zy8N7iLPbuHB9Lnc9X0d1Y/ex8YQxgb8oJxmA2mYd4FVqtmjgj3Arc5L5j3cV8eAb9dz5XC3AuBY/QNWRLs5fm4MjhJlJfSmdmUmxEz7pvLU4l32N3fz8mVqef7uVL1+0hm+/v4SUeAc3PljBDms8YW3e6PGERUmxpCXEaGaPUrNIA38UuOa8lSxblMiTVc0sSYsnO+XYqmPLFyWSGu/N6t1anBfS+fKtlM4Sa6wgGN+5vvdENSXOND52+jIykmL56iXrKDvYzjPVLWxZN348QUQozE7SzB6lZpEG/igQH2Pntss2AKNb++ANtCXOdOIcNs5ZnRXS+Xyt/LHnCrR0USJrclOwCXzrfRuxW5k/HzzFyekrMoHx3Tw+RdnJ41r81/+lnP/31Nsh1U8pNbmoWHpRwTmrs/nm5RvYEKRP/gsXrMLV3k9ibGj/HJYtSuT6i9dy+aZxD2GPcst7i2nqGhj1niLC9z9UyoNv1HOa9QEwVlF2Mg+85qJ7YJiU+Bhaugf5U9lhBDhvbU7Qa1BKhU4DfxT5+BnLgpZvXp7J5uWhn0dEuPpdUz/xfObK4N8gCjIT+dy7V014XGG2ldnT0ktpQTpPVTVhDCTFOfjqgxU8eM2Z/m8QSqnp064eNe8UZVuZPVZ3z47KJpwZCdz+vg2Uuzr5w8sHw1k9pRY8Dfxq3lmamYjdJtS29NA76OaFmla2FOeyrXQJZ63M4nuPV9PcNTD1iZRSQWlXj5p3Yh02lmUmUtfSy/NvtzDk9vifKL7t8g1c+OPneO/PXiAjMRbwPkvw7Q+UaPePUiHSFr+alwqtzJ4dlU2kJ8Zw6nLvg2UrspL48Yc3sakgnWWLEslOiePPr7m4V7t/lAqZtvjVvFSUncRzb7XQ3D3I+WtGP1h2ycbFXLJxMeCd2vkTd7/Cdx+v5sINeeSkxIerykotGNriV/NSUXYyQyMeOvqG2TrJNNG+7p/BEQ+3PVo1hzVUauHSwK/mJV9KZ6zDNuVqXCuykrjm3CIe2d3Ac2+1zEX1lFrQNPCrecmX0nnWyiyS4qbukfyPc4sozEri5of34NYF25WalAZ+NS9lJMXy6bNWhPSgGECcw851W9dwsK2PVw7okg9KTUYDv5q3brq0eMJpHYI5b202cQ5bSMs+KhXNNPCriJEY6+DsVVns2Dv1so9KRTMN/CqibCnOpb6jn6oj3VPvrFSU0sCvIsr5a3ORgGUfAXoG3Xzt4T0c6Qx9Hd8X3m7l18/VzUYVlQo7DfwqomSnxHHy0gx2VDb6y36wo5rfvXSQR3cfCekcrT2DfPbe17n9sSqeqtLxAhV5NPCriLO1OJe9DV3Ud/Szp76T3714AIDdro6Qjv/W36roG3JTkJnALQ/vpX9oZBZrq9TcCynwi8hFIlItIjUicn2Q1z8pIi0i8qb18+mA10YCyrfPZOWVCsa3stcTexr56oMVZCbFcebKRVTUd0557Iu1rfz1jXquOqeQ73+wlPqOfn6iK3+pCDPlkzEiYgfuALYALuBVEdlujKkcs+ufjDHXBjlFvzFm04lXVanQFGYnU5SdxA92VNM7NMJPrthEfUc/3328mo6+IdKtWT3HGnSPcNODeyjITODa81aREGvnQ6c4uev5Ot53Uj5rxiwMr9RCFUqL/zSgxhhTZ4wZAu4DLpvdail1YrYU59E7NMJZK7PYVrqEUmt94HLX6Fb/H14+yJf/vJsv/3k3n/rtq9S19nLbZRtIiLUDcMMl60iJd3DTQxWaIqoiRiiBPx84HLDtssrG+oCIlIvIAyJSEFAeLyJlIrJLRC4P9gYicpW1T1lLi861ok7cB09xUlqQzjcv34CI+NfpLQ/o5+8aGObr2/fyxN5G/lnTyv6WXj5z9grOXZPj3yczKZZrzl3Jqwfaqe8IPStIqfkslGmZg61uMbbp8wjwR2PMoIhcDfwOON96bakxpkFECoF/iEiFMaZ21MmMuRO4E2Dz5s3arFInbGVOMg9/9kz/dlpCDIVZSewOaPE/U93C8Ijhj585lc3LJ35C2Pf0cLmrE2dG4uxVWqk5EkqL3wUEtuCdQEPgDsaYNmPMoLX5a+CUgNcarD/rgGeAk06gvkodt43ONCoCAv/OyiYWJcVy0tKMSY9buziFGLuEnBWk1HwXSuB/FVglIitEJBa4AhiVnSMiiwM2twFVVnmGiMRZv2cBZwJjB4WVmhMlznQauwZo7hpgyO3hmX3NXLAud8olG+McdtYtTqX88NRZQUotBFN29Rhj3CJyLfAEYAd+Y4zZKyK3AmXGmO3A50RkG+AGjgKftA5fB/xKRDx4P2S+HSQbSKk5Uer09vPvdnUS57DRPej2p35OpcSZxsNvNODxGGy6tq9a4EJaetEY8xjw2JiyWwJ+vwG4IchxLwIbT7COSs2I9UvSsNuEclcHHX3DJMTYOWtVVkjHluSn87+7DrG/rde/VoBSC5U+uauiRkKsnVU5yex2dfJkVRPnrM4iPsYe0rElBeOzgpRaqDTwq6hS6kznnzWtHOkcYEtxXsjHrcxOJiHGzm7t51cRQAO/iiolBWmMeAw2gfPX5kx9gMVht7EhP1Vb/CoiaOBXUaUk3/sE7+blmWQmBZ+6YcJjnensbejSNX3VgqeBX0WVNXkpFGYl8eHNBVPvPEaJM41Bt4e3mnpmoWZKzZ2QsnqUihSxDhv/+NK5x3VsiX++nw6Kl6TOYK2Umlva4lcqRMsXJZIa7xg17YNSC5EGfqVCJCKUONN1gFcteBr4lZqGEmca1Y3dDAyHvirXvsYu3m7Sxd/V/KGBX6lpOGlpBm6P4fVD7SHtb4zhqnte42N3vUzXwPAs106p0GjgV2oa3lm0iFiHjZ2VoS3Cvq+xm0NH+2juHuSHO96a5dopFRoN/EpNQ1Kcg7NXZrGzsimkFbl2VjYhAu/ZuJh7Xjqg4wNqXtDAr9Q0bSnOxdXez77Gqfvtd1Y2sakgnf/+wEYWJcdx44N7GPHoWkMqvDTwKzVN716Xiwjs2Dt5d09DRz8V9Z1sLc4jNT6GWy4tpqK+k9+/dGBO6qnURDTwKzVN2SlxnFSQzs6qxkn3e7LK+8Hgm/P/0pLFvKNwEb98tm7W66jUZDTwK3UcthTnsae+i4ZJFmDfWdlEYVYSK3O88/eLCBcU59LYNUBT18BcVVWpcTTwK3Uctq73tuJ9rfqxOvuHeam2bdwKX75VwMr16V8VRhr4lToORdnJFGYnTdjP/0x1M26P8X9A+KxfkoZNdEEXFV46SZtSx2lLcS53P7+ff+xrwmEb3Yb6y+v1ZCXHsqkgY1R5Qqyd1bkpOt+PCisN/Eodp0s2LOZXz9bxqd+WBX39Y6cvxR5kYfYSZxo7rOcARHThdjX3NPArdZxKC9J54gvn0DMYbCoGoXhx8KmbS5zp3F/mwtXeT0Fm4uxWUqkgQgr8InIR8BPADtxljPn2mNc/CXwPqLeKfmaMuct67UrgJqv8m8aY381AvZWaF9bkpUz7mFJrXv/drg4N/CosphzcFRE7cAdwMVAMfEREioPs+idjzCbrxxf0M4GvAacDpwFfE5GMIMcqFTXW5KUQa7dpZo8Km1Cyek4DaowxdcaYIeA+4LIQz38hsNMYc9QY0w7sBC46vqoqFRliHTbWLUll92HN7FHhEUrgzwcOB2y7rLKxPiAi5SLygIj4FjQN9VilokpJfhp76jvx6Lw9KgxCCfzB0g7G/mt9BFhujCkBngR8/fihHIuIXCUiZSJS1tLSEkKVlFrYSpxp9A6NUNeqC7eruRdK4HcBBQHbTqAhcAdjTJsxZtDa/DVwSqjHWsffaYzZbIzZnJ2dHWrdlVqwSgusAd7D2s+v5l4ogf9VYJWIrBCRWOAKYHvgDiKyOGBzG1Bl/f4EsFVEMqxB3a1WmVJRrSg7mcRYuz7Bq8JiynROY4xbRK7FG7DtwG+MMXtF5FagzBizHficiGwD3MBR4JPWsUdF5Da8Hx4Atxpjjs7CdSi1oNhtwob8NH2CV4VFSHn8xpjHgMfGlN0S8PsNwA0THPsb4DcnUEelIlJJfhr37DrIiMcEfcJXqdmik7QpFSYrspMYcnto7tYpmtXc0sCvVJjkpycAUN8+8Zz+Ss0GDfxKhYkzwztdg0sDv5pjGviVChNnhrfF72rvC3NNVLTRwK9UmMTH2MlKjtMWv5pzGviVCqP8jATqJ1m3V6nZoIFfqTByZiRoi1/NOQ38SoWRMyOB+vZ+naxNzSkN/EqFkTM9gaERD609g1PvrNQM0cCvVBj5UjoPa3ePmkMa+JUKI03pVOGggV+pMMr3B35t8au5o4FfqTBKjHWQmRSrKZ1qTmngVyrMNKVTzTUN/EqFmTfwz20ff++gm20/e0EXgolSGviVCjNnRiL17f0YM3e5/LUtPZS7Onmxtm3O3lPNHxr4lQqz/PQEBt0eWnuG5uw9m7q8zw1oNlF00sCvVJiFI6WzsWvAek8dW4hGGviVCrNwzMvfbAV+XQQmOmngVyrMfLn800npNMbw1Qcr+PVzdcf1nk0BLf65HFtQ84MGfqXCLDnOQXpizLS6ev6+p5F7Xz7Enc/XHdcEb41WH3//8AhHe+dubEHNDxr4lZoHppPL3z0wzDce2UtirJ2W7kHePI6UzOauAWLt3v/++vBY9Akp8IvIRSJSLSI1InL9JPt9UESMiGy2tpeLSL+IvGn9/HKmKq5UJHGmJ4Yc+H+w4y2auwf55cdPwW4TdlY2Tfv9GrsG2OhMA3SANxpNGfhFxA7cAVwMFAMfEZHiIPulAJ8DXh7zUq0xZpP1c/UM1FmpiJNvzcs/VX97hauTe146wCfOWMY5q7M5fUXmtAP/wPAIHX3DnLIsA9CUzmgUSov/NKDGGFNnjBkC7gMuC7LfbcB3gYEZrJ9SUcGZkUD/8AhtU/S337J9D4uS4/jShWsA2FKcS01zD3UtPSG/V7PVv78yJ5mUeMdxtfjLXR188U9v4h7xTPtYFX6hBP584HDAtssq8xORk4ACY8yjQY5fISJviMizInJ2sDcQkatEpExEylpaWkKtu1IRY2VOMgDVjd0T7jMwPMIbhzr42OlLSY2PAbyBH5hWq7+p29s2y0uN9z81PF1/fb2ev75Rz75J6qvmr1ACvwQp838fFREb8CPguiD7HQGWGmNOAr4I3CsiqeNOZsydxpjNxiEkp+0AABdLSURBVJjN2dnZodVcqQiyMd/b3757koFaX8t82aJEf5kzI5HixanTC/xWKmduavxxTxDnq2dFfee0j1XhF0rgdwEFAdtOoCFgOwXYADwjIgeAM4DtIrLZGDNojGkDMMa8BtQCq2ei4kpFkvTEWJYtSqT88MSB1Jd943vgy2dLcS6vHWoPefnGxs5jLf78dO8EcdPJ5R8e8VDZ0AWgk7wtUKEE/leBVSKyQkRigSuA7b4XjTGdxpgsY8xyY8xyYBewzRhTJiLZ1uAwIlIIrAKO74kTpSJciTN90kDqG4T1TfHgs3V9LsbAU1WhtfqbuweJc9hITXDgzEigd2iEzv7hkOv5VlM3g24PDpuwe5IPKjV/TRn4jTFu4FrgCaAKuN8Ys1dEbhWRbVMcfg5QLiK7gQeAq40xR0+00kpFopL8NBo6B2jpDt5yd7X347AJOSnxo8qLF6eSn54QcndPY+cAuanxiMhxTRdR7vIG+4s25FHd1M3A8Ij/tb4hNzf8tZwjnZoiOp+FlMdvjHnMGLPaGFNkjLndKrvFGLM9yL7nGmPKrN//YoxZb4wpNcacbIx5ZGarr1TkKLHy6ivqg7f669v7WZKegN02ethNRDh9RSZ7re6XqTR1DZCX6v3wOJ4J4spdHaQlxHBpyRJGPGbU++6sbOKPrxw+rmcL1NzRJ3eVmic25KdhEybsPnG1943r5vFxZibS1DXAkHvq9Mrm7kFyUuO8xx3Hmr+7D3dS4kxjU0E6ABUB3VO+gF/bHHp6qZp7GviVmieS4hyszEmesJ/f1d5PfvoEgT8jAY85NnA7EWMMjZ3HWvxpCTEkx4Weyz8wPEJ1Uzcb89PITY0jOyXO3/Uz6B7hmWpvOnZda29I51PhoYFfqXnEO8DbOS7LZtA9QnP34LiMHh9nemhdNt2DbvqHR8i1Ar+IWJk9oQX+vQ1djHgMJc50RIRSZ5o/tXNX3VF6Bt1kJcdpi3+e08Cv1DxS4kyjrXdo3MRpDR3elvyEXT2+Qdoxx7na+/j+E9X+J2ybrG8EuWnxAceGvuav79tIaUGaVd906lp76R4YZmdlIwkxdq44tYCGzgH6htwhnfPVA0e575VDIe2rZoYGfqXmkRKnr998dD+/LzDnTxD489Liscn4vvqH32zgZ0/X8Mp+bzKdb8nF3JQ4/z5Oa56gUFS4OslOifN3FZU40zDGW76zsolzVmdRvMT7jGZdS2jdPb9+ro5vPVYV0r5qZmjgV2oeWbc4hRi7sHtM4PcF5ola/LEOG3mp8eNa7r4ulx3WoGtjwFO7PvkZCXQPukPK5d/t6qDUmYaIN7PI90H1h1cO0dQ1yNbiPIqyvdNP1IY4f1BtSw9dA276h0am3lnNCA38Ss0jcQ47a/NSxw3wutr7sdvE39IOJj/I9Au11iDrzsomjDGjpmvwOZbLP8X4wMAwda29/mAPkJkUS0FmAn8rP4LdJpy/NodlixKxSWgt/uERDwfbvO/b3K3zO84VDfxKzTMbnWlUuDpHrazlau9jcVo8DvvE/2XHTrhmjKGuuYeMxBjqO/qpPNJFc9cAqfEOEmLtAceFltJZUd+JMfjn8fcpyfd+EGxelkFGUizxMXacGYkhtfgPH+3DbV3nVBlJauZo4Fdqnil1ptE96GZ/27EWc33HxKmcPs6MBBq7BvwDuS3dg3QPuvnEGcsQ8bb6G7sGyEsb/a3Bd967X9jPTQ9VcNNDFbx2sH3c+X1pm6UBLX449uCZb6ZQgKLsJGpDaPEH7tM0wRPLauZp4FdqnjllWSYAL9a2+ctc7f0TpnL65KcnMOIxHLFazr6geuqKTE5ZmsHOyiaaugZHdfOAt7vm9BWZ1Db38PeKRv7yWj2fuaeM9oC1AXoH3dzz4gHWL0klMyl21PHvXpfLhvxULi1Z4i8ryk5mf2vPlOsBB64j0KQt/jmjgV+peaYoO4kVWUns2NsIwJDbQ2PXwIQDuz6+DwZfKqivq6UoO5ktxbnsbejirabucYFfRPjTv7+D127ewms3b+Gv17yTzv5hvvP4Pv8+P3nqbRo6B/j6tvXj3ndlTjKP/ufZo75JFGYnMzDsoWGKOXtqW3rISo4jIcbuH39Qs08Dv1LzjIiwpTiXXXVtdA0M09g5gDETp3L6jO2rr2vpJSHGTl5qvL8bpm9ohNzUuAnPAbBucSqfPmsF9716mFcPHKXqSBd3v7CfD28u4NTlmSFdQ1F2kr8Ok6lt6aUoO4nc1Dh/xpGafRr4lZqHthTnMjxieLa6ZcLpmMdanO5tcfv2r23poTA7CZtNKMxO9gfjsS3+YD5/wSry0xO48cEKbnywgrSEGK6/eG3I9S8MMaWzrqWHopxkclLj/UtCqtmngV+peejkpRksSoplZ2WTvwVfMEUff5zDTm5qnD+zp7alx59TD7B1fR4QWuBPjHXwjW3reauph9cPdfDVS9aRMaZvfzJZybGkxjsmDfxHe4do7xumMCuJvNR4/5KQavZp4FdqHvLlxD9d3cz+tl5swrhsnGCcGYm42vsZGB6hvqOfQquVD/C+k/IpyExg/ZJxq58GdUFxLh85rYBLNubxgZPzpz4ggIhQlJM8aVePfwwiJ9nb1dM5MK2VwNTxc4S7Akqp4Lauz+PPr7l4+I168lLjiZkkh98nPz2BNw63s7+1F2MY1eJfnZvC8185f1p1+O/3l0y73j6FWcm8UNMy4eu+jJ6V2cnUNvcw6PbQ1e8mLTHmuN9ThUZb/ErNU2etzCI+xkZD58CUqZw+zowEjnQM8HbzsYyecCnKSaKpa5CeweCTtdW29BLrsLEkPcHf/aQDvHNDA79S81RCrJ2zV2UDU2f0+DgzEnF7DC/VtgKwIitpiiNmT2GW90OnboJ+/trmHgqzkrxTUVjdWJrSOTe0q0epeWxLcS47K5umzOjx8X1APFvdQn56wqipGebayhzvh85vXzzAurxURLzr9Pq+vdS19rJucQoAuSna4p9LGviVmscuWJdLdkocJy1Nn3pnjqV8NnQOcM7q7Nms2pSWZnrz8//6ej1QD8C9rxzi758/G0E4dLSPS0sWA/iXgmzWwD8nNPArNY9lJsXy6o0XhLx/4Hw+hWHs5gHvVNEv/N/zGbTWAd5V28an7ynjV8/WccnGPEY8xj8GER9jJz0xxr9egJpdIfXxi8hFIlItIjUicv0k+31QRIyIbA4ou8E6rlpELpyJSiulgouPsZNtLbJSlBO+gV2fGLuN5DgHyXEOLijO5T0li/nZ0zU8VdUMMCrdNDclXrt65siUgV9E7MAdwMVAMfARESkOsl8K8Dng5YCyYuAKYD1wEfBz63xKqVnia/UXZYe3xR/MLZcWE2e38f0d1cCxJ3zBuxykdvXMjVBa/KcBNcaYOmPMEHAfcFmQ/W4DvgsE3rnLgPuMMYPGmP1AjXU+pdQs8fXzhzOVcyK5qfF86cI1DI8YclPjSI471tucm6Lz9cyVUAJ/PnA4YNtllfmJyElAgTHm0ekeax1/lYiUiUhZS8vED3wopaZW6kzHmZFATsrkk7GFy8fPWMYpyzLYVDB6wDovLZ6W7kFGppjKWZ24UAZ3JUiZ/86IiA34EfDJ6R7rLzDmTuBOgM2bN+tdV+oE/NtZK/jkmcv96+LON3ab8MfPnIFtTPVyUuPxGGjtGb9mgJpZoQR+F1AQsO0EGgK2U4ANwDPWP7Q8YLuIbAvhWKXUDLPZBFvQNtf8EesY39ngW0+4qWtAA/8sC6Wr51VglYisEJFYvIO1230vGmM6jTFZxpjlxpjlwC5gmzGmzNrvChGJE5EVwCrglRm/CqXUgudbJ0BTOmfflIHfGOMGrgWeAKqA+40xe0XkVqtVP9mxe4H7gUrgceCzxpiRE6+2UirSjJ2vx+MxVDd2h7NKESukB7iMMY8Bj40pu2WCfc8ds307cPtx1k8pFSWykuOwybGnd3/85Fv89B813PHRk3mP9YSvmhk6SZtSal6w24TsFO+8/DXNPfzi2VpE4BuP7KVrYDjc1YsoGviVUvNGXqr36d0bH6wgIcbO3VdupqVnkB88UR3uqkUUDfxKqXkjJzWel2rbeHn/Ua6/eB3nr83lX89Yxj27DlLu6gh39SKGBn6l1LyRlxqP22M4eWk6V5zqzQS/7sI1ZCfH8dUHK3CPeMJcw8iggV8pNW8sW5SIwybc/r6N2KwnvFLjY7jlvcXsqe/inpcOhrmGkUEDv1Jq3vjEO5bx9JfOZd3i0QvCv2fjYt61Opsf7KimsVPn8zlRGviVUvNGnMNOQeb49YVFhFsvW4/bY7j10b1hqFlk0cCvlFoQli1K4j/PX8ljFY08va853NVZ0DTwK6UWjKvOKWJlTjI3P7yHvQ2d7GvsorqxmyG3DvpOhy69qJRaMGIdNr55+QauuHMX7/npC/7yT75zOV/ftj6MNVtYNPArpRaUMwoX8eh/nsXho30A/O6lA/x9zxFuubTYnwmkJqeBXym14GzIT2NDfhoAfUMjXPfn3VTUd1I6ZnEXFZz28SulFrTz1+Zgtwk7KhvDXZUFQwO/UmpBy0iK5dTlGeysbJpwH2MMTV0DHOns50hn/7QnfXOPeCLqqWHt6lFKLXhbivO47dFKDrT2sjwradzrd7+wn2/+rcq/Heuw8cJXziMnxJW+PnbXyyTHObjrys3zdknL6dAWv1JqwdtanAswYav/xdo2nBkJfPv9G/nCBasYcnt4/VB7SOceGB6h7GA7T+1r5tHyIzNW53DSwK+UWvAKMhNZm5cSNPAbYyh3dXBG4SKuOG0pV7+rCIdN2O3qDOncexu6GPEYkmLt3PpoJZ39C39tAA38SqmIsLU4l7KDRznaOzSqvKFzgNaeIUqd3iyg+Bg7axenhDzNs2+/n1xxEm09g/xgx8JfG0ADv1IqImwpzsNj4Kmq0a3+8sPewL3ReSzVc2N+OuWuTjweM+V5y12d5KTEcUFxLv/6juX8ftdByg4cpX9oZMZ/BobnZklyHdxVSkWEDfmpLE6LZ0dlEx/aXOAv3+3qJMYurFuc4i8rdabxx1cOcfBoHyuCDAYHKnd1UGJ9W7hu62r+vucIH/zlS7NyDZsK0nnos2fOyrkDaeBXSkUEEWFLcS73lx2mf2iEhFg74A3ca/NSiXPY/fuWWK3/clfHpIG/e2CYutZeLtuUD0BKfAx/+PQZPFk1ceroichJiZuV846lgV8pFTG2FOdyz0sHeaGmlS3FuXg8hgpXJ9s2LRm13+rcZOJjbOw+3OkP6sFU1HdiDP4WP8DKnGRW5iTP2jXMhZD6+EXkIhGpFpEaEbk+yOtXi0iFiLwpIi+ISLFVvlxE+q3yN0XklzN9AUop5XP6ikWkxDnYaT3Fe6Ctl+5BN6XO0VM5OOw21i9Jm3KAt9zK/ClxRtZUEFMGfhGxA3cAFwPFwEd8gT3AvcaYjcaYTcB3gR8GvFZrjNlk/Vw9UxVXSqmxYh02zlubw1NVzYx4zLHAXZA2bt8SZxp7GjonfSK33NVBQWYCmUmxs1bncAilxX8aUGOMqTPGDAH3AZcF7mCM6QrYTAKmHipXSqlZsKU4l7beIV4/1M5uVwfxMTZWZo/vmilxpjEw7KGmpWfCc5W7OiOutQ+hBf584HDAtssqG0VEPisitXhb/J8LeGmFiLwhIs+KyNnB3kBErhKRMhEpa2lpmUb1lVJqtHPXZBNjF3ZWNlHu6mTDkjQc9vGhzj/Aezj4g1xtPYO42vspyR//bWGhCyXwB5uYYlyL3hhzhzGmCPi/wE1W8RFgqTHmJOCLwL0ikhrk2DuNMZuNMZuzs7NDr71SSo2REh/DGYWLeGJvI3sbJm6xr1iUREqcg90T9POX10dm/z6EFvhdQEHAthNomGT/+4DLAYwxg8aYNuv314BaYPXxVVUppUKztTiXg219DAx7KA3Svw9gswkbnWn+cYCxyg93IgIbndHZ4n8VWCUiK0QkFrgC2B64g4isCth8D/C2VZ5tDQ4jIoXAKqBuJiqulFITucCatA0mb7GXONPZ19jFoHv8E7Plrg6KspNJjou8rPcpA78xxg1cCzwBVAH3G2P2isitIrLN2u1aEdkrIm/i7dK50io/BygXkd3AA8DVxpijM34VSikVYHFaAiXONFLiHSxflDjhfqXONIZHDK8fHN3dM+ge4c3DHaPy9yNJSB9lxpjHgMfGlN0S8PvnJzjuL8BfTqSCSil1PG6+tJimroFJ588/a1UW2Slx/Pffq3jwmjOxW2v2/uKZWtp6h3j/Sc65qu6c0knalFIR6dTlmVxasmTSfVLiY7j50mLKXZ384eWDANS19PDzp2vZVrqEs1ZlzUVV55wGfqVUVHtvyWLOXpXF9x6vpqlrgJsf3kNcjI2bLl0X7qrNGg38SqmoJiLcdtkGBkc8XHHnLv5Z08ZXLlxDTkpoyzIuRBr4lVJRb3lWEteet5L9rb2UFqTz0dOXhbtKsyry8pSUUuo4/Pu7Chlye3j/yfn+Qd5IpYFfKaWAOIedL124JtzVmBPa1aOUUlFGA79SSkUZDfxKKRVlNPArpVSU0cCvlFJRRgO/UkpFGQ38SikVZTTwK6VUlBFj5te66CLSAhw8gVNkAa0zVJ2FJFqvG/Ta9dqjy0TXvcwYE9LatfMu8J8oESkzxmwOdz3mWrReN+i167VHl5m4bu3qUUqpKKOBXymlokwkBv47w12BMInW6wa99mgVrdd+wtcdcX38SimlJheJLX6llFKT0MCvlFJRJmICv4hcJCLVIlIjIteHuz6zSUQKRORpEakSkb0i8nmrPFNEdorI29afGeGu62wQEbuIvCEij1rbK0TkZeu6/yQiseGu42wQkXQReUBE9ln3/h1RdM//y/q3vkdE/igi8ZF630XkNyLSLCJ7AsqC3mfx+qkV98pF5ORQ3iMiAr+I2IE7gIuBYuAjIlIc3lrNKjdwnTFmHXAG8Fnreq8HnjLGrAKesrYj0eeBqoDt7wA/sq67Hfi3sNRq9v0EeNwYsxYoxft3EPH3XETygc8Bm40xGwA7cAWRe99/C1w0pmyi+3wxsMr6uQr4RShvEBGBHzgNqDHG1BljhoD7gMvCXKdZY4w5Yox53fq9G28AyMd7zb+zdvsdcHl4ajh7RMQJvAe4y9oW4HzgAWuXSL3uVOAc4G4AY8yQMaaDKLjnFgeQICIOIBE4QoTed2PMc8DRMcUT3efLgHuM1y4gXUQWT/UekRL484HDAdsuqyziichy4CTgZSDXGHMEvB8OQE74ajZrfgx8BfBY24uADmOM29qO1HtfCLQA/2N1c90lIklEwT03xtQD3wcO4Q34ncBrRMd995noPh9X7IuUwC9ByiI+T1VEkoG/AF8wxnSFuz6zTUQuBZqNMa8FFgfZNRLvvQM4GfiFMeYkoJcI7NYJxurPvgxYASwBkvB2cYwVifd9Ksf17z9SAr8LKAjYdgINYarLnBCRGLxB/w/GmL9axU2+r3nWn83hqt8sORPYJiIH8HbnnY/3G0C61QUAkXvvXYDLGPOytf0A3g+CSL/nABcA+40xLcaYYeCvwDuJjvvuM9F9Pq7YFymB/1VglTXKH4t34Gd7mOs0a6x+7buBKmPMDwNe2g5caf1+JfDwXNdtNhljbjDGOI0xy/He438YYz4GPA180Not4q4bwBjTCBwWkTVW0buBSiL8nlsOAWeISKL1b9937RF/3wNMdJ+3A/9qZfecAXT6uoQmZYyJiB/gEuAtoBa4Mdz1meVrPQvv17ly4E3r5xK8/d1PAW9bf2aGu66z+HdwLvCo9Xsh8ApQA/wZiAt3/WbpmjcBZdZ9fwjIiJZ7DnwD2AfsAX4PxEXqfQf+iHcsYxhvi/7fJrrPeLt67rDiXgXezKcp30OnbFBKqSgTKV09SimlQqSBXymloowGfqWUijIa+JVSKspo4FdKqSijgV8ppaKMBn6llIoy/x/J4xnFVRjykAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "scores = []\n", "for k in range(2,100):\n", " kn = KNeighborsClassifier(n_neighbors=k)\n", " kn.fit(X_train, y_train)\n", " scores.append(kn.score(X_test, y_test))\n", "\n", "plt.plot(scores)\n", "print(np.where(scores == np.max(scores)))" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[57 0 6]\n", " [ 9 1 10]\n", " [14 2 21]]\n", " precision recall f1-score support\n", "\n", " 1 0.71 0.90 0.80 63\n", " 2 0.33 0.05 0.09 20\n", " 3 0.57 0.57 0.57 37\n", "\n", " accuracy 0.66 120\n", " macro avg 0.54 0.51 0.48 120\n", "weighted avg 0.60 0.66 0.61 120\n", "\n", "0.6583333333333333\n" ] } ], "source": [ "kn = KNeighborsClassifier(n_neighbors=12)\n", "kn.fit(X_train, y_train)\n", "y_pred_kn = kn.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_kn))\n", "print(classification_report(y_test, y_pred_kn))\n", "print(accuracy_score(y_test, y_pred_kn))" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[49 4 10]\n", " [ 6 0 14]\n", " [ 6 5 26]]\n", " precision recall f1-score support\n", "\n", " 1 0.80 0.78 0.79 63\n", " 2 0.00 0.00 0.00 20\n", " 3 0.52 0.70 0.60 37\n", "\n", " accuracy 0.62 120\n", " macro avg 0.44 0.49 0.46 120\n", "weighted avg 0.58 0.62 0.60 120\n", "\n", "0.625\n" ] } ], "source": [ "lg = LogisticRegression()\n", "lg.fit(X_train, y_train)\n", "y_pred_lg = lg.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_lg))\n", "print(classification_report(y_test, y_pred_lg))\n", "print(accuracy_score(y_test, y_pred_lg))" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[25 25 13]\n", " [ 2 4 14]\n", " [ 3 5 29]]\n", " precision recall f1-score support\n", "\n", " 1 0.83 0.40 0.54 63\n", " 2 0.12 0.20 0.15 20\n", " 3 0.52 0.78 0.62 37\n", "\n", " accuracy 0.48 120\n", " macro avg 0.49 0.46 0.44 120\n", "weighted avg 0.62 0.48 0.50 120\n", "\n", "0.48333333333333334\n" ] } ], "source": [ "nb = GaussianNB()\n", "nb.fit(X_train, y_train)\n", "y_pred_nb = nb.predict(X_test)\n", "print(confusion_matrix(y_test, y_pred_nb))\n", "print(classification_report(y_test, y_pred_nb))\n", "print(accuracy_score(y_test, y_pred_nb))" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[47 7 9]\n", " [ 6 3 11]\n", " [ 4 5 28]]\n", " precision recall f1-score support\n", "\n", " 1 0.82 0.75 0.78 63\n", " 2 0.20 0.15 0.17 20\n", " 3 0.58 0.76 0.66 37\n", "\n", " accuracy 0.65 120\n", " macro avg 0.54 0.55 0.54 120\n", "weighted avg 0.65 0.65 0.64 120\n", "\n", "0.65\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "E:\\Programs\\Miniconda3\\envs\\myenv\\lib\\site-packages\\sklearn\\svm\\_base.py:231: ConvergenceWarning: Solver terminated early (max_iter=500). Consider pre-processing your data with StandardScaler or MinMaxScaler.\n", " % self.max_iter, ConvergenceWarning)\n" ] } ], "source": [ "from sklearn.ensemble import StackingClassifier\n", "\n", "clf1 = DecisionTreeClassifier(criterion='gini', max_depth=6, random_state=0)\n", "clf2 = SVC(gamma='scale', degree=7, probability=True, kernel='poly', max_iter=500)\n", "clf3 = KNeighborsClassifier(n_neighbors=12)\n", "clf4 = LogisticRegression()\n", "clf5 = GaussianNB()\n", "\n", "clf_stack = StackingClassifier(estimators=[('clf1', clf1), ('clf2', clf3), ('clf3', clf4)], final_estimator=clf2, n_jobs=-1)\n", "clf_stack.fit(X_train, y_train)\n", "y_pred_stack = clf_stack.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_stack))\n", "print(classification_report(y_test, y_pred_stack))\n", "print(accuracy_score(y_test, y_pred_stack))" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[60 2 1]\n", " [11 3 6]\n", " [14 3 20]]\n", " precision recall f1-score support\n", "\n", " 1 0.71 0.95 0.81 63\n", " 2 0.38 0.15 0.21 20\n", " 3 0.74 0.54 0.62 37\n", "\n", " accuracy 0.69 120\n", " macro avg 0.61 0.55 0.55 120\n", "weighted avg 0.66 0.69 0.65 120\n", "\n", "0.6916666666666667\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "E:\\Programs\\Miniconda3\\envs\\myenv\\lib\\site-packages\\sklearn\\svm\\_base.py:231: ConvergenceWarning: Solver terminated early (max_iter=500). Consider pre-processing your data with StandardScaler or MinMaxScaler.\n", " % self.max_iter, ConvergenceWarning)\n" ] } ], "source": [ "from sklearn.ensemble import VotingClassifier\n", "\n", "clf1 = DecisionTreeClassifier(criterion='gini', max_depth=6, random_state=0)\n", "clf2 = SVC(gamma='scale', degree=7, probability=True, kernel='poly', max_iter=500)\n", "clf3 = KNeighborsClassifier(n_neighbors=12)\n", "clf4 = LogisticRegression()\n", "clf5 = GaussianNB()\n", "\n", "\n", "clf_vote = VotingClassifier(estimators=[('clf1', clf1), ('clf2', clf2), ('clf3', clf5), ('clf4', clf3)], voting='hard')\n", "clf_vote.fit(X_train, y_train)\n", "y_pred_vote = clf_vote.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_vote))\n", "print(classification_report(y_test, y_pred_vote))\n", "print(accuracy_score(y_test, y_pred_vote))" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[49 7 7]\n", " [ 8 4 8]\n", " [ 8 3 26]]\n", " precision recall f1-score support\n", "\n", " 1 0.75 0.78 0.77 63\n", " 2 0.29 0.20 0.24 20\n", " 3 0.63 0.70 0.67 37\n", "\n", " accuracy 0.66 120\n", " macro avg 0.56 0.56 0.56 120\n", "weighted avg 0.64 0.66 0.65 120\n", "\n", "0.6583333333333333\n" ] } ], "source": [ "from sklearn.ensemble import BaggingClassifier\n", "\n", "base_clf = DecisionTreeClassifier(criterion='gini', max_depth=6, random_state=0)\n", "#base_clf = SVC(gamma='scale', degree=7, probability=True, kernel='poly', max_iter=500)\n", "#base_clf = KNeighborsClassifier(n_neighbors=12)\n", "#base_clf = LogisticRegression()\n", "#base_clf = GaussianNB()\n", "\n", "clf_bag = BaggingClassifier(base_estimator=base_clf)\n", "clf_bag.fit(X_train, y_train)\n", "y_pred_bag = clf_bag.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_bag))\n", "print(classification_report(y_test, y_pred_bag))\n", "print(accuracy_score(y_test, y_pred_bag))" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[51 7 5]\n", " [ 8 5 7]\n", " [ 8 3 26]]\n", " precision recall f1-score support\n", "\n", " 1 0.76 0.81 0.78 63\n", " 2 0.33 0.25 0.29 20\n", " 3 0.68 0.70 0.69 37\n", "\n", " accuracy 0.68 120\n", " macro avg 0.59 0.59 0.59 120\n", "weighted avg 0.67 0.68 0.67 120\n", "\n", "0.6833333333333333\n" ] } ], "source": [ "from sklearn.ensemble import RandomForestClassifier\n", "\n", "num_classifiers = 500\n", "rf = RandomForestClassifier(n_estimators=num_classifiers,\n", " criterion='gini', max_depth=6, min_samples_split=5)\n", "rf.fit(X_train, y_train)\n", "y_pred_rf = rf.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_rf))\n", "print(classification_report(y_test, y_pred_rf))\n", "print(accuracy_score(y_test, y_pred_rf))" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[49 5 9]\n", " [ 6 5 9]\n", " [ 4 3 30]]\n", " precision recall f1-score support\n", "\n", " 1 0.83 0.78 0.80 63\n", " 2 0.38 0.25 0.30 20\n", " 3 0.62 0.81 0.71 37\n", "\n", " accuracy 0.70 120\n", " macro avg 0.61 0.61 0.60 120\n", "weighted avg 0.69 0.70 0.69 120\n", "\n", "0.7\n" ] } ], "source": [ "from sklearn.ensemble import AdaBoostClassifier\n", "\n", "num_classifiers = 500\n", "learning_rate = 0.07\n", "base_clf = DecisionTreeClassifier(criterion='gini', max_depth=1, random_state=0)\n", "#base_clf = SVC(gamma='scale', degree=7, probability=True, kernel='poly', max_iter=500)\n", "#base_clf = LogisticRegression()\n", "#base_clf = GaussianNB()\n", "ada_clf = AdaBoostClassifier(base_estimator=base_clf,\n", " n_estimators=num_classifiers,\n", " learning_rate=learning_rate)\n", "ada_clf.fit(X_train, y_train)\n", "y_pred_ada = ada_clf.predict(X_test)\n", "\n", "print(confusion_matrix(y_test, y_pred_ada))\n", "print(classification_report(y_test, y_pred_ada))\n", "print(accuracy_score(y_test, y_pred_ada))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.12" }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }