{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": " Линейная_регрессия,_градиентный_спуск.ipynb", "provenance": [], "collapsed_sections": [] }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "8S0l1cfKEdcQ" }, "source": [ "# **План**\n", "\n", "- Реализация алгоритма стохастического градиентного спуска:\n", " - загрузка данных\n", " - создание и обучение алгоритма\n", "- Сравнение написанного алгоритма с реализованным в библиотеке Sklearn.\n", "\n", "- Рекомендованная литература" ] }, { "cell_type": "markdown", "metadata": { "id": "WBIx7HpJwCKa" }, "source": [ "ноутбук в колаб: https://drive.google.com/file/d/1aqRvKK0WRq8hn89bm1O__khM5cg1wuW7/view?usp=sharing" ] }, { "cell_type": "markdown", "metadata": { "id": "xvGoXa3CFAeV" }, "source": [ "# Реализуем алгоритм стохастического градиентного спуска" ] }, { "cell_type": "code", "metadata": { "id": "8JSYdc4UJD6s" }, "source": [ "import numpy as np\n", "\n", "import pandas as pd\n", "import matplotlib.pyplot as plt" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "4oW8ZZWAXN2E" }, "source": [ "## Загрузка и подготовка данных\n", "\n", "### **Постановка задачи:** \n", "Предсказать вероятность одобрения заявки на поступление в ВУЗ (вероятность от 0 до 1) по набору признаков: оценки за экзамены, наличие research и тд" ] }, { "cell_type": "code", "metadata": { "id": "vW1tcon9JTEQ" }, "source": [ "df = pd.read_csv('https://raw.githubusercontent.com/tixonsit/mmdad_materials/master/datasets_14872_228180_Admission_Predict_Ver1.1.csv')\n", "del df['Serial No.']" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "oUQRj1lsJlFR", "colab": { "base_uri": "https://localhost:8080/", "height": 204 }, "outputId": "f39b113b-4be6-4892-ac0b-6ee50245296b" }, "source": [ "df.head()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
GRE ScoreTOEFL ScoreUniversity RatingSOPLORCGPAResearchChance of Admit
033711844.54.59.6510.92
132410744.04.58.8710.76
231610433.03.58.0010.72
332211033.52.58.6710.80
431410322.03.08.2100.65
\n", "
" ], "text/plain": [ " GRE Score TOEFL Score University Rating ... CGPA Research Chance of Admit \n", "0 337 118 4 ... 9.65 1 0.92\n", "1 324 107 4 ... 8.87 1 0.76\n", "2 316 104 3 ... 8.00 1 0.72\n", "3 322 110 3 ... 8.67 1 0.80\n", "4 314 103 2 ... 8.21 0 0.65\n", "\n", "[5 rows x 8 columns]" ] }, "metadata": { "tags": [] }, "execution_count": 7 } ] }, { "cell_type": "code", "metadata": { "id": "TG3eV_4dJmZT", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "e16d8189-acfd-41be-c8cb-944f8cd3cb94" }, "source": [ "# мало :(\n", "len(df)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "500" ] }, "metadata": { "tags": [] }, "execution_count": 8 } ] }, { "cell_type": "code", "metadata": { "id": "kw2Nz3P1XgMH", "colab": { "base_uri": "https://localhost:8080/", "height": 747 }, "outputId": "0724c481-bfc6-493b-e7a5-07e8bfed8718" }, "source": [ "import seaborn as sns\n", "\n", "sns.clustermap(df.corr())" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 9 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAALJCAYAAACp99XTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde5huZ1kf/u+d7OAmJAgIIoZAICZaTgYyAcPJBChFiwSUU8Aqlp8b+xOkqNRYkWzx0FixFqjCbxckyGUltIKklHIQmnKGTGAnkAAhBJBwKAQEOeTE3vfvj1lbHnZnHybZ875z+Hy45pp3PetZa91rklx8557nXW91dwAAgCWHzbsAAABYSwRkAAAYCMgAADAQkAEAYCAgAwDAQEAGAIDBhg7IVfXnVfXFqvrwPvZXVb2wqq6oqkuq6j6zrhEAgLVlQwfkJOcmecR+9v9EkhOmr21JXjyDmgAAWMM2dEDu7rcn+cp+ppyR5C96yXuT3Kqq7jCb6gAAWIs2dEA+CMck+cywfdU0BgDAJrVlJZNvuPrKNfW51De73fFPy9LSiD12dPeOedUDAMD6t6KAnN27VqmMG2cKwzclEH82ybHD9h2nMQAANqmVBeRd316lMubm/CRPr6pXJblfkq919+fnXNOmsbCwcFaSrfOuA1g11y4uLp4z7yIAVmpFAbnXWUCuqr9KclqS21bVVUnOTnJEknT3S5K8IclPJrkiybeS/MJ8Kt20ti4uLm6fdxHA6lhYWNg+7xoAbowVdpBvWKUyVkd3n3mA/Z3kl2dUDgAA68BmX2IBAADfZUMvsQAAgJXa0EssAABgpSyxAACAgYAMAACDla1B3m2JBQAAG5sOMgAADLxJDwAABjrIAAAwWFlA/raADADAxrbCDwqxxAIAgI3NEgsAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAA89BBgCAwQo7yLtWqQwAAFgbLLEAAICBDjIAAAysQQYAgMGKAnLrIAMAsMFZgwwAAIMVLrHQQQYAYGPzJj0AABjoIAMAwEAHGQAABit7ioUOMgAAG5wOMgAADKxBBgCAgQ4yAAAMrEEGAIDBhu4gV9UjkrwgyeFJXtrd5+y1/05JXpHkVtOcs7r7DTMvFACANWOFHeTdq1XHIVdVhyf50yT/NMlVSS6sqvO7+7Jh2nOSvLq7X1xVd0vyhiTHzbxYAADWjI38Jr37Jrmiu69Mkqp6VZIzkowBuZPccnr9vUk+N9MKAQBYc1YWkHf3KpVx41TVtiTbhqEd3b1jen1Mks8M+65Kcr+9TrE9yZur6hlJbpHkYatUKgAA68TKlljsWltLLKYwvOOAE/ftzCTndvcfV9WpSV5ZVffo7rV1o8CKLCwsnJVk67zrIMctLCxsn3cRJEmuXVxcPOfA04BkxUss1lVu/GySY4ftO05jo6cmeUSSdPd7qmprktsm+eJMKgRWy9bFxcXt8y4C1gq/qMDKbNg36SW5MMkJVXWXLAXjJyZ50l5z/i7JQ5OcW1X/JEsdpy/NtEoAANaUFQbktbUGeX+6+9tV9fQkb8rSI9z+vLsvrarnJVns7vOT/FqS/1xVz8rSG/ae0t3r5yYBADjkVrjEYn1lx+mZxm/Ya+y5w+vLkjxg1nUBALB2bdgOMgAA3BgCMgAADFYYkFerDAAAWBsEZAAAGKwoIO8WkAHYxNbxh9Csxw9t8eEmzM0KP0mvVqsOAFgPfAjNjKzDQM8GssIOsoAMAMDGtrKArIMMAMAGp4MMAAADHWQAABgIyAAAMFhhQD5steoAAIA1YUUBeZcOMgAAG5wOMgAADKxBBgCAwcqWWOzWQQYAYGNbYUDWQQYAYGNb2RILARkAgA3OEgsAABisLCC3DjIAABubDjIAAAxWFpCjgwwAwMa2ooD8bUssAADY4HSQAQBgICADAPu0sLBwVpKtc7j0cQsLC9vncN1rFxcXz5nDdVlDBGQAYH+2Li4ubp93EbMyp1DOGrOyNcglIAMAsLGtsIMMAAAb28oCsg4yAAAb3AqXWKxWGQAAsDZ4kx4AAAx0kAEAYLDCNcirVcbqqKpHJHlBksOTvLS7/6/nGlbV45NsT9JJLu7uJ820SFhlc3yG6TzN6/mp8+TZrQCHyMo6yKtVxSqoqsOT/GmSf5rkqiQXVtX53X3ZMOeEJL+Z5AHd/fdV9f3zqRZW1aZ6hulmtQl/IQBYNRu5g3zfJFd095VJUlWvSnJGksuGOb+Y5E+7+++TpLu/OPMqAQBYU1YUkHevVhWr45gknxm2r0pyv73mnJgkVfWuLC3D2N7db5xNeQAArEXruoNcVduSbBuGdnT3jhWcYkuSE5KcluSOSd5eVffs7q8euioBAFhP1vUa5CkM7ysQfzbJscP2Haex0VVJ3tfdNyT5ZFVdnqXAfOGhrhUAgPXhsJVM3lVr6+sALkxyQlXdpapuluSJSc7fa87fZKl7nKq6bZaWXFy5kp8JAAAbywo/KGT96O5vV9XTk7wpS+uL/7y7L62q5yVZ7O7zp30Pr6rLsnR7z+7uL8+vamA9m/Mj9eb1aDuPlwM2nBUusejVqmNVdPcbkrxhr7HnDq87ya9OXwA31aZ7pJ7HywEb0YbtIAMAwI2xwo+aXl8dZAAAWCkdZAAAGKwwIOsgAwCwsW3oN+kBAMBKWWIBAAADSywAAGAgIAMAwMAaZJiROX7K2rw+YS3xKWsArEM6yDA7PmUNANYBARkAAAYrC8gtIAMAsLGtsIO8e7XqAACANcGb9AAAYGANMgAADFa4BtkSCwAANjYdZAAAGOggAwDAQAcZAAAGOsgAADDwHGQAABj4JD0AABis8INCdJABANjYrEEGAIDBigLybgEZAIANTgcZAAAGnmIBAAADHWQAABgIyAAAMKhewbONT7jdyWvqQcgf/9JFNe8aAADYWHSQN7kbrr5yTf3Ss9oee59fmXcJM3fsYbeYdwkzd4scNu8SZurtN3xh3iXM3IVfunzeJczUnW75/fMuYea+ecM18y5h5v7P1z6q8bdGrDAg71qtOgAAYE1YUZtlV+9eU18HUlWPqKqPVdUVVXXWfub9TFV1VS2s5OcBAMDGs7IO8u71s8Siqg5P8qdJ/mmSq5JcWFXnd/dle807Oskzk7xv9lUCALDWrKiD3Gvsfwdw3yRXdPeV3X19klclOWOZeb+b5A+TXLuSnwUAABvTRn6T3jFJPjNsX5XkfuOEqrpPkmO7+39U1bNnWRwAAGvTul5iUVXbkmwbhnZ0946DPPawJP8hyVNWoTQAANapdd1BnsLwvgLxZ5McO2zfcRrb4+gk90hyQVUlyQ8kOb+qHtXdi6tQLgAA68C67iAfwIVJTqiqu2QpGD8xyZP27OzuryW57Z7tqrogya8LxwAAm9u67iDvT3d/u6qenuRNSQ5P8ufdfWlVPS/JYnefP98KAQBYi1YUkHevo4CcJN39hiRv2GvsufuYe9osagIAYG1bWUBeX0ssAABgxVYUkG+4/rM+IxwAgA1tRR8UAgAAG52ADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAAOtAVe2qqp1V9eGq+u9Vdat517RHVV1QVQvzruNQEZABANaHa7r7pO6+R5KvJPnlWV68lmyK7LgpbhIAYIN5T5JjkqSqjq+qN1bVRVX1jqr6kWn8cVO3+eKqevs0dnhV/VFVXVhVl1TV06bxo6rqrVX1gar6UFWdMY0fV1Ufq6q/SPLhJMdW1W9Mcy6uqnOGmh5XVe+vqsur6kGz/GEcalvmXQAAAAevqg5P8tAkL5uGdiT5pe7+eFXdL8mfJXlIkucm+Wfd/dlhOcZTk3ytu0+pqu9J8q6qenOSzyR5THf/Q1XdNsl7q+r86ZgTkvx8d7+3qn4iyRlJ7tfd36qq2wylbenu+1bVTyY5O8nDVu2HsMpuVEBeWFg4K8nWQ1wLc/CeN7563iUAAEmqaluSbcPQju7eMWzfvKp2Zqlz/JEkb6mqo5LcP8l/rao9875n+v6uJOdW1auTvGYae3iSe1XVY6ft781SAL4qyR9U1YOT7J6ucftpzqe7+73T64cleXl3fytJuvsrQ317rnFRkuNWePtryo3tIG9dXFzcfigLYT5uuPrKs+ddAwCQTGF4x36mXNPdJ1XVkUnelKU1yOcm+Wp3n7TM+X5p6ij/8yQXVdXJSSrJM7r7TePcqnpKktslObm7b6iqT+U7zdBvHuQtXDd935V1vkrBGmQAgHVk6t7+SpJfS/KtJJ+sqscl//hGuh+dXh/f3e/r7ucm+VKSY7MUrP9VVR0xzTmxqm6RpU7yF6dwfHqSO+/j8m9J8gtTSM9eSyw2jHWd7gEANqPu/mBVXZLkzCRPTvLiqnpOkiOSvCrJxUn+qKpOyFLX+K3T2CVZWv7wgVpak/GlJI9O8pdJ/ntVfSjJYpKP7uO6b6yqk5IsVtX1Sd6Q5N+u2o3OiYAMALAOdPdRe23/1LD5iGXm//Ryp8lSoF0u1J66j0vfY6/znpPknL3GThteX511vgbZEgsAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABhsmXcBzNdj7/Mr8y5hpv7bB1447xJm7ttvfNm8S5i53R+5fN4lzNTubx417xJm7k3nnTbvEmbqbkd+dd4lzNzhR+yedwlsYjrIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMArANV9QNV9aqq+kRVXVRVb6iqE6vqhKp6/TD+v6rqwdMxT6mqL1XVzqq6rKp+cTjfo6uqq+pH5ndXa5OADACwxlVVJXltkgu6+/juPjnJbya5fZL/kWTHMP6MJHcdDj+vu09KclqSP6iq20/jZyZ55/SdgYAMALD2nZ7khu5+yZ6B7r44yYlJ3tPd5w/jH+7uc/c+QXd/Mcknkty5qo5K8sAkT03yxFWufd3ZMu8COHgLCwtnJdl6KM95TH7gUJ4OALiRqmpbkm3D0I7u3jG9vkeSi5Y57O5JPnCQ579rljrLVyQ5I8kbu/vyqvpyVZ3c3cudf1MSkNeXrYuLi9sP5QnPuNMjzz6U5wMAbpwpDO844MT9qKrXJjkhyeXd/dPT8BOq6oFJrkvytO7+SlWdmeQF0/5XZWmZhYA8EZABANa+S5M8dh/jD96z0d2PqaqFJM8f5pzX3U/fs1FVt0nykCT3rKpOcniSrqpnd3evSvXrjDXIAABr39uSfM+0DCNJUlX3SnJ5kgdU1aOGuUce4FyPTfLK7r5zdx/X3ccm+WSSBx3qotcrARkAYI2bOruPSfKw6XFulyb5d0m+kOSRSX6pqq6sqvckeU6S39vP6c7M0hMxRn8dT7P4R5ZYAACsA939uSSP38fun9zHMecmOXevsdOXmffCm1jehqKDDAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgMGWeRfAfB172C3mXcJMffuNL5t3CTO35RFPnXcJM3ft4jPnXcJM9devnXcJM3dEHz3vEmbq2us33/9d77pWD4/52Xz/xc3JwsLCWUm23sTTHHcISgEAYD8E5NnZuri4uP2mnGBhYeEmHQ8AwIH5+wUAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAwDpQVT9QVa+qqk9U1UVV9YaqOrGqTquq18+7vn2pqttV1fuq6oNV9aBl9t+2qm6oql/azzm2V9Wvr/C6756+H1dVT1rJsQIyAMAaV1WV5LVJLuju47v75CS/meT2863soDw0yYe6+97d/Y5l9j8uyXuTnHkoL9rd959eHpdEQAYA2GBOT3JDd79kz0B3XzwEzqOq6r9V1Uer6i+nQJ2qem5VXVhVH66qHcP4BVX1h1X1/qq6fE9nt6oOr6rnT/MvqapnTOMnV9X/njrXb6qqO+xd4NSpfdt03Fur6k5VdVKSf5/kjKraWVU3X+bezkzya0mOqao7Duf7ram2dyb54WH8gqr6k6parKqPVNUpVfWaqvp4Vf3eMO8b08tzkjxouv6zDuaHLSADAKx990hy0X723zvJv05ytyR3TfKAafw/dfcp3X2PJDdP8sjhmC3dfd/puLOnsW1Z6rie1N33SvKXVXVEkhcleezUuf7zJL+/TA0vSvKKPccleWF370zy3CTndfdJ3X3NeEBVHZvkDt39/iSvTvKEafzkJE9MclKSn0xyyl7Xur67F5K8JMnrkvzy9DN6SlV9315zz0ryjun6f7L8j++7bTmYSawZ1y4sLGw/lCf8sRx/KE8HANxIVbUtSwF1jx3dveMgD39/d181nWdnlkLuO5OcXlX/JsmRSW6T5NIk/3065jXT94um+UnysCQv6e5vJ0l3f6Wq7pGl8PmWqQF9eJLPL1PDqUl+enr9yix1jg/kCVkKxknyqiyF7z9O8qAkr+3ub033dP5ex+3Z/lCSS7v789O8K5Mcm+TLB3HtfRKQ15HFxcVzDvU5n37cE84+8CwAYLVNYXhfgfjSJI/dz+HXDa93JdlSVVuT/FmShe7+TFVtT7J1mWN2Zf+ZsLIUQk/dz5wb68wkP1BVT562f7CqTjiI4/bUvjvffe+7cwjyrSUWAABr39uSfM/UZU6SVNW9lnsqxGBPGL66qo7K/gP2Hm9J8rSq2jJd4zZJPpbkdlV16jR2RFXdfZlj352lZRFJ8uQky70h7x9V1YlJjuruY7r7uO4+Lsm/y1JofnuSR1fVzavq6CQ/dRC178vXkxy9kgMEZACANa67O8ljkjxseszbpVkKk1/YzzFfTfKfk3w4yZuSXHgQl3ppkr9LcklVXZzkSd19fZbC9R9OYzuT3H+ZY5+R5Beq6pIk/yLJMw9wrTOz9GSO0V8nObO7P5DkvCQXJ/mfB1n7vlySZFdVXXywb9KzxAIAYB3o7s8lefwyuz6e5IJh3tOH189J8pxlznXa8PrqTGuQp7XHvzp9jfN3JnnwAer7dJKHLDN+bpJzlxn/nWXGLknyT6bXv59l3gy4V+0X5Lvvfdx31PT9huXq2h8dZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGGyZdwGrYWFh4awkW+ddx16Om3cBy7nFJvsdafdHLp93CTN37eIz513CzG19zgvmXcJMXf+Cs+Zdwsy9Zut18y5hph553VHzLmHmvnp4zbuEmTtl3gXwjzZkQE6ydXFxcfu8ixgtLCxsn3cNAAAc2OZqHwIAwAEIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAA60BV3b6q/ktVXVlVF1XVe6rqMdO+06rqa1W1s6o+WlXPH457SlV9adq35+tuy5z/t6rq0qq6ZJpzv1ne31qyZd4FAACwf1VVSf4mySu6+0nT2J2TPGqY9o7ufmRV3TzJB6vqtd39rmnfed399P2c/9Qkj0xyn+6+rqpum+RmN7HmLd397ZtyjnnRQQYAWPsekuT67n7JnoHu/nR3v2jvid19TZKdSY5ZwfnvkOTq7r5uOsfV3f25JKmqU6rq3VV1cVW9v6qOrqqtVfXyqvpQVX2wqk6f5j6lqs6vqrcleWtV3aKq/nw67oNVdcaN/xHMjg7ynC0sLJyVZOu8rv/QnDCvSwMAg6ralmTbMLSju3dMr++e5AMHeZ5bJzkhyduH4SdU1QOH7VOnIL3Hm5M8t6ouT/K3Weo4/++qulmS85I8obsvrKpbJrkmyTOTdHffs6p+JMmbq+rE6Vz3SXKv7v5KVf1Bkrd197+sqlsleX9V/W13f/Ng7mVeBOT527q4uLh9Xhf/jePOPHte1wYAvmMKwzsOODFJVf1pkgdmqat8yjT8oKq6OEvh+D929xeGQ/a7xKK7v1FVJyd5UJLTk5xXVWcluSjJ57v7wmneP0zXf2CSF01jH62qTyfZE5Df0t1fmV4/PMmjqurXp+2tSe6U5CMHc5/zIiADAKx9lyb5mT0b3f3L0zrhxWHOnjXId0ny3qp6dXfvPNgLdPeuJBckuaCqPpTk57MUkFdq7A5Xkp/p7o/diPPMjTXIAABr39uSbK2qfzWMHbncxO7+ZJJzkvzGwZ68qn64qsZ1lycl+XSSjyW5Q1WdMs07uqq2JHlHkidPYydmqSu8XAh+U5JnTG8yTFXd+2BrmicBGQBgjevuTvLoJD9eVZ+sqvcneUX2HYJfkuTBVXXctP2EvR7zdv+95h+V5BVVdVlVXZLkbkm2d/f1SZ6Q5EXT8o23ZGmZxJ8lOWzqNJ+X5Cl73uC3l99NckSSS6rq0ml7zbPEAgBgHejuzyd54j72XZCl5RF7tq/Jd55ice70tb9zX5Rk79C8Z9+FSX5smV2/sMzc77rWVMfT9nfttUgHGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAIMt8y6A+Xr7DV+YdwkztfubR827hJnrr1877xJm7voXnDXvEmbqZs88Z94lzNyRr9xc/4yP3L173iXM3BFd8y6BTUxAnp1rFxYWti8zftyM6wAAYD8E5BlZXFxctsWzj9AMAMCcWIMMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMALDGVdX3VdXO6esLVfXZYftOVfW6qvp4VX2iql5QVTebjjutqr42zN1ZVQ+b9u3aa/y4af7rD1DLj1XV+6ZjPlJV22fwI5ipLfMuAACA/evuLyc5KUmmQPqN7n5+VVWS9yV5cXefUVWHJ9mR5PeTPHs6/B3d/chlTntNd580DlTVcQdRziuSPL67L56u98M34pa+S1Ud3t27bup5DhUdZACA9eshSa7t7pcnyRQyn5XkX1bVkat0ze9P8vk91+vuy5Kkqo6qqpdX1Yeq6pKq+plp/Mxp7MNV9Yd7TlJV36iqP66qi5OcWlU/W1XvnzrT/98UvudCQAYAWL/unuSicaC7/yHJ3yX5oWnoQXstpTh+Gr/5MPbaFVzzT5J8rKpeW1VPq6qt0/hvJ/lad9+zu++V5G1V9YNJ/jBLQf6kJKdU1aOn+bdI8r7u/tEkX07yhCQPmLrau5I8eQU1HVKWWMzftQsLC9vndfEjcvS8Lg0ADKpqW5Jtw9CO7t5xCE590EssDkZ3P6+q/jLJw5M8KcmZSU5L8rAkTxzm/X1VPTjJBd39pSSZjntwkr/JUgj+62n6Q5OcnOTCpVUjuXmSL660tkNFQJ6zxcXFc+Z5/VOPOf3seV4fAFgyheGVBuLLkjx2HKiqWya5U5Irktz30FT33br7E0leXFX/OcmXqur7bsRprh3WHVeSV3T3bx6yIm8CSywAANavtyY5sqp+Lll6s1uSP05ybnd/azUuWFX/fHpzYJKckKVO8FeTvCXJLw/zbp3k/Ul+vKpuO9V2ZpL/vY/7eGxVff907G2q6s6rUf/BEJABANap7u4kj0nyuKr6eJLLk1yb5N8O0/Zeg/zY5c41eGhVXTV8nbrX/n+RpTXIO5O8MsmTp07w7yW59fRmvIuTnN7dn09yVpL/leTiJBd19+uWuY/LkjwnyZur6pIshe07rOyncehYYgEAsI509/a9tj+T5Kf2MfeCJN+7j31H7WP+zQ9w/SfuY/wbSX5+mfG/SvJXB7p+d5+X5Lz9XXtWdJABAGAgIAMAwEBABgCAwY1dgzzXZ/cehOPmXQAAAOvTjQrI835274Gs8fAOAMAaZokFAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYLBl3gUwXxd+6fJ5lzBTbzrvtHmXMHNH9NHzLmHmXrP1unmXMFNHvvKseZcwcy9cPGfeJczUy+793HmXMHMnXH/DvEtgE9NBBgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEEaZ/l8AACAASURBVJABAGAgIAMAwEBABgCAgYAMAAADARkAYB2oqm8sM/a9VfUXVXVFVX1iev29077jquqaqtpZVZdN+46YfeXrj4AMALB+vSzJld39Q919fJJPJnnpsP8T3X1SknsmuWOSx8+hxnVny7wLAABg5arqh5KcnOQJw/DzklxRVccn2bVnsLt3VdX7kxwz2yrXJx1kAID16W5Jdnb3dwXhJDuT3H2cWFVbk9wvyRtnWuE6tWk7yAsLC2cl2TrvOgAAkqSqtiXZNgzt6O4dN/G0x1fVziR3SfI/uvuSm3i+TWHTBuQkWxcXF7fPu4h523KzY86edw0AQDKF4ZUE4suSnFRVh3X37iSpqsOSnDTtS6Y1yFV12yTvqqpHdff5h7TwDcgSCwCAdai7r0jywSTPGYafk+QD075x7tVJzkrym7OrcP0SkAEA1ocjq+qq4etXkzw1yYnTI94+keTEaWw5fzOd40GzKni92sxLLAAA1o3u3ldj82f3Mf9TSe4xbHeSHz30lW08OsgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYbJl3AczXnW75/fMuYabuduRX513CzF17/eb7z/yR1x017xJm6sjdu+ddwsy97N7PnXcJM/XUDz5v3iXM3Ocevm3eJbCJ6SADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADDYMu8CVsm1CwsL2w8w57gZ1AEAwDqzIQPy4uLiOQeacxABGgCATcgSCwAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAAGtcVR1XVR/ea2x7Vf36AY5bqKoXrnJt7x5qfNIKjz2tqr5WVTur6qNV9fyDOObRVXW3Yft5VfWwlVe+bwIyAMAG1d2L3f0rN/U8VbVlP9e4//TyuCQrCsiTd3T3SUnuneSRVfWAA8x/dJJ/DMjd/dzu/tsbcd19EpABANa5qrqgqv6wqt5fVZdX1YOm8dOq6vVVdVhVfaqqbjUc8/Gqun1V3a6q/rqqLpy+HjDt315Vr6yqdyV5ZVXdfTr/zqq6pKpOmOZ9YzrlOUkeNO1/VlW9vapOGq73zqr60X3dQ3dfk2RnkmOm+b841XPxVN+RVXX/JI9K8kfTdY6vqnOr6rHTMZ+qqt+pqg9U1Yeq6kem8dtV1Vuq6tKqemlVfbqqbruvWgRkAICNYUt33zfJv05y9riju3cneV2SxyRJVd0vyae7+/8keUGSP+nuU5L8TJKXDofeLcnDuvvMJL+U5AVTt3chyVV7Xf+sTN3g7v6TJC9L8pTpeicm2drdF++r+Kq6dZITkrx9GnpNd5/S3T+a5CNJntrd705yfpJnT9f5xDKnurq775PkxUn2LEE5O8nbuvvuSf5bkjvtq44k2We7nJVZWFg4K8nWedcBAKxPVbUtybZhaEd375he9z4OG8dfM32/KEvLHfZ2XpLnJnl5kidO20nysCR3q6o9825ZVUdNr8+fOrtJ8p4kv1VVd8xSeP34AW7pvyb57ap6dpJ/meTcfcx7UFVdnKVw/B+7+wvT+D2q6veS3CrJUUnedIDr7TH+HH56ev3ATL8cdPcbq+rv93cCAfnQ2bq4uLh93kWs1F1ve++zDzwLAFhtUxjesY/dX05y673GbpPkk8P2ddP3XVk+470nyQ9V1e2ytI7396bxw5L8WHdfO06eAvM3h/r+S1W9L8k/T/KGqnpad79tP/fzrap6S5Izkjw+ycn7mPqO7n5kVd0lyXur6tXdvTNLgfrR3X1xVT0lyWn7utZeDvRzOCBLLAAA1rju/kaSz1fVQ5Kkqm6T5BFJ3rmCc3SS1yb5D0k+0t1fnna9Ockz9swb1w2PququSa7s7hdmabnGvfaa8vUkR+819tIkL0xyYXfvt2vb3Z/M0jrm35iGjs7SPR+R5MkHuM6BvCtLIT1V9fD8379sfBcBGQBgffi5LC1Z2JnkbUl+Zx9rcPfnvCQ/m+8sr0iSX0myML3x7rIsrTVezuOTfHi6/j2S/MVe+y9Jsmt6U92zkqS7L0ryD1la1nEwXpLkwVV1XJLfTvK+LIXbjw5zXpXk2VX1wao6/iDP+ztJHj49Ku9xSb6QpaC9LEssAADWge6+LMnp+9h32vD66kxrkLv7giQXDPsWk9Rex16d5AnLnHP7XtvnZKnDu/e8o6bvNyR5yLivqn4wSw3ZN++j7r3ruybTUyyy9Ca7Fy9zzLsyPOYt0xsBp33HDa8X851lGV9L8s+6+9tVdWqSU7r7uuyDgAwAwCFXVT+X5PeT/Or0FI15ulOSV1fVYUmuT/KL+5ssIAMAcMh191/k/16GMRfTEzfufbDzrUEGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYLBl3gXM0bULCwvbD+H5jjuE55qZb95wzbxLmKnDj9g97xJmbte1m+/34K8eXvMuYaaO6M11v0lywvU3zLuEmfrcw7fNu4SZ+8E375h3CWximzYgLy4unnMoz3eIwzYAAHOy+VpLAACwHwIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAMA6UFW/VVWXVtUlVbWzqu5XVTerqv9YVVdU1cer6nVVdcfhmF3T3A9X1X+tqiPneQ/rhYAMALDGVdWpSR6Z5D7dfa8kD0vymSR/kOToJD/c3Sck+Zskr6mqmg69prtP6u57JLk+yS/Nvvr1R0AGAFj77pDk6u6+Lkm6++okX03yC0me1d27pvGXJ7kuyUOWOcc7kvzQbMpd3wRkAIC1781Jjq2qy6vqz6rqx7MUdv+uu/9hr7mLSe4+DlTVliQ/keRDM6l2ndsy7wI2kGsXFha2z7sIAGB9qqptSbYNQzu6e0eSdPc3qurkJA9KcnqS87K0vOJAbl5VO6fX70jyskNY8oYlIB8ii4uL58y7hhvj9t/7I2fPuwYAIJnC8I797N+V5IIkF1TVh5I8Lcmdquro7v76MPXkJK+fXl/T3SetUskbliUWAABrXFX9cFWdMAydlORjSV6R5D9U1eHTvJ9LcmSSt82+yo1DBxkAYO07KsmLqupWSb6d5IosLcf4epLnJ7m8qnYn+WiSx3R3z63SDUBABgBY47r7oiT338fuZ0xfyx131KoVtYFZYgEAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAIPq7nnXwCZVVdu6e8e865iVzXa/yea75812v8nmu+fNdr/J5rvnzXa/LE8HmXnaNu8CZmyz3W+y+e55s91vsvnuebPdb7L57nmz3S/LEJABAGAgIAMAwEBAZp422xqvzXa/yea75812v8nmu+fNdr/J5rvnzXa/LMOb9AAAYKCDDAAAAwEZAAAGAjLATVBVN6+qH553HbNSS362qp47bd+pqu4777o4dKrqNst8HTHvumCWBGSYoap6YFX96bzr4NCoqp9KsjPJG6ftk6rq/PlWter+LMmpSc6ctr+exL/TG8sHknwpyeVJPj69/lRVfaCqTp5rZYdYVW2tqn9dVf+pqp5WVVvmXRNrg4DMTFXVA6rqLVV1eVVdWVWfrKor513Xaqqqe1fVH1XVp5L8bpKPzrkkDp3tSe6b5KtJ0t07k9xlngXNwP26+5eTXJsk3f33SW4235I4xN6S5Ce7+7bd/X1JfiLJ65P8v1n6BWkjeUWShSQfytJ9/vF8y2Gt8JsSs/ayJM9KclGSXXOuZdVU1YlZ6rCdmeTqJOdl6akxp8+1sFVWVc/s7hccaGwDuaG7v1ZV49hGfzTQDVV1eKb7rKrbJdk935Jmq6oe0N3vmncdq+jHuvsX92x095ur6vnd/bSq+p55FrYK7tbd90ySqnpZkvfPuR7WCB1kZu1r3f0/u/uL3f3lPV/zLmoVfDTJQ5I8srsf2N0vygb+hWDw88uMPWXWRczQpVX1pCSHV9UJVfWiJO+ed1Gr7IVJXpvk+6vq95O8M8kfzLekQ6+qDq+qM6vq16vqHtPYI6vq3Un+05zLW22fr6rfqKo7T1//Jsn/mX4x2mi/DN2w50V3f3uehbC2eA4yM1FV95lePj7J4Ulek+S6Pfu7+wPzqGu1VNWjkzwxyQOytD71VUle2t0b8s/vVXVmkicleWCSdwy7jk6yu7sfOpfCVllVHZnkt5I8fBp6U5Lf6+5r51fV6qmqw5L8WJKvJHlokkry1u7+yFwLWwVVdW6SY7PUUbxfks9l6U/xZ3X338yxtFVXVbdNcnaW/ntOkncl+Z0kX0typ+6+Yl61HWpVtSvJN7P073KS3DzJt6bt7u5bzqs25ktAZiaq6n/tZ3d390NmVswMVdUtkpyRpaUWD0nyF0le291vnmthh1hV3TlLa2//XZKzhl1fT3LJRuzMTN20v93oy2b2VlUf7O57z7uO1VZVH05yr+7eXVVbk3whyfEb9C9ewF4EZFgl09rMOye5oru/WlW3TvK4JE/YqB3Vzaaq3prkp7v7a/OuZVaq6vlJ3pPkNb2B/w+kqj7Q3ffZ1/ZGNr2H4teTHJfhvUobtZGRJFV1epK7T5sf7u4L5lgOa4CAzExV1R8k+ffd/dVp+9ZJfq27nzPfyg6tqvp/srQu8xNZ6qxu6+4N+/ivqnpndz+wqr6e736T2ob+M2VVvS7JvbP0rv9v7hnv7l+ZW1GrbPpnfIssranfs5Rkw/0zrqpvJdmzlKCSHD9t7/l3+l7zqm21VdXFSV6Svd5M3d0Xza2oVVJVx2Rpyd+1WbrfJDk5S0stHtPdn51XbcyXgMxMLffn2Y3YmZn+PHt6d3+pqu6a5C+7+9R518WhVVXLvSkx3f2KWdfCoTUtG9qn7v70rGqZtaq6qLs31POO96WqXpvkdd197l7jP5fkZ7r7jLkUxtwJyMxUVV2S5JTuvm7avnmSxe6++/6PXF82+Z9nb52lNzeNf5rdUG/CHFXVzZKcOG1+rLtv2N/8jaCqHpXkwdPmBd39+nnWs5qq6i75zp/eL+vuDf3c9iSpqu1Jvpilp5WMb6b+yrxqWi1V9bHuXvaTMPe3j43Pc5CZtb9M8taqevm0/QtZelD7RnPHqnrhvrY36p/gq+p3s/RYtyvzncdBdZbeoLjhVNVpWfr391NZ+tP7sVX189399nnWtZqq6pwkp2Tpv+Ukeeb0XODfnGNZh1xV3TLJS7P05Iqd0/BJVXVRkqd29z/MrbjVt+cvI88exjrJXedQy2pb9nG30xNbDp9xLawhOsjMTC19msIds9SNedg0/JbuftP8qlod+/rT+x4b9U/wVfWxJPfs7uvnXcssTGHpSd39sWn7xCR/tZH/PD39Feik7t49bR+e5IMbbU3u9Ji3TyV53nCvleS3k/xQd//c/Krj/2/vzoPtquosjn+XYYYgohBAgSYgKoYZlFkFu+1qEEVRQBxaBe1qWxG6tBQHaLQbS2RoZ2nUJog4MBmbxhGaWaYwQwQRMTihKIiMGlf/sc81l+fLRM49m5y7PlWvcs8+pGqlwsv73X1/57fbIukESk/9O20/0KytChwPPNzXzYxYtOwgR2dsW9L/NqcWfat2nlHqawG8GG4E1qB8PDsOlh8UxwC2b5W0fM1AHVmDMgsZ4Mk1g4zQzrb/cXihmdpxlKTb6kQaLUm72z5P0ismu2/7zK4zdeBdlAeq75Q06CvfgPLJ0OHVUkV1KZCja7MlbW/7ytpBRknSLsB02zOb69OBNZvbH7Z9XrVwo3U0cE3zkOJw7+Le9SKN1FWSTgK+1FwfCFxVMU8XBn/H51PaSnbjsbOvx4EW/Z8sk14AnAe8dJJ7pkx76JutgOOADwKbAC+k/PlXAFZj/hvBGDNpsYhOSZpD+UfoTuafXtS7kUnNfNy32765ub6B0pu7KnC47b+vGG9kJN0EfA64gaEjaW1fUC3UCElaEXgb808cuwj49OAh1L6StC6lDxngCtu/rJlnFCSdTBnT+KHhec+SPgBsavt11cKNmKSNbN+xqLU+kDQbeLHt30rajXLq6dsphfNzbO9bNWBUkwI5OrWg0Ul9G5kk6Urb2w9dn2n7Fc3rS2zvXC/d6Ez8c/dd06v4sO15zfUUYEXbD9ZNNjqS9gHOGxyOImkN4IV9O365eUjv88A2DD2kB1xDeUivt4fDTDZ1p6+j3yRdZ3vL5vWngF/bPrK5vtb2VjXzRT1psYhODQphSWsDK1WOM0prDF8MiuPGtI6zdOkiSUcDs3hsi0Vfx7x9n/LA6R+a65WB7wA7VUs0ekfYPmtw0ZwSeQTQqwK5mVLxKkkbA5s1yzfbvl3SO4ET6qUbDUnPpjxE/eQJfcir099/r6dIWs72n4A9gLcM3UuNNMbylx+dauanHgusR3mQa0PgFubPGe2LOZL2tH3O8KKkvYAfLuD39MHgEJgdhtZ6O+YNWMn2oDjG9h8krVIzUAcmG4vV258ltm+ntFoMO4weFsjAs4C9KG/wh/uQ7wcOrpJo9E4DLpD0G+AhSpsUkjYBevspQSxaWiyiU80RprsD37O9taQXAa+1/ebK0VrV/ON6DnApMNg93Zays7iX7VtrZYv2SLqE0ms+u7neFvhkn09NlPQF4F7gU83SvwBPmTjxoc8kzbW9fu0coyJpR9uX1c7RFUk7AOsC3xka9bYpsFqPP/2KRUiBHJ2SdJXt7ZpCeWvbfx7uAeuLpkBeh3LC2mB3/CbgVuAXza5Ub0g6bGH3bR/XVZYuSdqe8lDPzykPnK4D7Gf76qrBRqjpu/4AQ7PMKZNZHqiXqluSfmp7g9o5RkXSSsCbKf92/aW1wvabqoWK6FhvPxaLJ6x7Ja1G+RjrVEl3U6ZZ9M0JwHttf2F4UdLmzb3Jxigty6Y2vz6LMt1gVnP9UuCKKok6YPvKpm9zcBxt74+abgrh98BfjhW/1z3caZF0P6U96K9uUXrN++wUYA7wEuAoyvjCW6omiuhYdpCjU83u00OUPsYDKYcMnGr7nqrBWrawaQ6SbmgOS+kdSRcCe9q+v7meCpxje7e6ydrV7BzPHYw3k/R64JWU8YVH2u7d7FRJHwS+ZntOM97uXGBLYB7lNMHvVQ0YrZF0TdMCd73tLZrDby6yvcMif3NET0x6BnnEqDS7T+tTxkKdDJwE9PFY4jUWcq/Pu0/TeOzf56P0c2rH52j+nM3s1I8AMykP9ZxYMdco7cf8B0zfQPn5sTblcIn/qBUqRmLwKci9kmZQNjLWrpgnonNpsYhOSTqYMkZnTWBj4OnAZynjdfrkKkkH2/6v4UVJBwG97U+lFIlXSBqMAXs55cjWvpkytEu8H3Ci7TOAMyRdu5Dftyx7dKiV4iXAac3851sk5WdJv5zYtM+8n9IutRql7zxibKTFIjrVFA/PAy63vXWz1ruWA0nTgLMou4yDgng7yvGl+/Tx5LEBSdsAuzaXF9q+pmaeUWiO0t7K9p+a0yHfYvvCwT3bM+ombJ+kHwAHAb+i7CRvOzhZTdIc28+umS9GS9IGtn9aO0dEV/KuP7r2iO1HJQHQ7Dz17l2a7V8BOzVj7AbF0jm2z6sYqxPNWKTZzQELr5H0Jdt9m3M9jrNTDwFOB9YCjh8qjv+Bcrpc9ICkHSmf7F1o+25JW1AeytyV0h4XMRaygxydkvRRygzV11POu/9nyulU76saLFohaT1Ky8FrgM2Bo4Ezbd9QNdgIZHZq9I2kYygHhVwLbAJ8m/KpwdHA52w/XDFeRKdSIEenJD2JMl/z7yjjkr4NnNTHMVHjRNJbgAMoO09fa76+YXujqsEiYrFJuhnYxvbDTQ/yXGCG7Z/UTRbRvRTIUY2kNYFn2L6+dpZYOpIeBS4D/tX2Vc3aj21Pr5ssIhaXpNm2txm6vmbwrEjEuEkPcnRK0v8Be1P+37sauFvSpbYPrRoslta6wKuAYyWtQ9lBXr5upOiKpOfbvrx2jlhq0yXNGrreaPja9t4VMkVUkR3k6NTQAPqDgPVtHzEYRl87W7RD0jMofcgHAKsCZ9k+vG6qdkl6tu05zesVbT8ydG8H2z+ol657fT96eVxIesHC7tu+oKssEbVlBzm6tpykdYFXA3kwr4ds3wUcS9lN3hTYv3KkUfgyMPgo+rKh1wCfnnA9DlQ7QCy9FMAR8+UkvejaUZQH8263faWk6cBtlTPFiNi+1fZRtXOMgBbwerLrcZCPIiOiV7KDHJ2y/XXg60PXPwZeWS9RxOPiBbye7LoXJH2Tyf9sAp7acZyIiJFKD3J0qvnI/TPANNszmiH0e9v+cOVosRQk7Wz7kon9uH0l6W7gK5TicL/mNc31q21Pq5VtVNKfOj4kbd7H2eURSyIFcnRK0gXAuyhD5wdHTffyaN5xIulq29tOHBPVV5LesLD7tk/uKktXctTw+JB0EbAi8N/Aqbb7ejpkxAKlxSK6tortKwZHTTf+VCtMtOaPkk4Eni7p4xNv2n5HhUwjs7ACWFJfpzmcTfPwoaQzbKc1qqds7yrpmcCbgKslXQF80fZ3K0eL6EwK5OjabyRtTNPLKGlf4Bd1I0UL9gJeDLyEMt+69yTtSDk58ELbdzftQu8BdgXWrxpuNIbf1eYAmJ6zfZuk9wNXAR8HtlbZ2Tjc9pl100WMXlosolPN1IoTgZ2A3wF3AAfavrNqsGiFpC1tX1c7x6hJOobypuBaYBPKZJaDgKMp7UMPV4w3EsPtM+PSSjOumjd7bwT2BL4LfN72bEnrAZfZ3rBqwIgOpECOKiStShkz+CCwv+1TK0eKFjSHhHwC2LlZugg4pJmN3BuSbga2sf2wpKcAc4EZtn9SN9noSJoHPEDZSV6Z8r1Lc23bq9fKFu1qnhU5CTjd9kMT7r3O9il1kkV0J3OQoxOSVpf0XkmflPS3lB+ubwB+RDk0JPrhi8AsYL3m65vNWt88PNgltv074LY+F8cAtqfYXt32VNvLNa8H1ymO++Us26cMF8eSDgFIcRzjIjvI0QlJ36C0VFwG7AGsTdl5OsT2tTWzRXskXWd7ywlr19reqlamUZB0L3Dh0NJuw9e29+481IhJ2t32ec3rjWzfMXTvFelL7Y/JWmgkXTOYPBQxDlIgRyck3WB78+b1FMqDeRv0sVdznEn6PmXH+LRm6QDgjbb3qJeqfeM4E3hhPcjpSe4HSQcArwF2obRHDUwF/ty37+OIhckUi+jKHwcvbM+TdFeK4156E6UH+XjKpJJLKQ/79EofC+DFkOO1++9SyubF04Bjh9bvB66vkiiikhTI0ZUtJf2+eS1g5eY6D/j0SDONpHftBRNJehnwDNufaq4vB9Zqbr/b9unVwo3O2B2vPW6a7987gR1rZ4moLQVydML2lNoZIlr0bmD/oesVge2BVSktJn0skKdLmkV5Uzt4TXO9Ub1Y0RZJF9veRdL9PPZNTzYyYuykQI6IWHIr2J47dH2x7XuAe5oRhn30sqHXH5twb+J1LINs79L8OrV2loja8pBeRMQSkvQj25ss4N7ttjfuOlNXJK1EORwF4Ed5lqB/mtNO77L9iKQXAlsAM23fWzdZRHcyBzkiWiNpmqTPSzq3ud5M0ptr5xqByyUdPHFR0luBKyrkGTlJy0n6KHAXcDIwE5gr6aOSlq+bLlp2BjBP0iaUk0/XB75cN1JEt7KDHBGtaQrjLwLvs72lpOWAawYj/vpC0trA2cAjwOxmeVtKL/LLbf+qVrZRkXQ8ZdzXobbvb9ZWp7RXPGT7kJr5oj2DsX2S3kU5FOcTmYMc4yYFckS0RtKVtrcf/mHax4NCBiTtDjy3ubxpcJBGH0m6DdjUE35oNHPN59h+Zp1k0bZmKssJwPuAl9q+Q9KNtmdUjhbRmTykFxFtekDSU2megJe0A3Bf3Uij0xTEvS2KJ/DE4rhZnCcpOy398kbgn4B/b4rjjYAcMR1jJTvIEdEaSdtQDgqZAdxImQ28r+0cMrCMk3Q2cKbtmRPWXwu8uo/Ha4+j5hOBmbYPrJ0loqYUyBHRqqbv+FmU2ak/tP3HRfyWWAZIWp8y3/kh4OpmeTtgZWAf2z+rlS3aJeliYHfbj9bOElFLCuSIaI2ktwGnDsZBSXoKcIDtT9dNFktr6MGtPYDNmuWbbX+/Zq5on6SZwHOAWcADg3Xbx1ULFdGxFMgR0ZrJHsjL0+/9kL/H8SHpiMnWbf9b11kiaslDehHRpimSNHiYq+lnXKFypmjHWpIOW9DN7C72x6AQlrSK7Qdr54moIQeFRESbvgV8VdIezUfxpzVrseybAqxGmYU82Vf0hKQdJd0MzGmut5SUNqkYK2mxiIjWSHoS8FZgj2bpu8BJtufVSxVtGPQg184Ro9fMQd4XmDU0zzxzkGOspMUiIlpj+8/AZ5qv6BfVDhDdsT1Xesxfed7kxlhJgRwRrZG0M3AksCHl3xdRDpiYLBsULgAAA19JREFUXjNXtGKPRf8n0RNzJe0EWNLywCHALZUzRXQqLRYR0RpJc4BDKXNy/7LjZPueaqEiYolIehrwn8CLKW9yvwMcku/jGCcpkCOiNZIut/382jki4vGTtJbtX9fOEVFTCuSIaI2kj1CmHZwJPDJYtz27WqiIWCKSbgV+AnwVOGNw8E/EOEmBHBGtkXT+JMu2vXvnYSLicZP0PGB/4OXAzcBXbH+pbqqI7qRAjoiIiEk1/cjHAQfanlI7T0RXMsUiIlolaU/gucBKgzXbR9VLFBFLQtLqwD6UHeSNgbOA51UNFdGxFMgR0RpJnwVWAV4EnEQ5bOCKqqEiYkldB5wNHGX7stphImpIi0VEtEbS9ba3GPp1NeBc27vWzhYRi0eSnOIgxlx2kCOiTQ81vz4oaT3gHmDdinkiYjFJOsH2O4FZkv6qQLa9d4VYEVWkQI6INv2PpDWAY4DZgCmtFhHxxHdK8+vHqqaIeAJIi0VEjISkFYGVbN9XO0tERMSSSIEcEa2StBPwNwx9QmV7ZrVAEbFEJO0MHAlsSPk+FmWe+fSauSK6lAI5Iloj6RTKWKhrgXnNsm2/o16qiFgSkuYAhwJXM//7GNv3VAsV0bH0IEdEm7YDNssT8BHLtPtsn1s7RERNKZAjok03AusAv6gdJCIet/MlHQOcCTwyWLQ9u16kiG6lxSIilpqkb1ImVkwFtqIcDjL8gzXjoSKWEZLOn2TZtnfvPExEJSmQI2KpSXrBwu7bvqCrLBEREUsrLRYR0YafAdNsXzK8KGkX0m4RsUyQdNiEJQO/AS62fUeFSBHVPKl2gIjohROA30+yfl9zLyKe+KZO+Fqd8uDtuZL2rxksomtpsYiIpSbpStvbL+DeDbY37zpTRLRD0prA92xvUztLRFeygxwRbVhjIfdW7ixFRLTO9m8ph4VEjI0UyBHRhqskHTxxUdJBlMMGImIZJelFwO9q54joUlosImKpSZoGnAU8yvyCeDtgBWAf27+slS0iFo+kGygP5g1bE/g58Hrbc7pPFVFHCuSIaE2z0zSjubzJ9nk180TE4pO04YQlA/fYfqBGnoiaUiBHRERERAxJD3JERERExJAUyBERERERQ1IgR0REREQMSYEcERERETEkBXJERERExJD/B5dvtr3ekREXAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "tDzaJx_AJoSZ" }, "source": [ "# перемешка\n", "df = df.sample(frac=1).reset_index(drop=True)\n", "# train test split\n", "df_train = df[:400]\n", "df_test = df[400:]\n", "# среднее и стандартное отклонение\n", "mean = df.mean(axis=0)\n", "std = df.std(axis=0)\n", "# 0 мат ожидание и 1 дисперсию\n", "df_train = (df_train - mean)/std\n", "\n", "X_train = df_train.drop(columns=['Chance of Admit ']).values\n", "y_train = df_train['Chance of Admit '].values\n", "df_test = (df_test - mean)/std\n", "X_test = df_test.drop(columns=['Chance of Admit ']).values\n", "y_test = df_test['Chance of Admit '].values" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "qvctLNtwKBzh", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "f9a2acac-6e2f-4072-c09e-e16510b6def6" }, "source": [ "X_train[:5]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([[-0.21885503, -0.3604156 , -0.09969289, 0.12714383, -1.06326701,\n", " -0.45706705, -1.12702343],\n", " [ 1.55181671, 1.94150887, 1.64930524, 1.13622188, 1.63812275,\n", " 1.59315411, 0.88551841],\n", " [ 1.02061519, 0.46170028, 0.77480617, 0.63168286, -1.06326701,\n", " 0.73338395, 0.88551841],\n", " [-0.21885503, 0.2972771 , 0.77480617, 0.12714383, 0.55756685,\n", " 0.32003291, 0.88551841],\n", " [ 1.19768236, 1.11939299, 0.77480617, 1.13622188, -0.52298906,\n", " 0.98139457, 0.88551841]])" ] }, "metadata": { "tags": [] }, "execution_count": 11 } ] }, { "cell_type": "code", "metadata": { "id": "aKV4jwOhKDpO", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "e186abb7-ee75-461d-c93b-29fbcfaabfe3" }, "source": [ "y_train[:5]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([-1.28765396, 1.5464034 , 0.62533476, 0.69618619, 0.97959192])" ] }, "metadata": { "tags": [] }, "execution_count": 12 } ] }, { "cell_type": "markdown", "metadata": { "id": "S8PPlFzDXSlU" }, "source": [ "## Создание алгоритма" ] }, { "cell_type": "code", "metadata": { "id": "yG3D8nw8JC2B", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "bb0e5293-10aa-4336-b5e6-78456631cc00" }, "source": [ "print('размерность пространства признаков:', X_train.shape[1])\n", "# инициализация весов под размерность пространства признаков\n", "w = np.ones(X_train.shape[1])\n", "# выбираем случайный индекс, по которому найдем частную производную\n", "ind = np.random.randint(X_train.shape[1])\n", "print('Случайный индекс', ind)\n", "# переможим выбранный столбец на нужный вес (получаем кол-во элементов train)\n", "len(np.dot(X_train[:,ind], w[ind]))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "размерность пространства признаков: 7\n", "Случайный индекс 5\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "400" ] }, "metadata": { "tags": [] }, "execution_count": 13 } ] }, { "cell_type": "markdown", "metadata": { "id": "tfobTgP5LHtZ" }, "source": [ "$$MSE = \\frac{1}{n}\\sum_{i = 1}^{n}(y_i - \\hat{y_i})^2$$\n", "\n", "Реализуем ошибку по формуле:" ] }, { "cell_type": "code", "metadata": { "id": "QWlm6k1ULb0X" }, "source": [ "mse = lambda y, y_pred:((y-y_pred)**2).sum()/len(y_pred) " ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "KTe3AIgPQ6Vc" }, "source": [ "# очистка экрана\n", "from google.colab import output\n", "# коэффициент детерминации\n", "from sklearn.metrics import r2_score" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "VAfKZM45XyT2" }, "source": [ "Стохастический градиентный спуск (англ. stochastic gradient descent, SGD) − оптимизационный алгоритм, отличающийся от обычного градиентного спуска тем, что градиент оптимизируемой функции считается на каждом шаге не как сумма градиентов от каждого элемента выборки, а как градиент от одного, случайно выбранного элемента." ] }, { "cell_type": "markdown", "metadata": { "id": "0QVx-8CgXc_h" }, "source": [ "![dssmall](https://github.com/m9psy/neural_network_habr_guide/blob/master/Part%203/images/stochastic.gif?raw=true)" ] }, { "cell_type": "markdown", "metadata": { "id": "7LMlKFjzMgdi" }, "source": [ "$$w_{t+1} = w_{t} - \\frac{2\\alpha}{n}X_i( - y_i) $$\n", "\n", "Реализуем шаг по формуле" ] }, { "cell_type": "code", "metadata": { "id": "x1JnXq6a_HRH" }, "source": [ "gradient_step_stah = lambda X, y, w, alpha, ind: w - (alpha* 2.0 / X.shape[0]) * X[ind] * (np.dot(X[ind], w) - y[ind])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "uz-WNufqNIpJ", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ed21396e-204c-4038-9f99-66e390a2aa91" }, "source": [ "# выбрали случайный индекс\n", "ind = np.random.randint(X_train.shape[1])\n", "# сделали один шаг (w = [1, 1, ...])\n", "gradient_step_stah(X_train, y_train, np.ones(X_train.shape[1]), 1e-4, ind)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([0.99999378, 0.99999473, 0.99999747, 0.99999958, 0.99999642,\n", " 0.99999523, 0.99999711])" ] }, "metadata": { "tags": [] }, "execution_count": 17 } ] }, { "cell_type": "code", "metadata": { "id": "NAK0BcE6gpnP", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "acf2591d-c4dd-48be-ed08-c50af935a8cb" }, "source": [ "X_train.shape[1]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "7" ] }, "metadata": { "tags": [] }, "execution_count": 14 } ] }, { "cell_type": "markdown", "metadata": { "id": "6jG4O7DRgStd" }, "source": [ "## Обучение алгоритма" ] }, { "cell_type": "code", "metadata": { "id": "wYMc6vvtJQVY", "colab": { "base_uri": "https://localhost:8080/", "height": 385 }, "outputId": "f2c98f38-957e-4380-db84-15041b6a9177" }, "source": [ "# стохастический градиентный спуск\n", "def sgd(X, y, w, alpha = 1e-4, max_it = 10e6):\n", " # номер итерации\n", " iter_num = 0\n", " # ошибки на трейне\n", " errors = []\n", " # ошибки на тесте\n", " errors_test = []\n", " # r2 на тесте\n", " r2 = []\n", " while (iter_num < max_it):\n", " # выбираем случайный элемент\n", " ind = np.random.randint(X.shape[0])\n", " # обновляем веса град спуском\n", " w = gradient_step_stah(X, y, w, alpha, ind)\n", " # отображаем каждый %\n", " if iter_num%(int(max_it/100))==0:\n", " # очищаем экран\n", " output.clear()\n", " print('Выполнено:', int(iter_num/max_it * 100), '%')\n", " # mse train\n", " error = mse(y_train,np.dot(X_train, w))\n", " errors.append(error)\n", " print('Mse train:', error)\n", " # mse test\n", " error = mse(y_test,np.dot(X_test, w))\n", " errors_test.append(error)\n", " print('Mse test:', error)\n", " # r2 test\n", " R = r2_score(y_test,np.dot(X_test, w))\n", " r2.append(R)\n", " print('R2:', R)\n", " iter_num += 1\n", "\n", " return w, errors, errors_test, r2\n", "\n", "w, mse_train, mse_test, r2 = sgd(X_train, y_train, np.ones(X_train.shape[1]))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Выполнено: 23 %\n", "Mse train: 0.22755895661893744\n", "Mse test: 0.17505321808539098\n", "R2: 0.7868441725625581\n" ], "name": "stdout" }, { "output_type": "error", "ename": "KeyboardInterrupt", "evalue": "ignored", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 35\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrors_test\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mr2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 36\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 37\u001b[0;31m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmse_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmse_test\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mr2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msgd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mones\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;32m\u001b[0m in \u001b[0;36msgd\u001b[0;34m(X, y, w, alpha, max_it)\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0miter_num\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0mmax_it\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0;31m# выбираем случайный элемент\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mind\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 14\u001b[0m \u001b[0;31m# обновляем веса град спуском\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 15\u001b[0m \u001b[0mw\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgradient_step_stah\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mind\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mKeyboardInterrupt\u001b[0m: " ] } ] }, { "cell_type": "code", "metadata": { "id": "oBcedhprSn3t", "colab": { "base_uri": "https://localhost:8080/", "height": 609 }, "outputId": "7fdf8871-321f-4aa4-8c9f-ce5e178e7ae1" }, "source": [ "from matplotlib.pyplot import figure\n", "\n", "plt.figure(figsize=(15,10))\n", "plt.grid()\n", "\n", "plt.plot(mse_train, label = 'train')\n", "plt.plot(mse_test, label = 'test')\n", "plt.legend()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 16 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAI/CAYAAADz4aFLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde5hkZ2Ee+Pervlf3TPfMSBrdEJK5CIQAYSZgLAijYAzYTmK8WTvxZb2Jn4jddTB2bD8mXpMszj4xm4tj42sIEBNnsWAhDo6NbbCtCbYgJpKMscRFQiCQhGYkzb3vt7N/dA309PTMdE931anu/v2ep5+uOnXOqbdbn0bz6vvqnFJVVQAAAKhXo+4AAAAAKGcAAABdQTkDAADoAsoZAABAF1DOAAAAuoByBgAA0AV6O/lml112WXX99dd38i3XZGJiIsPDw3XHYJszzmg3Y4xOMM7oBOOMTqhrnN1zzz1PVVV1+WqvdbScXX/99bn77rs7+ZZrcujQoRw8eLDuGGxzxhntZozRCcYZnWCc0Ql1jbNSypfP95pljQAAAF1AOQMAAOgCyhkAAEAX6OhnzgAAgJ1tbm4ujz76aKanp2vNMTo6ms9+9rNtO//g4GCuvfba9PX1rfkY5QwAAOiYRx99NLt27cr111+fUkptOU6fPp1du3a15dxVVeXo0aN59NFHc8MNN6z5OMsaAQCAjpmens6+fftqLWbtVkrJvn371j07qJwBAAAdtZ2L2RmX8jMqZwAAwI5x4sSJ/Oqv/uq6j/u2b/u2nDhxog2Jvk45AwAAdozzlbP5+fkLHvfhD384Y2Nj7YqVxAVBAACAHeTNb35zHnroodx6660ZGBjI4OBg9uzZk8997nN54IEH8p3f+Z155JFHMj09nTe96U25/fbbkyTXX3997r777oyPj+d1r3tdXv7yl+fjH/94rrnmmnzoQx/K0NDQhrOZOQMAAHaMt73tbXnGM56Ru+66K//qX/2r3HvvvfnFX/zFPPDAA0mSd7/73bnnnnty99135+1vf3uOHj16zjkefPDB/PAP/3Duv//+jI2N5YMf/OCmZDNzBgAA1OKt//X+fOarpzb1nDddvTv/7G8+b837v+QlLznrcvdvf/vb89u//dtJkkceeSQPPvhg9u3bd9YxN9xwQ2655ZYkyYtf/OI8/PDDGw8e5QwAANjBhoeHv/b40KFD+aM/+qN84hOfSLPZzMGDB1e9HP7AwMDXHvf09GRqampTsihnAABALdYzw7VZdu3aldOnT6/62smTJ7Nnz540m8187nOfy3//7/+9o9mUMwAAYMfYt29fbr311rz0pS/N8PBw9u/f/7XXXvva1+bXf/3X89znPjc33nhjvumbvqmj2ZQzAABgR3nve9+b06dPZ9euXWdtHxgYyO///u+vesyZz5Vddtllue+++762/Sd+4ic2LZerNQIAAHQB5QwAAKALKGcAAABdQDkDAADoAsoZAABAF1DOAAAAusCOL2c/9r5P5d9/eqbuGAAAQAecOHEiv/qrv3pJx/7CL/xCJicnNznR1+34cvbE6ekcmVysOwYAANAB3VzOdvxNqIf6ejOzUHcKAACgE9785jfnoYceyq233prXvOY1ueKKK/L+978/MzMzef3rX5+3vvWtmZiYyHd/93fn0UcfzcLCQt7ylrfkyJEj+epXv5rbbrstl112We68885Nz6ac9fdkdqGqOwYAANABb3vb23Lfffflrrvuyic+8Yl84AMfyCc/+clUVZW/9bf+Vj72sY/lySefzNVXX53f+73fS5KcPHkyo6Oj+fmf//nceeedueyyy9qSbceXs2Zfj5kzAACow++/OTn8V5t7ziufn7zubWva9SMf+Ug+8pGP5EUvelGSZHx8PA8++GBe8YpX5Md//MfzUz/1U/mO7/iOvOIVr9jcjOex48vZUH9PZsycAQDAjlNVVf7JP/knecMb3nDOa/fee28+/OEP52d+5mfyqle9Kv/0n/7TtudRzvp7MmvmDAAAOm+NM1ybadeuXTl9+nSS5DWveU3e8pa35Pu+7/syMjKSxx57LH19fZmfn8/evXvz/d///RkbG8s73/nOs461rLFNmn09WaiSuYXF9PXs+ItXAgDAtrZv377ceuuteelLX5pv//Zvz/d+7/fmZS97WZJkZGQk/+k//ad84QtfyE/+5E+m0Wikr68vv/Zrv5Ykuf322/Pa1742V199tQuCtMNQf0+SZHJ2IaNDyhkAAGx3733ve3P69Ons2rUrSfKmN73prNef8Yxn5DWvec05x73xjW/MG9/4xrbl2vFt5Ew5m56zthEAAKjPji9nzWUzZwAAAHXZ8eVsqG+pnE0pZwAAQI2Us/6lj91Nzc3XnAQAAHaGqtr+t7K6lJ9ROeuzrBEAADplcHAwR48e3dYFraqqHD16NIODg+s6bsdfrfHMZ84sawQAgPa79tpr8+ijj+bJJ5+sNcf09PS6y9N6DA4O5tprr13XMTu+nJ25WuOUqzUCAEDb9fX15YYbbqg7Rg4dOpQXvehFdcc4i2WNLggCAAB0gR1fzoYXTmVfTvrMGQAAUKsdv6xx12//QN7eN55Pzb2k7igAAMAOtuNnzsrAcJplxrJGAACgVspZXzPDZcayRgAAoFY7vpylfzjNzLhaIwAAUCvlrK+ZoTKdqdn5upMAAAA7mHLWP5yhWNYIAADUSznrH85gZs2cAQAAtVLO+ppppMrC7GTdSQAAgB1MOesfTpJUyhkAAFAj5ayvufR9dqLeHAAAwI6mnPWfKWdmzgAAgPooZ31Lyxob88oZAABQH+WsNXPWmJ+qOQgAALCTKWetz5z1LU5lfmGx5jAAAMBOpZy1rtY4nOlMzbkRNQAAUA/lrDVzNlRmMjWrnAEAAPVQzlozZ83MmDkDAABqo5y1Zs6amcmkmTMAAKAmylnvQBbTWFrWaOYMAACoiXJWSuYbA0vLGs2cAQAANVHOksw3BjKUacsaAQCA2ihnSRZ6BtO0rBEAAKiRcpZWOctMpmbn644CAADsUMpZksWegQz5zBkAAFAj5SxJ1VrWOGlZIwAAUBPlLEszZ67WCAAA1Ek5S7LYM5jhopwBAAD1Uc7y9as1WtYIAADURTlLstC6IMi0mTMAAKAmylmSxcZgBjOTyRmX0gcAAOqhnGVpWWNPFjM3N113FAAAYIdSzrK0rDFJqpmJmpMAAAA7lXKWZKFnKElSzY7XnAQAANiplLN8feYss5P1BgEAAHYs5SxLFwRJkjKnnAEAAPVQzvL1mbMyr5wBAAD1UM6ydLXGJGnMT9WcBAAA2KmUs3y9nPUtTGVhsao5DQAAsBMpZ0kWG0vLGptlJlNzCzWnAQAAdiLlLF+fORvKTKZmlTMAAKDzlLN8vZwNZ1o5AwAAaqGcJVls9KdKyZBljQAAQE2UsyQpJQu9Q2lmJpOz83WnAQAAdiDlrGWxVc4sawQAAOqgnLVUfU3LGgEAgNooZ2f0NVvLGpUzAACg85SzM/qG08y0mTMAAKAWyllLY2B4aVmjmTMAAKAGyllLY2DYskYAAKA2yllLo384Q3FBEAAAoB7KWUvpb2a4zGTKfc4AAIAaKGdn9A+n6VL6AABATZSzM/qaGfKZMwAAoCbK2Rn9zfRmIbMz03UnAQAAdiDl7Iy+4STJ4sxEzUEAAICdSDk7o7+ZJFmcVc4AAIDOU87OaM2cldnJmoMAAAA7kXJ2RmvmrJpTzgAAgM67aDkrpTytlHJnKeUzpZT7Sylvam3fW0r5aCnlwdb3Pe2P20b9SzNnDeUMAACowVpmzuaT/HhVVTcl+aYkP1xKuSnJm5P8cVVVz0ryx63nW1drWWNjXjkDAAA676LlrKqqx6uqurf1+HSSzya5JsnfTvKe1m7vSfKd7QrZEa1ljWbOAACAOqzrM2ellOuTvCjJnyfZX1XV462XDifZv6nJOq1vqZz1LkxlcbGqOQwAALDT9K51x1LKSJIPJvnRqqpOlVK+9lpVVVUpZdVGU0q5PcntSbJ///4cOnRoQ4HbYXx8PHfd/Ze5NUmzzOSjf3IoA73losfBeoyPj3fl+Gf7MMboBOOMTjDO6IRuHGdrKmellL4sFbP/t6qq/9zafKSUclVVVY+XUq5K8sRqx1ZV9Y4k70iSAwcOVAcPHtx46k126NCh3PqyA8nHk6HM5MXf9M25bGSg7lhsM4cOHUo3jn+2D2OMTjDO6ATjjE7oxnG2lqs1liTvSvLZqqp+ftlLv5PkB1uPfzDJhzY/Xge1ljU2M5Op2YWawwAAADvNWmbObk3yA0n+qpTyqda2n07ytiTvL6X8UJIvJ/nu9kTskEYjCz2Dac5PZ2pOOQMAADrrouWsqqo/S3K+D2C9anPj1GuhdyjNGTNnAABA563rao3b3WJvM80yk0nlDAAA6DDlbJmqr5mhzGRqbr7uKAAAwA6jnC3X12xdEGSx7iQAAMAOo5wtU/qHM1RmMjlr5gwAAOgs5WyZxsBwmpnOtKs1AgAAHaacLbNUzlwQBAAA6DzlbJmegTPLGpUzAACgs5SzZUr/0syZZY0AAECnKWfL9bnPGQAAUA/lbLn+4fRnPjOzM3UnAQAAdhjlbLm+ZpJkYXqi5iAAAMBOo5wt179UzhZnxmsOAgAA7DTK2XJ9w0mSam6y5iAAAMBOo5wt15o5y6xljQAAQGcpZ8u1PnMWM2cAAECHKWfL9S8tayxzUzUHAQAAdhrlbLnWzFnPvJkzAACgs5Sz5VozZz3zZs4AAIDOUs6WO1POFqZSVVXNYQAAgJ1EOVuutaxxKNOZmV+sOQwAALCTKGfLtWbOmpnJ5OxCzWEAAICdRDlbrtGThUZ/mmUmk7PzdacBAAB2EOVshYWeoQxlJtNzZs4AAIDOUc5WWOxrWtYIAAB0nHK2wmJvM80ynSnlDAAA6CDlbIXqzMyZZY0AAEAHKWcrlP5mmmXGzBkAANBRytkKpX84Q1HOAACAzlLOVigDw5Y1AgAAHaecrdAzMJyhMpNpM2cAAEAHKWcr9AyMpJlpl9IHAAA6SjlbodF/5mqN83VHAQAAdhDlbKX+4QyWuczMzNadBAAA2EGUs5X6mkmS+enJmoMAAAA7iXK2Uv9SOVuYnag5CAAAsJMoZyv1DSdJqhnlDAAA6BzlbKXWzFll5gwAAOgg5Wyl1sxZ5nzmDAAA6BzlbKXWzFlRzgAAgA5SzlbqO1POLGsEAAA6RzlbqX9pWWNjfrrmIAAAwE6inK3UmjnrnbesEQAA6BzlbKXWZ856F6dSVVXNYQAAgJ1COVupdbXGwWomM/OLNYcBAAB2CuVspd7+LJTeNMt0pmYX6k4DAADsEMrZKhZ6htLMTKbmlDMAAKAzlLNVLPQ2M5SZTJo5AwAAOkQ5W8ViXzPNMpNpM2cAAECHKGerqMycAQAAHaacraLqb2Y405mcna87CgAAsEMoZ6soljUCAAAdppytovQPW9YIAAB0lHK2isbAsEvpAwAAHaWcraJncDhDZcZNqAEAgI5RzlbROzCSpmWNAABABylnq2gMDKdZZjI1O1d3FAAAYIdQzlbT10ySzE9P1hwEAADYKZSz1fQPJ0kWZiZqDgIAAOwUytlqWjNni8oZAADQIcrZavpb5Wx2vOYgAADATqGcraZvaVljZn3mDAAA6AzlbDWtmbPMWdYIAAB0hnK2mtZnzsrcVM1BAACAnUI5W03rao2NecsaAQCAzlDOVtOaOVPOAACATlHOVtOaOeudt6wRAADoDOVsNa2Zs96FqVRVVXMYAABgJ1DOVtM7kMU0MpiZzC0oZwAAQPspZ6spJfM9Q2lmJlOzC3WnAQAAdgDl7DwWeocylOlMzs3XHQUAANgBlLPzWOhtplnMnAEAAJ2hnJ3HYm8zzcxkUjkDAAA6QDk7j6qvmaHMZHpOOQMAANpPOTuf/qVljWbOAACATlDOzqP0D1vWCAAAdIxydh6N/uE0M21ZIwAA0BHK2Xk0BoYtawQAADpGOTuPnoHhDGUmU2bOAACADuitO0C36h0cSV9mMjUzV3cUAABgB1DOzqNnYDgpVWZnJuuOAgAA7ACWNZ5P/3CSZH5mouYgAADATqCcnU9fM0myOK2cAQAA7aecnU9/q5zNKmcAAED7KWfn07e0rLGyrBEAAOgA5ex8WjNnmXVBEAAAoP2Us/NpzZxlXjkDAADaTzk7n9bMWZlTzgAAgPZTzs6ndbXGHjNnAABAByhn59O6z1lDOQMAADpAOTuf1sxZ78JUzUEAAICdQDk7n76hVCnpW5iuOwkAALADKGfnU0rmGoMZqKYzt7BYdxoAAGCbU84uYKFnKM3MZGpuoe4oAADANqecXcBC71CGykymZpUzAACgvZSzC1jobaaZmUwqZwAAQJtdtJyVUt5dSnmilHLfsm3/VynlsVLKp1pf39bemPWo+pppZtrMGQAA0HZrmTn7jSSvXWX7v62q6pbW14c3N1Z3qPqaS8sa5+brjgIAAGxzFy1nVVV9LMmxDmTpPv3DljUCAAAdsZHPnP2jUsqnW8se92xaoi5S+oczFBcEAQAA2q9UVXXxnUq5PsnvVlV1c+v5/iRPJamS/PMkV1VV9Q/Oc+ztSW5Pkv3797/4jjvu2JTgm2l8fDwjIyPnbL/2vl/KyJP35D/c+M687OreGpKxnZxvnMFmMcboBOOMTjDO6IS6xtltt912T1VVB1Z77ZIaR1VVR848LqX8+yS/e4F935HkHUly4MCB6uDBg5fylm116NChrJZr4uTvZuHJj+eGZz47B19yXeeDsa2cb5zBZjHG6ATjjE4wzuiEbhxnl7SssZRy1bKnr09y3/n23cp6BkbSzHTGp+fqjgIAAGxzF505K6X8VpKDSS4rpTya5J8lOVhKuSVLyxofTvKGNmasTf/QSBplMRNTU3VHAQAAtrmLlrOqqv7eKpvf1YYsXacxMJwkmZ44XXMSAABgu9vI1Rq3v75mkmR6UjkDAADaSzm7kP6lmbMZ5QwAAGgz5exCWjNnc9PjNQcBAAC2O+XsQvqXytm8cgYAALSZcnYhfUvLGhdnJmoOAgAAbHfK2YW0Zs4WZ5UzAACgvZSzC2l95qx/YTrTcws1hwEAALYz5exCWldrHCozOTk1V3MYAABgO1POLmRgV5JkdyaVMwAAoK2UswvpG8pCz2BGy7hyBgAAtJVydhGLA6MZzUROTipnAABA+yhnF1EN7clYmTBzBgAAtJVydhGNoT0Zs6wRAABoM+XsInqG9ywta1TOAACANlLOLqIM7c2ehnIGAAC0l3J2MUNjGc14TilnAABAGylnFzO0J0OZycTkRN1JAACAbUw5u5ihsSTJwsTxmoMAAADbmXJ2MUN7kiTVlHIGAAC0j3J2Ma1yVqZP1hwEAADYzpSzixlcWtbYO3ui5iAAAMB2ppxdTGvmbHjhdKbnFmoOAwAAbFfK2cW0ytlomXA5fQAAoG2Us4sZ2J0qJaNl3I2oAQCAtlHOLqbRyHz/aMainAEAAO2jnK3B4uBYRsuEcgYAALSNcrYWg2MZi3IGAAC0j3K2Bo3mnoz5zBkAANBGytka9Izsy24zZwAAQBspZ2vQGBrLHp85AwAA2kg5W4uhPdldJnJqcqbuJAAAwDalnK3F0J70ZDGzkyfrTgIAAGxTytlaDI4lSRYnjtccBAAA2K6Us7UY2rP0fVo5AwAA2kM5W4tWOSvTJ2oOAgAAbFfK2VoMLS1r7J31mTMAAKA9lLO1aM2cDS+czsz8Qs1hAACA7Ug5W4vWBUFGM+5eZwAAQFsoZ2vRN5j5nsGMlomcUs4AAIA2UM7WaKF/NGOZMHMGAAC0hXK2RouDezJWLGsEAADaQzlbq6GxjBYzZwAAQHsoZ2vU09y7dEGQSeUMAADYfMrZGvWO7M1YmcjJqfm6owAAANuQcrZGjeaejLogCAAA0CbK2VoNjqVZZjIxMV53EgAAYBtSztZqaE+SZH7ieM1BAACA7Ug5W6tWOVucUs4AAIDNp5yt1dBYkqQoZwAAQBsoZ2vVmjnrmTlRcxAAAGA7Us7W6kw5mz1ZcxAAAGA7Us7WanBpWePwwunMzC/UHAYAANhulLO1GtidxTQyWsbd6wwAANh0ytlaNRqZ69+d0UzklHIGAABsMuVsHRb6RzNWJsycAQAAm045W4dqcCxjsawRAADYfMrZOpTm3oyaOQMAANpAOVuHnuaejGY8JyeVMwAAYHMpZ+vQO7K39Zmz+bqjAAAA24xytg49zb3ZnYmcnJypOwoAALDNKGfrMTSWnlJlZuJE3UkAAIBtRjlbj6E9SZKFiWM1BwEAALYb5Ww9WuWsmjJzBgAAbC7lbD0Gx5IkjanjNQcBAAC2G+VsPVozZz0zZs4AAIDNpZytR6uc9c2drDkIAACw3Shn6zG0tKxxaOF0ZucXaw4DAABsJ8rZevQOZL5nsHUj6rm60wAAANuIcrZOc/1jGcu4cgYAAGwq5WydFvpHM2rmDAAA2GTK2TpVQ2MZK+M5pZwBAACbSDlbp0Zzb0Zj5gwAANhcytk69QzvtawRAADYdMrZOvWN7HFBEAAAYNMpZ+vU09yboTKb8YnxuqMAAADbiHK2XkN7kiRz48dqDgIAAGwnytl6DY4lSRYnj9ccBAAA2E6Us/VqzZxVU8oZAACweZSz9WqVs8b0iZqDAAAA24lytl5DS8sae2aUMwAAYPMoZ+vVmjnrnz1ZcxAAAGA7Uc7Wa2B3FtOTocXTmVtYrDsNAACwTShn61VKZvt2ZSwTbkQNAABsGuXsEsz3j2asjCtnAADAplHOLsHC4FhGzZwBAACbSDm7FINjGTVzBgAAbCLl7BI0mnszlomcUs4AAIBNopxdgt6RvRktljUCAACbp7fuAFtR/8jeDGQiJydm6o4CAABsE8rZJehp7k1KlZnx43VHAQAAtgnLGi/F0J4kydyEcgYAAGwO5exSDI0lSRYnj9UcBAAA2C6Us0vRmjnLlJkzAABgcyhnl6JVzhrTJ2oOAgAAbBfK2aUYXFrW2Dt7suYgAADAdqGcXYrWZ876lTMAAGCTXLSclVLeXUp5opRy37Jte0spHy2lPNj6vqe9MbtM70BmG0NpLp7O3MJi3WkAAIBtYC0zZ7+R5LUrtr05yR9XVfWsJH/cer6jzPXtzmgmcmpqru4oAADANnDRclZV1ceSrLxm/N9O8p7W4/ck+c5NztX15gfGMlbGc1I5AwAANsGlfuZsf1VVj7ceH06yf5PybBmLg2MZLRPKGQAAsClKVVUX36mU65P8blVVN7een6iqamzZ68erqlr1c2ellNuT3J4k+/fvf/Edd9yxCbE31/j4eEZGRtZ1zNP/4ucyc/yxfOT5v5AXXN7bpmRsJ5cyzmA9jDE6wTijE4wzOqGucXbbbbfdU1XVgdVeu9RWcaSUclVVVY+XUq5K8sT5dqyq6h1J3pEkBw4cqA4ePHiJb9k+hw4dynpznTxyR6ZOfD5Pf9Zzc/CWa9oTjG3lUsYZrIcxRicYZ3SCcUYndOM4u9Rljb+T5Adbj38wyYc2J87W0Te8N2MZd0EQAABgU6zlUvq/leQTSW4spTxaSvmhJG9L8upSyoNJvqX1fEfp37Uvg2Uu4+On644CAABsAxdd1lhV1d87z0uv2uQsW0rv8NJH7GbHV17IEgAAYP0udVkjQ0vlbH5COQMAADZOObtUrXK2OHm85iAAAMB2oJxdqsHWnQSmTtSbAwAA2BaUs0vVmjnrmVHOAACAjVPOLlWrnPXOnqw5CAAAsB0oZ5dqYFcW05OBuVN1JwEAALYB5exSlZLp3l1pLpzK/MJi3WkAAIAtTjnbgLn+0YyWiZyanq87CgAAsMUpZxuwMDCasYznqfGZuqMAAABbnHK2AY3m3oyWiRw+OV13FAAAYItTzjagb2RvRjORw6eUMwAAYGOUsw0Y3LUvY2U8R8ycAQAAG6ScbUDP8N7sLpM5cnKy7igAAMAWp5xtxNCeNFLl5PGjdScBAAC2OOVsI5r7kiSzpw7XHAQAANjqlLON2H11kqRx+vGagwAAAFudcrYRrXI2PHMkM/MLNYcBAAC2MuVsI3YtlbMrcyxPnHIjagAA4NIpZxvRN5jZgb25qhzLEfc6AwAANkA526DFXVfnqnI0j7vXGQAAsAHK2Qb1jF1r5gwAANgw5WyDelvl7LCZMwAAYAOUsw0qo1dnrIznqRMn6o4CAABsYcrZRu2+NkmycPzRmoMAAABbmXK2Ua17nZXTX605CAAAsJUpZxvVKmeDk0eyuFjVHAYAANiqlLONapWzy6uncmxytuYwAADAVqWcbVTfUGb79+SqctQVGwEAgEumnG2C+ZGrcqV7nQEAABugnG2Cxug1ubocy+NmzgAAgEuknG2C/r1Py5XlqJkzAADgkilnm6Axek32lvE8dfxk3VEAAIAtSjnbDLuvSZLMuhE1AABwiZSzzTC6VM7KqcdqDgIAAGxVytlmaM2c9U0erjkIAACwVSlnm2HXVUmSPXNPZmJmvuYwAADAVqScbYb+Zmb7RpduRO2KjQAAwCVQzjbJ3MjVSzeidq8zAADgEihnm6TsviZXlWNmzgAAgEuinG2S/r3X5qpyNI+bOQMAAC6BcrZJevdcm33ldI6ecCNqAABg/ZSzzdK6nP7MMfc6AwAA1k852yy7r06SVCcfrTkIAACwFSlnm2X3tUmS3onHaw4CAABsRcrZZtm9dCPqkZknMr+wWHMYAABgq1HONkv/cGb6dmd/jubJ8Zm60wAAAFuMcraJ5ppX5epyzOX0AQCAdVPONlG1+5pcWY7miHIGAACsk3K2ifr2XJMry7EcPqWcAQAA69Nbd4DtZGDv0zJYTuXJ46fqjgIAAGwxZs42URldupz+9DH3OgMAANZHOdtMrRtRL554rOYgAADAVqOcbabd1yRJeia+WnMQAABgq1HONlNr5mxo8nCqqqo5DAAAsJUoZ5tpYCQzvbtyWXU0J6fm6k4DAABsIcrZJptpXpmrXE4fAABYJ+Vsk1W7rs5V5WgedyNqAABgHZSzTdaz59pcVY7liHIGAACsg5tQb7LBvddlpJzMEydO1x0FAIZQYwEAABptSURBVADYQsycbbLesaXL6U8dfaTmJAAAwFainG220aVytuBG1AAAwDooZ5utdSPqxmk3ogYAANZOOdtsrRtRD04drjkIAACwlShnm21gV2Z6RjI692Sm5xbqTgMAAGwRylkbTLduRH3EjagBAIA1Us7aYGHkqlxZjuawe50BAABrpJy1Qc/YNbm6HMthM2cAAMAauQl1Gwzuuy67cjJPHHcjagAAYG3MnLXBwN6npVGqTBx9tO4oAADAFqGctUPrcvrzx5UzAABgbZSzdth9bZKkuBE1AACwRspZO7RmzvonHq85CAAAsFUoZ+0wuDszPc3smn0iC4tV3WkAAIAtQDlrk6nBK7M/R3N0fKbuKAAAwBagnLXJ/MjVucq9zgAAgDVSztqkMXp1rizHcvikcgYAAFycctYmA3uflityIkdOuBE1AABwccpZmwxd9vQ0SpXxpx6rOwoAALAFKGdt0hi9Jkky50bUAADAGihn7dK611lOKmcAAMDFKWft0po5K6fdiBoAALg45axdBnZntqeZ5vSRjM/M150GAADocspZu5SS2eaVuaoczYNHXLERAAC4MOWsjXrGrs3V5al8/rByBgAAXJhy1kaDV96YZ5bH8/nDJ+uOAgAAdDnlrI3KlTdnpEzl+FcfqjsKAADQ5ZSzdrrieUmSxhOfrTkIAADQ7ZSzdrriuUmSq2ceytHxmZrDAAAA3Uw5a6eBkUyNPC3PaXwlDxwZrzsNAADQxZSzNiv7b85zyiN5wOX0AQCAC1DO2mzgmptzQ+PxPPT4U3VHAQAAuphy1mZl/83pSZWpxz5TdxQAAKCLKWfttv/mJMnAsc+mqqqawwAAAN1KOWu3vTdkvjGY6+e/lMdPTtedBgAA6FLKWbs1ejK959m5sTySz7soCAAAcB4bKmellIdLKX9VSvlUKeXuzQq13fRd/byly+kfVs4AAIDVbcbM2W1VVd1SVdWBTTjXtjRw9QtyeTmVxx77ct1RAACALmVZYyfsvylJsvD4/TUHAQAAutVGy1mV5COllHtKKbdvRqBtqXXFxuETn8/Cois2AgAA5yobubx7KeWaqqoeK6VckeSjSd5YVdXHVuxze5Lbk2T//v0vvuOOOzaSty3Gx8czMjLS1vc48Kc/mN+feUEWXvZjuXLYhOVO1Ilxxs5mjNEJxhmdYJzRCXWNs9tuu+2e830krHcjJ66q6rHW9ydKKb+d5CVJPrZin3ckeUeSHDhwoDp48OBG3rItDh06lHbnOv3A8/OcR7+Sx55+Uw7efGVb34vu1Ilxxs5mjNEJxhmdYJzRCd04zi55CqeUMlxK2XXmcZJvTXLfZgXbbgaf9oI8qzyWBw8frzsKAADQhTYyc7Y/yW+XUs6c571VVf3BpqTahvquujl9ZS7HH/18kufWHQcAAOgyl1zOqqr6YpIXbmKW7W3/85a+H7k/yXfWGgUAAOg+rkzRKZfdmMX0ZO/4g5mZX6g7DQAA0GWUs07pG8zErutzY76SLz45UXcaAACgyyhnHVRdcVNuLF/JA0dO1x0FAADoMspZBzWf9oJc13gyX3rscN1RAACALqOcdVDvVc9Pkkw+6o4DAADA2ZSzTrripiRJ31OfqTkIAADQbZSzThq7LrM9w7li6qFMzMzXnQYAAOgiylknlZKJsWfnOY1H8uAT43WnAQAAuohy1mG9Vz0/zy1fyQOPn6o7CgAA0EWUsw4bftoLsrtM5quPPFR3FAAAoIsoZx3WuPLmJMn8V/+q5iQAAEA3Uc46bf/SFRsHj3+u5iAAAEA3Uc46bXA0pweuzNPmvpRjE7N1pwEAALqEclaD2X3PzXPKV/LAkdN1RwEAALqEclaDgWuen28oj+cLjx+tOwoAANAllLMaDF/3wvSVhRz/8n11RwEAALqEclaDsv95SZLqyGdqTgIAAHQL5awO+56Z+dKXXSc/n6qq6k4DAAB0AeWsDj19OTXyDfmGhYdz+NR03WkAAIAuoJzVZOHym3Jj45F8/rArNgIAAMpZbUaue2GuLMfzxa88UncUAACgCyhnNRm69vlJkiMP3ltzEgAAoBsoZ3XZf3OSZPHwpzM9t1BzGAAAoG7KWV1G9meqeU3+WnV/7n74eN1pAACAmilndSklvTe+Ot/cuD93ff6rdacBAABqppzVqO/Zr85Imc7Rz/1p3VEAAICaKWd1+oZXZqH05BtOfCJPnHa/MwAA2MmUszoN7MrUlS/JKxt/mbu+8FTdaQAAgBopZzVr3vSaPLfxlXzq/s/VHQUAAKiRclazxrO+Zen7F/8ki4tVzWkAAIC6KGd1239zpgYuz4vn7s7nDp+uOw0AAFAT5axupSTPfFVe0fir/NnnH687DQAAUBPlrAsMPfc1GS2Tefwzd9UdBQAAqIly1g2ecVsW08hlRz6WqdmFutMAAAA1UM66wdCenL7slrw8f5k//9LRutMAAAA1UM66RPOm1+SFjS/mns88UHcUAACgBspZl+i78VuTJPMP/FHNSQAAgDooZ93iqlsy1TeWZ49/ModPTtedBgAA6DDlrFs0Gpl9+m35641P52MPHKk7DQAA0GHKWRfZ/fzXZl85nYfv+3jdUQAAgA5TzrpIeea3ZDElI1+5M4uLVd1xAACADlLOusnwZTk59ry8ZOEvcv9XT9WdBgAA6CDlrMv0P+db86LyYP78Mw/VHQUAAOgg5azLDN/02vSUKqc/89G6owAAAB2knHWba16c6Z5dedqxuzI+M193GgAAoEOUs27T05vxa16RV5S/zJ8/9FTdaQAAgA5RzrrQ6Atel/3lRB749J/XHQUAAOgQ5awL9T371UmSni/+Uc1JAACATlHOutHuq3Js5Nl5/tTdefipibrTAAAAHaCcdan+57w6Bxqfz/vu+kzdUQAAgA5QzrrUyAtfn76ykPl735tT03N1xwEAANpMOetW1x7IxBUvyg9U/zX/3ycfrjsNAADQZspZtyolwwf/ca5rPJkv/ekdmV9YrDsRAADQRspZN3vOt2di5On5OzP/OX943+G60wAAAG2knHWzRk+G/vqP5JbGF/PxO3+n7jQAAEAbKWddrnHL92a6b08OHn1f7vny8brjAAAAbaKcdbv+Zhov/Yd5dc+9+b0/+W91pwEAANpEOdsC+l/2hsyVgTz7i7+RR45N1h0HAABoA+VsKxi+LLPP/558V+NP84H/dk/daQAAgDZQzraI4Vf+aHrLQpqfendOuyk1AABsO8rZVrHvGTn99G/N9+QP85///MG60wAAAJtMOdtCRl/14xkrEzn2Z+9yU2oAANhmlLOt5LqX5vi+F+V/mvlQPnrfY3WnAQAANpFytsXsftU/znWNJ3P/n7y37igAAMAmUs62mJ7nfHtONq/Ltxy/I/d++VjdcQAAgE2inG01jZ4MvuJHckvji7nzI/+l7jQAAMAmUc62oIED35/J3rEc+Mp/yEfue7zuOAAAwCZQzraivqH0H/yJvLLn0/nEB38xT5yarjsRAACwQcrZFtX7zT+cyWtuzU8uvjv/+rd+L4uLVd2RAACADVDOtqpGI83veWca/UP5Xx772fzmXQ/UnQgAANgA5Wwr2311Br7rV3Jz4+EsfPSt+dzhU3UnAgAALpFytsWV535Hpm75+/kHjd/Lb/7muzI9t1B3JAAA4BIoZ9vA0Lf/XMZHn5UfHf/5/Mp//UTdcQAAgEugnG0HfUMZ+d73ZKwxnW/8i5/Oxz5/pO5EAADAOiln28X+5yXf+n/ntp6/zN3v+7kcm5itOxEAALAOytk20vdNt+f001+df7Twm/nl934wVeXy+gAAsFUoZ9tJKdn13f8ucwNj+b5H3pp33Xl/3YkAAIA1Us62m+F9Gfrud+WGxuE8487/I2/70N1uUA0AAFuAcrYNNZ55MPmOX8gre+7L37znH+Sn/+MfusQ+AAB0OeVsm2oc+F9TvveOPKvvyfzIl/73/PSv3eEiIQAA0MWUs22sPPtb0/8PP5I9zb787NEfz9t+6Zfz5aMTdccCAABWoZxtd1c+P0P/251p7H16/sX0P89//OW35t6vHK87FQAAsIJythOMXpPmGz6a2etembdU/y7/451vyh/81VfrTgUAACyjnO0Ug7vT/MEPZOoFP5A3ND6Uuff//fzMHXfl8ZNTdScDAACS9NYdgA7q6c3Q638pc/ueke+48615+We/J7/6me/K0Mtuzz+87cbsGuyrOyEAAOxYZs52mlLS98ofS7n9UIae9sL8n4335PWf+K7883/5c/nNj38pcwuLdScEAIAdSTnbqa6+JYM/9LvJ930gV+4by79c/De56Q/+5/zjf/Pv8of3H05VuXE1AAB0knK2k5WSPOvVGXrjJ1L9zbfn5ubJ/NLkm7N4x/fnh9/+vvzHTzycp8Zn6k4JAAA7gs+ckTR6Ul78gxl4/t/Jwl2/nG/5s1/I646/Iff+/jPzK793a5647tvyym98Xl7zvCszOuRzaQAA0A7KGV/XP5ye234qPX/t7yefem9u+ov35RuPvicLX/3N3PXI8/Iv/sutmX7m6/Lqb3x2Xvnsy11ABAAANpFyxrlGrkhe/qMZfPmPJk98No2/+kBe+qn356+f/vXMPPyu/PFDL8rPLt6So2O35PLrb8oLr9uXF103lmfv35WeRqk7PQAAbEnKGRd2xXNTXvWWDPyNn0keuyd9n35/vuXTH8y3TX8ymUhO3j+cv/j0M/P7i8/Kv+65MQtXfWNuvP6aPOOykVy3r5nr9w3nil0DaShtAABwQcoZa1NKcu2BNK49kP7Xvi05+mDyyCez+5FP5pu/8sm88ugHU1Jl8UjJQ4evzoOL1+Te6sp8sLoyj5WrMj/2Ddl92dV5+mUjuXbPUC4bGci+kf5cPjKQfSMDGRvqU+AAANjRlDPWr9FILr8xufzGlG/8gfQnyfTJ5LF70nj07jzj0btz/ZNfSM/Je9Oo5peOGU8mx4fy8Jf258uLl+eJaiyfr8byZMbyZDWaY2Usc0NXpDFyeUaazYwM9mbXYG92DfRm12Df156PDPSm2d+bwb5Ghvp6Mtj6GurvyVBfTwZ6G+nvbaS3UVKKsgcAwNahnLE5BkeTZ/yN5Bl/I4207tGwMJ+c/Epy7IvJ0S+meeyhPPfYF3PjsYeT8QfTM3Pi7HPMJzmRTJwczniGcyrNnKyGcnyhmVNp5lTVzMNpZqoayGQGMpWBTFaDmcxAptOfyWogkxnMbHozl74s9vSn6hlI1ehP6e1PX29P+nsa6WmU9PYsFbjenrL0vdFIb09JT6Okp5Q0GkvbG63nPY2SRinpaSQ9reLXU0oaJUuPG0uPG2XptTOPz7zeKCVf/vJs7lt8MKWUlDP75swxS/uVLE1SliSNxtLznDlPzhy39Dhn9vva8V/fZ+mw5ecrXzvv0uvLny/bb8W5l7/WOuprx+Wc8y577Tzv0zrsnKxZti0rsyz7nSw/77nby1n7LD82Zx174X2/tvt53//sn+PM7+VrG8/afu4xWeW4lf8fYbX8K8+12vncnxAAtjbljPbp6U32fsPS1zOXNpUkPWden59Jxp9ofR352tfw1PEMT5/M/ulTyfTJVNMnUk0/kUyfTGPm1PpzVEnmkrn5/syX3iykt/W9J/Otx/NnHqcn8+nJQtXIQhqZT6O1rZHFqpH51vaFlKXv1dI+C1Uji1X52vbFKq1jShZb+9+QRha+VJKU1val16osPT7zfTGtfb72erLYuiXh8udVkqp1XFWVs56fOcdZ+yw7brXXlj/OKtvP/CrPfr7yvXPWaznnvOe+Z1bkyXnOtfz4rDjX8mzn37/kTHdZedzKvMu35zzbV/uZvv76+Y9Zy/as2L6uc/3h756V+2JF8ex9ljfZnHe/5fuWcw9ZtUCucuqz91vreVbZf/mrq59nxbErzrbyd7Tyvc//vms/97lJz32fc95xHceu/vrac13M8vcaH5/Krr/80wvse5Hn5/6kF9n/Ai+uPHY95zpn3/Wd+9zjV+6/9jdf77nPPf78O1z02Dae+2Iu9Ds/fmw6/+GLn2zLe1/8933pY2Hji3bW9+/H5p15o+duY+4NHPvip+/ND738hks/QQ02VM5KKa9N8otZ+vv2O6uqetumpGJn6B1Ixp629HUBJcv+QFlcTOanktnJZG7Z1/Ln87PJwsxS+ZufSeank4XZ9M1Pp29hLlmYSxZb3895PN/6Wlj6XrW+L84uzQRWi61tC63vrefVYmvb4tlfiwupWsdUi2cqmNkN6rN6EWxtW6XwrLbf17dXZ/1Xc3kRXnnsyuJ59nErz32+91v+g5RzH1bn+bnO+lduDdmWv83Kwrz8bwlVzvlbw8p/u8899/n3v9C+K39Hq73Xhfa/aK4VP1d1/l9J5hcW0jPRu2z3tf/NafWf42I/1/n33czXL/Q/R1r/sC9wbHKh3/eFjq9WHUdr/50s/fO6YF04/6EX/rFW/ed11rHn7H+Bf19WOfmF/nnMzc+nd+b8f01dz7hb8ba5eE252PGXfu4Lj8ELH7+Rvz2sLduFXrvoQLnkc1/82Ev/nc5MH0he/tYNvHvnXXI5K6X0JPmVJK9O8miS/1FK+Z2qqj6zWeHgHI1G0j+89LVFnPkj478dOpSDBw8u/RetWprnWV7ilp4v27by+VnHVMv2Wf7amamh6uxzZOV7rtx25ntW337mnKsec6F9c3auc/bPRd7nfMdn2b4X2mfFOVd9vHK/tTxf73tv4Pmq+2TVY7708Jdyw/XXr/hb09nnOaearOX9ztq2WtbzbFvtPOcct973XkOui55nrefO+V+72LHrfr2bz322J596MpdfdvnazrXSht67g+fe8M91kfOv59h2nntDP1c7f2dVTp2azO7duy9w7CWOlQ0v/27n73vdYdZx7naOo4vZwPEb/Z1eM3fp712TjcycvSTJF6qq+mKSlFLuSPK3kyhncCFnPlCVZNkiT9iwLx86lBsOHqw7Btvc/Wf+RxO00b3GGTtUYwPHXpPkkWXPH21tAwAAYJ3afkGQUsrtSW5Pkv379+fQoUPtfst1Gx8f78pcbC/GGe1mjNEJxhmdYJzRCd04zjZSzh5LsvxKDte2tp2lqqp3JHlHkhw4cKDqxinqQ6bO6QDjjHYzxugE44xOMM7ohG4cZxtZ1vg/kjyrlHJDKaU/yd9N8jubEwsAAGBnueSZs6qq5ksp/yjJH2bpqgbvrqrq/k1LBgAAsINs6DNnVVV9OMmHNykLAADAjrWRZY0AAABsEuUMAACgCyhnAAAAXUA5AwAA6ALKGQAAQBdQzgAAALqAcgYAANAFlDMAAIAuoJwBAAB0AeUMAACgCyhnAAAAXUA5AwAA6ALKGQAAQBdQzgAAALqAcgYAANAFlDMAAIAuoJwBAAB0gVJVVeferJQnk3y5Y2+4dpclearuEGx7xhntZozRCcYZnWCc0Ql1jbOnV1V1+WovdLScdatSyt1VVR2oOwfbm3FGuxljdIJxRicYZ3RCN44zyxoBAAC6gHIGAADQBZSzJe+oOwA7gnFGuxljdIJxRicYZ3RC140znzkDAADoAmbOAAAAusCOLmellNeWUj5fSvlCKeXNdedheyilPK2Ucmcp5TOllPtLKW9qbd9bSvloKeXB1vc9dWdl6yul9JRS/qKU8rut5zeUUv689efa+0op/XVnZGsrpYyVUj5QSvlcKeWzpZSX+fOMzVRK+bHWfy/vK6X8Vill0J9lbFQp5d2llCdKKfct27bqn11lydtb4+3TpZRvrCv3ji1npZSeJL+S5HVJbkry90opN9Wbim1iPsmPV1V1U5JvSvLDrbH15iR/XFXVs5L8ces5bNSbknx22fP/J8m/rarqmUmOJ/mhWlKxnfxikj+oquo5SV6YpfHmzzM2RSnlmiQ/kuRAVVU3J+lJ8nfjzzI27jeSvHbFtvP92fW6JM9qfd2e5Nf+//bunsWuKgzD8P3iGDARDCoEzShREFtjIQFFQrTSYCxEC8UQ8AdYiKCNWNiJWAg2iRJBFIlB5wcoaGPQmELQRqLmg8kHSKIoGMXHYq3gOHEKc87M2Z5zX82ctfYu3mLzzLz7rLVmjWq8xMw2Z8BdwLdJjia5ALwL7JpwTZoCSRaTfNk//0z7Q2Yz7fna32/bDzw8mQo1LapqHngQ2NvHBewADvRbfM40kqq6BrgX2AeQ5EKSc5hnGq854KqqmgPWA4uYZRpRkk+AH5dNr5Rdu4C30nwGbKyqG9am0n+a5eZsM3B8yfhEn5PGpqq2AFuBQ8CmJIv90ilg04TK0vR4FXgW+LOPrwPOJfmjj801jeoW4CzwZl8+u7eqNmCeaUySnAReBo7RmrLzwGHMMq2OlbJrMH3BLDdn0qqqqquB94Gnk/y09FraMakelarLVlU7gTNJDk+6Fk21OeBO4PUkW4FfWLaE0TzTKPqen120FwE3Ahu4dCmaNHZDza5Zbs5OAjctGc/3OWlkVXUlrTF7O8nBPn364lfk/eeZSdWnqXA38FBVfU9blr2DtjdoY18aBOaaRncCOJHkUB8foDVr5pnG5X7guyRnk/wOHKTlm1mm1bBSdg2mL5jl5uxz4LZ+GtA62ubThQnXpCnQ9/3sA75J8sqSSwvA7v55N/DhWtem6ZHkuSTzSbbQ8uujJI8DHwOP9Nt8zjSSJKeA41V1e5+6D/ga80zjcwzYVlXr++/Pi8+YWabVsFJ2LQBP9lMbtwHnlyx/XFMz/U+oq+oB2p6NK4A3krw04ZI0BarqHuBT4Cv+3gv0PG3f2XvAzcAPwKNJlm9Ulf6zqtoOPJNkZ1XdSvsm7VrgCPBEkt8mWZ/+36rqDtqhM+uAo8Ae2std80xjUVUvAo/RTjs+AjxF2+9jlumyVdU7wHbgeuA08ALwAf+SXf3FwGu0JbW/AnuSfDGRume5OZMkSZKkoZjlZY2SJEmSNBg2Z5IkSZI0ADZnkiRJkjQANmeSJEmSNAA2Z5IkSZI0ADZnkiRJkjQANmeSJEmSNAA2Z5IkSZI0AH8Be9axgxmFN7kAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "etXEv-uWVAvm", "colab": { "base_uri": "https://localhost:8080/", "height": 609 }, "outputId": "92efb630-0f21-4fd1-ea4c-88d28512815c" }, "source": [ "from matplotlib.pyplot import figure\n", "\n", "plt.figure(figsize=(15,10))\n", "plt.grid()\n", "\n", "plt.plot(r2, label = 'r2')\n", "plt.legend()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 17 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA28AAAI/CAYAAADgNuG/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df5Sld10n+Pe3qrr6Z9L53YF0SCcmgIAKoY2ggI3oDorK6ugs7swwIzqRVVdd9bg6eHZ2nMM5OuOOOzs6Kouos+uaQQRBR3D81RpRlIQgECA/CAE6pDu/q7pTt6puVX33j6rqVDrVXT/ufe5Tt/r1OqdO1X2ee5/nU1Xf3K53vp/n+ZZaawAAANjaRtouAAAAgLUJbwAAAENAeAMAABgCwhsAAMAQEN4AAACGgPAGAAAwBMbaLmClyy67rB46dKjtMp7hySefzN69e9sug23OOGMQjDMGwTijacYYg9DWOLv99tsfqbVevtq+LRXeDh06lNtuu63tMp7h6NGjOXLkSNtlsM0ZZwyCccYgGGc0zRhjENoaZ6WUz51tn7ZJAACAISC8AQAADAHhDQAAYAhsqWveAAAANqrb7ebYsWOZnp7u2zH379+fT33qU3073pl27dqVgwcPZseOHet+jfAGAAAMtWPHjuWCCy7IoUOHUkrpyzFPnjyZCy64oC/HOlOtNY8++miOHTuWa6+9dt2v0zYJAAAMtenp6Vx66aV9C25NK6Xk0ksv3fBMofAGAAAMvWEJbss2U6/wBgAA0EdTU1N53etel+c///l54QtfmJ/8yZ/sy3GFNwAAgD6qteZHf/RH8+lPfzp33HFHPvjBD+b9739/z8cV3gAAAHp0//3353nPe17e+MY35qabbsr111+fJBkfH8+NN96YY8eO9XwO4Q0AAKAP7rnnnnz/939/7rzzzlxzzTVJkieeeCK///u/n9e85jU9H99SAQAAwLbxr3//znzyi5M9H2d+fj6jo6NJkhc8+8L8q2954Zqvueaaa/Kyl73s9OO5ubl813d9V37oh34o1113Xc81mXkDAADog7179z7t8c0335wbbrghP/IjP9KX45t5AwAAto31zJCtR6+LdP/0T/90JiYm8va3v70v9SRm3gAAAPrq2LFjeetb35pPfvKTufHGG/PiF7+4LyHOzBsAAECPDh06lE984hNJkoMHD6bW2vdzmHkDAAAYAsIbAADAEBDeAAAAhoDwBgAADL0mrjFr0mbqFd4AAIChtmvXrjz66KNDE+BqrXn00Ueza9euDb3O3SYBAIChdvDgwRw7diwPP/xw3445PT294XC1Ebt27crBgwc39BrhDba4WmtqTRZqzUJNap56vHJ7lrflqX3Lz60r9i0fr555rCxuy+ltT3/98tfJ0x+fPuYZr3/q8Zn7F1+bFc9ZeayVz8+q512xf8Uxs/J5K4575uuSlcdasX/Fa59Ww4qNK5//9Oc+c/9Tr31mravVv/K8K6127LXOfablY9aafPb+2Xxs/p41X7uy/jO3Pf1566srqxz7zN/HGU975jHW+P6eUfs6jnO25652tJXj8Km9q/++VqvlmfvO8eRVazv793mu86712tXPffaf+VrfR63JI49M5//7/G1nnuUstZ2zsjXOfeb+s79+rd/1uc987mOvdrynH3vzrz1z/1pj7pnnXsMG6u7nz2zt1557/+RkJ79w5wc39eJezr3W73Ita/33dM7Xrnnszde29jhq7thrvv6cx+7x4En+8IdfmZ1jo6vuO3r0aF7ykpf0fI5+Et7YlhYWaqa685manUtndj5TSx+d2fnMzM1nZm4hM3PzmZ1bWPy6u5DZ+YXMdOfTXajpzi1kbqFmdn4hc/MLmZtf/Hp+oWZuoS5uW6inHy9/Xlj6eqEufp6vS9tqzcJCTm9fqMuha/FxrUl3bi7lT9+/GJrqU88Zktl/hsk9d2/o6aWs+Pr0trLKtuXH5Wk7zty/8jnrPXZWed7K5648zpnPedrznrZt9Wef61jP+B43cO7VPOO1zzjWRmorZ9232oa16nzm/vXX9WSnZuqxqTMrWNe5zvHjWnX/WrWc7TyrPXmNU/f2u97gsZ/5+rP8R7DGsctaxy7P/Bk+7awbHKPPOMJa3/g5X3v2F891Si7avWPT513797H53+Vannno9R+x9+/rXK9t8ne5+dcma9TW5LG3IOGNLW1hoebxqdk8dHJm8WNyOo8+OZvJTjcTnW4mp+cWP3e6mZxe/HxqZi7T3YVNn3PHaMmO0ZGMjZSMj41kbGQkO8ZKdoyMZHSkZGxp3+hIOf15947RjIyUjJZkdKRkpCxuX9xWVmxLRsri9pGSjJaSsrT/2LEv5NBznpNSFveNLH0uZfEfz9Glz4v7F79efl6SFduWnpenXluyfKylbVn5+anXJDl9/pKVx0mSM46b5eMtvfGttm/FP/xLh3jaH+1Pr/Gp8zxj/+l9ZxzzjD+kl+tZ+bynAsFT38PyWZaP/dT5nhkEVv5cnqpu9XOv3L/a93rmcc4ads441nrP/YyaVwkYf/kXf5Gv/dqvPfsxe/0XFrL4f6uPHHlV22WwjS2OsZvaLgMGTnijVbXWPHxqJp99+Ml89pGnPk5MTuehkzN55NRMuvPPnHoaGynZv3tHLlz+2DWWqy7enf27d2TfzrHs3jGavTtHs3t8LHt2jGbP+Gh2j49mz/hYdu0YyfjYSHaOjWbn2PLXi493jJbW/ng9evShHDnypa2cm/PH8v+AAACGj/DGwHRm5/P3x57I7Z97PHefOLkY1B5+Midn5k4/Z3xsJNdcsidX7t+V66+4IFdcuDNXXLAzV1yw6/TXl+3bmT3jo2YIAAA4rwhvNOahyenc/rnHc9vSx50PTGRuYXEW7eDFu3Pd5fvyD196ca69bO/pj2dftDujI0IZAACcSXijb2qt+fgDE3n3Rx7In376RL7wWCdJsnNsJF9x8KL8i1ddl8PXXJyXXnNxLtoz3nK1AAAwXIQ3evbAE5383h0P5N0fOZbPPPxkxkdH8qrnXp43vuxQXnro4rzo2fszPuYaGwAA6IXwxqacnO7m/R8/nnffcSwfuu+xJMlXHro43/vK6/JNX/as7D/H7XsBAICNE97YkKnZufzKX9yX//sv70unO59rL9ubH/2G5+bbXnJVrr5kT9vlAQDAtiW8sS4LCzW/99EH8nMf+HROTM7km7/8WfmeV1ybF199kbs+AgDAAAhvrOn2zz2en/mDT+bvv/BEvvzg/vzS/3hjDh+6pO2yAADgvCK8cVYPPNHJz73/03nf338xV1ywM//Hd35Fvu0lV2XErfwBAGDgGg9vpZTXJvkPSUaTvL3W+rNNn5PeLCzU/Kej9+Y//tm9SZL/+euuz5u/9kuyd6esDwAAbWn0r/FSymiSX0ryDUmOJflwKeV9tdZPNnleNq87v5D/9V0fy7vveCDf+KIr85bXfWkOXuxGJAAA0Lamp1JuSnJvrfW+JCml3JLk9UmEty3oyZm5/E+/9ZH85d0P58e+4bn5wa+73s1IAABgi2g6vF2V5AsrHh9L8lUNn5NNeOTUTN70Gx/OnV+czM9++5flDTc9p+2SAACAFUqttbmDl/IdSV5ba/3epcf/NMlX1Vp/cMVzbk5yc5IcOHDgpbfccktj9WzWqVOnsm/fvrbLaMxDUwv5+dum88R0zfe/eGdefIVr29qw3ccZW4NxxiAYZzTNGGMQ2hpnr371q2+vtR5ebV/Tf6U/kOTqFY8PLm07rdb6tiRvS5LDhw/XI0eONFzSxh09ejRbsa5++Pixifz4b/xd5jKWW978lbnxORe3XdJ5azuPM7YO44xBMM5omjHGIGzFcTbS8PE/nOSGUsq1pZTxJG9I8r6Gz8k63XrPw3nD2/4mO8dG8643f7XgBgAAW1ijM2+11rlSyg8m+aMsLhXwjlrrnU2ek/X5ozuP5wd+6yO5/op9+c033ZQDF+5quyQAAOAcGr+4qdb6h0n+sOnzsH4PPNHJj7/z7/PCZ1+Y/+d7vyoX7trRdkkAAMAamm6bZItZWKj5sXd+NAu15j9+142CGwAADAnh7Tzzjg9+Nh+677H8b9/ygjznUotvAwDAsBDeziN3HT+Zf/uBu/L1X3og/+jw1Wu/AAAA2DKEt/PEzNx8fuS/fDQX7h7Lz/7DL0sppe2SAACADbAa83niF/74nnzqwcm8/Y2Hc9m+nW2XAwAAbJCZt/PA3332sfzqX34m33XT1fn6FxxouxwAAGAThLdt7uR0Nz/6zo/m6ov35Kdf94K2ywEAADZJ2+Q29zO//8l88YlOfufNL8/enX7dAAAwrMy8bWMf+MTx/M7tx/L9R67PS6+5pO1yAACAHghv29Sjp2byL9/z8bzoqgvzQ6+5oe1yAACAHumj26Z+86/vz+NTs/ntf/GyjI/J6AAAMOz8Vb8NTXfn8//+7efzmucfyPOuvKDtcgAAgD4Q3rah9370gTz25Gze9IpDbZcCAAD0ifC2zdRa846/uj/Pv/KCvPy6S9suBwAA6BPhbZv56888mrtOnMz3vOLalFLaLgcAAOgT4W2b+bW/+mwu2zeeb/mKZ7ddCgAA0EfC2zZy38On8meffij/+Kuuya4do22XAwAA9JHwto38xl/fn/HRkfyTl13TdikAAECfCW/bxMRUN79z27F864ufncsv2Nl2OQAAQJ8Jb9vELR/+fDrd+bzpa65tuxQAAKABwts2MDe/kN/86/vz8usuzQuefWHb5QAAAA0Q3raBD9x5PF+cmM6bXmHWDQAAtivhbRt4x199Ntdcuidf9/wr2i4FAABoiPA25O74/OP5yOefyHd/9aGMjliUGwAAtivhbcj9+gfvzwU7x/Idh69uuxQAAKBBwtsQe3Cikz/8+IN5w01XZ9/OsbbLAQAAGiS8DbH//Defy0KteePLD7VdCgAA0DDhbUjNL9Tc8nefz3/3gitz9SV72i4HAABomPA2pD527Ik8PtXNN3/Fs9ouBQAAGADhbUjdes8jKSX5mi+5rO1SAACAARDehtSt9zycL7tqfy7eO952KQAAwAAIb0Po5HQ3d3z+ibzyBrNuAABwvhDehtCH7nsscws1r7zh8rZLAQAABkR4G0K33vNw9oyP5sbnXNx2KQAAwIAIb0Po1nseycuuuzTjY359AABwvvDX/5D5wmNT+ewjT7reDQAAzjPC25D5q3sfSRLhDQAAzjPC25C59Z6H86z9u/Ill+9ruxQAAGCAhLchMr9Q88F7H80rb7gspZS2ywEAAAZIeBsiH39gIhOdbl5hiQAAADjvCG9D5Na7H04pySuud70bAACcb4S3IXLrPY/kRc/en0v2jrddCgAAMGDC25A4NTOXj3z+8bzCXSYBAOC8JLwNiQ995tHMLVRLBAAAwHlKeBsSt97zcHbvGM1Lr7m47VIAAIAWCG9D4tZ7H8lXXXdJdo6Ntl0KAADQAuFtCBx7fCr3PfxkXmmJAAAAOG8Jb0Pgr+55JEnyKte7AQDAeUt4GwK33vtIDly4M9dfsa/tUgAAgJYIb1vc/ELNB+99JK+84fKUUtouBwAAaInwtsV94oGJPDHVtUQAAACc54S3Le7Wex5OknzN9cIbAACcz4S3Le7Wex7JC599YS7bt7PtUgAAgBYJb1vYqZm5fOTzj1siAAAAEN62sr+979F056vr3QAAAOFtK/urex/Jrh0jeek1F7ddCgAA0DLhbQu784HJvOjZ+7Nrx2jbpQAAAC0T3raoWmvuOnEyz7vygrZLAQAAtgDhbYt66ORMJjrdPPeA8AYAAAhvW9Zdx08mifAGAAAkEd62rLtPLIe3fS1XAgAAbAXC2xZ11/GTuWzfzlxqcW4AACDC25Z194mTed6VZt0AAIBFwtsWtLBQc/eJU653AwAAThPetqBjj3fS6c7necIbAACwRHjbgu5avlmJNd4AAIAlwtsWtHynyRuucM0bAACwSHjbgu46fjJXXbQ7F+za0XYpAADAFiG8bUGLd5rUMgkAADxFeNtiuvML+czD7jQJAAA8nfC2xdz/yJPpzldrvAEAAE8jvG0xp+80aeYNAABYQXjbYu4+fjIjJfmSy828AQAATxHetpi7TpzMocv2ZteO0bZLAQAAthDhbYu5+8SpPE/LJAAAcAbhbQuZ7s7n/kefdL0bAADwDMLbFnLvQ6dSa6zxBgAAPIPwtoXc7U6TAADAWQhvW8hdJ05mfHQkhy7d03YpAADAFiO8bSF3Hz+Z6y7fm7FRvxYAAODppIQt5O4Tp1zvBgAArEp42yJOTnfzwBMd17sBAACrEt62iLtPnEoSa7wBAACrEt62iOU7TWqbBAAAViO8bRF3HT+ZPeOjueqi3W2XAgAAbEHC2xZx94mTueHABRkZKW2XAgAAbEHC2xZx94mTed6BfW2XAQAAbFHC2xbwyKmZPHJq1p0mAQCAsxLetgA3KwEAANbSWHgrpfzvpZQHSikfXfr4pqbONezuPr4U3sy8AQAAZzHW8PF/odb68w2fY+jddeJULtqzI5dfsLPtUgAAgC1K2+QWcPeJk3nugQtSijtNAgAAq2s6vP1gKeVjpZR3lFIubvhcQ6nWmruPn9QyCQAAnFOptW7+xaX8SZIrV9n1liQfSvJIkprk3yR5Vq31Tasc4+YkNyfJgQMHXnrLLbdsup6mnDp1Kvv2NXMb/0c7C/mxv+jkjS8Yz9c9Z0cj52A4NDnOYJlxxiAYZzTNGGMQ2hpnr371q2+vtR5ebV9P4W29SimHkvxBrfVF53re4cOH62233dZ4PRt19OjRHDlypJFj//ldD+W7f/3Deef3vTw3XXtJI+dgODQ5zmCZccYgGGc0zRhjENoaZ6WUs4a3Ju82+awVD78tySeaOtcwW77T5HMt0A0AAJxDk3eb/LellBdnsW3y/iTf1+C5htZdJ07mwIU7c9Ge8bZLAQAAtrDGwlut9Z82deztZPlOkwAAAOdiqYAWzS/U3HPilPAGAACsSXhr0ecfm8rM3IJlAgAAgDUJby2658TizUpucLMSAABgDcJbix6cmE6SHLx4T8uVAAAAW53w1qIHJ6azY7Tk0r3uNAkAAJyb8NaiE5PTueKCXRkZKW2XAgAAbHHCW4uOT0znyv272i4DAAAYAsJbi45PTufKC4U3AABgbcJbS2qtZt4AAIB1E95aMjk9l0533swbAACwLsJbS05MLi4TcMDMGwAAsA7CW0uW13gz8wYAAKyH8NaSE0vh7Vlm3gAAgHUQ3lpyfKlt8ooLd7ZcCQAAMAyEt5Ycn5zOJXvHs3NstO1SAACAISC8teT4hDXeAACA9RPeWmKNNwAAYCOEt5acmJzOATNvAADAOglvLZiZm8+jT85qmwQAANZNeGvBQ5MzSSwTAAAArJ/w1oLlZQIOCG8AAMA6CW8tOL60QLe2SQAAYL2EtxacDm9m3gAAgHUS3lpwfHI6u3eM5sJdY22XAgAADAnhrQXHJxfXeCultF0KAAAwJIS3FpyYmM6BC3e2XQYAADBEhLcWPDgxnWft3912GQAAwBAR3gZsYaHmoZPTOeBOkwAAwAYIbwP22NRsuvM1V2qbBAAANkB4G7CnlgnQNgkAAKyf8DZg1ngDAAA2Q3gbsOOTS+HNNW8AAMAGCG8DdmJyOiMluWzfeNulAAAAQ0R4G7AHJ6ZzxQW7MjbqRw8AAKyfBDFgJyanc8D1bgAAwAYJbwN2fGLaMgEAAMCGCW8DdnxyOs+yTAAAALBBwtsAPTkzl5PTczngTpMAAMAGCW8DdHqZgP3aJgEAgI0R3gboxNIC3WbeAACAjRLeBmh55s01bwAAwEYJbwP04NLM25Vm3gAAgA0S3gboxOR0Ltw1lt3jo22XAgAADBnhbYCOT0znSgt0AwAAmyC8DdDxyelc6Xo3AABgE4S3ATo+MZ0rL7RMAAAAsHHC24DMzS/kkVMzblYCAABsivA2IA+fmslCjbZJAABgU4S3ATm9TMB+bZMAAMDGCW8DcmIpvB3QNgkAAGyC8DYgxyct0A0AAGye8DYgxyenMz46kkv2jrddCgAAMISEtwE5PjGdA/t3ppTSdikAAMAQEt4GZHGNNy2TAADA5ghvA3JictoyAQAAwKYJbwNQa82DE9O58kLLBAAAAJsjvA3ARKebmbkFywQAAACbJrwNwOllAvYLbwAAwOYIbwNwfGmB7mcJbwAAwCYJbwOwHN60TQIAAJslvA3ActvkFRcIbwAAwOYIbwNwYnI6l+3bmfExP24AAGBzpIkBOD4xnSv3WyYAAADYPOFtABbXeNMyCQAAbJ7wNgAnJqfdrAQAAOiJ8Naw6e58Hp/qWiYAAADoifDWsIcmZ5JYJgAAAOiN8NawByc6SZIrzbwBAAA9EN4atrzGm7ZJAACgF8Jbw04shTdtkwAAQC+Et4Y9ODGdveOjuWDXjrZLAQAAhpjw1rATk9M5oGUSAADokfDWsOMT0653AwAAeia8NezE5Izr3QAAgJ4Jbw1aWKg5MTmdK4U3AACgR8Jbg07OzGVuoeaSveNtlwIAAAw54a1Bk51ukmT/bneaBAAAeiO8NWhCeAMAAPpEeGuQ8AYAAPSL8Nag0+Ftj/AGAAD0RnhrkJk3AACgX4S3BglvAABAvwhvDZrodLNjtGT3jtG2SwEAAIac8NagiU43+3fvSCml7VIAAIAhJ7w1aKLTzYVaJgEAgD4Q3ho0uTTzBgAA0CvhrUETwhsAANAnwluDhDcAAKBfegpvpZTvLKXcWUpZKKUcPmPfT5VS7i2l3FVK+Qe9lTmchDcAAKBfxnp8/SeSfHuSX125sZTygiRvSPLCJM9O8iellOfWWud7PN/QWFiornkDAAD6pqeZt1rrp2qtd62y6/VJbqm1ztRaP5vk3iQ39XKuYXNqdi4L1QLdAABAfzR1zdtVSb6w4vGxpW3njYmpbpJYKgAAAOiLNdsmSyl/kuTKVXa9pdb63l4LKKXcnOTmJDlw4ECOHj3a6yH77tSpUxuu6/6JxQ7RL3zmrhw99ZkGqmK72cw4g40yzhgE44ymGWMMwlYcZ2uGt1rr12/iuA8kuXrF44NL21Y7/tuSvC1JDh8+XI8cObKJ0zXr6NGj2WhdH7z3keRv/jZf85Uvycuuu7SZwthWNjPOYKOMMwbBOKNpxhiDsBXHWVNtk+9L8oZSys5SyrVJbkjydw2da0ua6Cy2TbrmDQAA6Idelwr4tlLKsSQvT/JfSyl/lCS11juTvDPJJ5N8IMkPnE93mkyENwAAoL96Wiqg1vqeJO85y763JnlrL8cfZsIbAADQT021TZ73JjrdjI2U7BkfbbsUAABgGxDeGjLR6eaiPTtSSmm7FAAAYBsQ3hoy0ela4w0AAOgb4a0hk52u690AAIC+Ed4aMiG8AQAAfSS8NUR4AwAA+kl4a4jwBgAA9JPw1oCFheqaNwAAoK+Etwacmp3LQrVANwAA0D/CWwMmprpJYqkAAACgb4S3Bkx0FsObmTcAAKBfhLcGTApvAABAnwlvDTDzBgAA9Jvw1gDhDQAA6DfhrQHCGwAA0G/CWwMmOt2MjZTsGR9tuxQAAGCbEN4aMLG0QHcppe1SAACAbUJ4a8ATS+ENAACgX4S3Bkx2uhboBgAA+kp4a8CEmTcAAKDPhLcGCG8AAEC/CW8NEN4AAIB+E976bGGhZlJ4AwAA+kx467NTs3NZqBboBgAA+kt467OJqW6SZP8e4Q0AAOgf4a3PJjpL4c3MGwAA0EfCW59NCm8AAEADhLc+M/MGAAA0QXjrM+ENAABogvDWZ8IbAADQBOGtzyY63YyNlOwZH227FAAAYBsR3vpsYmmB7lJK26UAAADbiPDWZ8vhDQAAoJ+Etz6b6HRzofAGAAD0mfDWZ5Nm3gAAgAYIb32mbRIAAGiC8NZnwhsAANAE4a2Paq2ZnJ4T3gAAgL4T3vro1Mxc5heq8AYAAPSd8NZHE51ukghvAABA3wlvffTE1GJ4s1QAAADQb8JbH02aeQMAABoivPWRtkkAAKApwlsfnQ5ve4Q3AACgv4S3PjLzBgAANEV466OJTjejIyV7x0fbLgUAANhmhLc+muh0c9HuHSmltF0KAACwzQhvfTTR6WqZBAAAGiG89dFEp2uNNwAAoBHCWx9NmnkDAAAaIrz1kbZJAACgKcJbHwlvAABAU4S3Pqm1ZnJ6TngDAAAaIbz1yamZucwvVOENAABohPDWJxOdbpIIbwAAQCOEtz5ZDm+WCgAAAJogvPWJmTcAAKBJwlufTApvAABAg4S3Pjk987ZHeAMAAPpPeOsTbZMAAECThLc+meh0MzpSsnd8tO1SAACAbUh465OJTjf7d+9IKaXtUgAAgG1IeOuTJ6a6WiYBAIDGCG99MtHpWuMNAABojPDWJ5MdM28AAEBzhLc+mRDeAACABglvfbIY3sbaLgMAANimhLc+qLVmcnrOzBsAANAY4a0PTs3MZX6hCm8AAEBjhLc+mOh0kyQX7R5vuRIAAGC7Et76YDm8WSoAAABoivDWB8vhTdskAADQFOGtDyaFNwAAoGHCWx+cnnnbI7wBAADNEN76QNskAADQNOGtDyY63YyOlOwdH227FAAAYJsS3vpgotPN/t07UkppuxQAAGCbEt76YKIzp2USAABolPDWBxOdrjXeAACARglvfbDcNgkAANAU4a0PJoU3AACgYcJbHyzOvI21XQYAALCNCW89qrVqmwQAABonvPXoydn5zC9U4Q0AAGiU8NajJ6Zmk0R4AwAAGiW89Wii000ivAEAAM0S3nq0HN6s8wYAADRJeOvRpJk3AABgAIS3HmmbBAAABqGn8FZK+c5Syp2llIVSyuEV2w+VUjqllI8uffxK76VuTcIbAAAwCL2uLP2JJN+e5FdX2feZWuuLezz+ljfR6WZ0pGTfTot0AwAAzekpcdRaP5UkpZT+VDOEJjrdXLhr7Lz+GQAAAM1r8pq3a0spd5RS/qKU8soGz9Oqic5cLtoz3nYZAADANldqred+Qil/kuTKVXa9pdb63qXnHE3y47XW25Ye70yyr9b6aCnlpUl+L8kLa62Tqxz/5iQ3J8mBAwdeem9U/rsAABIaSURBVMstt/Tw7TTj1KlT2bdv36r7/v1t0zk5W/Ovvnr3gKtiuznXOIN+Mc4YBOOMphljDEJb4+zVr3717bXWw6vtW7Ntstb69Rs9Ya11JsnM0te3l1I+k+S5SW5b5blvS/K2JDl8+HA9cuTIRk/XuKNHj+Zsdf3yXX+T3UmOHHn5QGti+znXOIN+Mc4YBOOMphljDMJWHGeNtE2WUi4vpYwufX1dkhuS3NfEudrW6c5n9/ho22UAAADbXK9LBXxbKeVYkpcn+a+llD9a2vWqJB8rpXw0ybuSvLnW+lhvpW5Nndn57BHeAACAhvV6t8n3JHnPKtt/N8nv9nLsYTE1O5/dOywTAAAANKvJu02eFxbbJv0YAQCAZkkdPVpsmzTzBgAANEt468HCQk2nO59dO1zzBgAANEt468H03HySuGEJAADQOOGtB51Z4Q0AABgM4a0HU0vhTdskAADQNOGtB52umTcAAGAwhLceaJsEAAAGRXjrgbZJAABgUIS3HnS6c0linTcAAKBxwlsPOrMLSbRNAgAAzRPeejA1uzjztlvbJAAA0DDhrQfTS3eb3G3mDQAAaJjw1oMpd5sEAAAGRHjrwem7TY4JbwAAQLOEtx5Md+eza8dIRkZK26UAAADbnPDWg6nZeTcrAQAABkJ468HU7Lw13gAAgIEQ3now3Z13p0kAAGAghLceTM3OaZsEAAAGQnjrwdSsmTcAAGAwhLceTHfnrfEGAAAMhPDWA3ebBAAABkV464G2SQAAYFCEtx5omwQAAAZFeOuBtkkAAGBQhLdNqrWm053Pbot0AwAAAyC8bdJ0dyFJzLwBAAADIbxt0tTsXJK45g0AABgI4W2TOt35JHG3SQAAYCCEt03qzC6FN22TAADAAAhvmzS1FN60TQIAAIMgvG2StkkAAGCQhLdN0jYJAAAMkvC2SU+1TVrnDQAAaJ7wtknLbZOueQMAAAZBeNukztI6b7u0TQIAAAMgvG2Su00CAACDJLxt0nLbpJk3AABgEIS3TerMzmfn2EhGR0rbpQAAAOcB4W2TOt15LZMAAMDACG+bNDU7b403AABgYIS3TerMzme3mTcAAGBAhLdNWmybtEA3AAAwGMLbJk3NzmmbBAAABkZ42yRtkwAAwCAJb5vkbpMAAMAgCW+b5G6TAADAIAlvm6RtEgAAGCThbZO0TQIAAIMkvG1CrTWdrrZJAABgcIS3TZjuLqTWZLd13gAAgAER3jah051Pkuze4ccHAAAMhvSxCVOzc0mSPWbeAACAARHeNmF6eebNDUsAAIABEd42YWp2uW1SeAMAAAZDeNuE5fBmqQAAAGBQhLdN6GibBAAABkx424TOrPAGAAAMlvC2CafbJne42yQAADAYwtsmaJsEAAAGTXjbhM7SOm/CGwAAMCjC2yZYKgAAABg04W0TOt35jI+NZHSktF0KAABwnhDeNqEzO2+NNwAAYKCEt02Ymp3PHi2TAADAAAlvm9DpzmeXmTcAAGCAhLdN0DYJAAAMmvC2CZ3ZeQt0AwAAAyW8bcKUtkkAAGDAhLdN6MzOuWEJAAAwUMLbJnS6rnkDAAAGS3jbhM6stkkAAGCwhLdNsM4bAAAwaMLbBtVa0+nOZ7eZNwAAYICEtw2amVtIrRHeAACAgRLeNmhqdj5JtE0CAAADJbxtUKe7GN7MvAEAAIMkvG1QZ3YuSbJ7fKzlSgAAgPOJ8LZB2iYBAIA2CG8b1JnVNgkAAAye8LZBU655AwAAWiC8bdD0ctuk8AYAAAyQ8LZBy9e87XbNGwAAMEDC2wZpmwQAANogvG3QU22TlgoAAAAGR3jbIG2TAABAG4S3DZrqzmV8bCSjI6XtUgAAgPNIT+GtlPLvSimfLqV8rJTynlLKRSv2/VQp5d5Syl2llH/Qe6lbw/TsvFk3AABg4HqdefvjJC+qtX55kruT/FSSlFJekOQNSV6Y5LVJ/lMpZVsknqnZecsEAAAAA9dTeKu1/rda69zSww8lObj09euT3FJrnam1fjbJvUlu6uVcW8VUd96dJgEAgIHr5zVvb0ry/qWvr0ryhRX7ji1tG3raJgEAgDaseb/7UsqfJLlylV1vqbW+d+k5b0kyl+S3NlpAKeXmJDcnyYEDB3L06NGNHqJxp06dOl3XFx/qZL5mS9bJcFs5zqApxhmDYJzRNGOMQdiK42zN8FZr/fpz7S+l/PMk35zkNbXWurT5gSRXr3jawaVtqx3/bUneliSHDx+uR44cWbPoQTt69GiW6/qFOz+Y/bt35MiRbdEFyhaycpxBU4wzBsE4o2nGGIOwFcdZr3ebfG2Sn0jyrbXWqRW73pfkDaWUnaWUa5PckOTvejnXVrHYNmmFBQAAYLDWnHlbwy8m2Znkj0spSfKhWuuba613llLemeSTWWyn/IFa63yP59oSprpz2TPe648NAABgY3pKIbXW68+x761J3trL8beizuyCu00CAAADp/9vgzqzc+42CQAADJzwtgG11kx1LdINAAAMnvC2ATNzC6k12WXmDQAAGDDhbQM6s4v3XDHzBgAADJrwtgFTXeENAABoh/C2Acszb9omAQCAQRPeNuCptknrvAEAAIMlvG3A1OxcEm2TAADA4AlvG9DpapsEAADaIbxtgLtNAgAAbRHeNmBKeAMAAFoivG3Actvkbm2TAADAgAlvG7DcNrnbzBsAADBgwtsGLLdNmnkDAAAGTXjbgE53PuOjIxkb9WMDAAAGSwrZgM7snJZJAACgFcLbBnS681omAQCAVghvGzA1O2+ZAAAAoBXC2wZ0Zue1TQIAAK0Q3jZA2yQAANAW4W0Dpsy8AQAALRHeNqDjmjcAAKAlwtsGaJsEAADaIrxtwGLb5FjbZQAAAOch4W0DOrNz2iYBAIBWCG/rVGvVNgkAALRGeFunmbmFLNS42yQAANAK4W2dOrPzSWLmDQAAaIXwtk6d7mJ4c80bAADQBuFtnaaWZ96ENwAAoAXC2zpNd7VNAgAA7RHe1ml55m2Pdd4AAIAWCG/rNDU7l0TbJAAA0A7hbZ20TQIAAG0S3tbpqbZJ4Q0AABg84W2dhDcAAKBNwts6LbdN7hLeAACAFghv63R6nTfXvAEAAC0Q3tZpanY+O0ZLdoz6kQEAAIMniazTdHferBsAANAa4W2dpmbnLNANAAC0Rnhbp6nZeQt0AwAArRHe1knbJAAA0CbhbZ2mZuet8QYAALRGeFunTlfbJAAA0B7hbZ06s9omAQCA9ghv66RtEgAAaJPwtk7aJgEAgDYJb+u02DZpnTcAAKAdwts61FqXFuk28wYAALRDeFuHuZos1GibBAAAWiO8rcPM3OJnd5sEAADaIrytw8x8TWLmDQAAaI/wtg6z84ufXfMGAAC0RXhbh9Mzb9omAQCAlghv6zCzNPOmbRIAAGiL8LYOs0szb9omAQCAtghv63B65s0i3QAAQEuEt3WYXVj8rG0SAABoi/C2DjNz2iYBAIB2CW/r4IYlAABA24S3dZhdsFQAAADQLuFtHWbmkh2jJTtG/bgAAIB2SCPrMDNfs8usGwAA0CLhbR1mF9ysBAAAaJfwtg4zczV7xq3xBgAAtEd4W4eZ+WibBAAAWiW8rcPsQtU2CQAAtEp4W4eZOde8AQAA7RLe1sHdJgEAgLYJb+vgbpMAAEDbhLd1mJkX3gAAgHYJb+swq20SAABomfC2hlqrmTcAAKB1wtsaZucXslCT3WbeAACAFglva5ieXUiS7B4fa7kSAADgfCa8rWGqO5dE2yQAANAu4W0NU7PzSbRNAgAA7RLe1tBZDm9m3gAAgBYJb2vodBfDm7ZJAACgTcLbGrRNAgAAW4HwtgZtkwAAwFYgvK2hc/puk5YKAAAA2iO8rWHX2Giu3FOyd6eZNwAAoD2mk9bwjV/2rOx+9K5cccGutksBAADOY2beAAAAhoDwBgAAMASENwAAgCEgvAEAAAyBnsJbKeXflVI+XUr5WCnlPaWUi5a2HyqldEopH136+JX+lAsAAHB+6nXm7Y+TvKjW+uVJ7k7yUyv2fabW+uKljzf3eB4AAIDzWk/hrdb632qtc0sPP5TkYO8lAQAAcKZ+XvP2piTvX/H42lLKHaWUvyilvLKP5wEAADjvlFrruZ9Qyp8kuXKVXW+ptb536TlvSXI4ybfXWmspZWeSfbXWR0spL03ye0leWGudXOX4Nye5OUkOHDjw0ltuuaWnb6gJp06dyr59+9oug23OOGMQjDMGwTijacYYg9DWOHv1q199e6318Gr71gxvayml/PMk35fkNbXWqbM852iSH6+13nauYx0+fLjedts5n9KKo0eP5siRI22XwTZnnDEIxhmDYJzRNGOMQWhrnJVSzhreer3b5GuT/ESSb10Z3Eopl5dSRpe+vi7JDUnu6+VcAAAA57OxHl//i0l2JvnjUkqSfGjpzpKvSvIzpZRukoUkb661PtbjuQAAAM5bPYW3Wuv1Z9n+u0l+t5djAwAA8JR+3m0SAACAhghvAAAAQ0B4AwAAGALCGwAAwBAQ3gAAAIaA8AYAADAEhDcAAIAhILwBAAAMAeENAABgCAhvAAAAQ6DUWtuu4bRSysNJPtd2Hau4LMkjbRfBtmecMQjGGYNgnNE0Y4xBaGucXVNrvXy1HVsqvG1VpZTbaq2H266D7c04YxCMMwbBOKNpxhiDsBXHmbZJAACAISC8AQAADAHhbX3e1nYBnBeMMwbBOGMQjDOaZowxCFtunLnmDQAAYAiYeQMAABgCwtsaSimvLaXcVUq5t5Tyk23Xw/ZQSrm6lPLnpZRPllLuLKX88NL2S0opf1xKuWfp88Vt18pwK6WMllLuKKX8wdLja0spf7v0nvZfSinjbdfIcCulXFRKeVcp5dOllE+VUl7uvYx+K6X8L0v/Xn6ilPLbpZRd3s/oVSnlHaWUh0opn1ixbdX3r7Lo/1oabx8rpdzYRs3C2zmUUkaT/FKSb0zygiTfVUp5QbtVsU3MJfmxWusLkrwsyQ8sja2fTPKntdYbkvzp0mPoxQ8n+dSKxz+X5BdqrdcneTzJ97RSFdvJf0jygVrr85N8RRbHm/cy+qaUclWSH0pyuNb6oiSjSd4Q72f07jeSvPaMbWd7//rGJDcsfdyc5JcHVOPTCG/ndlOSe2ut99VaZ5PckuT1LdfENlBrfbDW+pGlr09m8Y+dq7I4vn5z6Wm/meS/b6dCtoNSysEkr0vy9qXHJcnXJXnX0lOMMXpSStmf5FVJfi1Jaq2ztdYn4r2M/htLsruUMpZkT5IH4/2MHtVa/zLJY2dsPtv71+uT/Oe66ENJLiqlPGswlT5FeDu3q5J8YcXjY0vboG9KKYeSvCTJ3yY5UGt9cGnX8SQHWiqL7eH/TPITSRaWHl+a5Ila69zSY+9p9OraJA8n+fWl9ty3l1L2xnsZfVRrfSDJzyf5fBZD20SS2+P9jGac7f1rS+QC4Q1aVErZl+R3k/xIrXVy5b66eCtYt4NlU0op35zkoVrr7W3XwrY2luTGJL9ca31JkidzRouk9zJ6tXTN0euz+D8Lnp1kb57Z6gZ9txXfv4S3c3sgydUrHh9c2gY9K6XsyGJw+61a67uXNp9YnoJf+vxQW/Ux9L4mybeWUu7PYsv312Xx2qSLltqOEu9p9O5YkmO11r9devyuLIY572X009cn+Wyt9eFaazfJu7P4Huf9jCac7f1rS+QC4e3cPpzkhqW7GY1n8eLY97VcE9vA0rVHv5bkU7XWf79i1/uS/LOlr/9ZkvcOuja2h1rrT9VaD9ZaD2XxvevPaq3/OMmfJ/mOpacZY/Sk1no8yRdKKc9b2vSaJJ+M9zL66/NJXlZK2bP07+fyOPN+RhPO9v71viRvXLrr5MuSTKxorxwYi3SvoZTyTVm8bmQ0yTtqrW9tuSS2gVLKK5LcmuTjeep6pH+Zxeve3pnkOUk+l+Qf1VrPvJAWNqSUciTJj9dav7mUcl0WZ+IuSXJHkn9Sa51psz6GWynlxVm8Kc54kvuSfHcW/+ew9zL6ppTyr5P8D1m8W/MdSb43i9cbeT9j00opv53kSJLLkpxI8q+S/F5Wef9a+h8Hv5jFlt2pJN9da71t4DULbwAAAFuftkkAAIAhILwBAAAMAeENAABgCAhvAAAAQ0B4AwAAGALCGwAAwBAQ3gAAAIaA8AYAADAE/n93+P5bnbXZBAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "8ACfnz1FRV8E", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "123ca904-521d-4bd7-944f-787a2e8346e3" }, "source": [ "print('веса', w)\n", "print('R^2 = ', r2_score(y_test, np.dot(X_test,w)))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "веса [0.17588524 0.17370687 0.07285169 0.04868733 0.15837534 0.3460636\n", " 0.08444862]\n", "R^2 = 0.8374580182018496\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "xLZZmoD_QQyB", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "185043e1-bf2d-4661-d5be-2fa9a511a85a" }, "source": [ "# массив результатов\n", "r2_shuffles = []\n", "# проверим, зависит ли изначальная перемешка от результата\n", "for i in range(20):\n", " print(f'Итерация {i+1}')\n", " # перемешка\n", " df = df.sample(frac=1).reset_index(drop=True)\n", " # train test split\n", " df_train = df[:400]\n", " df_test = df[400:]\n", " # среднее и стандартное отклонение\n", " mean = df.mean(axis=0)\n", " std = df.std(axis=0)\n", " # 0 мат ожидание и 1 дисперсию\n", " df_train = (df_train - mean)/std\n", " X_train = df_train.drop(columns=['Chance of Admit ']).values\n", " y_train = df_train['Chance of Admit '].values\n", " df_test = (df_test - mean)/std\n", " X_test = df_test.drop(columns=['Chance of Admit ']).values\n", " y_test = df_test['Chance of Admit '].values\n", "\n", " w, mse_train, mse_test, r2 = sgd(X_train, y_train, np.ones(X_train.shape[1]))\n", " print(f'Итерация {i+1} | R^2 = ', r2_score(y_test, np.dot(X_test,w)))\n", " r2_shuffles.append(r2_score(y_test, np.dot(X_test,w)))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Выполнено: 36 %\n", "Mse train: 0.17666929498762216\n", "Mse test: 0.3019622514937877\n", "R2: 0.7218339837286702\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "jlFxViMwXAjH" }, "source": [ "fig1, ax1 = plt.subplots()\n", "ax1.set_title('R^2 в результате перемешки')\n", "ax1.boxplot(r2_shuffles)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "GiNTmE6ngdhx" }, "source": [ "Вывод: сильно зависит от перемешки -> данные нерепрезентативны -> накапливаем еще или аугментируем" ] }, { "cell_type": "markdown", "metadata": { "id": "tvxt53hHFQZJ" }, "source": [ "# Сравнение с Sklearn" ] }, { "cell_type": "markdown", "metadata": { "id": "q8GU1nh3mj7n" }, "source": [ "## Обучение с помощью sklearn" ] }, { "cell_type": "code", "metadata": { "id": "aIrbw9wGmjRc", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "15673994-49f5-4f7c-c617-6737c89020c1" }, "source": [ "from sklearn.linear_model import SGDRegressor\n", "# инициализируем и обучаем\n", "reg = SGDRegressor()\n", "reg.fit(X_train, y_train)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "SGDRegressor(alpha=0.0001, average=False, early_stopping=False, epsilon=0.1,\n", " eta0=0.01, fit_intercept=True, l1_ratio=0.15,\n", " learning_rate='invscaling', loss='squared_loss', max_iter=1000,\n", " n_iter_no_change=5, penalty='l2', power_t=0.25, random_state=None,\n", " shuffle=True, tol=0.001, validation_fraction=0.1, verbose=0,\n", " warm_start=False)" ] }, "metadata": { "tags": [] }, "execution_count": 21 } ] }, { "cell_type": "code", "metadata": { "id": "4dBg3GWs0Jq_", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "6d751e4b-df2b-409a-800b-d98dbddcd5a1" }, "source": [ "print('Mse sgd (sklearn): ', r2_score(y_test, reg.predict(X_test)))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Mse sgd (sklearn): 0.8099229508452628\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "i181uCF21Bs4" }, "source": [ "Настраиваем тонко" ] }, { "cell_type": "code", "metadata": { "id": "lmS4w6U8nHA7", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "8d927a25-b103-4b57-f491-02cee4c29d56" }, "source": [ "from sklearn.model_selection import GridSearchCV\n", "\n", "grid = {'penalty': ['l1', 'l2'],\n", " 'alpha': [1e-4, 1e-5, 1e-6, 1e-7]}\n", "\n", "reg = SGDRegressor()\n", "gs = GridSearchCV(reg, grid, cv=5)\n", "\n", "# Обучаем его\n", "gs.fit(X_train, y_train)\n", "gs.best_params_, gs.best_score_" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "({'alpha': 0.0001, 'penalty': 'l2'}, 0.8097119654612956)" ] }, "metadata": { "tags": [] }, "execution_count": 23 } ] }, { "cell_type": "code", "metadata": { "id": "buEMyrmz0n58", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "42529b2f-a940-4e9e-c935-08307e4056d7" }, "source": [ "from sklearn.linear_model import SGDRegressor\n", "# инициализируем и обучаем\n", "reg = SGDRegressor(alpha = 1e-05, penalty = 'l2')\n", "reg.fit(X_train, y_train)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "SGDRegressor(alpha=1e-05, average=False, early_stopping=False, epsilon=0.1,\n", " eta0=0.01, fit_intercept=True, l1_ratio=0.15,\n", " learning_rate='invscaling', loss='squared_loss', max_iter=1000,\n", " n_iter_no_change=5, penalty='l2', power_t=0.25, random_state=None,\n", " shuffle=True, tol=0.001, validation_fraction=0.1, verbose=0,\n", " warm_start=False)" ] }, "metadata": { "tags": [] }, "execution_count": 24 } ] }, { "cell_type": "code", "metadata": { "id": "PWG_X64r0o1V", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "c792db64-b042-4b5e-de24-25af6bfc76f4" }, "source": [ "print('Mse sgd (sklearn): ', r2_score(y_test, reg.predict(X_test)))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Mse sgd (sklearn): 0.8085748782624143\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "s5PlFmWT1Lb3", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "e3a0ca9d-01e9-4668-b068-6dd43ae94d1c" }, "source": [ "fin_score = []\n", "\n", "for i in range(30):\n", " print(f'Итерация {i+1}')\n", " # перемешка\n", " df = df.sample(frac=1).reset_index(drop=True)\n", " # train test split\n", " df_train = df[:400]\n", " df_test = df[400:]\n", " # среднее и стандартное отклонение\n", " mean = df.mean(axis=0)\n", " std = df.std(axis=0)\n", " # 0 мат ожидание и 1 дисперсию\n", " df_train = (df_train - mean)/std\n", " X_train = df_train.drop(columns=['Chance of Admit ']).values\n", " y_train = df_train['Chance of Admit '].values\n", " df_test = (df_test - mean)/std\n", " X_test = df_test.drop(columns=['Chance of Admit ']).values\n", " y_test = df_test['Chance of Admit '].values\n", "\n", " # обучение\n", " grid = {'penalty': ['l1', 'l2'],\n", " 'alpha': [1e-4, 1e-5, 1e-6, 1e-7]}\n", "\n", " reg = SGDRegressor()\n", " gs = GridSearchCV(reg, grid, cv=5, scoring = 'r2')\n", "\n", " # Обучаем его\n", " gs.fit(X_train, y_train)\n", " print(gs.best_score_)\n", " fin_score.append(gs.best_score_)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Итерация 1\n", "0.8109177706435201\n", "Итерация 2\n", "0.8198261986327733\n", "Итерация 3\n", "0.8092944066597569\n", "Итерация 4\n", "0.8078472515506094\n", "Итерация 5\n", "0.8119252273501584\n", "Итерация 6\n", "0.7969072937292928\n", "Итерация 7\n", "0.7956026082377654\n", "Итерация 8\n", "0.8036936558068124\n", "Итерация 9\n", "0.8060383820051505\n", "Итерация 10\n", "0.8032334665712701\n", "Итерация 11\n", "0.8063744122411365\n", "Итерация 12\n", "0.814178695781138\n", "Итерация 13\n", "0.8113545721445821\n", "Итерация 14\n", "0.8000055833354516\n", "Итерация 15\n", "0.7936631228843594\n", "Итерация 16\n", "0.8128613722777185\n", "Итерация 17\n", "0.8174970129769239\n", "Итерация 18\n", "0.7978412281418048\n", "Итерация 19\n", "0.8081217313303494\n", "Итерация 20\n", "0.8204904791626919\n", "Итерация 21\n", "0.8159335375063641\n", "Итерация 22\n", "0.8187848150067663\n", "Итерация 23\n", "0.8072786430431321\n", "Итерация 24\n", "0.8033467813126485\n", "Итерация 25\n", "0.8249546871718272\n", "Итерация 26\n", "0.8244556550113333\n", "Итерация 27\n", "0.8117539794599405\n", "Итерация 28\n", "0.7904734177162176\n", "Итерация 29\n", "0.8014776460898485\n", "Итерация 30\n", "0.8105552181734911\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "tlUhmuP74t8G", "colab": { "base_uri": "https://localhost:8080/", "height": 417 }, "outputId": "0bcca010-aebe-4481-d985-0ec7de53c3ae" }, "source": [ "fig1, ax1 = plt.subplots()\n", "ax1.set_title('R^2 в результате перемешки (+ регуляризация)')\n", "ax1.boxplot(fin_score)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "{'boxes': [],\n", " 'caps': [,\n", " ],\n", " 'fliers': [],\n", " 'means': [],\n", " 'medians': [],\n", " 'whiskers': [,\n", " ]}" ] }, "metadata": { "tags": [] }, "execution_count": 27 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAcuklEQVR4nO3dfbQcZYHn8e/PQECR8JarqwmB7BBXorKgLeoZHQcRDEGFHR1NBIEdBBWDRwZf0AGNCLuD6x5nXAEFByOoxBwVzYy4MI6goqC5kdfgwIbXBFAu8o4DMfG3f9Rzoej0vd33Jbnc1O9zTp/bXfXU00/1rapf1VPVXbJNREQ0z7MmugERETExEgAREQ2VAIiIaKgEQEREQyUAIiIaKgEQEdFQCQBA0v6STpG0/US3JWK8SXqTpO9NdDtibCT9StJLxrPOLTYAJN0u6T8kPSrpt5KWSHpuh3KvA74LzAcukjS1bfxHJN0g6RFJt0n6yGaahYjxcjrw9xPdiBizzwGnjmeFW2wAFG+x/Vxgb2Af4OP1kZL2ApYBC4G/AB4CLpBU/1wEHAHsBMwDFklasBnaHjFmkl4J7GD7qiHGL5F01OZtVYzScmA/Sf9pvCrc0gMAANu/BS6hCgIAJO0OfAc43PbFtv8IvBNYD/xjbdrP2v617fW2bwK+D/x5p/eRtLskl6OOR8uRw192KXuspLsl3SPpw7Xxz5J0kqRbJP1e0jJJO5dxfybppnJU8jtJp5Xhryyvp9Tq+StJ19ZeHyVpQ62NlrRHGXe5pPd0aOs7a+U3SHp88HUZv6+kKyU9WObji4NHUpL+uZR9rO2z+VIZ/0JJ35E0UI6wPjjU/7FsrNbV6nhMkmvjd5D0T6UNd0k6bfCzKPP989K2hyT9u6T9RzCtJZ1QKz+/DDutNuzNkq4pn8Mvyg7G4LjbJb2xPN+qTLt7eb2HpPW1sk/+H8r/eo2kt9backWt7EdLXW8c4mM7CPjJUJ9pr8a4vLavF49KOkPSX0ta2fY+fyvp+7XXiyX9sW0Z2qqMe235vB8p4/+ksr6VZeVLkv61jP+JpN1q9daX+1mqegu+Xl53XL/KuIMlXS3p4fJ/WdzhM9qqNuzrbWWGfN8y7L2S7ui0fNt+HFgJvGnE/8AhNCIAJM2kWhFWDw6zfbvtObb/rTZsve3DbB8/RD0CXges6vKWOwLbUx1dfK5L2f2AOcCBwMdqK/LxwKHA64EXAg8AZ5Zx91J1WU0DXg28R9LLbK8Afl/qGvRu4Pza62cBv7D93HJ01JXtb9XK/wxY1Db9BuAEYDrwGmB/4Lgy7eBR2GDf5Y5l2vepOtL6Z+BaYEaZ7kOShlvAP1t77//aNm4JVYDvQXXEdyBQD7RXAbeUdn4K+O7gRqqHaVcDR9Zevwf4zeALSfsA5wHvBXYBvgwsl7TNMPMyLEkvoNpx+YTt5R3G7wx8EHhwmGpeBtw02jZ0MJrlddDg//65tj9GtUc7W9KetTKdltelbcvQoM8BFwHTyvi728YfBnyG6v99DfCNIebpM1TrzaCO61cZ9xhVj8COwMHA+yUdOkS93TztfSVtB5wFHDnE8g3VMtdp+Khs6QHwPUmPAGuo/qmfGmN9i6k+s6/2UFbAFJ6+YHXyaduP2b6+1LuwDH8f8He219p+orz32yVtZfsR27e4+iEnAb/jqYX/a8Dh8OQG4k3AN2vvNxVY10P7e2Z7pe2rSoDeTrXxe30Pk74S6LN9qu11tm8FzgVG3MUm6flUK+2Hyud5L/D5trruBf7B9h9tf4tqw3hwj9P+Drhd0mtK+d2AX9XGHwt82fYvbW+w/TXgCaoNyGjsRLXx/4btC4Yo8wmq0HlomHp2BB4ZZRs6GfHyOlRFpdy3eGp5fQmwO/AvtWLdltcpVOtAJz+w/dPyPn8HvEbSrvUC5SjtNVTrzWC7hly/bF9u+3rbf7J9HXAhvS3rT9Ppfam2LX8ChvzMqP6XO470/YaypQfAoba3B/4SeDHVnsCoSFpElfwHlwVqOPcBjwIfAs7oUnZN7fkdVHtPUG1gLirdCQ9SJf8G4PmlPbMkPUS1Z3oFT63kXwfeUvYm3gH8zPY9tffYmWrvbChf0FNdOedJ2rZL+5H0Ikn/oupk+8PA/6C3z3o34IWD81jm8xOD8zhCuwFbA/fU6voy8Lxambv89F8/HPy8e5kW4CtUe/5H8vS91MH3P7FtXnblqf/nSH2aahl6QznyfJrSnfEO4H91qecBqqPR+rTX1dr4LuCsWrvP6lLfqJbXYXwNeFeZx3cDy9rWr+GW1+OBtwKPl/ds/6yfbKvtR4H7O5Q5AzgF+GN94FDrl6RXSbpMVZflQ1TBN5rtykbva/sR4GjgfEl/AH7dYbrtGf6Ib0S29AAAwPZPqA7xu3XHdCTpb4CTgP1tr+1hkum2nwMcAnxH0rOHKVvfI5nFU3vya4CDbO9Ye2xr+y4A23fa3oGq6+T1VAsOZfyVwF9RrVDte48vAm4epj0ftL0j1eH2K4Cjus4tnA38OzDH9jSqjfhQe2V1a4Db2uZxe9vze5i2U11PUH32g3VNs13vNpjRtjEd/Lx7mRbgh1Tnf45k4891DXB627w8x/aFo5gXqLoPX0v1OXbqkvwMVXdYt73766j+50+yvddgG6mODo+rtfm4LvWNankdiquT0+uoulbfxQiW19LleQfVkceObNwF9GRbVV0BuHNbmTdQddct61B3x/WL6vNaDuxaxn+J3pb1uiHfF/geVSjsB7y8w/g9qbpMx0UjAqD4B+AASSPqP5N0GNUe7QGli2IkNgA7UB3GDuUUSc8ph7//neqQGKoF6/TBE1eS+iQdUp7PrPVdT6U6DP6PWp3nAx+l6v/9bm1e/pyqn7aXa8L/QLVR7GUZ2R54GHhU0ouB9/cwDVRdKI9I+pikZ0uaIumlqq5cGZFylHMp8L8lTVN1UvLPJNUPz58HfFDS1pL+mmplurjHabG9gWrP7eu2729rwrnA+8oeoiRtp+qE4Wi/W3KF7T8BfwN8UtJ/ro3bg+p8xpd7qOdiRtFFMYwRL689OB/4IvBH21eU6VWmb1EF70YkvYMqhD4/RL3zVZ0onkoVmFfZrh/BLAY+2nZU2G392h643/bjkvalCq2R6vi+xd8Dy23/sn1EORp/BfCvo3jPjhoTALYHqBa0T45w0tOo0nqF2q5gGcaDqq6QOR94r+3h+mh/QnWY+W/A52xfWob/I9WexqXlPMZVVCs9VBv2q8vwX1Ct5PU9p4soh+S2/wAgaS7V4faHbdf7rtt9VtJa4NbSrl7Od3yYakV4hGpD+K3hi1fKBvXNVFdn3UbVdfYVqtAcjSOoVtgbqboNvg28oDb+l1QnMO+jujb+7bZ/3+O0g23+qu3/2WF4P3AM1YbsAarP7qi2YheWz/b28vrK8vpnQ82Q7ZupNgpfqR29PB842dWVa8Oy/WvgIUmv6la2R6NZXru5AHgpVffloHlU695hbRttACTtRLXhP8b2+vbxxTepzvvdT7XhPLxt/NW2L+8w3XDr13HAqWXcJ+m8F3+7pLXlf/vfgL8tOxzDvm/ZQTuY6gi6k7cAl9tuP9IZNXUOodjUVF0CeBuw9TAL8Fjqv4UqfH403nVPRqqudX+P7ddOdFs2N0kHUnXzjPZqlU26vJYu0nuBl9v+f+NU5xJgre2Tx6O+MbZlMXC77SVjrOeXwNG2bxiPdsHwZ5tjkpL0NsDAjye6LTHxyl76pV0LTpz3AyvGa+P/DHQr8NuxVmJ7vI7inpQA2MJIuhyYC7y79CFHPGNJup3qJOqoj06e6Wy3XzH2jJEuoIiIhmrMSeCIiHi6SdUFNH36dO++++4T3YyIiEll5cqV99nuax8+qQJg9913p7+/f6KbERExqUi6o9PwdAFFRDRUAiAioqESABERDZUAiIhoqARARERD9RQAkuapukXaakkndRg/q/xG9tXlt8bnl+EHSFop6fry9w21aS4vdV5THu2/vR4REZtQ18tAVd0X9UzgAGAt1a9iLrd9Y63YyVQ3cji7/OrkxVR39rmP6sbsd0t6KdUdjmbUpjus/IpiRERsZr0cAewLrLZ9q+11wFKqG53Umer+mVD9lO/g7dOurv106Srg2RrDPVIjImL89PJFsBk8/TZwa9n4d74XU/0O+PHAdsAb2djbgF+33e7tq5I2AN8BTut0gwRJx1Ldb5VZs2b10NyIsdPGd2HcJPJbXDGRxusk8EJgie2ZVDfXvkDSk3WXuwedAby3Ns1htl9GdSu411HdvnAjts+x3bLd6uvb6JvMEZuE7RE9RjNNNv4x0XoJgLt4+n1AZ5ZhdUdT7oxj+0pgW8qNkiXNpLpD1RG2bxmcwE/d2/YRqjv37Du6WYiIiNHoJQBWAHMkzS731lxAdeu3ujuB/QEk7UkVAAOSdgR+AJxk++eDhSVtJWkwILamui3guN3lJiIiuusaAOX2b4uoruD5DdXVPqsknSrpraXYicAxkq4FLgSOKv35i6huYP3Jtss9twEukXQdcA3VEcW54z1zERExtEl1Q5hWq+X8Gmg8E0lKn348Y0laabvVPjzfBI6IaKgEQEREQyUAIiIaKgEQEdFQCYCIiIZKAERENFQCICKioRIAERENlQCIiGioBEBEREMlACIiGioBEBHRUAmAiIiGSgBERDRUAiAioqESABERDZUAiIhoqARARERDJQAiIhqqpwCQNE/STZJWSzqpw/hZki6TdLWk6yTNL8MPkLRS0vXl7xtq07yiDF8t6QuSNH6zFRER3XQNAElTgDOBg4C5wEJJc9uKnQwss70PsAA4qwy/D3iL7ZcBRwIX1KY5GzgGmFMe88YwHxERMUK9HAHsC6y2favtdcBS4JC2Mgamlec7AHcD2L7a9t1l+Crg2ZK2kfQCYJrtq2wbOB84dIzzEhERI7BVD2VmAGtqr9cCr2orsxi4VNLxwHbAGzvU8zbg17afkDSj1FOvc0avjY6IiLEbr5PAC4EltmcC84ELJD1Zt6SXAGcA7x1pxZKOldQvqX9gYGCcmhsREb0EwF3ArrXXM8uwuqOBZQC2rwS2BaYDSJoJXAQcYfuWWp0zu9RJqe8c2y3brb6+vh6aGxERveglAFYAcyTNljSV6iTv8rYydwL7A0jakyoABiTtCPwAOMn2zwcL274HeFjSq8vVP0cA3x/z3ERERM+6BoDt9cAi4BLgN1RX+6ySdKqkt5ZiJwLHSLoWuBA4qpzcXQTsAXxS0jXl8bwyzXHAV4DVwC3AD8dzxiIiYniqttOTQ6vVcn9//0Q3I2IjkphM61I0i6SVtlvtw/NN4IiIhkoAREQ0VAIgIqKhEgAREQ2VAIiIaKgEQEREQyUAIiIaKgEQEdFQCYCIiIZKAERENFQCICKioRIAERENlQCIiGioBEBEREMlACIiGioBEBHRUAmAiIiGSgBERDRUAiAioqESABERDdVTAEiaJ+kmSaslndRh/CxJl0m6WtJ1kuaX4buU4Y9K+mLbNJeXOq8pj+eNzyxFREQvtupWQNIU4EzgAGAtsELScts31oqdDCyzfbakucDFwO7A48ApwEvLo91htvvHNgsRETEavRwB7Austn2r7XXAUuCQtjIGppXnOwB3A9h+zPYVVEEQERHPIL0EwAxgTe312jKsbjFwuKS1VHv/x/f4/l8t3T+nSFKnApKOldQvqX9gYKDHaiMiopvxOgm8EFhieyYwH7hAUre6D7P9MuB15fHuToVsn2O7ZbvV19c3Ts2NiIheAuAuYNfa65llWN3RwDIA21cC2wLTh6vU9l3l7yPAN6m6miIiYjPpJQBWAHMkzZY0FVgALG8rcyewP4CkPakCYMj+GklbSZpenm8NvBm4YeTNj4iI0ep6FZDt9ZIWAZcAU4DzbK+SdCrQb3s5cCJwrqQTqE4IH2XbAJJupzpBPFXSocCBwB3AJWXjPwX4EXDuuM9dREQMSWU7PSm0Wi339+eq0XjmkcRkWpeiWSSttN1qH55vAkdENFTXLqCIyW7nnXfmgQce2OTvM8SVzONmp5124v7779+k7xHNkgCILd4DDzywRXTPbOqAieZJF1BEREMlACIiGioBEBHRUAmAiIiGSgBERDRUAiAioqESABERDZUAiIhoqARARERDJQAiIhoqARAR0VAJgIiIhkoAREQ0VAIgIqKhEgAREQ2VAIiIaKgEQEREQ/UUAJLmSbpJ0mpJJ3UYP0vSZZKulnSdpPll+C5l+KOSvtg2zSskXV/q/IJyu6OIiM2qawBImgKcCRwEzAUWSprbVuxkYJntfYAFwFll+OPAKcCHO1R9NnAMMKc85o1mBiIiYnR6OQLYF1ht+1bb64ClwCFtZQxMK893AO4GsP2Y7SuoguBJkl4ATLN9laubtZ4PHDr62YiIiJHqJQBmAGtqr9eWYXWLgcMlrQUuBo7voc61XeoEQNKxkvol9Q8MDPTQ3IiI6MV4nQReCCyxPROYD1wgaVzqtn2O7ZbtVl9f33hUGRER9BYAdwG71l7PLMPqjgaWAdi+EtgWmN6lzpld6oyIiE2olwBYAcyRNFvSVKqTvMvbytwJ7A8gaU+qABiyv8b2PcDDkl5drv45Avj+KNofERGjtFW3ArbXS1oEXAJMAc6zvUrSqUC/7eXAicC5kk6gOiF8VDm5i6TbqU4QT5V0KHCg7RuB44AlwLOBH5ZHRERsJirb6Umh1Wq5v79/opsRk4wkJtNyPpQtZT5i85O00narfXi+CRwR0VBdu4AiJjt/ahos3mGimzFm/tS07oUiRiABEFs8ffrhLaLrRBJePNGtiC1JuoAiIhoqARAR0VAJgIiIhkoAREQ0VAIgIqKhEgAREQ2VAIiIaKgEQEREQyUAIiIaKgEQEdFQCYCIiIZKAERENFQCICKioRIAERENlQCIiGioBEBEREP1FACS5km6SdJqSSd1GD9L0mWSrpZ0naT5tXEfL9PdJOlNteG3S7pe0jWScqPfiIjNrOsdwSRNAc4EDgDWAiskLbd9Y63YycAy22dLmgtcDOxeni8AXgK8EPiRpBfZ3lCm28/2feM4PxER0aNejgD2BVbbvtX2OmApcEhbGQODNyzdAbi7PD8EWGr7Cdu3AatLfRERMcF6CYAZwJra67VlWN1i4HBJa6n2/o/vYVoDl0paKenYEbY7IiLGaLxOAi8EltieCcwHLpDUre7X2n45cBDwAUl/0amQpGMl9UvqHxgYGKfmRkRELwFwF7Br7fXMMqzuaGAZgO0rgW2B6cNNa3vw773ARQzRNWT7HNst262+vr4emhsREb3oJQBWAHMkzZY0leqk7vK2MncC+wNI2pMqAAZKuQWStpE0G5gD/ErSdpK2L+W3Aw4EbhiPGYqIiN50vQrI9npJi4BLgCnAebZXSToV6Le9HDgROFfSCVR9+0fZNrBK0jLgRmA98AHbGyQ9H7hI0mAbvmn7/26KGYyIiM5Ubacnh1ar5f7+fGUgRkYSk2k5H8qWMh+x+UlaabvVPjzfBI6IaKgEQEREQyUAIiIaKgEQEdFQCYCIiIbqehloxJagXHI8qe20004T3YTYwiQAYou3OS6dzCWaMRmlCygioqESABERDZUAiIhoqARARERDJQAiIhoqARAR0VAJgIiIhkoAREQ0VAIgIqKhEgAREQ2VAIiIaKgEQEREQyUAIiIaqqcAkDRP0k2SVks6qcP4WZIuk3S1pOskza+N+3iZ7iZJb+q1zoiI2LS6BoCkKcCZwEHAXGChpLltxU4GltneB1gAnFWmnVtevwSYB5wlaUqPdUZExCbUyxHAvsBq27faXgcsBQ5pK2NgWnm+A3B3eX4IsNT2E7ZvA1aX+nqpMyIiNqFeAmAGsKb2em0ZVrcYOFzSWuBi4Pgu0/ZSJwCSjpXUL6l/YGCgh+ZGREQvxusk8EJgie2ZwHzgAknjUrftc2y3bLf6+vrGo8qIiKC3W0LeBexaez2zDKs7mqqPH9tXStoWmN5l2m51RkTEJtTLXvoKYI6k2ZKmUp3UXd5W5k5gfwBJewLbAgOl3AJJ20iaDcwBftVjnRERsQl1PQKwvV7SIuASYApwnu1Vkk4F+m0vB04EzpV0AtUJ4aNc3SF7laRlwI3AeuADtjcAdKpzE8xfREQMQdV2enJotVru7++f6GZEbEQSk2ldimaRtNJ2q314vgkcEdFQCYCIiIZKAERENFQCICKioRIAERENlQCIiGioBEBEREMlACIiGioBEBHRUAmAiIiGSgBERDRUAiAioqESABERDZUAiIhoqARARERDJQAiIhoqARAR0VAJgIiIhkoAREQ0VAIgIqKhegoASfMk3SRptaSTOoz/vKRryuNmSQ/Wxp0h6YbyeGdt+BJJt9Wm23t8ZikiInqxVbcCkqYAZwIHAGuBFZKW275xsIztE2rljwf2Kc8PBl4O7A1sA1wu6Ye2Hy7FP2L72+M1MxER0btejgD2BVbbvtX2OmApcMgw5RcCF5bnc4Gf2l5v+zHgOmDeWBocERHjo5cAmAGsqb1eW4ZtRNJuwGzgx2XQtcA8Sc+RNB3YD9i1Nsnpkq4rXUjbDFHnsZL6JfUPDAz00NyIiOjFeJ8EXgB82/YGANuXAhcDv6A6KrgS2FDKfhx4MfBKYGfgY50qtH2O7ZbtVl9f3zg3NyKiuXoJgLt4+l77zDKskwU81f0DgO3Tbe9t+wBAwM1l+D2uPAF8laqrKSIiNpNeAmAFMEfSbElTqTbyy9sLSXoxsBPVXv7gsCmSdinP9wL2Ai4tr19Q/go4FLhhbLMSEREj0fUqINvrJS0CLgGmAOfZXiXpVKDf9mAYLACW2nZt8q2Bn1XbeB4GDre9voz7hqQ+qqOCa4D3jcscRURET/T07fUzW6vVcn9//0Q3I2IjkphM61I0i6SVtlvtw/NN4IiIhkoAREQ0VAIgIqKhEgAREQ2VAIiIaKgEQEREQyUAIiIaKgEQEdFQCYCIiIZKAERENFQCICKioRIAERENlQCIiGioBEBEREMlACIiGioBEBHRUAmAiIiGSgBERDRUAiAioqF6CgBJ8yTdJGm1pJM6jP+8pGvK42ZJD9bGnSHphvJ4Z234bEm/LHV+S9LU8ZmliIjoRdcAkDQFOBM4CJgLLJQ0t17G9gm297a9N/B/gO+WaQ8GXg7sDbwK+LCkaWWyM4DP294DeAA4enxmKWLsJI3oMZppBqeLmCi9HAHsC6y2favtdcBS4JBhyi8ELizP5wI/tb3e9mPAdcA8VUv+G4Bvl3JfAw4dzQxEbAq2N8sjYiL1EgAzgDW112vLsI1I2g2YDfy4DLqWaoP/HEnTgf2AXYFdgAdtr+9WZ0REbBpbjXN9C4Bv294AYPtSSa8EfgEMAFcCG0ZSoaRjgWMBZs2aNb6tjYhosF6OAO6i2msfNLMM62QBT3X/AGD79HJ+4ABAwM3A74EdJQ0G0JB12j7Hdst2q6+vr4fmRkREL3oJgBXAnHLVzlSqjfzy9kKSXgzsRLWXPzhsiqRdyvO9gL2AS111fl4GvL0UPRL4/lhmJCIiRqZrF5Dt9ZIWAZcAU4DzbK+SdCrQb3swDBYAS/30M1tbAz8rVzs8DBxe6/f/GLBU0mnA1cA/jcscRURETzSZrkRotVru7++f6GZEREwqklbabrUPzzeBIyIaKgEQEdFQk6oLSNIAcMdEtyOig+nAfRPdiIgh7GZ7o8soJ1UARDxTServ1Mca8UyWLqCIiIZKAERENFQCIGJ8nDPRDYgYqZwDiIhoqBwBREQ0VAIgIqKhEgARYyDpPEn3SrphotsSMVIJgIixWQLMm+hGRIxGAiBiDGz/FLh/otsRMRoJgIiIhkoAREQ0VAIgIqKhEgAREQ2VAIgYA0kXUt0H+79IWivp6IluU0Sv8lMQERENlSOAiIiGSgBERDRUAiAioqESABERDZUAiIhoqARARERDJQAiIhrq/wOnlUHa7SJynQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "WAJIdMiW3fpS" }, "source": [ "# Вывод\n", "\n", "- Реализован стохастический градиентный спуск (СГС) для задачи линейной регрессии\n", "\n", "- Приведено сравнение написанного СГС и реализованного в библиотеке sklearn\n" ] }, { "cell_type": "markdown", "metadata": { "id": "NUmxK0kjNwXe" }, "source": [ "# Рекомендованная литература\n", "\n", "\n", "- [Линейная регрессия в подробностях](https://habr.com/ru/company/ods/blog/322076/)\n", "\n", "\n", "- [Функция ошибок в задачах регрессии](https://alexanderdyakonov.files.wordpress.com/2018/10/book_08_metrics_12_blog1.pdf)\n", "\n", "- Градиентный спуск:\n", " - [Часть 1](https://habr.com/ru/post/307312/)\n", " - [Часть 2](https://habr.com/ru/post/308604/)\n" ] } ] }