{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "s4wplUzDYJcx" }, "source": [ "

Многослойная сеть на PyTorch

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "J2msuyHTYJcx" }, "source": [ "В этом ноутбке мы научимся писать свои нейросети на фреймворке PyTorch, конкретно - рассмотрим, как написать многослойную полносвязную сеть (Fully-Connected, FC)." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "9xJnMEZrYJcz" }, "source": [ "

Компоненты нейросети

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "InwacmvIYJc0" }, "source": [ "Здесь самое время напомнить о том, какие вещи играют принципиальную роль в построении любой ***нейронной сети*** (все их мы задаём *руками*, самостоятельно): \n", "\n", "- непосредственно, сама **архитектура** нейросети (сюда входят типы функций активации у каждого нейрона);\n", "- начальная **инициализация** весов каждого слоя;\n", "- метод **оптимизации** нейросети (сюда ещё входит метод изменения `learning_rate`);\n", "- размер **батчей** (`batch_size`);\n", "- количество итераций обучения (`num_epochs`);\n", "- **функция потерь** (`loss`); \n", "- тип **регуляризации** нейросети (для каждого слоя можно свой); \n", "\n", "То, что связано с ***данными и задачей***: \n", "- само **качество** выборки (непротиворечивость, чистота, корректность постановки задачи); \n", "- **размер** выборки; " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "tXujEOB0YJc1" }, "source": [ "

Многослойная нейронная сеть

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "mnxH-DajYJc3" }, "source": [ "Как можно понять из названия, многослойная нейросеть состоит из нескольких **слоёв**. Каждый слой состоит из **нейронов**. Ранее мы уже писали свой нейрон на NumPy, вот из таких нейронов и состоит ***MLP (Multi-Layer Perceptron)***. Ещё такую многослойную нейросеть, у которой каждый нейрон на предыдущем уровне соединён с нейроном на следующем уровне, называют ***Fully-Connected-сетью*** (или ***Dense-сетью***). \n", "\n", "Расмотрим их устройство более подробно:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "onjJUneMYJc5" }, "source": [ "* Вот так выглядит двухслойная нейросеть (первый слой - input layer - не считается, потому что это, по сути, не слой):" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "owRRulLzYJc6" }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "tFNxGGBEYJc8" }, "source": [ "* Так выглядит трёхслойная нейросеть:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zRaKX35eYJc9" }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "6w1FTkO1YJc-" }, "source": [ ".. и так далее для большего случая слоёв." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "8iKV7m5YYJc_" }, "source": [ "**Обратите внимание:** связи есть у нейронов со слоя $L_{i-1}$ и нейронов $L_{i}$, но между нейронами в одном слое связей **нет**." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "URV9qWkfYJdA" }, "source": [ "**Входной слой** -- это данные (матрица $(n, m)$)." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zK8tWuHHYJdB" }, "source": [ "Слои, которые не являются входными или выходными, называются **скрытыми слоями (hidden layers)**." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "fz9clUlCYJdC" }, "source": [ "При решении ***задачи регрессии*** на **выходном слое** обычно один нейрон, который возвращает предсказанные числа (для каждого объекта по числу). \n", "\n", "В случае ***задачи классификации*** на **выходном слое** обычно один нейрон, если задача бинарной классификации, и $K$ нейронов, если задача $K$-класовой классификации." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "nJblXqY5YJdE" }, "source": [ "#### Forward pass в MLP" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "D87xoAl8YJdF" }, "source": [ "Каждый слой многослойной нейросети - это матрица весов, столбцы которой -- это нейроны (один столбец - один нейрон). То есть один столбец -- это веса одного нейрона." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "collapsed": true, "id": "RyXUqCfVYJdG" }, "source": [ "Допустим, мы решаем задачу $K$-классовой классификации (на последнем слое $K$ нейронов). Рассмотрим, как в таком случае выглядит `forward_pass` нейросети:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "YO6gHbOjYJdH" }, "source": [ "* Вход: $$X =\n", "\\left(\n", "\\begin{matrix} \n", "x_{11} & ... & x_{1M} \\\\\n", "... & \\ddots & ...\\\\\n", "x_{N1} & ... & x_{NM} \n", "\\end{matrix}\n", "\\right)\n", "$$\n", "\n", "-- матрица $(N, M)$" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "collapsed": true, "id": "XdImZiQkYJdI" }, "source": [ "* Структура сети - много слоёв, в слоях много нейронов. Первый слой (после входного) выглядит так:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "U2c2M4MJYJdJ" }, "source": [ "$$ W^1 =\n", "\\left(\n", "\\begin{matrix} \n", "w_{11} & ... & w_{1L_1} \\\\\n", "... & \\ddots & ...\\\\\n", "w_{M1} & ... & w_{ML_1} \n", "\\end{matrix}\n", "\\right)\n", "$$\n", "\n", "-- матрица $(M, L_1)$" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "UUCdeLN0YJdK" }, "source": [ "То есть это в точности $L_1$ нейронов, каждый имеет свои собственные веса, их $M$ штук." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ixHtlMwKYJdL" }, "source": [ "Мы помним, что нейрон - это линейное преобразование и потом нелинейная функция активации от этого преобразования. Однако в многослойных нейростеях часто отделяют `Linear` часть и `Activation`, то есть слоем считаем набор весов нейронов, а следующий слой всегда функция активации (у всех нейронов из слоя она одна и та же, обычно фреймворки не позволяют задавать конкретному нейрону в слое отличную от других нейронов в этом слое функцию активации, однако это легко сделать, объявив слой из одного нейрона)." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AUt1NgTvYJdN" }, "source": [ "* Другие слои выглядит точно так же, как первый слой. Например, второй слой будет такой:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "IdtSTvTmYJdN" }, "source": [ "$$ W^2 =\n", "\\left(\n", "\\begin{matrix} \n", "w_{11} & ... & w_{1L_2} \\\\\n", "... & \\ddots & ...\\\\\n", "w_{L_11} & ... & w_{L_1L_2} \n", "\\end{matrix}\n", "\\right)\n", "$$\n", "\n", "-- матрица $(L_1, L_2)$" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "R3nGnHoLYJdP" }, "source": [ "То есть это в точности $L_2$ нейронов, каждый имеет свои собственные веса, их $L_1$ штук." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "bQhOAQjSYJdR" }, "source": [ "* Выходной слой: \n", "\n", "Пусть в нейросети до выходного слоя идут $t$ слоёв. Тогда выходной слой имеет форму:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "fWvqm-K0YJdT" }, "source": [ "$$ W^{out} =\n", "\\left(\n", "\\begin{matrix} \n", "w_{11} & ... & w_{1K} \\\\\n", "... & \\ddots & ...\\\\\n", "w_{L_t1} & ... & w_{L_tK} \n", "\\end{matrix}\n", "\\right)\n", "$$\n", "\n", "-- матрица $(L_t, K)$, где $L_t$ - количество нейронов в $t$-ом слое, а $K$ -- количество классов." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "2z5tO89NYJdU" }, "source": [ "В итоге *для `forward_pass` нам нужно просто последовтельно перемножить матрицы друг за другом, применяя после каждого умножения соответсвующую функцию активации*." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "IZT4GgsCYJdV" }, "source": [ "*Примечание*: можно думать об умножении на очередную матрицу весов как на переход в **новое признаковое пространство**. Действительно, когда подаём матрицу $X$ и умножаем на матрицу первого слоя, мы получаем матрицу размера $(N, L_1)$, то есть как будто $L_1$ \"новых\" признаков (построенных как линейная комбинация старых до применения функции активации, и уже как нелинейная комбинация после активации). Здесь уместно вспомнить, что Deep Learning является пообластью Representation Learning, то есть позволяет выучивает новые представляения данных." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "4RhJ4fsHYJdW" }, "source": [ "**Backward pass в MLP**" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "MYN043DbYJdX" }, "source": [ "Обучается с помощью метода \"Error Backpropagation\" - [\"Обратное распространение ошибки\"](https://ru.wikipedia.org/wiki/%D0%9C%D0%B5%D1%82%D0%BE%D0%B4_%D0%BE%D0%B1%D1%80%D0%B0%D1%82%D0%BD%D0%BE%D0%B3%D0%BE_%D1%80%D0%B0%D1%81%D0%BF%D1%80%D0%BE%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B5%D0%BD%D0%B8%D1%8F_%D0%BE%D1%88%D0%B8%D0%B1%D0%BA%D0%B8), принцип распространения очень похож на то, как мы обучали один нейрон - это градиентный спуск, но по \"всей нейросети\" сразу. " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "collapsed": true, "id": "oK7Vi4bxYJdZ" }, "source": [ "Backpropagation работает корректно благодаря ***chain rule*** (=правилу взятия производной сложной функции): \n", "\n", "Если $f(x) = f(g(x))$, то: \n", "\n", "$$\\frac{\\partial{f}}{\\partial{x}} = \\frac{\\partial{f}}{\\partial{g}} \\frac{\\partial{g}}{\\partial{x}}$$" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "collapsed": true, "id": "WKvMsaEBYJda" }, "source": [ "Более подробно про backpropagation можно прочитать здесь (на английском): https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "z5dyjPVNYJdc" }, "source": [ "

Многослойная нейросеть на PyTorch

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "T9ufk3ECYJde" }, "source": [ "Ешё раз напомним про основные компоненты нейросети:\n", "\n", "- непосредственно, сама **архитектура** нейросети (сюда входят типы функций активации у каждого нейрона);\n", "- начальная **инициализация** весов каждого слоя;\n", "- метод **оптимизации** нейросети (сюда ещё входит метод изменения `learning_rate`);\n", "- размер **батчей** (`batch_size`);\n", "- количетсво **эпох** обучения (`num_epochs`);\n", "- **функция потерь** (`loss`); \n", "- тип **регуляризации** нейросети (для каждого слоя можно свой); \n", "\n", "То, что связано с ***данными и задачей***: \n", "- само **качество** выборки (непротиворечивость, чистота, корректность постановки задачи); \n", "- **размер** выборки; " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "9KrWarqTYJdf" }, "source": [ "Cоздадим двухслойную нейросеть из 100 нейронов:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": {}, "colab_type": "code", "id": "bLjkPg19YJdg" }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "import torch" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "iCDVRvQJYJdl" }, "source": [ "Генерация и отрисовка датасета:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": {}, "colab_type": "code", "id": "k0J27RcLYJdm" }, "outputs": [], "source": [ "N = 100\n", "D = 2\n", "K = 3\n", "X = np.zeros((N * K, D))\n", "y = np.zeros(N * K, dtype='uint8')\n", "\n", "for j in range(K):\n", " ix = range(N * j,N * (j + 1))\n", " r = np.linspace(0.0, 1, N)\n", " t = np.linspace(j * 4, (j + 1) * 4,N) + np.random.randn(N) * 0.2 # theta\n", " X[ix] = np.c_[r * np.sin(t), r * np.cos(t)]\n", " y[ix] = j" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "X9FHhqX_YJdp" }, "source": [ "Не забываем оборачивать данные (без этого градиенты не посчитать):" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": {}, "colab_type": "code", "id": "pQINaQqZYJdq" }, "outputs": [], "source": [ "X = torch.autograd.Variable(torch.FloatTensor(X))\n", "y = torch.autograd.Variable(torch.LongTensor(y.astype(np.int64)))" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": {}, "colab_type": "code", "id": "Who9mS8oYJdu", "outputId": "1ec6e30a-2cd9-4bd6-d0e0-3b7c936d1b8a" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "torch.Size([300, 2]) torch.Size([300])\n" ] } ], "source": [ "print(X.data.shape, y.data.shape)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "I-9dFW5CYJd0" }, "source": [ "Сама ячейка с нейросетью и обучением:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": {}, "colab_type": "code", "id": "kf-YapleYJd1", "outputId": "d71315ae-670a-48e3-bcf7-b425ebe198fe" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "c:\\users\\izakharkin\\anaconda3\\envs\\vision\\lib\\site-packages\\torch\\nn\\functional.py:52: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.\n", " warnings.warn(warning.format(ret))\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "0 330.8737487792969\n", "1 328.732177734375\n", "2 326.73291015625\n", "3 324.8538513183594\n", "4 323.0766296386719\n", "5 321.386962890625\n", "6 319.77215576171875\n", "7 318.22314453125\n", "8 316.7312316894531\n", "9 315.2906799316406\n", "10 313.89508056640625\n", "11 312.5408935546875\n", "12 311.2237854003906\n", "13 309.9412536621094\n", "14 308.69000244140625\n", "15 307.4679260253906\n", "16 306.2733154296875\n", "17 305.10418701171875\n", "18 303.9598083496094\n", "19 302.8381652832031\n", "20 301.7386779785156\n", "21 300.6603088378906\n", "22 299.60205078125\n", "23 298.5636901855469\n", "24 297.5439758300781\n", "25 296.5423583984375\n", "26 295.5587158203125\n", "27 294.5920104980469\n", "28 293.6417541503906\n", "29 292.70819091796875\n", "30 291.79022216796875\n", "31 290.88787841796875\n", "32 290.00054931640625\n", "33 289.1278991699219\n", "34 288.2695007324219\n", "35 287.4254150390625\n", "36 286.59490966796875\n", "37 285.7780456542969\n", "38 284.9742431640625\n", "39 284.18316650390625\n", "40 283.4047546386719\n", "41 282.638671875\n", "42 281.8844909667969\n", "43 281.1424255371094\n", "44 280.4117736816406\n", "45 279.6929016113281\n", "46 278.98492431640625\n", "47 278.2878723144531\n", "48 277.6017150878906\n", "49 276.9258117675781\n", "50 276.2603759765625\n", "51 275.60498046875\n", "52 274.9594421386719\n", "53 274.3237609863281\n", "54 273.69744873046875\n", "55 273.0805358886719\n", "56 272.4727783203125\n", "57 271.8741149902344\n", "58 271.28436279296875\n", "59 270.7031555175781\n", "60 270.13055419921875\n", "61 269.5662536621094\n", "62 269.0101623535156\n", "63 268.46209716796875\n", "64 267.9218444824219\n", "65 267.3895568847656\n", "66 266.8648681640625\n", "67 266.3474426269531\n", "68 265.83758544921875\n", "69 265.3347473144531\n", "70 264.8389892578125\n", "71 264.35028076171875\n", "72 263.8682556152344\n", "73 263.3929443359375\n", "74 262.9244689941406\n", "75 262.462158203125\n", "76 262.0062255859375\n", "77 261.5567321777344\n", "78 261.11297607421875\n", "79 260.6752624511719\n", "80 260.24371337890625\n", "81 259.8177795410156\n", "82 259.39764404296875\n", "83 258.9831237792969\n", "84 258.5740051269531\n", "85 258.17047119140625\n", "86 257.77239990234375\n", "87 257.3795166015625\n", "88 256.9918518066406\n", "89 256.6094055175781\n", "90 256.2320861816406\n", "91 255.85975646972656\n", "92 255.4922332763672\n", "93 255.12928771972656\n", "94 254.7711181640625\n", "95 254.41737365722656\n", "96 254.0686492919922\n", "97 253.7240753173828\n", "98 253.38406372070312\n", "99 253.048095703125\n", "100 252.71652221679688\n", "101 252.38906860351562\n", "102 252.06591796875\n", "103 251.74659729003906\n", "104 251.43118286132812\n", "105 251.1195526123047\n", "106 250.81195068359375\n", "107 250.50804138183594\n", "108 250.20799255371094\n", "109 249.91146850585938\n", "110 249.61854553222656\n", "111 249.32940673828125\n", "112 249.04347229003906\n", "113 248.76107788085938\n", "114 248.4823760986328\n", "115 248.2066192626953\n", "116 247.93441772460938\n", "117 247.66543579101562\n", "118 247.3993377685547\n", "119 247.13636779785156\n", "120 246.8765106201172\n", "121 246.61959838867188\n", "122 246.36578369140625\n", "123 246.11471557617188\n", "124 245.86680603027344\n", "125 245.62159729003906\n", "126 245.3793182373047\n", "127 245.1397705078125\n", "128 244.90293884277344\n", "129 244.668701171875\n", "130 244.4371795654297\n", "131 244.2081298828125\n", "132 243.9817657470703\n", "133 243.7580108642578\n", "134 243.53665161132812\n", "135 243.31785583496094\n", "136 243.10150146484375\n", "137 242.8875274658203\n", "138 242.67599487304688\n", "139 242.46681213378906\n", "140 242.2598876953125\n", "141 242.05548095703125\n", "142 241.85304260253906\n", "143 241.65306091308594\n", "144 241.455322265625\n", "145 241.25962829589844\n", "146 241.06610107421875\n", "147 240.87461853027344\n", "148 240.68505859375\n", "149 240.497802734375\n", "150 240.3124542236328\n", "151 240.12908935546875\n", "152 239.94769287109375\n", "153 239.76808166503906\n", "154 239.5904541015625\n", "155 239.4147491455078\n", "156 239.24102783203125\n", "157 239.06932067871094\n", "158 238.8993377685547\n", "159 238.73109436035156\n", "160 238.56472778320312\n", "161 238.39988708496094\n", "162 238.23672485351562\n", "163 238.07516479492188\n", "164 237.91534423828125\n", "165 237.7571258544922\n", "166 237.60055541992188\n", "167 237.4454345703125\n", "168 237.2916259765625\n", "169 237.1393585205078\n", "170 236.9887237548828\n", "171 236.8394775390625\n", "172 236.69174194335938\n", "173 236.5454864501953\n", "174 236.40061950683594\n", "175 236.25697326660156\n", "176 236.1148223876953\n", "177 235.97398376464844\n", "178 235.83470153808594\n", "179 235.69656372070312\n", "180 235.55975341796875\n", "181 235.4245147705078\n", "182 235.29046630859375\n", "183 235.1576690673828\n", "184 235.02626037597656\n", "185 234.8958282470703\n", "186 234.7667694091797\n", "187 234.6388702392578\n", "188 234.51202392578125\n", "189 234.3862762451172\n", "190 234.26170349121094\n", "191 234.13832092285156\n", "192 234.0160675048828\n", "193 233.89483642578125\n", "194 233.7747344970703\n", "195 233.6556396484375\n", "196 233.53765869140625\n", "197 233.4207763671875\n", "198 233.30471801757812\n", "199 233.1900177001953\n", "200 233.07615661621094\n", "201 232.9632110595703\n", "202 232.85122680664062\n", "203 232.74021911621094\n", "204 232.63035583496094\n", "205 232.5215606689453\n", "206 232.4134521484375\n", "207 232.306396484375\n", "208 232.20028686523438\n", "209 232.09500122070312\n", "210 231.9905242919922\n", "211 231.8870391845703\n", "212 231.78427124023438\n", "213 231.68247985839844\n", "214 231.58154296875\n", "215 231.48130798339844\n", "216 231.38198852539062\n", "217 231.28353881835938\n", "218 231.18572998046875\n", "219 231.08877563476562\n", "220 230.99240112304688\n", "221 230.8970489501953\n", "222 230.80233764648438\n", "223 230.7084503173828\n", "224 230.6151885986328\n", "225 230.5227508544922\n", "226 230.43099975585938\n", "227 230.33998107910156\n", "228 230.24993896484375\n", "229 230.16046142578125\n", "230 230.07156372070312\n", "231 229.98330688476562\n", "232 229.89585876464844\n", "233 229.80921936035156\n", "234 229.72317504882812\n", "235 229.63787841796875\n", "236 229.55319213867188\n", "237 229.46926879882812\n", "238 229.38583374023438\n", "239 229.30311584472656\n", "240 229.22097778320312\n", "241 229.13951110839844\n", "242 229.05859375\n", "243 228.97821044921875\n", "244 228.89834594726562\n", "245 228.81919860839844\n", "246 228.7405242919922\n", "247 228.66250610351562\n", "248 228.5851287841797\n", "249 228.50808715820312\n", "250 228.43177795410156\n", "251 228.35577392578125\n", "252 228.2805633544922\n", "253 228.20558166503906\n", "254 228.1313934326172\n", "255 228.05772399902344\n", "256 227.9844512939453\n", "257 227.9118194580078\n", "258 227.83944702148438\n", "259 227.76771545410156\n", "260 227.6964874267578\n", "261 227.62564086914062\n", "262 227.55517578125\n", "263 227.48536682128906\n", "264 227.41592407226562\n", "265 227.34690856933594\n", "266 227.27841186523438\n", "267 227.21031188964844\n", "268 227.1426239013672\n", "269 227.07553100585938\n", "270 227.00869750976562\n", "271 226.9425048828125\n", "272 226.8767852783203\n", "273 226.81141662597656\n", "274 226.74658203125\n", "275 226.6818389892578\n", "276 226.61788940429688\n", "277 226.55419921875\n", "278 226.4910430908203\n", "279 226.4283447265625\n", "280 226.36598205566406\n", "281 226.30397033691406\n", "282 226.2423553466797\n", "283 226.1811981201172\n", "284 226.12063598632812\n", "285 226.060302734375\n", "286 226.00042724609375\n", "287 225.94081115722656\n", "288 225.88162231445312\n", "289 225.82281494140625\n", "290 225.76438903808594\n", "291 225.7063446044922\n", "292 225.64842224121094\n", "293 225.59112548828125\n", "294 225.5340576171875\n", "295 225.47738647460938\n", "296 225.42083740234375\n", "297 225.36495971679688\n", "298 225.30914306640625\n", "299 225.25381469726562\n", "300 225.19866943359375\n", "301 225.1439666748047\n", "302 225.08958435058594\n", "303 225.03536987304688\n", "304 224.98138427734375\n", "305 224.9280242919922\n", "306 224.874755859375\n", "307 224.8217315673828\n", "308 224.7690887451172\n", "309 224.7168426513672\n", "310 224.66493225097656\n", "311 224.6132354736328\n", "312 224.5619354248047\n", "313 224.5107879638672\n", "314 224.4600372314453\n", "315 224.4093017578125\n", "316 224.35906982421875\n", "317 224.30905151367188\n", "318 224.25927734375\n", "319 224.20980834960938\n", "320 224.16049194335938\n", "321 224.11146545410156\n", "322 224.0626983642578\n", "323 224.0141143798828\n", "324 223.9657440185547\n", "325 223.91778564453125\n", "326 223.87010192871094\n", "327 223.82252502441406\n", "328 223.77523803710938\n", "329 223.72813415527344\n", "330 223.6814727783203\n", "331 223.63462829589844\n", "332 223.58831787109375\n", "333 223.5421600341797\n", "334 223.4960174560547\n", "335 223.45028686523438\n", "336 223.40487670898438\n", "337 223.35946655273438\n", "338 223.31427001953125\n", "339 223.26914978027344\n", "340 223.22439575195312\n", "341 223.1798553466797\n", "342 223.13540649414062\n", "343 223.09117126464844\n", "344 223.0472412109375\n", "345 223.00338745117188\n", "346 222.9597930908203\n", "347 222.91632080078125\n", "348 222.8732147216797\n", "349 222.8301239013672\n", "350 222.7873077392578\n", "351 222.7447509765625\n", "352 222.70228576660156\n", "353 222.659912109375\n", "354 222.61798095703125\n", "355 222.5760498046875\n", "356 222.53427124023438\n", "357 222.49267578125\n", "358 222.45132446289062\n", "359 222.41017150878906\n", "360 222.3690948486328\n", "361 222.32810974121094\n", "362 222.2873077392578\n", "363 222.24671936035156\n", "364 222.20628356933594\n", "365 222.16615295410156\n", "366 222.12588500976562\n", "367 222.08590698242188\n", "368 222.04608154296875\n", "369 222.00643920898438\n", "370 221.96688842773438\n", "371 221.92747497558594\n", "372 221.88816833496094\n", "373 221.84906005859375\n", "374 221.81005859375\n", "375 221.7711944580078\n", "376 221.73239135742188\n", "377 221.6939697265625\n", "378 221.65554809570312\n", "379 221.61721801757812\n", "380 221.57923889160156\n", "381 221.5411834716797\n", "382 221.50323486328125\n", "383 221.46563720703125\n", "384 221.42796325683594\n", "385 221.39044189453125\n", "386 221.35302734375\n", "387 221.31573486328125\n", "388 221.27880859375\n", "389 221.24176025390625\n", "390 221.20477294921875\n", "391 221.16806030273438\n", "392 221.13133239746094\n", "393 221.09475708007812\n", "394 221.05816650390625\n", "395 221.02174377441406\n", "396 220.98556518554688\n", "397 220.94918823242188\n", "398 220.91311645507812\n", "399 220.87728881835938\n", "400 220.84132385253906\n", "401 220.80552673339844\n", "402 220.76998901367188\n", "403 220.7344512939453\n", "404 220.69898986816406\n", "405 220.66366577148438\n", "406 220.6285400390625\n", "407 220.59347534179688\n", "408 220.55848693847656\n", "409 220.52359008789062\n", "410 220.4888153076172\n", "411 220.45404052734375\n", "412 220.41934204101562\n", "413 220.38478088378906\n", "414 220.35031127929688\n", "415 220.31590270996094\n", "416 220.28167724609375\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "417 220.2473907470703\n", "418 220.2133331298828\n", "419 220.1792449951172\n", "420 220.14540100097656\n", "421 220.11141967773438\n", "422 220.0778350830078\n", "423 220.04403686523438\n", "424 220.01052856445312\n", "425 219.97705078125\n", "426 219.943603515625\n", "427 219.910400390625\n", "428 219.87709045410156\n", "429 219.84405517578125\n", "430 219.81103515625\n", "431 219.7781982421875\n", "432 219.7453155517578\n", "433 219.71253967285156\n", "434 219.67999267578125\n", "435 219.6472625732422\n", "436 219.6147003173828\n", "437 219.58233642578125\n", "438 219.5499267578125\n", "439 219.51748657226562\n", "440 219.48532104492188\n", "441 219.45327758789062\n", "442 219.4212646484375\n", "443 219.38925170898438\n", "444 219.35740661621094\n", "445 219.3255157470703\n", "446 219.29376220703125\n", "447 219.26210021972656\n", "448 219.2305450439453\n", "449 219.19888305664062\n", "450 219.1672821044922\n", "451 219.13607788085938\n", "452 219.1046905517578\n", "453 219.07330322265625\n", "454 219.04208374023438\n", "455 219.0108184814453\n", "456 218.97976684570312\n", "457 218.94859313964844\n", "458 218.91758728027344\n", "459 218.88671875\n", "460 218.85562133789062\n", "461 218.8248291015625\n", "462 218.7939453125\n", "463 218.7631072998047\n", "464 218.73252868652344\n", "465 218.7017364501953\n", "466 218.67117309570312\n", "467 218.64051818847656\n", "468 218.61012268066406\n", "469 218.57965087890625\n", "470 218.54920959472656\n", "471 218.51881408691406\n", "472 218.48854064941406\n", "473 218.45822143554688\n", "474 218.4281005859375\n", "475 218.39797973632812\n", "476 218.36767578125\n", "477 218.3377685546875\n", "478 218.3079833984375\n", "479 218.2783203125\n", "480 218.2487335205078\n", "481 218.21925354003906\n", "482 218.1898651123047\n", "483 218.1603546142578\n", "484 218.13104248046875\n", "485 218.10177612304688\n", "486 218.07244873046875\n", "487 218.04319763183594\n", "488 218.01397705078125\n", "489 217.9847869873047\n", "490 217.95562744140625\n", "491 217.92665100097656\n", "492 217.89743041992188\n", "493 217.868408203125\n", "494 217.8395233154297\n", "495 217.81057739257812\n", "496 217.78160095214844\n", "497 217.7528533935547\n", "498 217.7239990234375\n", "499 217.6951446533203\n" ] } ], "source": [ "# N - размер батча (batch_size, нужно для метода оптимизации); \n", "# D_in - размерность входа (количество признаков у объекта);\n", "# H - размерность скрытых слоёв; \n", "# D_out - размерность выходного слоя (суть - количество классов)\n", "N, D_in, H, D_out = 64, 2, 100, 3\n", "\n", "# Use the nn package to define our model and loss function.\n", "two_layer_net = torch.nn.Sequential(\n", " torch.nn.Linear(D_in, H),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(H, D_out),\n", ")\n", "\n", "loss_fn = torch.nn.CrossEntropyLoss(size_average=False)\n", "\n", "learning_rate = 1e-4\n", "optimizer = torch.optim.SGD(two_layer_net.parameters(), lr=learning_rate)\n", "for t in range(500):\n", " # forward\n", " y_pred = two_layer_net(X)\n", "\n", " # loss\n", " loss = loss_fn(y_pred, y)\n", " print('{} {}'.format(t, loss.data))\n", "\n", " # зануляем градиенты (чтобы не было остатка с редыдущего шага)\n", " optimizer.zero_grad()\n", "\n", " # backward\n", " loss.backward()\n", "\n", " # обновляем\n", " optimizer.step()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zTQiN0bcYJd6" }, "source": [ "**Обратите внимание:** несмотря на то, что это задача 3-х классовой классификации и столбец $y$ нужно по-хорошему кодировать OneHotEncoding'ом, мы подали просто столбец из 0, 1 и 2 и всё отработало. Дело в том, что PyTorch сам делает OneHot в таком случае." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "apsWWq17YJd8" }, "source": [ "Проверим, насколько хороша наша сеть из 100 нейронов:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": {}, "colab_type": "code", "id": "X0ICB6Z-YJd-" }, "outputs": [], "source": [ "# Обратно в Numpy для отрисовки\n", "X = X.data.numpy()\n", "y = y.data.numpy()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab": {}, "colab_type": "code", "id": "JvjfvgbPYJeB", "outputId": "cf5545ff-3315-45c9-c674-af44f63a346b" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnIAAAH2CAYAAAD052HGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl83Hd17//XmdFotEteZFleZMmOpeyJiRNIcEhYQoELhFKgC0t7gYal3F5Ke3vLrzvQ3t5e2t4WaGkKlAstUJYLbm7ZTAsmzgJxoiy2432RbMmyvGjfZuZ7fn98R7KWGVm2JY1Gej/z0MPWzHdmPiM71luf5Rxzd0REREQk/0RyPQARERERuTIKciIiIiJ5SkFOREREJE8pyImIiIjkKQU5ERERkTylICciIiKSpxTkRGRRMLM3mtl/mFmXmQ2b2UEz+5iZrcz12ERE5oqpjpyI5Dsz+wvgg8A/AtuBHuB64L3AUXf/2RwOT0RkzijIiUheM7PXAf8KvMvdPzfpvijwSnf/Tk4GJyIyx7S0KiL57jeApyaHOAB3T7n7d8ys3sw828fo9WZ2b/q2+nG3vTp924/G3fZHZnbWzF5sZk+Z2ZCZPW1m28a/vpm9w8x2mdl5M7tgZj80s62TxznudSd/3Ju+f3T8r52Fr5eILCIKciKSt8wsBtwFfHeGD/kt4M5xH39xiec34H8AqQx3lwD/BHwaeDPQBXzHzFaPu6Ye+EL6/l8CTgI/NrONWV7yrelxvXVmb0dElrqCXA9AROQqrADiQMsMrz/g7o+PfpJpdmySXwLWAw8ByybdVwz8rrt/Kf1cP0yP44PA7wC4+0fGvVYE2AHcDrwN+Mi454qlf33K3febWd8M34+ILHGakRORxWDWN/uaWSHwUeBPge4sl31zbADufYRB7Y5xz3GdmX3TzDoIZ/USQBPQOOl5itO/Dl9iWBEzK0jPFIqIKMiJSF47Rxh+6ubgud9DuGrxqSz397n74KTbzgC1AGZWDnyfcEbvQ8DdhLNxzwBFkx63PP3ruUuMaTthGBwys31m9sszeB8isohpaVVE8pa7J8zsEeBngN+bxacuSz/ff3f3oSwTYGVmVjwpzK0C2tO/vxNYB9zn7vtHLzCzygzPtRG44O49lxjXbwC7CGfw3gH8o5k95+5PzeRNicjioxk5Ecl3/xvYmml2yswiZvaqK3jO3wI6CQ8qTGesPp2ZlQH3AT9N3zRludTM7iI8ADF+jAa8EnhsBuM67O673f1hwlk+A26aweNEZJHSjJyI5DV3f8jM/hL4rJm9mHD5sQ+4lrAg8HHCmazL8XbgDe4eTHPNIPAn6QDXRhj+CoG/Tt//eHoc/2Bmf044O/dHwKnRJzCzRuCPCffVvXoG41pnZtcShsS3E+4NfGbmb0tEFhsFORHJe+7+m2b2KPAB4EuEQec4YaHgjzN1T9qlPObuD13imgHC5c1PANcB+4HXuHt7ekwdZvbm9OtvBw4RBsvfHvccbwAagJ919+/NYFx/l/51EDgMvMPdn57ZWxKRxUidHURELpOZ/RHwAXdXH1cRySntkRMRERHJUwpyIiIiInlKS6siIiIieUozciIiIiJ5SkFOREREJE8tmfIjKysrvX51Ta6HMad8pI+AgKGqCjxaSGTp/PGKiIgsKs8/+cxZd6++1HVL5jt9/eoadj/4yVwPY+61H6B7uIWOLZs4fcM9VNjiDq8iIiKL0ZboihMzuW7JBLklo7aJynYoaW4DdtJdt5FYxdZcj0pERETmgILcYlTbRKwdGprbOAZ018Fg+XrNzomIiCwyCnKLVW0TMZpoaN5FFzGO1wEVCnIiIiKLiU6tLnKxeDUVVpHrYYiIiMgcUJATERERyVMKcotdbRORp56jcfsOEj276fGOXI9IREREZomC3BIQq99GCcU0bt/B6r07FeZEREQWCQW5JSJWv43KeB01zUcU5kRERBYJnVpdSlRjTkREZFHRjNxSU9tELF5NQ3MbNVakmTkREZE8piC3FNU25XoEIiIiMgsU5Jaw4hOHKO5tzfUwRERE5Appj9wSFYtXU9V8gnrguFp4iYiI5CXNyC1VtU3E6rdR1XyCxu07KO5t1X45ERGRPKMgt8SN1pirsaJcD0VEREQuk4KciIiISJ5SkBMAKr/1kJZXRURE8oyCnKiFl4iISJ5SkBNALbxERETykcqPyEVq4SUiIpJXFORkotomYu3Q0NxGFzGO16EwJyIiskApyMlU6TBXYRW5HomIiIhMQ3vkRERERPKUgpxkFXnqOepbWkn07NbhBxERkQVIQU4yUwsvERGRBU9BTqY1WpakvqWV4t7WXA9HRERExlGQkxnRwQcREZGFR0FOZiTy1HMAWl4VERFZQBTk5NJqmwDUwktERGSBUZCTGVELLxERkYVHBYFl5tTCS0REZEFRkJPLU9tEjCYamncx0HyEg/fDYPl6Kqwm1yMTERFZcrS0KlckVr+NEoqpsaJcD0VERGTJUpATERERyVMKcnJVik8cUtcHERGRHFGQkyumFl4iIiK5pSAnV2W0LInCnIiIyPxTkJOrV9ukgw8iIiI5oCAnIiIikqcU5GRWxOLVVH7rIXV9EBERmUcqCCyzI931geYjAOr6ICIiMg8U5GT2jGvhdQzorkNhTkREZA4pyMnsUgsvERGReaM9cjInRlt41be0UtzbmuvhiIiILEoKcjJnYvFqKqwi18MQERFZtBTkZF7oJKuIiMjsW3BBzsw+Z2ZnzGxPlvvvNbNuM3s6/fEH8z1GmaHaJiJPPaeuDyIiInNkwQU54PPAqy5xzcPufmv64yPzMCa5QuNbeKnGnIiIyOxacKdW3f3HZlaf63HILBpXlgR2qsaciIjILFmIM3IzcaeZPWNm3zGzG7JdZGYPmNluM9vd2d09n+OTyWqbiMWraWhuo76llUTPbs3OiYiIXKV8DHJPARvc/RbgE8C3sl3o7g+6+1Z331pdWTlvA5QsapuI1W+jqvkE9S0qSSIiInK18i7IuXuPu/elf/9tIGZmK3M8LLkMsXh1rocgIiKyKORdkDOz1WZm6d/fQfgezuV2VHIlVChYRETk6iy4ww5m9mXgXmClmZ0E/hCIAbj7p4E3Ae8zsyQwCPyCu3uOhitXoraJquZdNDbvVwsvERGRq7Dggpy7/+Il7v8k8Ml5Go7MkVj9NirbD9C4fQcdWzZx+oZ7FOZEREQu04ILcrKEpMuS0HwEQGVJRERELpOCnOTWuBpzx4DuOhTmREREZijvDjvIIpQuS9LQ3EaNFam+nIiIyAwpyImIiIjkKQU5WVCKTxxSWRIREZEZ0h45WTBi8eqw6wNwvE5lSURERC5FM3KycIxr4dW4fQfFva3aLyciIjINBTlZcGL126iM19G4fQer9+5UmBMREclCS6uyMI0rSwI7VWNOREQkA83IycJV20QsXs0GW5HrkYiIiCxICnIiIiIieUpBTha8yFPPUdlylETP7lwPRUREZEHRHjlZ2GqbiLVDg1p4iYiITKEgJwtfbRMxmmho3sVA8xEO3q8acyIiIqClVckjKksiIiIykWbkJL+ky5LQfASA0zfco5k5ERFZsjQjJ/mntonKeB3LiOV6JCIiIjmlICciIiKSpxTkJG8VNu/XXjkREVnStEdO8pNaeImIiCjISR4bV2OuixjH61SWRERElhYtrUp+q20iVr+NquYTNG7fQXFvq5ZaRURkyVCQk0VhfI05hTkREVkqFORk8ahtooRiaqwo1yMRERGZFwpyIiIiInlKQU4WlVi8mspvPaSyJCIisiTo1KosLpNaeKksiYiILGYKcrL4jKsxdwzorkNhTkREFiUFOVmcapuI0URD8y4Gmo9w8H7VmBMRkcVHe+RkURtflkT75kREZLHRjJwsfto3JyIii5Rm5GRpqG2iMl5HQ3MblS1HNTMnIiKLgoKcLB21TcTi1SwjluuRiIiIzAoFOREREZE8pT1ysuRUNZ9gNQntlRMRkbynICdLS20TsXZoaG6jixjH61SWRERE8peWVmXpqW0iVr+NquYTNG7fQXFvqw4/iIhIXlKQkyVLNeZERCTfaWlVlrZx7bxgp/bNiYhIXlGQE9G+ORERyVNaWhUB7ZsTEZG8pCAnMo72zYmISD7R0qrIZNo3JyIieUIzciKZpNt5bbAV1FiRZuZERGRBUpATERERyVMKciLTiDz1HMUnDuV6GCIiIhlpj5xINumyJOrNKiIiC5WCnMh0apuI0URD8y4Gmo9w8H7VmBMRkYVjwS2tmtnnzOyMme3Jcr+Z2d+Y2WEze9bMXjDfY5SlJ1a/jRKKVZZEREQWlAUX5IDPA6+a5v5XA5vTHw8AfzcPYxIZqzFX03xEYU5ERBaEBRfk3P3HwPlpLrkf+IKHHgeqzKx2fkYnS15tE5XxOhqa21i9dyeJnt25HpGIiCxhCy7IzcBaoHXc5yfTt01hZg+Y2W4z293Z3T0vg5MlIF1jrqG5jcbtO0j07NbsnIiI5EQ+BjnLcJtnutDdH3T3re6+tbqyco6HJUtKujdrCcXUt7Re+nq5pNJHfkLdL7+Phtf9Iss/+0/Y4GCuhyQisuDl46nVk8D6cZ+vA9pyNBZZ4mLx6lwPYVGo+eP/yaqPf4LI4BDmTtnOXVT/1ac49Oj3CCoqcj08EZEFKx9n5P4VeEf69OqLgG53b8/1oGRpK+5t1fLqFSo8epyaP/8bogODmIeT69H+AeJHT7DqLz6V49GJiCxsCy7ImdmXgceAJjM7aWbvMrP3mtl705d8GzgKHAb+AXh/joYqArVNVDWfoHH7DoW5K1T50HfJtDsiMjzMsi99ff4HJCKSRxbc0qq7/+Il7nfg1+ZpOCKXFKvfRmX7ARq376BjyyZO33CPCgZfBrdM217TprlLREQW4IycSF5SWZIr1vP6V2e8PSiKc/5tb5nn0YiI5BcFOZHZMq4sSX1Lq8qSzNBIfR2nf/+3CUqK8Uj4T1KqtIThzZvo/JAm30VEprPgllZF8lq6N2tV8y4am/dz8P776ClHS62XcOZ3Pkjvy1/Cis9+kej5Lnpe9yq63vIGPB7P9dBERBY0BTmROaB9c5dv8PYXcPJ2tU4WEbkcCnIic6W2icp2KGluA3bSXbeRWMXWXI9KREQWEQU5kblU20SsHRqa2zgGdNfBYPl6zc6JiMis0GEHkbmWbuc12ptV9eZERGS2KMiJzJNY/TYq43U0bt/B6r07FeZEROSqaWlVZD6l983RfARA++ZEROSqKMiJzLdxhyBG980pzImIyJVQkBPJhXS9uYbmXQw0H+Hg/ToEISIil09BTiSHRuvN1be08vwN6+f1tYue3UvVV7+JJZN0/+xrGbjjNpiu76mIiCw4OuwgsgTV/u5HabzrZ6j5879h1cc/yaZX/Czr3/3r4E7h4aM03P9Wbi5Zw02Vdax7zweJXujK9ZBFRCQDzciJLACFzfsprls/J+28ip98mor/9z28sJCuN72egvMXWPk3f09kcHDsmujAAFVf+xZ9L7mLtR/6XaI9vVgQwPAwy7/wFcoefoz9z+yCWGxWxyYiIldHQU4k12qbKDneSeP2HbPbm9Wd9e/+daq++k0iQ8N4NMLqj32cgVtvJDI0POXyaP8ANX/ycSIDg2GIS4uMJIidaqfqm/9G11vecPXjEhGRWaOlVZEFYC5qzFV+8/9R9bVvEU0Hs0giSWRoiNInmicEtQnjOHOWyMjIlNujff2UPvLYVY9JRERml4KcyEJR20RlvI6G5jZW791Jomf3VT3digc/T7R/YOod7gSxqZPxqdIShjfW4xmeK4jHGVm75qrGIyIis09BTmQhqW0iFq+mobmN+pZWEj27r3h2LtLXn/mOICC5ahWp0pKxm1LFxQw3XkP7n/0hQUnxlId4NMKFt//8FY1DRETmjvbIiSw06RpzVc27aGzef8X75rredD8lT++ZcKgBICgrpfXTf0msvYPl//hPWDLJhbe+hXPvfjteXMzpj/4utb/7MbwwfbDBneNf+RzJ2tWz9Q5FRGSWmHumhZTFZ2tTo+9+8JO5HobI5Wk/QPdwCx1bNnH6hnsuK8xF+vpovP1lxFpOER0aAiBVUkL/XXdw9Dtfg0j2CfnohS7KfvgwQVERfS+7Gy8quuq3IiIiM7cluuJJd79k2x/NyIksZOPaecHOy+rNGpSVcfAnP2Dlpz7Dsq/8Xzwe59y73sa5d75t2hAHkFpWRfcbXzcLb0BEROaSgpzIQlfbRKwdGprb6CLG8brM7bwKjxyj9g/+lPIdPyJVXsbZ976Tzg++lzMf/hBnPvwhAArOdLLqf32C4qefZeim6zn37neMLZna8DCV33iIsh8/QmLtGs7/8i+SqFs3729XRERmTkurInkkcXwXAwxy8P77JoS5wmMnaLrtXqy3j0i6tEiquJi+l97NsYe+DEDRM3vYfO9rsZEEkaEhgqI4XlDA4R98i5FNDWy+62eItZ8m2tdPUFgI0QjHv/QZel7/6py9X8msiz666KeIGKuoIqJzayKLjpZWRRah0d6sjdt3TNg3V/PR/0Wkr39Cfbjo4CBlP3qY4ieeYvD2F7DhHe8l0tPLaDfVsCjwMPVve4Dee7dReKKFyEgivC9dS27D2x5gz+kDeEkJknspAn5i+7lAL44TwYgQ4S6/gQr0ZySyFOnHOJF8M6neXI93UP6DnVgqNeVSSyYpe/gxCk53ED90ZCzEjRc7dZplX/7GWIibIBql/Ac7Z/89yBV53lo4Tw8pCwjMSVrACEket+fxjBUARWSxU5ATyUfpenPLCEuEpJZVZbzMY7HwviAAyxTjwI2MITC8EyxTwJOcaOEMgU0KbAZJkpynNzeDEpGcUpATWQQ6f/2BCQV+x+t64+tIrqllZENdxvtT1SvpefV9BBlOsloiQd/LXzKrY5UrlyRL4MZIkJzXsYjIwqAgJ5LHqppPsHrvTjp+romuN7+BoKiIoLiIVHkZqdJSjn3jCwSVFQC0fP5TpMpKw4MMQBCLkSot4cT/+TvaPv5RgqpKgngcADcjVVJM25/+ftbZPpl/lZRmvD0goIqyeR6NiCwEOrUqks/aD5AY7uTYljV0122krL2Ssh/uIlVZQc/rfoagbOI391jrKVb+7WcofupZhm6+nrO/9quM1IczdQUdZ1j5yX+g/Ac7SayrpfPX30v/3Xfm4l0tGSkCDnOKFjtDgLOaZTT5eooozHj9OXp43J4nRcDohseoR1jLClb5MoooZBllDDJMH0OUUUQJKuYsko9mempVQU4kHwUBfGM7fOVr0NWFr17OwJu3se8P35uxxpwsPI6zy/bSTd/YvjdziBHjpX4L8fT+x8ku0Mvz1ko3fRQSI0aUbvqJEiUgwLD0idYIAQErqWSrN1JAdD7fnohcJZUfEVnM/vYf4P/9GwwNA2CnOin524e4Ppni1K+++rLbecn866SLHvonHF5wg6QnOWrtXOeZ9zQuo5y7/HoA9tPKYWsLHze6f84dDIL052e9m2fsCLd549y+IRHJCe2RE8k33T2w/aGxEDfKRpKU/ssuap46PFaWRBauTusmZcGU2wNzOrgwo+c4ZqcJJj/HpMPJgTntnNdhCJFFSjNyIvnm6FEoLIREhrIg57qotNor6s0q86vQC4hgU8uJAIWT/ml2nLN0M8gIVZRSkT70MNNwZhjDJIjpn3yRRUf/V4vkm+XLIZXlG3gsBnU3EItEaGjexUDzEQ7en7k3q0yUIqCdc3RaN3EKqfNqyiies9dbSzUHODnl9qhHaPDasc/7GeJR28tIOrQ5sIJybvYGYkRJZC1JMlEx8VkZt4gsLFpaFck1d9i3Hx7/KXR1X/r6DXWwfh1MrvtWWAj/6VVjt8fqt1FCMTWmU4uXkiDJTnuWZ+wordbJEU7xI3uWk3TO2WuWEOdW30TEjahHiLgRcWMDq1jNMiCciXvcnmeQEVIWpDs6BJylmx/ZszMKcVGPsNnXEtU/9yKLkmbkRHLpRAv8998L971FLFwu/bk3wAPvytqJAYA//Qj81ofhzBmwCCSTcNsWeM+752/si8gBa6WfITy9zOkGTsDTHKXGl836kmQfgwwwzAoqeKXfxmkukCJgFVWUjisX0sMAQ4xM2ffmBikPptw+mblxra9nI7XTXygieUtBTiRXkin4jd+GC13hrNyobz4EdXXw6ldmf2z1Svj8g7D/AHR0wqaGcJYug+IThyiuG6KnHC2vZtFK51iIGy8CdNDFOlbOyuskSPJTO8AF+sL9cQTUsIwX+OaMM2bDJLBsaW0GIe46X88m1szCyEVkodJcu0iu/PSJ8OTp5FqOQ0Pw5a9Ovf5UG3z0f8D9b4ZfeAd85euw+Rq49+6sIS5Wv42q5hM0bt9BcW/rkj/J6jhd9HGGLoYJD4sEBFmXKAN8VpvR77ZDXKCXwAKSlho7obrHjmW8vpJSAqaebAWYbljmECfGBhTcRRY7zciJ5MrZc5CtWf35SeUn2k/DAx+AwcGwGHB3D3z+i/DMs/BnH532ZWL126hsP0Dj9h10bNm0ZGvM9THIT2w/Q4xg6dmwBlazzMsxMueiAGcVmVuUOT52EnQm+8+GGOEc3VNOqQbmtPpZbqRhyvPEidHAao54+yVn4NKDwoD1VHOtr6eLfrroo4hCalmuosAii5CCnEiuNDVm/+bceM3Ez7/wpYshbtTwMDz9bLi8em3T9K9V20RlO0u2LInjPGr7puw3O+4dDDKc9XERLGOHheN0sN9axorwrqOam3xqEBtviJF0t4VM4d1JkCSaoTXXNb6GI9Y+9SGj6dMm3hbxCOt9FT+xA/QxSEBAhAjPcYy7/Hr1ZBVZZLS0KpIrTZvhumvD06bjxePwrl+ZeNvuJyeGuFGJRBjmZqK2iVi8mobmNupbWkn07F50S60DDHGQk+y143TSPbYs2kl3GLomBeeUBZyzXiKZ/il0WEHFlJtbOMMeO8aIJQnMwxk1zvCkHZx2bKUUZV0mjRLN2pIrSZA9IGb4QcBxnrcT9DJAyoLwYER6Kfcntv+yl4odp49Buum/7McmSNJKJyeYPjCLyJXTjJxILv3ZR+HvPwvf/l44w7ZpI3zgvXDDdROvKy+HzrNTH59KTd1jN53aJmI0UdW8i8bm/Ry8/75FcwiihTM8a0dxwvBx3DpYRhkv8usYYiRrCEmQpIxi+nxwwrJnlAhNvn7K9fvsxJTlUTc47Rf4iT1Pra9gLSunhK8YBTSwmuPeMaGjQ9QjXOvrsh5qKCZOBJthtbiw+G/XpNZfo1KkOEcPK6mc0XN1089uOxjOZBJ+TW7xTdSy/JKPPcVZmu3I2LtynE2+huvI3HpMRK6MZuREcikeh19/P3x3O/z7t+Ezfwu33jz1ule+PPtzHDh02S8bq99GZbyOxu07FkU7ryFGeNaOEpiHp0/Ts1Dn6eMYp6mkNOtcUiWlvNhvoJblRNwwh1Iv4nZvYjnlE651fKwwbyYd1sVzdowf2TMZuy5c7xvY7OuIeRQc4h7jRq+nYZryIBGM630DUZ/4z7V5eDI1kyDru7WLPVkvIUGSR2wv/QyN1bAbsSRP2SG66Z/2sQMM02yHCdKPC+vfOUetnTN0zej1RWRmNCMnslBMLvA7XkUFRCOQyrA0t/vJK3u9RbRvrp3zGW8PLOAEZ9jka1hBOee8Z8JMVcQjXO8biFHAbd5IQECAZz0UkD0gMbbMmbKAAR/mgLVyozdMusRoZC2bfQ2OZ17SzWADNRR6jAOE9e5KiNPk6zhtF2jz84CPzei90K9lv7Vynt4M4w+mhNNsTtIZzmJOXo4m4LCd4jZvzPrYVjozfqVSFnCUdlb5xQMkSVK0cY4+G6TcS1jDChUvFrkMCnIi+aAoHs7eDQxOvS+WeW/VjNQ2EWuHhuY2jgHddfnZzis1TcRKkuRhe44L9AHhTJYDZRRzo9dP2AcXuUS0OkDrjMbj5pz0c9xIQ8b7Lf3f5ahlObU+cUlzja9kMwOco4cYBdSwjAKi3OhRHmEvKS4WDY56hGt8DYVZ9uJN1mdDE5aAxw2eXs/w93CcEUtkrMsHcJ5eztPLcsrpY5BdtocU4axdlAj7OMHdfiMl6cLIAc7IZZwOFllqFORE8sGL7sg8G1dYOH3h4JlI75traN5FFzGev2HqvrCFbhVV7Kd1yj44c2OIBEOWuHijQxWl3O03XVaYukAvx+z0zMqAjL7QPCinhHJKJtxWRRl3+00ctJOc916KKOQaX8MaVsz4eSu9lCiRqWEu/fWbzkqvpIUzGYNgkhSP2T5u9U0cslPhUvW42cyUBzxph9nmN3CENg7aKQICDGMVVSRJ0c8Q5ZTQ6OtYplO4ssTpxxuRfFBSAr/3O+Gs3OgMXHEx1G+At791Vl4iFq+elefJhQpKWMfKCfvIIp45prlBL4MZlx6n02JnwhmuKU849SZzZnQgYC5VUMJWb+SVfhsv8ZsuK8QB6SXO6JT3FyXCJl877WMjWBiqM2XZ9P7FZ+wofQxODcYG3fRxgJMcsJNjhZNTFtDOeTrpZsCG6eACj9peOriQ4UVElo4FNyNnZq8C/hqIAp9x9z+bdP+vAP8LOJW+6ZPu/pl5HaRILtx9F3zpH2HHf8CFC3DrLXDHVojObpHX4t5WqMivpVWAW3wjq6jiMKcYYJhi4vQzRGBTN/cHOBfoG1tWHSFBC510Wz/lXsIGVk0pB5IkS2/T8dWELVzCjFHAtZ7fpzMLiHK330izHeaC92GEJ2hv9U2UU5z1cWfp5gk7mPHU7HhhKZbMpZid8GDElBm9STXzUgQ8y1Fe4S+47KVqkcViQQU5M4sCnwLuA04CT5jZv7r7vkmX/ou7f2DeByiSaytWwC+8eW6eu3Z8WZL82ytnGMMk6GEQJzxhmW11M4JRlC6+28MAj9geUjiBBUQwDnNqSvHcWl/Oac5PCRdRj7DRaxmwYYY9wSqvYgOriC2sf16vSClFbPMbSZAkwLPWuhtvn7UQZNpbN4kDBVjGynpxYtOeDh5vmARDjFBMfEbXiyw2C21p9Q7gsLsfdfcR4CvA/Tkek8iSEavfRgnFeVmWZJBh9tozQ44uAAAgAElEQVRxgnQRXCDbhA8RjFqWAdBsh0mQGgsfgTlJS/GkHZqw5241y6mkdMLybdQjlFJEI+u4zTdzl1/PNaxZFCFuvBgFMwpxAL0MXPoihzKKuNWvCb+efvH2qEe42TfNeIbNCQsqiyxVC+1fm7Uw4VjYSeCFGa77OTN7CXAQ+A13n9lRMpGFYHgEvvYN+LfvwkgiXDJ9+y/BitzuqRo12puV5iMAedObNVsJktEwF5YUcaJEeaFfS5QoQ4yEwSNDZhhihAGGKU2fnoxg3OnX00IHLenSHOu8mnpqdJpynEJi03dxcCikgNu9iTKKuctv4LCdotcHqKCUzb6WSkppYDXH/PTEGdDJLckcVlBO4YL7ViYyfxba3/6Me5Mnff4Q8GV3Hzaz9wL/B3hZxiczewB4AKCuZtVsjlPkygQB/NaH4eDBMNABPPRt+PEu+NzfQ9XMKu7PuTysMRdM00CqmMJ0L9Qo5RQTm+EMzuRnjBKhgVoaPHsB36XuGl/DPk5kPLFqDiup5A5vGptFW0YZt/vUXsHXeh0JS9HqZ4gQIUUqLNriRirdtqyQArb4NVMeK7KULLQgdxIYX/tgHdA2/gJ3Pzfu038A/me2J3P3B4EHAbY2Nc5PLQCR6ex+Cg4fvhjiIGyz1dsH3/gWvOuXcze2ycbVmBtoPpKzfXOOh4cWCCinJOuSWw1V7ONEpidgJZUEOM/aYUYIS5GsZjm3+EZKiNPH0JSHxSgYm42Tmaunhn4GOeangYs/iUcwNlLLdV43o2XTCMYNvoFyiumyPpZ5GWupppMu+hiinGJWs2zGRZXHO0cPx+00Q4xQ7VXUUzPj+noiC81CC3JPAJvNrIHwVOovAL80/gIzq3X39vSnrween98hisxAMgWPPAo/eQIqK+FV98GGOnjyKRicGhpIJOCxxxdWkIOxGnMlx3fRuH3HvPdmzdTr81a/htXp/W3jTdd1oZt+2uzchFmidj/PgA1xq2/iMZ4PZ/TMMQ8LA2/xa3QS8goYxo3ewGbW0UUfMaLEKaSIwstagu5lgF22lyBdLLid8xzkFHf7Taxl5RWP7wht7LfWsWLJF9Jt3O7xm8cOwIjkkwUV5Nw9aWYfAL5HWH7kc+6+18w+Aux2938Fft3MXg8kgfPAr+RswCKZDA3Bf/1v0NIShrZoFP7vt+D974Xy8rAOXCIx9XHlM2udlAuj++Yat++gY8umedk3N0KCR2wvyXHlQ1IEPMlBtvmNVE4qSnuBPiIw9RSkQY9P3Qfn5ukOBcZL/RaO2mm6vY9yStjotZRNU2JDLi1OjJoMgXumnrCDYb/accWCAw9otsO82G+4ouccJsHz1jKhNEpgzoiHt2uZVvLRggpyAO7+beDbk277g3G//zDw4fkel8iM/cs34NhxGEkvn6ZS4cen/g4+8ZfwhS9NfUxREbzxDfM6zMs2z/vmWkd7fU4SEHDE2niBb55we5wYEaIEM2wKP6qXAZZTw41efzXDlVnUx2B4YGJK+Ibz3kuC5BWdDO6kKz3LOvHvlVs4Q7vlKsYskis6aiUy2777/YshbjyLwP6D8N8+GLbWKopDYSz8/WteBdvunP+xXq7aJmLxahqa22jcvoNEz+45K1HSZ4MZN8y7hd/oJ6uhKvtS6DQrpCWqP7bgBBmry4UMMnfYmIHplsq1jC75asHNyInkvVSWGSH38L77Xg63b4Vdj4aB746tsG76lkcLyjztm8vW69OcKcuqEO5re5Ffx8M8l/38+7jbzcNZvJUskJPCMqaMkvRJ1amBrYjCCTXtkqQ4xVn6bJByL2HtaGuxDFZRlblrmBvrrmLfnUguKciJzLZ77oZv/iskM1Smf9Ed4a9VlfDaV8/vuGbZXO+bW0c1+2kl5RNbY0WIsM6rec6O0cEFCohS7zVsoCZjwBsVlq4IHx/glFPM7d6kmZgFKIJxszfwNEfGDiXg4WGXW8YVC558ICJKhOdpYZvfmPHEcYwCbvWNPM1RnLBwdNQjFFFIk6+fcr1IPlCQE5ltb/vFsC5cV9fFMiNFRfDG+2HNIqs/Nof75sJenzfxlB2mK93rs4Qirvc6nrADJEjh6U3rezlBJ93c7k2soopO77rY3WHc873Ct9DHIIXEVFpkgVvLSko8ziFro88HqaSEzb6WinFhfXeGAxGp9IGIbX5jxuddRzVVXkaLnWHIw/Ija1ihos6StxTkRGZbZQV87tNhod9HHgvLj7zhtbD1tlyPbG6MqzfXRYzjdZdfb26QYdo5T0DAKpZRQQkQ9vq8229khCSOU0gBj9m+sA/nuKCWsoAz3kUXfdzsG/mxPUfKU6QswNyIYNzmm4lRwDIW7ulgmWgZ5dyRoVgwQD9D9Gc4EIFBl/cxQiJrbbgyirneN8zyaEVyQ0FOZC6UlobN7eeqwf1Ck943V9W8i3rg+Rtmvkx1nA722DEg3Ma2n5OsZyU3+8axJbTRFkzP2lHO0pNxD1xAQCfdbGYtL/dbaaWT8/RS6kVsoEaHGhaZgGDaRfHp6gqKLCYKciIya2Lx6su6vo9B9tixCXW9HOekn6WacMlr1CDDtHAm6wlUI5LupxruhdpILRvVSmvRKqM4PAwzgwMRowICEqQopEB7I2XRUJATkVlX3Ns6o5OsJzmbsVZcygKO08EaX8EICc7SwwX6MtYAu8gnBD9Z3AzjFt/EUxya9kAEhAFun53gBGdwnChRmnwdDaxWoJO8pyAnIrOnNlxebWzeP6OyJAlLZo1lCZIcpo391oJh09cOc9ji12SchZHFq5blvNhv4JCdos8HqaCUBq8hQoQhRsZabj1jRznF2bGZ34Akz9OCOTSgWVvJbwpyIjKrLqcsSY0vo8XOTAlpETcqvIQD1pr+5pt9v5O5Ue81V9V/U/JXFWXc7k04zj47waO2L11iJqCaSm7w+gkhblTKAg5wknrXrJzkNwU5kfk0WhS4IMP/ekEAzzwHZ87A5mtgY8P8j2+2pMuS0HwEIGtZkmoqqaKMC95HkC78a27EiTFow9ln4dLfk6NEWE4516MTiEvdIU5xnA4C87E2bWe8myE7lA52Uwt1J0iRJHVF7b5EFgr97RWZa4ePwCc/HYY0TyeQFSvgPe+EV74i/Px0B/zGb0NXN+AQONx4PfzpH0M8T09bjqsxdwzormNKmDOMO/06jnGaE5whIGANK7jG1/Co7cveWsvCwPdiv4Eqyub8rcjC5jiHrW1KFxA3p8f7s864RbGxAzIi+UpBTmQunTwF/+U3YXBSb9Bz5+Djfw0Dg3D2LHzla5CcNGPw3F741IPwof8yf+OdbemyJA3NuxhoPsLB+6fWmIsQYRNr2ORrJjx0JZX0+EDWMBfBFOIECHuvpjLMuAFEiVJGEd0+MFZAGsKODg1eq2VVyXsqZS0yl77wzzA8nPm+kRH4xN/ClzOEuNH7v/v9cMk1z8Xqt1EZr6Nx+w5W791Jj3dkvC5FQAcXaOMcdb6KAiJZt8dpJkVGRYlkXR4NCNji11BNJRE3CjxKxI31VHMtassl+U8zciJzac++6YNY6hIhLZmERCJ/l1fHu8S+uTN0sdsOpj9zApyVXskZ68r4dMMkaLbDrPAK1rBCwW4JM4wmX88+TkxYXo24sZrllFPCi/w6BhlmiBFKKR4rMi2S7/Q3WWQurVwBbe1X/vja1YsjxI3Ksm9uiBGesANT9jh10j3t07VaJ22cYz+tvMRvGis3IUtPPTUEHnCQk6TS07jrWcmNfvHQUDFxirN0+BgmwTl6KCDCSiqJaMFK8oSCnMhc+vk3wYFD2ZdXpxOPwwfeO/tjyrUM++YOV5RlLAyc6bYx4xqlBz7CHjvOVm+co0HLQmcYm1hDg9cyQoIYBURnGMb208phOzUW3gy4w69lBRVzOGKR2aEfOUTm0ovvhNe/5vIfV7ce/uSP4M4XzvqQFopY/TZKKKZx+w6i51qm1PkCwu+o4/eiZykp5wbtnJ8++MmSEMEoonDGIe4knRyyUwTmJC1F0lIkLMXj9jwJknM8WpGrpyAnMtfe+SsQy9JxIBqFWAEUFl78ePd/hi98Bra+YF6HmQujhyA2722h4FL7BSF7ORIuMXsnkkGSFE/bkQmnWcc7xbl5HpHI5dPSqshcKy6C1/wMfHfHxCXWeCG84mXwjrfCI4+Ft734TqhZlZtx5kptE1tOH+AnvcOcryjCI+m05kwb3CbwsLiwSknI5ThBB0GWHwBSBAwzMs8jErl8CnIi8+G/vC/s6PC9H0DEwtOqL70XPviBcLbujfeH152/AJ/6+zDYFRfDG14XhsDo4j6RGV3dxDv3OjtWnmJvwwpSsYLpOqsScQu/AVv4+wgRbvI87oQhOdFhF7L+sGDAMsrndTwiV0JBTmQ+BEEY0gDMwhZdz+2Bc+dhdbo47vkL8K73QW9vWHYE4FOfhieehI/8fm7GPY+Kk8brT6/j1Y/vop9BPv3WuxkpmBpgIxh3+vUctw4GfIgVVLDRa3ViVQBIkOSgneIUZ3Gctayg0ddRyNTtDYUUZJ75dYhRQDWV8zJmkauhPXIi8+Ef/hF2PxXWhBsahqEhaD8Nv/P7F9t2feVrE0MchNf+9AnYfyA3486BWP02quJ1vOyRA0Qm1eCLuHGtr2cFFdzmm7nbb+J636AQJ0C4HPqw7eEY7QzZCMOW4Bgd/NieI5mh80O9r856KOIOv1ZL9ZIXFORE5po7PPTtsFPDeEEQ9lg9cjT8/JHHJoa4USOJcFZukegodb5+XZK/uSPBF29OcrRq6iJqX10jqYoarmm5QOlwgqhDmRdxi29kLStJTbvwKktVG+cYZHjCCWg3Z5gELZyZcv1KKtnkteHyvBsRjxBx4yavZ7mWVSVPaGlVZK6lUtnryEUjcCHduaC4OPM1BQVQUjI3Y5tnrRUB/3RziqSBR6Cr2DlZkeK+I87W9nAZ9dDygK9fnyJgBakoxBIpll3oo6LYeLboKAEXT6iWU8yNXk81VTl8V7JQdNiFKUWlIaw12MEFNnrtlPuupY46X0UHXRiwmuUznuHtYYDjdppBRqj2StZTnbVVmMhc0YycyFwrKIC1azLfl0jA5k3h79/wWijK0sXh3pfMzdjm2b9tTpGIhiFuVCIKOzYFJCJOIuJ847rwmlR6e1wiFuVcVQnHClOkzMNSEen6cr02yGP2PM9wVOVHhDixzL15nYx75EaVUEQDq6ln9YxDXCtn+LE9x3E66LALPG8t/NCeYUgnXWWeKciJzIdfe8/UVltFcXjta6AqPZv06p+BF90R3h6JXKwr999+A1Ysn/8xz7JExOkszXxfxKGt3DmyLHMY80gEj2b558rghHVwgo5ZGun8cIfhTmOo3Tizo4CzPyogSOR6VPmtzldlbK0VJUKDr56110mQ5Fk7RmDBhA4jw4ywz07M2uuIzITmgEXmw50vhI/9Ifz9Z+H4CVhWBW95E/zc/eHJ1aPHoHol/NHvwf6D8ORT4VLrPXcvihAHYVgzD7swTOZAYcroLbzCWTWDA5ykfha/Wc+lsz8q4On3lNJ/LAIpIArRYicSg61f6mPVK9VR4EpUUsoNXsdeTowdVHCczb5uVve8naU740EIN2j387P2OiIzoSAnMl9uvy38GJVMwcf/N+z4j3DmLZmE+jr4Hx+Ft/5C7sY5R6JuNJ01Dqx0gvGTJg7FSVjdBxXDRjLTxNsMigMPk8DxBX/SsGdPlMdfV05qYNw4U5DqM1LAT99Uzt0Pd3Ps74po+3q4zLfmTSNc99FB4tVaPr6UBmqp9RV0cAEHaqiimCxbFmbAcRIkiRIdO+GqPwVZSBTkRHLli/8M//6jcJ9cIr2mdvhoWJLkwU/mdGhz5T8dinKmLElvIYxEoTAVztT9/J4Chgpgb3WGOvvpGyIpCKapixwntiBC3Gg1GcsylIP/s4jUUPbHBwnY9dIKUgOGJ8Inafl8nDPfjfGyPd0UlM3ygBehIgrZQM1VP88pzrLXTjBC+P9nLSu4xTdSTWXGPZnmUMvimEGX/KEgJ5IL7vD1b009zZpKwYkWOHYcGupzMLC5VZI03vdEAYdWOB2lTuWw0dQJP2pI8VStk7KJByEAMChIwrpeaK2ElF28fVQkFbDeKubrbWTU+3yE5z5YQucPY0QKoPaNI9z0VwNTZtG6m6MQZA+cPmKkAvDkxWs8YYyci9DyhTgb35/lBPQ0BtuMtq8Xkuw1ql+RYNkdqaxBU0JnuMDTdmTCKdh2P8eADbPVN1NFKee8N7zDIOIRColyvW/I0YhlqVKQE8mFVAADA5nvKyiAzrOLMshB2Jmh6ZzRlO5HvnNDiuZaJznNbFsyChu6I7xlb4TTpQE7NgacLoeCACDgRc+eYk3BKXrqeohVbJ2PtzHB4Enjxy+uINlj4EaQgravF3Lh8QJetreb6LiVvfKmgL79WTYLAkR9QogblRowznwvNiHIJQfg+N/HOfmlOFbg1P3nETb852Ei4w5otv5zIU8/EJ4yCUbg0J8VU/3KEe74aj+2uDu/XZXnrWVKKZPAnG7v44f2DClSF3+YcCgkyj1+M3EVp5Z5piAnkgsFUahZFRYEniwxAhuXRt9Qx3l8XUDiEoGiMAU1fUZRyqjvifKrT0cZKHD6C6FqCGKnEiSGO+kixvE6GCxfT4Vd/dLaTB35qyJSgzYhnHnCGO4MZ8LWv/ViSYprfnuQM9+PkcqY4z29HzBT0HN69kboOxShbHNAahAefnEF/Yei4WsDvXuitH21kLu+14tFYei08fQDpQRDF58rNQCd3y/k+GeTNDwwTJCEM9+J0X80Qvn1KapfnsRUz4A+Mq9/BzjB+BAHYDDkCf7DnuFuv5EystSEFJkD+t9VJFfe866pJUnicbj3Hli5IjdjmmfJSLhXbjqRAEoT0HhuYrApSRrVA0YsMKhtClt7NZ+gcfsOintb6fH5K0dy9scxfCTDLFpfhPOPTfx5efkLU2z5xz4ihc7UbfNGuuJxBsbg8Sg7X1hB/7EIrf8Up//wxRAHkBqIcOGJAjq+E07JnfxSIUGGldjUgHH803EGjkf4waZKnnx7Kfs+XMJP31TOf9xUwXCn1l2nrSeX6ctjYVmS3XZwzsYkkomCnEiuvPQe+O+/Catrwp3xJSXwpp+F3/5Qrkc2bwoCKMlWO83BAth83nhncwGRGRxkiNVvozJeR+P2Hazeu5Me75jwMVdK6lPhLNokFnWKaqZ2Glj7pgTRymxHcY0Ja3bjuZHsMw7+SRGnvlo48eRrWqrPaP9mGOROfbUw6xHLZI/x0zeXMdgWIdkbIRg2Un1G/5Eoze/OUvBvCdnsa4lO2rAZ1qKe5u+hhTN5fQzO8ehELtLSqkguveye8CORCPfGLbEd6IZx77EI379m4vJqQQo2nzN+dn+Ugmx7ybKpbaKyHUqa21jGo2M3XyDB6RvumZMl100fHKbjO4UEk75/ewpO/kshm39naMK+NYBYmZPovNQzZ3jvKaPz32OUXpOt36zTfzzCD28rp+fpgszPgVOyMcn5RwunHLzwhHHm+zESvRBbwu1G11PNkI9wiFMYRoBTQTG1vpwDnJzQz3W8CEaS1DyPVpYyBTmRhSCWvX3QYnfb6SiBwY/qA0YKwlmPF7Qb9x2NEr3cEDeqtolYO1Tvu7gRrWq4E9hJd93GWT8QseLFSapfMULHQ4VM3jw12Brl9PYYq9+Q4Mz3YgydjFBcn2L5nQkGjkW4ZIG8DAqXO9Hi0TXYqY8/93Bs3PHezCq3BHT91AkyPN4ikOo3YuVLp2JaH4O0c36s32oZxTSyjo1eSy8DFBKjlCIcp5t+2vx81j+6csLeyOfo4aCdpIcBSimiydepL7DMOgU5kfmUTMEPd8J3vx+WIHnlK+AVLw1n4xa5U+UBP2gIaC93ipPwotYId7RFMIzb26NsbY8wWADxVFg8eCTiPFOT4mSFs2zQuKXDOLrM2VUX0BOHFYPw0mNRGs9n2SFS2zTh01g7NDS3MdB8hIP3z+6BCA+gb3/m2a9Un3H0b+M882ulpIbC/Wlkm0yb+sxTnjNa6mz8r0Oc/Mrk0DjKuOSEkEHDA0OceDBzodzClQHxmqUT4vbZCY7SPrYKvZ9WrvG1XMt6CoiybFxXCMPY6k0c4zR7OB7Wk0v/MUQ9wvW+gSgRTnOe3XaIgLCN1zAJfsoBbvIG6lg1/29SFq3F/91DZKEYHob3fzCsE5dMt2Datx++8334iz8LT7IuUq0VAV+8OTVWYmSkAP5jY8CZUud1h8J/hgyjJP1l6Sl0PvOCJMNRSBRAQcrZWR/O1qXSz9FRBl+/PsXrDzg3ds7ga1fbRIwmSo7vonH7Dg7efx895VxRmAtGwv1npx+KEVvmDLZG6D+abcuxc27n6IzrZcy+mROJg0UcizqeAg+M9W8bZv3bRxg8GeHcztiEE6mjrzf96zjL7kxSusm54c8H2POh0gl77aIlzs2fGFgyq/xn6eYYpycslTpwhDZWeVXW1l4NrKbSSzlgrfT6ACUU0ejrWEUVjl/sxTpOygL2cJx1vjJjT1iRK6EgJzLbnt0Dn/8iHD0OtTXw9l+CdWvh/f8V+vonXjs0BAcOwo8fhpfdm4PBzo8dG4MpdeISUXiuxnlJS1gYeLx/a0zRX3ixAke2GnPJKHx/U8ANnZEZd3WI1W+jsv0Ajdt30LFl02Xvm0v2w8MvSZf96Ld0qRDIHp6uJBE5q183wnUfG6KkPkXHt2Mke4yVL01SujEMB/UPDHPkL4vCU6ljS9CX7mUWKYStX+4Ln+NXRyhe6xz4WBEDx6KUX5/i2j8aZMXdS6fX6wk7QyrDFGmKgBY7w3KfGORGSHKSTvptiEov5XZvooCJf0GHGBnrBjGZ4/QxSAU6UCKzQ0FOZDY9/lP4w49d7NjQ1QUf+dOwrMjkEDdqaChs1fWye+dpkPOvPcteq4jDyQqncly5iwDnyDLPWi93ssEYDMTCEiUzNu5AxOXumzvy10X0PR8lGA2fV7qPbzoGt32xf6wd19o3T31z8ZXOSx7r4Zn3lXJ2Z8HF06kZv9QOESgoc7b+Sx8l6y5eVPOaBDWvuZwv3uKSIJm9nIhPDLRd9PGo7cNxUhYQJcLztHC330TJuH6uk4PdeI5ToG+9Mos0tysyW9zhf39yatutoWHo7pn+sYv8sEN8mgmeyeVH2st8ug5WUwQGqSwnCKdV20QsXk1DcxuN23eQ6Nk9oxIlR/6q6GKImysRsJl8rzcYOWdYIdglatBaAQQJ45n3lU6pbbeUrfEVU8qMQLjfrdYv9k11nN12kKSlxjo+pCxghARP2+EJj41RwAoqplak8fAgxPjQJ3K1FOREZkt3D5w7f/mPi8fhhbfDl78Kn/sC7H3+Yuf1RWJrW4SCyRvwPezYsKErDEUjEef/3JLkC7emwm+Amb4EGW5z4J9vSoWbyi9XupBwCcUzKiTc+k+FJC7MdYhzVt2XIFo0/WWpYdh1TwU9e6L4kOEDo50lMqQHDB8xgsGwqPCjrypn8NTF99GzL0LrPxdy9scFi+2v3iWtZSUlxImMm1mNuFFGMWu4WJi7j0GGMyyXusE5eqeUHNni11BMnKhHMIcCj1BEIVu9ce7ejCxJ+rFMZLbE45cfwMzCdlx/+TfhY1Mp+Oo34IVb4Q/+P4gujgMQd7dE6Ch1jiz3dFFViAXwtmcvFvr9zuYUJ8t97DADMLblywKIOiTH18odZdBZCn/1ohRv2Qfrey7/59NM++bGK0vV0PzOUk5+OXuB3cwm71lLv/kIWBQisbCvqkWd1ECEaFlArNK59cEsy/DjnP7XGMl+ptSBG19MOFrmYeeHSSHaE3D8wTiNHx7ip28q49yPYlAQ/tkU1jgv3tFLyYYrCMZ5KEqEu/0mjtLOSToBY71X08DqCQcSUhkLtVwU/iBx8S9vEYW83Ldwhi56GaCUYmqo0iEHmXWXDHJmdq2775+PwYjkteIieNHt8PgTF0+lQhjWllXBwEC4zDoqGoFf/Hn4l6+HBYFHDQ3BT3bD9/8dXv3K+Rv/HIq68fP7CugscU6VO6UJ2HTexkJc0pw91ZNCHISZxKHxHLykpYDPbklmnncz6I+HM3Pvf8KoyNAu65ImFRIe3LAZgA4fYs8nV9D+zanFc6cVcaq2JBk+E2GoLYIVwNqfH+Gmv+4nWgyJLiO2zEn2Gie/VEj/0QhVW1KsedPIJWfjAPqPRgkGpxlPkWO1KTg0ddk+GDZan47Q/ptxen8Yg6GLCTnZ5/zgtnLWPHqW2PrZCXO3FC/spcQCojTy/7N33vFVluf/f9/Pc1Z2QiCBsAkQtoIgIooWRa2KRa1Vsa3Fn9pWrVZtq7a1/Vrbql3OOuto3XWhdeNAEReIIDNhJJDFSMhOznqe+/fHnXVyzskgO9zv1+u8Qp51ricJ53zOdV/X5xrBRDki6jGqOSHyzzseDy7Cf84CQToppJPSVaFqNGG0JyP3tRDiYeD/pJRl3R2QRtOv+eV1cM0vYN9+JeacDjV6656/q+7Up5+HAwdgwnhY9kPY8E3kLJ7XC6+9MWCEXANDatV81JYEzOg9A24Lji0wGVYtSPFCaWz06/tNuGtuEI8FM4sFJ+421SzW9lJvJJz89W6Sv96tNgEf3HlcxHFY4ajfZcJUi6l31JH+bSXQgzVguMFo9orrHqKOdaVIxl0ZYSBqGyROsTBiJFZVlLi8Bj4khlMiAi2mN7gkFYMEzsdjEWH1fgK7zKBgVhp1D9Qgh3durdWXWgkTfX1ezLXFfsqwsEKTrBIMDI6Qmb0ZmuYwpz1C7mjgTmC7EOIW4H4ppZ4/otFEIikRHnsQ1q2HvN0wbCjMPVp5xGUMU/NVm/PFl6HZu+Z4vd0fbx/BE4Q4P1RFyERZAtJq1DvniXkmr2VZIeO8QqhPLHkNWDNcUpRg8cNvzHZbkwCNfnPN8Ve0Ms1omuMAACAASURBVEC9EYlrsORbGyrwDA0VP442nCY21HVcyMkTfYjBMVAtIqpgM0YyY7HBlgchGFLaJXE64ZQzXax4MvK1BQK8kszH4jj61g6HFsKafA/r2E/uYC9jB3W+vrA3BKGNzTqxI+KHjSRionrNaTQ9QZtCTkq5EThZCLEE+CvwUyHE9VLKt7o9Oo2mP2IYMHuWerTF0bPhvy+HizanE46f3z3x9UEEglN3Grw6KXTmqtOCY/cYuOvHTU09YFDrkLybaWO1Md0qaEJRolrKHREta9VO0jNrKM5JavUYwwULPq8ME3FtsaHOx7ocG3dpYofjErd7cf3Zg7lB/dAaBKtwSDypMPp0SJkE626T1BSoc+JHwaxfQ2w6xA2ncXsYUrB/Tec7H+YkOyA/je2lXjZ38lq+1MpGQdiTgq6M6sg7BJTLGmxsXfum6TXa3ewgpVwuhHgTuBZ4TgixGriuq+vnhBCnAXejqkb/JaW8vcV+N/Af4CigFDhfSpnXlTFoND3G1Ckw5yhY81WTmHM6VU3deWer7wuLYNVqtQR77DEwelTvxduNTCkxcW4RfDjGoiQW4v0wqBa+Gmbz9TCbGfsE8/NN5hSbTDgoeGB2fWauFY1mA4WJkhFVTdskkj1JknKPyvQNq25b5E0+YR/7/BZTjnyCsSUrcVTY7C4+hS0lP8BLMoYH5jxXTdzYjtWUNRdxS13xHToXgKHAPVCWDVsegtINEsMBwxfC1J+C4YTkLFj4BHhLAQGeJkcNpl0Fa34rsYORfwZmF7nizEl2MIdDuL8WrMn3sL3US25WFQzq/8u1Gk1XIGQHuuyEEMnAbOAs4EpUL9SDwM1SyopOByOECeQAi4ACYA1woZRyS7NjrgBmSCl/IoS4ADhbSnl+W9eenTVRrn34vs6GqNF0PZYF774Hr76umiEWHKdE3N59cOvtsCdfHWcYaibrOd+Bn1zauzF3M35D8tDsIJVuVOYNMC0Y5IXLvnLgkIKieJtnp6kJEEBkQSchJgBnbDeYUmJS6ZL854gg1Q3NpwKGVguWbjQbs37VTklhosQThFEVAq8DHpQBvn/5FSTl7cNV5w9pXLVwUjl0KMVHTiO2opyaQams/cEPyTu27YxqWYlkWn6aylr1EkWfwJrfErY8a7gkY5fAtCt7J67WeMZfjS+1kpTBnV+qbU92z8bmHbGWgGjZ/gtDSGKenNLpODSalsw0U7+SUrbpVN6mkBNC/ByYU//IBPzAeuCz+q8XAdOAc6SUX3QmaCHEPFRTxan1398EIKW8rdkx79Qf85kQwgHsBYbINm5ECzlNv2LDRvjlr8HvD9/n8cBtt8DMI3s+rh5iTYYVcayXw4JjCgQn7Db5bITNqtF29Hq5Zhg2jC2HUg+Uewhx0DRtmHxAcPY2kxXjLL4cLnHUJ9ZMCQFg4S8eZvYDb+AIRC8PbqiBl0DQHcvGs67hq+/f0mZsvSnicl+FbY9CoBqkBcIAaYMZA/Ej4bi7wdFKc0lvsqa8a8aIbRq5n5TBgiXDW28V3k8Za0QONhIpJKY0Gq1L4mhHm7FG00HaK+Ta8wpyPUq0PQB8DnwlpWz+7vIfIcQNwGPA1EMJthnDgfxm3xcAc6MdI6UMCiEqgFSgpJPPrdH0He66L7KIg/qO1jcHrJALCslHo8NFHKi6t9UjJWszglgGkUVchHGjtgE7B0XeZxmwKU1iEWRrmtrvbxB69R8Ppz+zslURR7PLCsDpq+XIV+/Ec+6l+IeNbfW83mLPW7D5n2A171oVkriRMP0qVT9XtQcSx4LZB1cwu0wA56exif0sp63mohgwJyHiS8DhI+CLI1CTygoJY/Uyr6YXaU+zw8h2XOdx4M+dDyfa4kiHj1EHCnE5cDnAqPS0zkWm0fQUXi/s3tP6MdVRiq8HAB+OsahrpTZLGqorNaoxb2urba3MtW8QcZGON6xD8FOTksRPX6fk3J91/NweYOu/Wog4QFqC2r2SzQ9CTREYpsrQTb4Mhh0PtXshfgR4UqNctB/S0IwRkkJolVCvue2xumZP07t0VU7/ALCwC65TADQXjiOAoijHFNQvrSYBEeciSSkfBh4GtbTaBfFpNN2Pw6Hq4ewo4sHlhBOO69mYepCvMmT75tB39aSsVq6XfdbRTH9yJWa030mkywV8pC5/gNpJc6idekwXBNh12MH65ocIyCBU5QFSNJovb7pPsvkBlZmz/ZBxIsy8QTVTDAQ6k92bQzzPZMO6er+8jqCFn6Yr6BIhV1+f9lEXXGoNMEEIMRYoBC4AlrY45jXgYtRy73eBD9qqj9No+hUOByyYDx+vjuwxN3QoLDqp5+PqASQSfx+cSvbBny9m/FtfEXegInwQehQE4NmzjfFXn8ju3z5JxbfO69YYO4IwwRkPgaoIOxu6QEK2CWSw6c+x6GOJMx5m/LybA+0nLHXFsybfw+bSjnk/5mZV9biVimbg0admrdbXvF0FvIOyH3lMSrlZCPEHYK2U8jXgUeBJIcQOVCbugt6LWKPpJn5+FezMhf37m8Z6CQHfORMuW6bmug5ABIIhtWp2al+iOiOVB7Y8wDF/f4VZD79NbGmTAmprJVcE/Yz5v/PJsyUVJ30v8oHBIIlfvo2zpJDarNnUZR3VpfGHxSVg/IWQ8x+J5W12B4ZUQq4NsWr7BHlvSKb+tG/Wz/UGh2Kx0p5MnhZ5mrbokP1If0Z3rWr6Hbat/OV27oL0NDhuPrjbM2Ggf7Mzxeb5qVZos0PDy1T7p2S1//iOICWDa3xs3zKLXzzxIhkP/xphta97UgIHT/0h+Tc9rpRUPe68rYy/diGGtxZhBZFCUDt5LrvueB3pjuniG2gWjw2bH4Dc5WqJ1A5Awhio3kOouIt+BU54BJIndluIhwXP+FW9a/qo8J95VUylztgdxnSZ/chAQQs5jab/kJts8/5Ym/1xktgAzNgrKEiS7E6OvPJn2OAOqs3jygRDq+CDsRI7Wldr/VcDsJvmxUc+rh6nBTM35ZJiC54RS1hWlUHsli8YedsyPHu2tVtjlp5xKQXXPwCmCVIy+fxxuPbvQTR7LbZdHkoXX0bh1Xe346qdI1AN1fmqgcEzBFZeqmrkZBST4CYkg2bA8fd2e4gDnmhWKttjvfhSK5k10dBi7jBEC7kWaCGn0fR/bCSfjrD4eIzErh8x6rBh2j7BmdtDZ6quGmnxSXOfOalWDo8pEMzLN4gNCjakS16fYIULPgkziwWn7zARUmIZgppd77HH5+fFhcsYuWVqU4G8lIy5cTFJn7/ZbjHnHz6e7fetwrU3j8zrFmHWhXchW544Nr4TqYite/FXwdd3wN5V0FZK03BKTnkRXElQuQOsACRPGDhNEH2BNeXBRq+7ttDZu4FFV/rIaTQaTZ/AQHBcgYMpJZItQ9S81YmlRsQxW8fnm6TWCVaNsqhyqwkOJ+YZjKhqcgM+cp9gT5LN5jTZKPhMGxJ8sGiXiSkFFOdQ7NtFzqTxLHctYVlRBiQ3eyIhyLvjddL/dTNDn7oNpN1m3ZyrKJfRt1zAge9erTqUI92rtwbP9vV4J/SsX6ArAab+GA6sAauN2n1hQukG2HivyuyJ+uzmEb+AEV3hY9BPsbGp9dThtBy4A50TVu21R9ke622st9Ni7vBCZ+Q0Gs1hjUSydbDkqwwbnymZfMBgdrHROLKL4mxypjj5p7WYZVUZrV7LqK1ixB2XkrLyBaD1fJYEpNONsIIIO9xsWALSHcPei3/H/otuOLSbO0S8B2HF98AOtJ4FciZK1c1aG3qc6ZbMvwdSJnVnlH2T/LQCNmVuxTZspJCkVCZz1NaZeDop6NpD8+zd2EGRf3da5PUfdEZOo9Fo2oFAMKVEMKUkcmasI9ixCey55XkqPjiHMbdc2MbzKq85aZhIwwwTcwIQvjqGPnEL5Secg3/EhE7H1148gyB5EpRtkUgrsiAw3ZKh86BwZfg+yw87noc5v+/eOPsaRanFrJ+4EWk0JUgOJh5k9RGfs3DtgpCl/+6gIXu3vdTL5gj7famV5A72tjmOTNO/0EJOo+lPlJXD6s/ACsLco2Foem9HNLApzqbQt4vCuvHQzoZh4a1FSEn1jOOJ/2ZVm2/dwrawTQfS4cIIRhjLZlskf/QS+y+6scPhd4bZv4dPfgb+CokdQHWGSHDGQcJoyPoRFH6krEjCkIKa/MNjtaeBwsFFfDV5fbgFnwE+l5eS5FKGlA/u9jhas0FZk+9pHEfWPGOns3T9Gy3kNJr+wutvwT33q5oqacM/H4Lzz4P/d3FvRzYwqRdxm8dm8oZrSZvLqgCxmz4l81dnqDo5bx0QcbxrGLYnDishGffe3WH7hG0jgoFDuIHOETMETn4a9n0JNQUQPxLS5qi6uAZq90J+jMSqC71DYUpSJvdwwL1IrbuWr7O+ifqLtoWkOqamR4Rca0TL2Glj4v6NFnIaTX9gT74Scf4WGZsXXoIjpsPsWeHn7N4DX2+A2BiYPw/i+pjLbh+nwreHDxcu4pvs41lW1bbRq/D7GHfDmZg1FVGPiSbqjICf0uPPYfCrD2D6QzsMpNNN5bGLOxh91yBMGDov+v7hC9XMVtsfugRruGD8YWTVnp9eiGxl5IeQgvi6vvH/r2XGbk15kO3Z6Hmx/Rgt5DSa/sDbK8AKL4jH64Pl/wsVcrYNf7kTPlyplINpwj/ugVtuhrlzeiriAUGRezgTaj3tWlZNWPNOxKaF5gjCxZztdFMzbR77Lvk/kr54C/btwfTVAsqCpOzkpdT1cOdqezHdsOAB+PovUPK1mgqROA6OuB7ihvd2dD2Hz+ULqYsLQYLH72FweWrPBtVOGoTdM9mwmXpBFwGdseu7aCGn0fQHyssjCzmA8hYZoHdWwMqPwdcie/e7W+HFpyEhIfJ1apV4IDa2c7EepphVZdBOFwBpGNiuGIQdpGrWyey++Sns2ARyHl7DoDcfI3nli1ixCZSedXmPZeMOboItj0DFdnCnqBFeo88IGULRSE0RFH4AwTpInwvz/gq2H2wLnIfhn8+QssHkpxViOVr8H5UQ441h/oZjur3RobMsdcUrY+KvwxshtLVJ30YLOY2mr1JdAyveh9w8QIDHA94Wxl4uF8ybG7rtpeXhx4F6R/5oFZx5euj2vN1wx98hZ4f6PmsC/Oo6GDO6q+7ksKBmxvFtjutqyMZJKRG2xZ5fPEL5qd9v3G/HxFFy7s8oOfdn3RtsC0q+hs9vAKu+cSFYA5vulVTvgWlXhB676xXYfL8q05RB2PUiDJkNR/0WjEiTNA4D0kvTiPPGUR1TjW3aaqMNzqCTBV/Pxx3sH6P1Gk2uW24nnmdKYV2O6nptQGfp+gZayGk0fZHcPLj6eggE1PKpxw3+ADgdEKgXCw4HJCXCWWeEnlsZPiUAUNeqarGvrByuvFZl4xqySVuz4apr4anHIDk5/DqaiPgzxnFw0UWkvP8spldlN1suozb8W0iJ8HvJeOgGyhctBcPALD/AoHefxnmggJpp86iYfxY4emZEwsb7mkRcA5ZXkPuKZMJScNf/GdQUKxFn+0Wz42Dvaskb31afFVKmKkPgxDE9EnqfwMDguPXHkDNqB/nphdiGzbCSdLJ2T+w3Iq4tlrriWZPvCTEmXjdyv7Yz6QNoIafR9EX+cJvKyDWIK69PvUsmJCgBZ1lw4vHwg6WQ0KIQ/+ij4K13w5dinU44YkbottfegIA/dElQSiUaX38Lvt+6F9qApFm3amHhYFI6cGrBLx6idvIc0v57F+6C7W3WzDmqK3DtzcO1N4+xN30HEfRjBAPI/4J0uCi85m5KF18eeX2zi5A2VO6MvM9wQvk2SD8GAjWw5WGwIyUdpYpPSji4SbLqCjjpSTW/9XDBYTuYkjeJKXkD1wU5LGNX3wG7nCptQNyLaCGn0fQ19u2HwqLweispVUbtuX9DaivvkD9YqmrkautU4wOA2626WydnhR67dZsSbS3x+2HLts7dRz8kkPcJ+6lSliMJZyvLkY4kJQ2Dg4sv5+Diy5l42Wxic9a1frxtIQ2TMTd/F9Nb07hZACLoZ8RdP8MsO8D+i397SPfTLgSYHrDqwndJG5yJsPdTWHuLsi/EbuN6UmD7JbtehimXdUfAmr7CnGQHlHvYXt8o0RJtQNwzaCGn0fQ1AgEwomRghGhaWo1Geho8cj88/h/48itlP3LWGfDds8MzO6NHwdp1EGxxTYcDRo089Hvop9RS1yHLkdYoOevHDL/v2sZl1pZIwDsyC3d+DiJKI4uwggx9+jZKzrsGOzZKk0onEUI1NeS9JkOWTBESVxLEDoNPfx6+9NoadkBwcNPhZQh8uNKWAXG0jJ3O1HUdWshpNH2N4RkQn6CWU1syKEUJtbYYNhR+/au2j1uyGF79X2Qh950z2xfvAKMjliOtcfDby0ha/T8SP3s9ar9ixfFLMPwRUmHNsB0uYnZ+Q830+Z0LqBWmXA5VuWpZFAHCAMMNE38AX9wEVqt+xBHc8QxJ/OH3OUDTgmgZu1jtWdelaCGn0fQ1hIAbr4Pf3qKWPW1becE5HHDDdR2vl9q5C77ZBImJMP8Y1f3awLCh8Kdb4I+3NwlHjxtuvkntO5wozqaaINWVskN1cVFxOCg5+woSP3s96iHBQenUzFiACEQQ7fUYwQDBpO6dCGC64dh/QHkOVOSAOxXy34FN96pmhqg4pOrCbfE5wHTCuHO7M2JNfyFSxu6ZbLSdSReihZxG0xeZfRQ8eC8896LqYM0cC+d/Vy2FtpegBbf8Cb5cq+rrTBP+LuD2W2HGtKbjjpoJLz4Du3LV95nj1Biww4hA3ifkWhV8PP8UvHmZUW0YOkrMjvWt7q+YfxbOfbuRDieyfs5qc5kuhYF3xAR8o7IiX6CLSRoPZVthwz/Au79lNKEIUzLmLBh2PKz9v/p5rKhs3swbIXFsj4Ss6Yc0dMCuY3+InUlztLVJ+9FCTqPpq4wZDTdef+jnv/CyEnG+Ftmem26Gl59TDRANmCZMGH/oz9WPaRBxTy1Yhjcvk6WuztXGNUeK1gWx7fKQef2pIY0OoBYrpekkMGQ4uX9e3mXxtMW6P0PxKmU90iqGxBELE5aqmaynvQJl2SAtSJkMhn5n0bRBw9zX5nYmDWgD4o6h/7tpNAOV5a+FizhQ2bnPv4QTju/5mPooOYtPxbuha0UcQOW8M8h46CaUNAtFOpzEb/48Yo2cAKy4BLY+u6PHsqNVeVD0MdhtNDUIh2T4Qph8iRJxoGayDprS/TFqBhatGRBHy9jpDthwtJDTaPoitXVqqsPXG2BomprGMKKDwytraiJvt+1wY2BN1xAMkPTJqyR++Q7BxFQOnr4M7+hJeHZvDZ2vajooO3kpZnX0sV6Gr65Hl7hLWl8FBsD0SGb9BjIWdH88msObSBm7TSP3sxxtZ9ISLeQ0mr5G6UH48VXKENjrVU0OL76ipiy4XMoI+ILzVPNCaxw5A1Z/Hi4UbKk85TSNy6p7D3beKsOoq2H81Sfgzs/BrKvGNh0Meelein90M4NfuR/nwX0IaWObDuqyZlN4zT04yvZHHetVM3luxO3dhTNe1bdFw4yRDD4ChnVf86xGE4I2IG4fWshpNH2N+x9So7MavMUarEFKStXXF16B91fCow9CfFz061y6DL5ar8Rgg5jzuGHBcTByRLeF31+oyFvBbp+fFxcuY+SWqSztZIPDkGf/iidvC4ZfLQUZVhCsIMMe/R0YJkLaatKDwwlCTW7wD8+k/MTzSProJUxf01gv2x1L0U/u6Owtdoihx0bZYUgSRsGkZaqxQbRznmrdfqjYATFpqolCo+ksDR2wz0QxIAbIzao67BoltJDTaPoaqz4NH6/VnEBACb1XX4eLzo9+nGHAFZfBsy9CcbHaNnYM/L+LuzLafkkg7xOyp4/habGEaVvSuqRLNfXtJxpFXHOEFQzJupl+LzHb1jL4xXs4sPSX7LnxMQaPP5K0F+/GrDpI7ZS5FF1+G3WTZnc6po7giIW5f4Ivfq285GRQZeiGzIY5f2h/A4MdhHW3QfHHasSXtCBuBMz7y8Ac2SWR7Byey67heQQcAVKqkpmSm0VytZ5T3F1Eq2VdUx5ke7YScwwKrQ8eyMJOyCj1GQON2VkT5dqH7+vtMDSatll0phJrbTFporIoacnqz+Av/4DKqvBlVSHUkuwTD0PK4ftGE8j7hHdOP4aPNxzfZQ0OU84dgaukKGx7BLtcALwZmWx7dnuXPHdXEqhRnauBKkg9ApInhu4v+hhynlL2JElZKlOX0my86OaHIPelFpMghCQxE771aM/cQ0/yVdZ69g7ei2XWzy6TYNomx26YS4oWc73CM35VA5w+Sv0NVsVU9sss3Uwz9SspZZuf6A4vsyiNpj9wzNHtK3KPjyBAHvyXMhKuqIxcRC+laoJ48ZXOx6kJoeKEc7Ed4eMgovWAOqrLujegQ8QZB6NOg7gM2PB3eHsJfHodHNwEOU/Duj9BRbbAVybY/wWsvrqpUUJKyFseYZyXFFTuUkutA4lqTw3FzUUcgADLtNgy9vCbVdxXWOqKZ0Kth8RtbhK3uanNTmBdjs3ywtbcrfsvemlVo4mAlCAtgeHohYz1VT9Rkxjq6tTw+kh4PGqE1rr18OHHYBqQNUEJtLay7MGg8pe7bFnXx34Ys+/860lZ8TTUVGJYwaiZOGias9pX2fUibHmkyU/uwFeS0o0gbZDBZnclBZYPNt4t+dbjajk2GHm0LNhQ8N7AqpcrSyxHyMi/5fLEih6ORtOc5uUSDXYmm1Bdr5Hojxm7BrSQ02ia4asxefu+LDa+NwwraDB0fBXfvnobo6aX91wQ6Wnwn3/B628qoeb1QvZ2laWzbSXaFi1UDQ+ff6n2C6H2t1Zb15zDeFm1OUYwQOqbD5D6v0cwAj7KTjyPkvN+jpXQsZ9PzPb1ZF57EiLgR9hWqyJOIShdfHknIu8+gt5QEacQ2H5ARP6QUJkHlh9MFzjiIBjR+UZQsn5glfK4A9EH8jqD+u21L9GaATHAupHKs64/WpvovzSNph4p4d/XzmZ/bjxWQLXm7d2eyFO/OIpl933JsAlVPRdMUiJcdIF6AOw/AKtWq9q5uXOgsAj+eIcScQ3Bt1fEGQacu6R74u4HVOStYJ/lZ2+pzbKbv0fGps8wvSqNlP7sHQx690myH/0aO64Ne5cGbJuxN52Fo6p9S6USgT99FGWLLjrUW+hWKne2YkMSRYcZpnqAqqvb92kUKTuwdByDy1IxbRNLWiG3a1oGYwvH9Fpcmsi02tQUxdqkP2TpdI2cRlNP3voUSvPjGkVcAwG/wcrHM3spqnrShijxdcF5qvP0rXebRFxHWXiiEoOHIQ2WI08tWMawF5yM3/xFo4gDMPw+HKXFDL/zSiZeMpNpi4eQec1C4jaujnrN2G1rMKvbl7GVQOUxp7P9wc+VP2AfxBkPdmufCVpk5YRTMvykJluSCReobtWWGG7JiJO6Ls6+gIHBvI1H4wq4cARNzKCJYRmkl6aTWaCHzfYn5iQ7mFDroTY7gc2fxTc+lhd62VAXYUJOH6JvvpJoNL1AcU4iwUCEzzZSULQtqecDao1gZBNZoGkJNmSbUGmW754NP72se2PrqxRns29mJo9ai1lWlEHGxusw6sInXJh+L4PeexYh1c8wYf1K4q4/ldxbX6Rq7mlhxxu1Ve1qTrHdMez55SOUL1ra+XvpRhJGg+kG29cyqyaJHwv+MrD9EiugLEniR8L0q5uOGjQDhs6HfZ/LxuVZ0y2JHQajF/forfQISTWJnPLFQg4kl+Bz+RlUmUx8XdeOetP0DA0+dQ002Jk0zH2NRm9n7bSQ02jqSRzixeG08QfD35QT0/pYt9PJC2HDxvCsnMcDF54HO3epMV9zj4bkehF61EwYlNKx5wkEYPtOcDkhc5yqxRsgWLEJSIcDEUEUN4i4BgxfLcPvvpptz+SEbHcV7SJ26xcIb7QKf5WFkw4Xu259meq5p3ZJ7N2JvwqsOghfGhX4SiSL/gsH1kDdPkiaqJZSm/9ZCAGzfw9FKyHvfxLLC8NPgtFngCOmx26jRzGkQXpZWm+HoeliGg2IS2HzZ5GPia33rOtNMaeFnEZTz6T5B3jzLhvqQjMRTk+Q4y7K7b3AIvGtBfC/NyBnO3jrPyl6PGos1w+Wds2MzhUfwF331o8asFXd3q2/g4kTmo4pr1ANF9JWorGjQrEHqfDtobBuDNTXp5cvuoj05/4Wlt2M1qjg3puHUVuFHZsAQNpTtzP0338A20ZIGfU8AdhON9LdP4qoq3LBcIEdwcowUAXvfheOuB4yvxf9GsKA4QvVQ6Pp77TmNflMs4xdb4k5LeQ0mnocbpuL71rLszfOpK7KiTAkVsDg+O/nMvn4/b0dXigOB/zjDnjvQ1jxvhJup52i5rB2hYjbvBX+dhf4mi0neL1w7Q3w/JNqNNirb8A/H1RdtBKw74XLLoHzzun883clxdkU+naRM2k8H6bOZ9qWNEgG38iJFF1+OxkP3aDEmB1Emi5E0BfRwkUKgXSqF+qYnHUM/fetYZMcWutWdZYWd/GNdQ+ewZFFnEIQrIH1d0hi02HQtJ6MTKPpeyx1KWuTdaiu1wZ60s5ECzmNphnp46q55vlVFG1LxFvjYPikSjzxrdSj9SYOB5y2SD26mudeiOxhZwXh/Q9g2lQ1E7blMf96AqZNgcn1Vv91Xvj0M2VQfMR0tTzbk9SLuM1jM/k6byLX3HI37oId1EybR8m5V1Py3Z9ROX8xSStfwAj4qMmaQ+YNp4ddRgK1k49GOl3EbVjF2N8sQfjr2h2GsALUZvXsyK1DJS4DkrOgbKsM9YxrhuVX5sDH3KZmqvrKIH7UwF061WhaI5K1SU/amWghp9G0QAgYPrmyt8PoXQoKIxsLe31QVAybNkcWen6/mgE7eRJ8sxFu/J26TjCorgd1qwAAIABJREFU7FGkVAJ0wXy48idqKTZowcFSNakiNrbzsUup4vSoT8M1M7MoeMfglzcvgYAPw7aI3fYlg197iO33rsKbOZ0DF/4SgCH/vRPpcCECoYXNAhDBIJ7czYz71bdDOl1bHhfeIgC1E2fhH9F/nHCP/iN89iuo2CHBimQjIqjKk3xyNZRtrZ+pGoQJ34eJPxhQpZQaTbsIszZpZmcSja4SeVrIaTSacCZlwe494d2vMR74+hvYsTP6CLB1G2DvXiXiaiMInkAAVq6CjZvhgu/B4/9RAjAQUArAsmBCJlzxY5h5RPtjllJNtnjyWaipVqLwzIUw/mQu/MNvMHzNbEYCfmTAz4h//JQd//ykcbtZVRYm4hr31VaQ9tRtCH/0xhfbMBC2pLlhmgBidm1C+H1IV9/0pJISDqyF3W+qyQwZJ8Dx98Gul2DbYxLb30KZCYm/UmXjZLDeLBjY/rTEk6oaGzSaw5k5yQ4o98DXkcXa9tgmz7rOLsEK2dY4nwHC7KyJcu3D9/V2GBpN/2D3HrjsytCsm2GA262ya4GoRVTquNgYJcjqWun2dTqbsnWRcLvhr3+CGdPbF/NTz6qHt0mI2S4HxQuPYtCqzcTUhFuNSMPkm7erGhsR4td9yNhffwezhS2JbRiULLmSxDXv4MnPCbsOgBQGSDtijZwVm0juH1+i+qi+aaS28R7Y/QZYXgCB6ZHEj4R5/4CVl4C3FLCb7sxwSaW5W85UBWKHShY932OhazT9kjXlQbbHeonNCjUghiY7k5lm6ldSyjZrMrQhsEajCWXffvjtLfXrifUvMAKYdaQSXq2JOFBZvDov+KLMiW0gEGjdD8/ng4cfg1258KvfwGlnwdnnw2P/Dl/W9fvh6edDRByA4Q+S9sF6XNEMbhtGm9VTPfNEarNmY7doGBG2TfJHLxKaa2tCCkHFvDOaxhuEPQ/h2c0+QsUO2P16w0gu9fu2vILqPZD/Fiy4HwbPBOGQGE5J3AjJ+PObDIBb4i3pudg1mv7KnGQHS13xYQbEuQdlhw2I9dKqRqNpQkq48WY1Aqy58HC6YNxYWLuufddp77iwtsjZAVf+vCmz5/XBcy/Clm3wt9uajjtQQrT5T4bpwnY4MVskB23DpHrWQqSz2bxMIdj3w98w7lef0rheiJI3ztJinAf3hV1fAlZ8MntufprRt1xA4hdvh/nQYdvUzDiu/ffdgxSvAiuCNrd8gvx3lWib/w8I1KhuVleSsijZ+d/I14vN6N54NZqBREtrk2faYUDcEp2R02g0TezKheK94dkjvx/+9yaMGtn+ayUnNTYcHDJChGXZ8Pth0xbYlt20LSU5qngUAS/7v3ctlisG26FmR9mmAzs2gfzrHww7PvGLtxERlI1AGQU3NDRIw8TyxOHPGMeOuz/Ejo2n8Jp7sBJSsF2e+ucxsd2x5P/yYaS7b7Z0CiN6c0JzPeqMA3eyOjZxHCRPBsMZKp5Nt2TyYTo4RKPpCpa64pmWn8bmz9o/HUQLOY1G00TpQTCjrZl51Xgvtyvy/uZ4PHDVT+Anl8GI4RATo5YwPR71gOjP04Dbrc6JVMdrWUrMNRAbi3/OBCxX6CKDRCBsm6HP/Q0j6AfLUl2lVhARDDDs0ZvDrm+7PNGXSOsRqJq4nX9/l63PbMebOQMAf8Y4tj65lb0/+A1VR53EwdMvIeeBzyg/6YLW77UXGbZAjdoKR1K3t74+LgLH3AYZJyoxZ7gk7hTJjOsgY0E3BqvRHAY0LLu2F720qtFompgwHgJRatuGZ8C8ufCnW+ChRyFvt5r2cOQM+OiTphmvhgEnnQgnfUulb5bUD9isrYX138ATT6mu10gZNCGU0AsG4dST4ZtN6nla4nCoLFwz6i5exL7EdMa99xnC6VIzUJEI28KsqVCXb3a86a0hadVyEj9/k8p5TW2WZSdfSNrzf0dYbfgHmiaB9FFh6SwreTD7f/gb9v/wN62f34tICfs+g7zXIFgDiROhbGNL4xSB5ZNsexyO/EXTVssH+e9C0UfgjIXZtyjfOc8gld3TaDQ9ixZyGo2miZRkOP00eOvd0KkObjdccbn69+xZ6tGcK8vh41VqGXTuHBgzOvzasbFw9Gy4+ZbIhf8xHvjp5ZCSApOzYHAqvPkO3HN/+ExZIWD+vNBtLifLn7iTEf9L4IR1LzP0sf/D9Na0erumt4aMe68jdvPnlJ28FN+YyfjGTKFs4fkMevvfUac0AAQTUwkM7p8FYd/cCfnv0DjUXjgi1xdKS1C8SjYKuWAdfHwF1BY1nCvZ94WyG5l+dQ8Fr9FoQtCfnzQaTShXXwGX/FCZ9ZomjBsDf7gZjj0m+jkpyfCdxXD+dyOLuAakjNaToNI5Y0bB8ccqEQfw7VNUZs7lVMuzsbHKOPivf1KZu6Ji+OfdVP3l1+SU+yksSKVmyEisxNR2u9K6C7eT9swdZF0+myHP/hWA2qnHIF2R/Z8kYLtjKbjun/3S+bZiB+S/3STigKgTHECZ/Taw62WoKWx+rsDyCna/DpW7uilgjUbTKjojp9FoQjEMJcjO/+6hnR+01PzXN99WS6QnL4Qzv62yek6nGuG1cXN47ZugabRX4zYB1/4MLvyeWmZNiFfZQKcTXn0ded8DYNvEWzYzPtzKH2ZsJ/fPr1I7YWa77T4EqmYOK8iwx35P5XHfoW7cdGSUOjkrLonc2//XZ7tQ22LfZ2BHXDUOF3OGSzLq203fF6wAO4J3nB2E4tWqCUKj0fQsOiOn0Wi6DtuGm26Gu+9TYm1rtvKCu/LaJl+5669RmTVnfapHCHC54Bc/b9rWkqHpcMpJqkbP6YR9+5H3PYAIBBGW6iR119USv+EjUt/4F97xR1A79ZjG7tEG2rQ/t4Ikf/A8tVPn4Rs9GdsZ2thhu2PJ++NL/VbEQb3/W7RXfiER9Z2ohlsSlwHjlzbb3co7Rj9MTmo0AwIt5DQaTdex5is1h7W5ZYjPBwUFsOI99f3oUfCfR1QTREJCUyLozvvg7Xfb9TR1rzxKpKk0preW1NceBmDXba9x8LSLsV0xSMPEO3w8+y/4BZWzF2HFRO4IE7alRnAJwY5/rKDi+LOxnS5spwt/2ijybn6K6lkL2/3j6ItknBhZdBluycQfQvq8ppq52r2wcplajgUYdZqyGGmJMFX3q0aj6Xn6zNKqEGIQ8DwwBsgDvielLItwnAVsrP92j5TyrJ6KUaPRtMHHn0Qey+X1wfsr4czT1feDBsHXG6CuDmypvOH8fiXmEhNbrccL5H1CyaB4MuwoBsC+OgCkJ5aC6x+g4Np/IgK+EB+3tKduJ/3ff8BsMTfVdsdSeazqsrXjk9j9+2cRvjoMby1W4qABkXaKy4BJyyD7CamMgG2BGSNJngBDZsLO51TNnKxffq0tlqy+Bha9AGO+A4UroXKXxKoTYEhMJ4y/EBJG9eZdaTSHL30pI3cj8L6UcgLwfv33kaiTUh5Z/9AiTtOnqS13crAwBruLBh30eRq836Lta2DzFigoDB/R5fPBY/9p82lyz1mE5Qw3G7adbspPbFHbZxhhZrwl51xJIH0UdrPtlieOymMXUztlbsix0h2DldT+5on+wISlsOAByDwPRp8pmf07mH8X7PivshcJRWAFoOgDMN1w3D0w6yYYfrJkzGKYf7cShhqNpnfoMxk54DvAifX//jewEriht4LRaDpDdamLl/80nT0bUzBMG6fb5rSrtjF90d7eDq17OeUkeOPtUOsSUB2mZzSrmt9TEP0ahYWtPkUtdVTOGM/G487iyNWvN1qM2C4PwZQ0Dnzv2jbDtGMTyHl4DamvPUzyh//FjomjdPHllH/rewNKsEVCWuCvgvjRMO2K0H01BRCp6cH2CSp21tfOOSDjBPXQaDS9T18SculSymIAKWWxECItynEeIcRaIAjcLqVc3mMRajTtQNrwxM/ncLAoBmkZWAGDgBf+97epxKb4yZx9sLdD7D4mZcG5S+Cl5RAIqOYHjweOnw/HNfN9G5ERSS8ohg6NvL04mwrfHnb7/HxZkcHIq/5N6vyXGbz8AYzaSioWnEvJOVdhJSRHPr8FdmwCBy64ngMXXN+xe+ynSAm5L8G2J8Dyqrq2MWfBlB83TXZImgDVe1oaAwNIyrPRaDR9kB4VckKI94BIr9IdsUAfJaUsEkKMAz4QQmyUUu6M8nyXA5cDjEqPpgs1mq5l17pBVJW4kVboEmPAZ7Ly8cyBLeQALr9ETXb4YKWyIlkwH6ZMDs10TZ8GaWlqebX5hAePG5b9IPyaxdkU+naxeWwmbySczbKiDBgE5Sdd0KfHX/Ulcl+BLY8084ALQN6rEn8VzKovZBlyNBS+H+lsQXm2RMoBn7DUaPodPSrkpJQnR9snhNgnhBhWn40bBuyPco2i+q+7hBArgZlARCEnpXwYeBhgdtbENp0HNJquoHRPHLYV+d2uND+uzfMHxJtl5jj1iIYQcOdf4Nbb1MxUhwMMAZddAgtCrT0CeZ+Qa1Xw8fxTyMlfwLKq9s8g1CikDdseDzUBBrB8gsL3JVN/DO4UiEkF06MydmHXsFD+Lf39b1OjGWD0paXV14CLgdvrv77a8gAhRApQK6X0CSEGA/OBv/RolBpNG6SOrMUwI39uGDSiNuJ2acPqZ8fw6X/HUFfhInlYLSdfvp2p39rXnaH2LoNSlJgrPQhVVWqWawsfuYq8Feyz/I0iriODpDVNBGsgGPlPD8MJ1flKyA2aFuUCQjJ4pp6lqtH0RfrSf8vbgUVCiO3AovrvEULMFkL8q/6YycBaIcQG4ENUjdyWXolWo4nCuKNKiR/kQ5ihkwWcbosTfxQxeczb92Xx8ZOZ1FUoA9ry4lhevX0a36yIUi82kEgdpMZ6tTQDLs6mmiBPLViGzD5Fi7hO4IgNHbXVHDsAsfV/Zo4YNTPVdEsQ6sOIcEgcsTD9mh4KVqPRdIg+k5GTUpYCJ0XYvha4tP7fnwLTezg0jaZDCAN+dPdaXrp1OgVbkjAcEofD5pQrsxl/dGnY8bUVTr763wisQOhIqIDP5L2HJjL95L39f6n1EKmZmUVF1RDOSe4zL1X9EmHCuHNg14sSq9mILeGUDD4SYpqVEI8+AxLGwM4XlIdc6gxlUxKjy4w1mj6JfnXUaLqBhME+fnT3WqoPuvDVOEgZVofhiLzcuj83HofLDhNyADVlLvx1Ju7Yw8WIromA7wDlwXRdk9VFTLoEAtWw5y2J4VSZuMGzYPbvwo8dNFU9NBpN36cvLa1qNAOO+EF+VTMXRcQBJA72YgUi/1c0HRKn+/ATcRV5K9jqq+D1xPlMy9epoK7AcMAR18GpL0HWD+utRgrgqz9B0ar6ZgaNRtPv0Bk5jaaXGTSijqHjqyjKTsRuZlnicFnMOjMfIzxRN3CptxnJmTSe5a4lTNuSxhy9rNohgl4oXgXeA5A0EYbMCm1SyH0Ftj/T1MFaWyDZtxqciXD0rTD4yF4KXKPRHBL6FVKj6QNc8Kf1PHPjTA7sjsMwJVbAYMK8A5z84+29HVqPEcj7hP1UKa841xKWVWVA+7x9NfVUbIfV14IdBNsPhgvihsNxd4MzHnzlkPMk2IHm69Xq34FK+PwGycnPgCe1d+LXaDQdRws5jaYPEJfi57KHvmDfrngq9npIG1dN8tAIZl4DFO0V13mkDZ/fBIGqJpFm1UHVbsnG+5Tpb+l6Guvjol1j9xtq6VWj0fQPdI2cRtOHSB9XzcRjSw4rEQdqfuryMy/RNiOdoGwrBKvDt8uAoPA9ZTRtelq/hu0XVOd3T3wajaZ70EJOo9FoBgDBWqK+ottBwFZdqq296pseqbtVNZp+hhZyGo2mVwnkfcI+y091pZ6i1xlSJtcLtggkT1RecqYL5v4RDI9Ezdtq+pkLQ+KIgRGn9Ei4Go2mi9BCTqPR9BoVeSsaa+O8eZm6Q7UTOONh0sUqq9aIkJgeGTKVYfBMOPUFmPAD8AwBDIkwJYNnwYIHwRnb46FrNJpOoF81NRpNr1CRt4LdPj8vLlzGyC1TWapFXKeZcBHEj4HtT0m8JZA8CSb9CBIzQ49zJcKUS9XD8gFCZes0Gk3/Q79yajSanqXeK27z2EzeSDibZUXaZqQrGTZfPdqL6e6+WDQaTfejhZxGo+kx+pPNSLAW8t+D0g0Qmw6jF0PcsN6OCqrzoTxbLYumTg81+9VoNIcfWshpNP2QoF8Q9Jm444OIfjaLNGfxqeRsOL5P24x4S+Gjy9VsUssrEA7Jrhdh9u9haAeyXV2JHYC1t8C+L9S4LSnBnQTH3glxGb0Tk0aj6X20kNNo+hHeagdv3DmZrR+nIyUkDPZx+tVbmXhsSW+H1mVYfihZpzJig2eCO6XnY9h4L/jKQFpKJcugwArCV3+UnLa8d5Yj1/8NilcDtsD2q221dZKV/w+O+h2kz9XZOY3mcEQLOY2mnyAlPPmLo9i3MwEroN6xK/bG8MItR3DR7esYM7OslyNsnYYRXHsPRrcZObAOvvwtja4YdhDGXwiTL+mZGBvY+0mTiGtJ6TeQNqf18/2VkL8CagohJQsyTuyc+CvdAPlvQ8M4rSYEwVpY83vJ4Bkw93aVrdNoNIcP+vObRtNPKNicxIG8uEYR10DQZ/LhY+N7Kar20VAb99YxJ5OTvyDisqq/Er64CYI1gmCteth+wc7n6zNRPYi0o+wQIK3Wzz24BVacD1segtyXBBvuhPcugroDhx7PN/fUP3kUbJ+gdCMUvHfoz6HRaPonWshpNP2EfTsTkDLym/n+3L5bb1aRt4KtvgqeWrCs1RFchR9EPt/yKjHXk6QdDRjhmUNpQeoR0c+TNqy5GSVCfep3ZdUJvKWw4W+HFosdhMqdbR9neQW73zi059BoNP0XnYTXaPoJSWl12FGyQYlpfXA2a73NSM6k8Sx3LWHalrRWDX99ZWBFuQ1vaTfFGIXpV8PBH4Plldh+AYbEdMKM68ARE/28iu0QqImwwxbsXyOx/B33axOmWi6NNui+OTLKZAeNRjNw0UJOo+knZH86BDsYnkQ3HBYLfrCrFyKKTkM93OaxmbzhWsKyqra94lKmgBkDVl3odmFKBh/ZfbFGIi4DTnoS8l6FA+skscMg81xImtD6eZaf6F3EspUl21YQAkYsgoIVEjsQfXnVdEtGnNrx62s0mv6NFnIaTT+gpszF+neGE6lOyh1rMfVb+zp1/bpKB9+8N4yKvR4yJlUy+fj9mM5Dm30aySvO8sO2RyHvdQjWqNmf064MXaZMmwPxI6Eqt5lgERLTDRO+36nbOyTcyZB1sXq0l+Ss6PsSx4O3BGqLIWE0xKS1/7rTfgaVu6Bqt1SZOdlQq6d+TqZHEj8KRn+7/dfUaDQDAy3kNJp+wN4dCTicNpbfDNtXV+XECgpMx6EJr/xNSTz1y6OwbeVN54oJ8v4jE7j0/i+JS/F36FrRxm59+VsoWU9j3Vh5Nnz2S8mxd8KgqepcYcBxd8PWRyH/bbUMOWQWTP1p3zDibQ+mC2ZcCxv+puJHCoQpEU5AwoeXNC2TDjsOZt7UvqVWZ6yag1r6DVTk1M9Ipf7n5IMRJ6uHntKg0Rx+aCGn0fQD4lN92FHsMFweC8NUIq60IJbVz4yhYEsyKcNqOfbCPEbPKI96XduC528+En9d00uBv85B0G/w5t2TOOParax7fTgFW5MYPKqG2WcVkDy09Xq8t5dcwsjPpzTWw1XuhNJmIq4ByyfY+ohk/l1N2xyxMP1n6tGfsHxQ8rX6mj4Xjr0Ldj4H1YWSlClQtQvKskEGBLZPnVO8WuJ6AGZc0/q1GxACBh+hHg0MP7HLb0Wj0fQztJDTaPoB6eOqGTS8lv258Ui7qU7O4baYfVY+QkDRtkSeuHY2QZ+BtA0O5MWRuy6Vb1+9lZlnFEW8buHWJAK+8CyfbRlsW5XGzrWpWAGDoM9kh8Piy5dHceGf1zN21sF2x162jajOGeXZ7b5MrxCsVdMdPKmq6SAS+9fAmt8T4n036Ucw5w/q+7r9yn5Etqhvs32CPW9Ipl0BhrPbbkGj0QxwtP2IRtNPWHrb1wweWYvTE8QdG8Dhsph4zAEWXroDgDfumkygztFM6AkCPpO375tEwBf5v3rQbyKIvCRrWwJftUmwXuhZQZOA18FLt06PWLRfkbeCfZaf6srQ63lSo08ccLXRANFbBGth7R/grbPg/e/DW9+B3FfDj/OVqWXjlt532f9RAg+Uf1w0oWZb6rk0Go3mUNEZOY2mn5CY5uOnT3xKUXYilQc8DM2sIiVDtXgG/YLinMSI5wlDUrg1iTFHhk9+GDGlHNuOlC5rEGPhCizgNdm7I4FhE6sat7WsjWtuMzJkNpgeCNZJaOaDZ3ok489vx433Al/8Bg5upLHpwvLB5vsljjgYeXLTcfkrIneiWl7BjucknlRweFQ3aySkpSZ2aDQazaGiM3IaTT9CCBhe31XaIOIADAOEiKwIpC1weiIb0Dk9NqddmY3TbdEg3gzTxumxEBEMcZtfE1BecXlv8cXY0Tw651ecU3REmFec4VCD3WOGgCNG4oiVGE7JqNNhzHc6cPM9RFUeHNxMmNWH5RVsezT0WF8JymcuAgfWwcdXwEc/aWhoCP95CkNZnGg0Gs2hojNyGs0AwHBIJs47QM5nQ7Ct0M9nrtggGRMro547a3EhqaNr+PS5MZQXxzByWjnzL8zj5T9Op2BrUkgWDcDhshk6oTKizUg0EsfAoueVQPJXQMpkteTakkAtlG8DZ7zybIvqydaNVO0Bw4RIlm+1e9XXQBXseQtKNwGmhLBGFAm2aPTEs/2SSIWC0hLs/VR2yOJEo9FomqOFnEYzQDjjuq3svSKB2goX/joHTncQYcIFf1wftUatgdEzyhk9Y33ItsW/3MxjVx1NMGBg+U0Mh41pSs7+zUaq8lU9XIOIizZ2qznCgNTp0fdvfw6yHwPhUMuV7mSYexskjm3P3XcdccOjG/d6UqGmCD7+iZpCYfkEKtPWXKhFEm0iynZwRV4R12g0mnahhZxGM0CIH+TnqidXs3VVGsU5iSQP9TL9pGI8CYc2tyltbA1XPbmata+NoHCLsh+ZsyQfM/haRK+4zlC8GrIfrxdG9fYctXWS1dfAKS+E+qNJC3a8ALtehEAlJGXB1B/DoGmdDgOApExIGAsV2yUyGFrTl/ld+PJ34K+kWaayScAJp8oi2u203zM9krHndE3cGo3m8ETIw6TSdnbWRLn24ft6OwyNpv9SPzt189hM3kg4W43d6iI+/imUbQnPVjliJEf+CoYvbNq27jYo+rAhG6Yw3ZJj/9F1Ys5fobpWSzeojlPbUsvBZVsaRFqEWGMl8/4Gmx+Eg9+EH2O4JNJWNYOyPok39mxleNwbS8gajaZvM/ME4ysp5ey2jtMZOY1G0y4CvgN8ffoiVhQc0aUiDqAuyoSxoE/5sDVQWwyFH4Q3GFg+wab7JQvu75p4XElw7N/BW6osRsq3w8a7ojc2NFCwQom9lghTdbAe/0/Y+6ma7JA+Vy3jajQaTWfQQk6j0XQId2kitDJWSlpQXahsN9o7TzQpC7yfRqghkxDbTOyUbVU1dERYuqzIad9zdQRPqnqs+5PqWm0NKWHPm4Qsx9bvIWkCzP2zutaYxV0fp0ajOXzRQk6j0bRNcTb7qaK8lelctftg22NQ+CFqqLtUjQqzfw/xI1q/fNaPYN+nEXZIKP4IMo5X37qSow6JwNl2v0UIgVqo3A7OJNVV2xre0lZ2ConpUsu6B76KtB8Sx0Xu0tVoNJrOooWcRqNplUDeJ+ynis1jM1lTcgTLInSo7nkLNvxdLRk2l1oVOZJPrlLWI4YL8t+BHc8qYZScBZMvVbVnSDDcNM4hbUJQ9KFk1q9V1+vgI9Q81mBtaPbOcEvGnh0eu+WHgveg8H31/KPPhKHHQs5TkPNkfb2aBbHDVMYsLsqKcXIW7P8yQsbQkAydD1MuhS2PAJHMlaXAV3Z41CJrNJqeRws5jUYTlfZ4xXlLYcM/wg10FYKgV1L0EVTlwq6Xm5YoD6yVHNwI8/6qmgkMRyQhp/ZJSwk5YcK8v8Gn14HllUip9g2dBxMuCj3P8sEnP4Oq3U3PWbJOLXOW56hZpw3PV5WnOmQXPRd5puqk/6caH6xm8RlOSXIWHH2ralYYeqy6p5ZLsKZHkn5s1B+xRqPRdAot5DQaTURajt2KZjNStLL161h1gvLtkrxXWoo9geWDjfdK5t8DMopLSmJm6KzSxLFw6otqcoLvIKRMgfiR4eftfitUxIH698FNMszkGCkIVEsOfAVpR4dfK2WSEpAb75FU7FCTGkaeFtpxOuJklW2s3Ssb79NwSjyDYeQprf+MNBqN5lDRQk6j0YRSbzOSM2k8H6bOD5ud2hLLp7Ji0TBjJIahljbV0msoFTtUY0TWJZD9eLOMVn3t2Yyfh58jTEib0/ptFLwbpUEhyiqntJomN0QidQac+K96s2ARbhliumHBg2rZtuA99STDT4Ks76v702g0mu5ACzmNRtNEc6841xKWFWVAcuunpM2B7CfAiijmJI4YSJ8HuVFmipputWw64QJIGAk5T0vq9qu6tEk/UqO6DgWjo69uQmX/2jyslSkZzniY+hP10Gg0mp5ACzmNRgO0rx4uEkkTIONEKPqoeX2YykglZcGcWyA2XY2iqvOGLmsaLsmobzdda+h89egKRp0O5dnhNWuGEyQS2WyZ13BKEsepZVqNRqPpT2ghp9FoAKiljuVnXkLK51M6PHZr5o0qM7druSRYDYNnqqkFCaObjjnmL7D6GjVA3rbU0mTyJJjSTdmrESdD4XtQukli1YnGpdoxS2DILPjmbkndPpVhG34STL9GT1jQaDT9Dy3kNBpNpxEGjFikHtFIHAOnvgT7vgDvAbV0mjK5+2IyHEo87v8SCj+UmB7VdDBoqtq/6BgI1qravQ74ETNtAAAPC0lEQVQvw2o0Gk0fQb98aTQaAnmfsM/yU10pSenG5zEcMKyLlk7bgzAg/Rj1iIQjtudi0Wg0mu5ACzmN5jCnIm8F+yw/H88/BW9eZqsdqhqNRqPpW7TSf6XRaAY6DV5xTy1Yhsw+haURpjZoNBqNpu+iP3prNIcjzW1GEs5ul82IRqPRaPoeWshpNIcZzW1GNtYeybKqKANGNRqNRtPn0UJOozmMCPOK00upGo1G06/RQk6jOUxo7+xUjUaj0fQf+kyzgxDiPCHEZiGELYSY3cpxpwkhsoUQO4QQN/ZkjBpNfyWQ9wnZ/7+9+42t667vOP75qMyp1CUbNHVKwCUlDS1NUJKpZKuiVZC1Kc1om3RUKnkwcGEZk9AeTBNCq7RNfcQeTdvEtmaAYOuAokpJMxpaXChkAQHJ2pK0Dg5puaSurTjJiOvi2HXMdw/u9TiJfe3r+vr8fb+kyvfPyfW3Px37fHzO73x/71mlz7/3U7pnYD13pgJASeQmyEl6XtI9kg4028D2ZZI+K+kOSTdK+rBtFtUBWjDUda2WnF2WdRkAgDbKzZ/lEXFMkjz7GjmbJJ2IiJca235V0t2Sehe9QKDARnVe58ayrgIA0G65CXItepuklxPP+yX9bka1APnXaDNy/IbrdHx8pdaMXi51ZF0UAKBdUg1ytp+SdPUMbz0QEY+18hEzvBazfL9dknZJ0jUrOluqESiNZK+4ju30igOAEko1yEXErQv8iH5JXYnnb5c0MMv32y1ptyTddP27mgY+oGymtRkZoc0IAJRRnm52aMUhSWtsX2u7Q9J9kvZlXBOQK8O1Hh0bH2bZLQCogNwEOds7bPdLulnS47afbLy+0vZ+SYqIC5I+KelJScckfS0iXsiqZiB3Bvt0auNqPbH9fnX1rqXNCACUXG5+y0fEHkl7Znh9QNK2xPP9kvanWBpQOK+8slxbCXEAUHq5OSMHYOGGx0/q3IXJrMsAAKSEIAeUxNQSXF9ftlnrXuYubQCoAq69AEWXbDOydAdtRgCgQghyQIEl24wcHd2g7pGVWZcEAEgRQQ4oqGm94mgzAgCVQ5ADCmhqPtyjW7rV1btWO7lDFQAqid/+QJEwHw4AkECQAwpkYvy0nt12m3r61zMfDgBA+xGgiJacXZZ1CQCAHCDIAUUx2KchjejcWNaFAADygiAHFEFibtyhM+u1kztUAQBijhyQe9PajIwQ4gAAdQQ5IMeSbUbGaqs5EwcAuAhBDsijwT4Nj5/UMzes0tNXblZX71q9l15xAIBLcGQAcurUxtXaO3knveIAAE1xswMAAEBBEeSAHBoeP6lXzk9mXQYAIOe4tArkSXIJro7trN4AAJgVQQ7IiWSbkaOjGwhxAIA5EeSAHJjWK442IwCAFhDkgIwle8V19a7VTtqMAABaxBEDyEpyPtzSHbQZAQDMG0EOyADz4QAA7UCQA1LGfDgAQLsQ5IAUMR8OANBOHEWAlEzUDqrvPav0n96udb2drJ0KAFgwVnYAUjTUda2WnF1GiAMAtAVBDkjJqM7r3FjWVQAAyoTTAsBiG+zT8PhJPXPDKh0fX6k1o5dLHVkXBQAoA4IcsJguXTuVXnEAgDYiyAGLZFqbkRHajAAA2osgByyCZJuRsdpq7aRXHABgERDkgHZqzIdL9orjDlUAwGLhCAO0C2unAgBSRpAD2qER4p7ecpuO9P0+8+EAAKmgjxzQJr/ceL1OaEO9vQgAACkgyAFt9IszkXUJAIAKIcgBCzRRO6jjoyd0YEmXJHFzAwAgNRxxgAWY1iuONiMAgBQR5IA3KNkrrqt3rXZyJg4AkDKOPMB8XdJmZF1vJ5dTAQCZ4OgDzMNE7aCGNPLrXnEj9IoDAGSHIAe0iLVTAQB5w12rQAumQtzDt3Qr+raydioAIBcIckCLjt95u8Zqq5kPBwDIDYIcAABAQXFqAZjDVJuRHw2vzLoUAAAuQpADmhns0/D4yYt6xXFZFQCQJ7k5Ktm+V9LfSnq3pE0RcbjJdjVJI5ImJV2IiJvSqhEVckmvuO4B2owAAPInN0FO0vOS7pH0UAvbvj8izixyPago2owAAIoiN0EuIo5Jku2sS0GFDdd6dGrydT18S7fGaqtpMwIAyLXcBLl5CEnftB2SHoqI3VkXhBJgPhwAoIBSPVLZfkrS1TO89UBEPNbix2yOiAHbnZJ6bP8kIg40+X67JO2SpGtWdL6hmlEBzIcDABRUqkEuIm5tw2cMNL4O2d4jaZOkGYNc42zdbkm66fp3xUK/N8qH+XAAgCIrVENg21fYXjr1WNJW1W+SAOZtuNZzUYhjPhwAoGhyE+Rs77DdL+lmSY/bfrLx+krb+xubrZB00PaPJf1I0uMR8UQ2FaMM9n7wftZOBQAUVm5mc0fEHkl7Znh9QNK2xuOXJK1PuTQAAIBcys0ZOSBNE7WDOjX5ul57lamTAIDiys0ZOSAtU2un0mYEAFB0HMFQHY02I8dvuE57O7ZrXW8nIQ4AUGgcxVAJE7WDGtJIvVdcx3Z1j9ArDgBQfAQ5lB694gAAZUWQQ6ldOh9uJ5dSAQAlwlEN5ZSYD/f0lZu5qQEAUEoc2VA+ybVTO7azdioAoLQIcigV5sMBAKqEhsAoj8E+DWlED9/SzbJbAIBKIMihVH658XoNj1zFfDgAQCUQ5FAaE+Onde7CZNZlAACQGoIcSmG41qOfTQ7ryBWrtOTssqzLAQAgFQQ5FN5UrzjmxgEAqoaJRCiuZJuRpTtoMwIAqByCHAop2Wbk6OiG+tqpAABUDEEOhTOtVxyXUgEAFUWQQ6GwdioAAL/GURDFwHw4AACmIcgh95gPBwDAzAhyyDXmwwEA0BxBDrnFfDgAAGbHkRH5c8l8uHW9naydCgDADDg6Incmxk/r2W23qad/fX0+HDc1AAAwI5boQm6xZioAALMjyCFfBvs0pBGdG8u6EAAA8o8gh/xIzI07dGa9dnKHKgAAs2KOHHJhWpuREUIcAABzIcghc8k2I2O11ZyJAwCgRQQ5ZGewT8PjJy/qFUebEQAAWsdRE9lg7VQAABaMIIfUMR8OAID2IMghdaM6r70fvF9njryT+XAAACwA7UeQmTWjl2ddAgAAhUaQQ6omagd1avJ1vfZqZF0KAACFR5BDapJz48Zqq7lDFQCABSLIIRVTIe7hW7oVfVuZGwcAQBs4ohqXuGyflvTzrOvIyHJJZ7IuIqcYm+YYm+YYm+YYm5kxLs0xNjN7R0RcNddGlQlyVWb7cETclHUdecTYNMfYNMfYNMfYzIxxaY6xWRgurQIAABQUQQ4AAKCgCHLVsDvrAnKMsWmOsWmOsWmOsZkZ49IcY7MAzJEDAAAoKM7IAQAAFBRBroRs32v7Bdu/st30TiDbNdtHbT9n+3CaNWZlHmPzAdt9tk/Y/nSaNWbF9lts99j+aePrm5tsN9nYZ56zvS/tOtMy1z5ge4ntRxrv/9D2qvSrzEYLY/NR26cT+8nHs6gzC7a/YHvI9vNN3rftf2yM3RHbv5N2jVloYVzeZ3s4sc/8ddo1FhVBrpyel3SPpAMtbPv+iNhQoVu/5xwb25dJ+qykOyTdKOnDtm9Mp7xMfVrStyJijaRvNZ7P5Hxjn9kQEXelV156WtwHPibpFxFxnaS/l/R36VaZjXn8fDyS2E8+l2qR2fqipA/M8v4dktY0/tsl6V9SqCkPvqjZx0WS/juxzzyYQk2lQJAroYg4FhF9WdeRRy2OzSZJJyLipYh4XdJXJd29+NVl7m5JX2o8/pKk7RnWkrVW9oHkeD0q6Q9sO8Uas1LVn4+WRMQBSf87yyZ3S/r3qPuBpN+2/dZ0qstOC+OCN4ggV20h6Zu2/8f2rqyLyZG3SXo58by/8VrZrYiIQUlqfO1sst3ltg/b/oHtsoa9VvaB/98mIi5IGpZ0ZSrVZavVn48/alw6fNR2VzqlFUJVf7+04mbbP7b9Ddtrsy6mKFi1vKBsPyXp6hneeiAiHmvxYzZHxIDtTkk9tn/S+Kup0NowNjOdVSnF7d2zjc08Puaaxn7zTknftn00Il5sT4W50co+UNr9ZA6t/H//l6SvRMS47U+ofuZyy6JXVgxV3W/m8ozqS1K9ZnubpL2qX37GHAhyBRURt7bhMwYaX4ds71H9kknhg1wbxqZfUvIMwtslDSzwM3NhtrGxfcr2WyNisHGpZ6jJZ0ztNy/Z/o6kjZLKFuRa2Qemtum3/SZJv6VqXDqac2wi4mzi6b+pIvMHW1Ta3y8LERGvJh7vt/3PtpdHBGuwzoFLqxVl+wrbS6ceS9qq+o0AkA5JWmP7Wtsdku6TVNq7MxP2SfpI4/FHJE07e2n7zbaXNB4vl7RZUm9qFaanlX0gOV4fkvTtqEZjzjnH5pI5X3dJOpZifXm3T9IfN+5e/T1Jw1NTGqrM9tVTc0xtb1I9n5yd/V9B4oxcKdneIemfJF0l6XHbz0XE7bZXSvpcRGyTtELSnsbPzZskfTkinsis6JS0MjYRccH2JyU9KekySV+IiBcyLDstn5H0Ndsfk3RS0r2S1GjT8omI+Likd0t6yPavVP9F+5mIKF2Qa7YP2H5Q0uGI2Cfp85L+w/YJ1c/E3ZddxelpcWz+3PZdki6oPjYfzazglNn+iqT3SVpuu1/S30j6DUmKiH+VtF/SNkknJI1K6s6m0nS1MC4fkvRnti9IOi/pvor8YbRgrOwAAABQUFxaBQAAKCiCHAAAQEER5AAAAAqKIAcAAFBQBDkAAICCIsgBAAAUFEEOAACgoAhyAAAABUWQA4B5sH2v7XHb70i89g+2X7S9IsvaAFQPKzsAwDw01oM8JOnZiPgT238p6VOSNkfET7OtDkDVsNYqAMxDRITtv1J9rd4XJT0gaQshDkAWOCMHAG+A7e9L2iTpzoj4Rtb1AKgm5sgBwDzZ3iJpvSRLOpVxOQAqjDNyADAPttdL+q6kv5D0h5J+MyJuz7YqAFVFkAOAFjXuVP2+pIci4kHb6yQdUX2O3HcyLQ5AJRHkAKAFtt8i6XuSDkTEnyZef0TSNRFxc2bFAagsghwAAEBBcbMDAABAQRHkAAAACoogBwAAUFAEOQAAgIIiyAEAABQUQQ4AAKCgCHIAAAAFRZADAAAoKIIcAABAQf0fVCWri4qr8e4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "h = 0.02\n", "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", "\n", "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n", " np.arange(y_min, y_max, h))\n", "grid_tensor = torch.FloatTensor(np.c_[xx.ravel(), yy.ravel()])\n", "\n", "Z = two_layer_net(torch.autograd.Variable(grid_tensor))\n", "Z = Z.data.numpy()\n", "Z = np.argmax(Z, axis=1)\n", "Z = Z.reshape(xx.shape)\n", "\n", "plt.figure(figsize=(10, 8))\n", "\n", "plt.contourf(xx, yy, Z, cmap=plt.cm.rainbow, alpha=0.3)\n", "plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.rainbow)\n", "\n", "plt.xlim(xx.min(), xx.max())\n", "plt.ylim(yy.min(), yy.max())\n", "\n", "plt.title('Спираль', fontsize=15)\n", "plt.xlabel('$x$', fontsize=14)\n", "plt.ylabel('$y$', fontsize=14)\n", "plt.show();" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "G2wNgoh5YJeG" }, "source": [ "Подберём гиперпараметры получше:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "X = torch.autograd.Variable(torch.FloatTensor(X))\n", "y = torch.autograd.Variable(torch.LongTensor(y.astype(np.int64)))" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "c:\\users\\izakharkin\\anaconda3\\envs\\vision\\lib\\site-packages\\torch\\nn\\functional.py:52: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.\n", " warnings.warn(warning.format(ret))\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "0 329.6793212890625\n", "1 327.8143615722656\n", "2 326.1004943847656\n", "3 324.5147399902344\n", "4 322.9456787109375\n", "5 321.3118896484375\n", "6 319.5968933105469\n", "7 317.75592041015625\n", "8 315.75390625\n", "9 313.5726318359375\n", "10 311.1824645996094\n", "11 308.5584411621094\n", "12 305.68798828125\n", "13 302.5540771484375\n", "14 299.15460205078125\n", "15 295.4794006347656\n", "16 291.5202331542969\n", "17 287.2996826171875\n", "18 282.85137939453125\n", "19 278.21435546875\n", "20 273.4761657714844\n", "21 268.6789245605469\n", "22 263.89453125\n", "23 259.163818359375\n", "24 254.56350708007812\n", "25 250.1515655517578\n", "26 245.91949462890625\n", "27 241.89158630371094\n", "28 238.07766723632812\n", "29 234.496826171875\n", "30 231.16111755371094\n", "31 228.04730224609375\n", "32 225.1465301513672\n", "33 222.45156860351562\n", "34 219.9502716064453\n", "35 217.6254119873047\n", "36 215.45823669433594\n", "37 213.44068908691406\n", "38 211.57652282714844\n", "39 209.91677856445312\n", "40 208.7138671875\n", "41 208.98919677734375\n", "42 214.6230010986328\n", "43 240.55369567871094\n", "44 314.333984375\n", "45 303.1317443847656\n", "46 260.13043212890625\n", "47 220.28932189941406\n", "48 214.5643768310547\n", "49 212.7597198486328\n", "50 213.45379638671875\n", "51 222.7734375\n", "52 235.19393920898438\n", "53 258.9502868652344\n", "54 235.3881072998047\n", "55 222.33709716796875\n", "56 205.35061645507812\n", "57 201.09136962890625\n", "58 197.60365295410156\n", "59 196.84959411621094\n", "60 196.8997802734375\n", "61 203.0827178955078\n", "62 210.37013244628906\n", "63 233.3351287841797\n", "64 223.5889129638672\n", "65 227.52284240722656\n", "66 199.73655700683594\n", "67 195.20997619628906\n", "68 192.6437225341797\n", "69 197.1069793701172\n", "70 223.88955688476562\n", "71 250.8140106201172\n", "72 302.1340026855469\n", "73 234.83021545410156\n", "74 192.74407958984375\n", "75 184.17343139648438\n", "76 178.7435760498047\n", "77 175.32286071777344\n", "78 173.30526733398438\n", "79 173.79928588867188\n", "80 178.83067321777344\n", "81 195.47805786132812\n", "82 224.01499938964844\n", "83 254.59860229492188\n", "84 197.6997528076172\n", "85 184.15213012695312\n", "86 178.74087524414062\n", "87 177.8366241455078\n", "88 196.80772399902344\n", "89 216.80592346191406\n", "90 267.1333312988281\n", "91 198.99403381347656\n", "92 174.82217407226562\n", "93 172.68698120117188\n", "94 166.46255493164062\n", "95 171.16470336914062\n", "96 170.53395080566406\n", "97 183.16978454589844\n", "98 180.73060607910156\n", "99 191.34986877441406\n", "100 179.35696411132812\n", "101 175.2283172607422\n", "102 172.541748046875\n", "103 171.90914916992188\n", "104 184.98558044433594\n", "105 205.13856506347656\n", "106 205.8452911376953\n", "107 224.54994201660156\n", "108 158.3968505859375\n", "109 156.24256896972656\n", "110 147.74032592773438\n", "111 150.6797637939453\n", "112 150.395751953125\n", "113 159.10755920410156\n", "114 170.22654724121094\n", "115 180.6762237548828\n", "116 191.1690216064453\n", "117 163.69064331054688\n", "118 163.85519409179688\n", "119 139.80825805664062\n", "120 140.6570587158203\n", "121 144.08004760742188\n", "122 158.80630493164062\n", "123 188.1263885498047\n", "124 195.49148559570312\n", "125 192.2193603515625\n", "126 153.9271240234375\n", "127 132.1674041748047\n", "128 124.47053527832031\n", "129 116.3742904663086\n", "130 114.36947631835938\n", "131 110.64471435546875\n", "132 111.86954498291016\n", "133 110.50808715820312\n", "134 116.09949493408203\n", "135 116.46121978759766\n", "136 128.56101989746094\n", "137 129.96461486816406\n", "138 145.54727172851562\n", "139 147.5155487060547\n", "140 151.8369140625\n", "141 163.9568634033203\n", "142 143.47850036621094\n", "143 166.7104949951172\n", "144 153.11297607421875\n", "145 158.97427368164062\n", "146 162.9993896484375\n", "147 142.5881805419922\n", "148 129.16439819335938\n", "149 110.0616683959961\n", "150 99.67420196533203\n", "151 91.19140625\n", "152 85.92176055908203\n", "153 82.2587890625\n", "154 79.37641143798828\n", "155 77.57473754882812\n", "156 75.71873474121094\n", "157 74.76695251464844\n", "158 73.61392974853516\n", "159 73.7373046875\n", "160 73.32074737548828\n", "161 74.79610443115234\n", "162 75.15473937988281\n", "163 78.76358032226562\n", "164 80.19961547851562\n", "165 87.09074401855469\n", "166 91.6589126586914\n", "167 105.99201965332031\n", "168 132.52023315429688\n", "169 199.60317993164062\n", "170 285.26385498046875\n", "171 186.7301025390625\n", "172 85.42977905273438\n", "173 71.33895111083984\n", "174 66.64602661132812\n", "175 63.440467834472656\n", "176 60.96210479736328\n", "177 58.90080261230469\n", "178 57.13398361206055\n", "179 55.62022018432617\n", "180 54.284812927246094\n", "181 53.065895080566406\n", "182 51.94929122924805\n", "183 50.92884063720703\n", "184 50.01615524291992\n", "185 49.2723388671875\n", "186 48.88428497314453\n", "187 49.18997573852539\n", "188 51.66621780395508\n", "189 56.960655212402344\n", "190 74.84833526611328\n", "191 96.08964538574219\n", "192 175.0045623779297\n", "193 258.491455078125\n", "194 184.846435546875\n", "195 83.15020751953125\n", "196 69.83650207519531\n", "197 51.21598815917969\n", "198 48.549232482910156\n", "199 46.50739669799805\n", "200 45.08198547363281\n", "201 43.89152145385742\n", "202 42.86724853515625\n", "203 41.932674407958984\n", "204 41.0839958190918\n", "205 40.29037094116211\n", "206 39.5553092956543\n", "207 38.86370086669922\n", "208 38.21051025390625\n", "209 37.590450286865234\n", "210 37.0007438659668\n", "211 36.43656921386719\n", "212 35.89463806152344\n", "213 35.36964416503906\n", "214 34.8619499206543\n", "215 34.37443542480469\n", "216 33.90838623046875\n", "217 33.463294982910156\n", "218 33.03306579589844\n", "219 32.62254333496094\n", "220 32.2387580871582\n", "221 31.8708438873291\n", "222 31.53622055053711\n", "223 31.22540855407715\n", "224 30.986581802368164\n", "225 30.75267219543457\n", "226 30.672260284423828\n", "227 30.66810417175293\n", "228 31.07099723815918\n", "229 31.388729095458984\n", "230 32.891693115234375\n", "231 33.72759246826172\n", "232 37.617942810058594\n", "233 38.177757263183594\n", "234 45.63814163208008\n", "235 43.17341613769531\n", "236 52.84267807006836\n", "237 45.67348861694336\n", "238 53.69830322265625\n", "239 43.98344039916992\n", "240 47.329803466796875\n", "241 39.6602668762207\n", "242 39.626487731933594\n", "243 35.451988220214844\n", "244 34.44646072387695\n", "245 32.15733337402344\n", "246 31.359241485595703\n", "247 29.95440101623535\n", "248 29.509944915771484\n", "249 28.538223266601562\n", "250 28.30735206604004\n", "251 27.50522804260254\n", "252 27.50213623046875\n", "253 26.80497932434082\n", "254 26.98086929321289\n", "255 26.31241226196289\n", "256 26.672256469726562\n", "257 25.982328414916992\n", "258 26.511144638061523\n", "259 25.812694549560547\n", "260 26.602962493896484\n", "261 25.768213272094727\n", "262 26.81290054321289\n", "263 25.837427139282227\n", "264 27.06724739074707\n", "265 25.927724838256836\n", "266 27.30327606201172\n", "267 25.987638473510742\n", "268 27.363279342651367\n", "269 26.01933479309082\n", "270 27.32394027709961\n", "271 25.930755615234375\n", "272 27.011472702026367\n", "273 25.699975967407227\n", "274 26.50739097595215\n", "275 25.408933639526367\n", "276 25.86797332763672\n", "277 25.097475051879883\n", "278 25.256912231445312\n", "279 24.849056243896484\n", "280 24.65272331237793\n", "281 24.661697387695312\n", "282 24.140518188476562\n", "283 24.619226455688477\n", "284 23.73824119567871\n", "285 24.717910766601562\n", "286 23.54086685180664\n", "287 24.9752197265625\n", "288 23.630443572998047\n", "289 25.48272132873535\n", "290 24.02497100830078\n", "291 26.01576805114746\n", "292 24.780500411987305\n", "293 26.469703674316406\n", "294 25.79985809326172\n", "295 26.6370906829834\n", "296 26.846729278564453\n", "297 26.385406494140625\n", "298 27.688945770263672\n", "299 25.86838150024414\n", "300 28.163406372070312\n", "301 25.150251388549805\n", "302 28.116533279418945\n", "303 24.517921447753906\n", "304 27.596027374267578\n", "305 23.859285354614258\n", "306 26.56470489501953\n", "307 23.3745059967041\n", "308 25.282028198242188\n", "309 22.86705207824707\n", "310 23.85765266418457\n", "311 22.422100067138672\n", "312 22.59556770324707\n", "313 22.051166534423828\n", "314 21.59400177001953\n", "315 21.73813247680664\n", "316 20.74985122680664\n", "317 21.347631454467773\n", "318 20.129060745239258\n", "319 20.965091705322266\n", "320 19.616971969604492\n", "321 20.59461212158203\n", "322 19.323686599731445\n", "323 20.284162521362305\n", "324 19.114233016967773\n", "325 20.032251358032227\n", "326 18.95831871032715\n", "327 19.813901901245117\n", "328 18.809864044189453\n", "329 19.587026596069336\n", "330 18.633268356323242\n", "331 19.3393611907959\n", "332 18.45094108581543\n", "333 19.10856056213379\n", "334 18.29006004333496\n", "335 18.963394165039062\n", "336 18.093769073486328\n", "337 18.76136016845703\n", "338 17.90596580505371\n", "339 18.58025360107422\n", "340 17.717267990112305\n", "341 18.402299880981445\n", "342 17.53641700744629\n", "343 18.24544906616211\n", "344 17.375978469848633\n", "345 18.122867584228516\n", "346 17.215097427368164\n", "347 18.01441192626953\n", "348 17.10610008239746\n", "349 17.929285049438477\n", "350 17.017274856567383\n", "351 17.8439884185791\n", "352 16.94769859313965\n", "353 17.71332550048828\n", "354 16.843292236328125\n", "355 17.566287994384766\n", "356 16.777320861816406\n", "357 17.416345596313477\n", "358 16.69339942932129\n", "359 17.225542068481445\n", "360 16.65900230407715\n", "361 17.003944396972656\n", "362 16.512805938720703\n", "363 16.707984924316406\n", "364 16.380939483642578\n", "365 16.481826782226562\n", "366 16.23741340637207\n", "367 16.224605560302734\n", "368 16.116172790527344\n", "369 16.01961326599121\n", "370 16.0587215423584\n", "371 15.799931526184082\n", "372 15.911197662353516\n", "373 15.567693710327148\n", "374 15.816048622131348\n", "375 15.395733833312988\n", "376 15.712669372558594\n", "377 15.229640007019043\n", "378 15.624187469482422\n", "379 15.078832626342773\n", "380 15.593881607055664\n", "381 14.984262466430664\n", "382 15.602352142333984\n", "383 14.913544654846191\n", "384 15.634354591369629\n", "385 14.903152465820312\n", "386 15.721226692199707\n", "387 14.883771896362305\n", "388 15.738739013671875\n", "389 14.934103012084961\n", "390 15.740025520324707\n", "391 15.051740646362305\n", "392 15.747553825378418\n", "393 15.295642852783203\n", "394 15.794942855834961\n", "395 15.707908630371094\n", "396 15.878894805908203\n", "397 16.251039505004883\n", "398 15.95378303527832\n", "399 16.912405014038086\n", "400 16.121116638183594\n", "401 17.57708740234375\n", "402 16.237180709838867\n", "403 18.057422637939453\n", "404 16.25248146057129\n", "405 18.1812801361084\n", "406 16.16710662841797\n", "407 18.015974044799805\n", "408 16.019521713256836\n", "409 17.67422103881836\n", "410 15.807389259338379\n", "411 17.291345596313477\n", "412 15.551980018615723\n", "413 16.870981216430664\n", "414 15.28683090209961\n", "415 16.45754051208496\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "416 15.046100616455078\n", "417 16.132770538330078\n", "418 14.80198860168457\n", "419 15.807101249694824\n", "420 14.587957382202148\n", "421 15.535334587097168\n", "422 14.381044387817383\n", "423 15.229350090026855\n", "424 14.211679458618164\n", "425 14.969184875488281\n", "426 14.052995681762695\n", "427 14.728175163269043\n", "428 13.93752670288086\n", "429 14.523200988769531\n", "430 13.845670700073242\n", "431 14.32778549194336\n", "432 13.768282890319824\n", "433 14.132892608642578\n", "434 13.77458381652832\n", "435 14.004599571228027\n", "436 13.83255386352539\n", "437 13.862512588500977\n", "438 13.929853439331055\n", "439 13.736422538757324\n", "440 14.140033721923828\n", "441 13.69277572631836\n", "442 14.489221572875977\n", "443 13.784162521362305\n", "444 15.030045509338379\n", "445 14.053020477294922\n", "446 15.616470336914062\n", "447 14.566765785217285\n", "448 16.124311447143555\n", "449 15.121051788330078\n", "450 16.293582916259766\n", "451 15.538890838623047\n", "452 16.134872436523438\n", "453 15.629945755004883\n", "454 15.781015396118164\n", "455 15.652036666870117\n", "456 15.342903137207031\n", "457 15.427617073059082\n", "458 14.877690315246582\n", "459 14.979843139648438\n", "460 14.347681045532227\n", "461 14.468661308288574\n", "462 13.883293151855469\n", "463 14.067682266235352\n", "464 13.53783130645752\n", "465 13.715188026428223\n", "466 13.294391632080078\n", "467 13.479684829711914\n", "468 13.086114883422852\n", "469 13.345418930053711\n", "470 12.981571197509766\n", "471 13.237014770507812\n", "472 12.89077091217041\n", "473 13.044453620910645\n", "474 12.756986618041992\n", "475 13.012262344360352\n", "476 12.732881546020508\n", "477 12.824119567871094\n", "478 12.588905334472656\n", "479 12.721162796020508\n", "480 12.548263549804688\n", "481 12.599344253540039\n", "482 12.476131439208984\n", "483 12.607063293457031\n", "484 12.505374908447266\n", "485 12.477144241333008\n", "486 12.475809097290039\n", "487 12.60508918762207\n", "488 12.559510231018066\n", "489 12.446664810180664\n", "490 12.48794937133789\n", "491 12.326469421386719\n", "492 12.461620330810547\n", "493 12.414360046386719\n", "494 12.549243927001953\n", "495 12.268865585327148\n", "496 12.52327823638916\n", "497 12.245977401733398\n", "498 12.541507720947266\n", "499 12.180692672729492\n", "500 12.569136619567871\n", "501 12.120124816894531\n", "502 12.593598365783691\n", "503 12.06467056274414\n", "504 12.618692398071289\n", "505 12.018089294433594\n", "506 12.653897285461426\n", "507 11.985318183898926\n", "508 12.665456771850586\n", "509 11.875547409057617\n", "510 12.63729476928711\n", "511 11.951704025268555\n", "512 12.78012466430664\n", "513 11.966864585876465\n", "514 12.805120468139648\n", "515 11.917511940002441\n", "516 12.759134292602539\n", "517 11.91922664642334\n", "518 12.74145221710205\n", "519 11.870694160461426\n", "520 12.648548126220703\n", "521 11.858591079711914\n", "522 12.608646392822266\n", "523 11.837157249450684\n", "524 12.503793716430664\n", "525 11.827508926391602\n", "526 12.392881393432617\n", "527 11.836199760437012\n", "528 12.351949691772461\n", "529 11.827587127685547\n", "530 12.291703224182129\n", "531 11.845207214355469\n", "532 12.238033294677734\n", "533 11.851987838745117\n", "534 12.129100799560547\n", "535 11.895650863647461\n", "536 12.006834983825684\n", "537 12.071447372436523\n", "538 11.862838745117188\n", "539 12.36300277709961\n", "540 11.838622093200684\n", "541 12.923316955566406\n", "542 12.066741943359375\n", "543 13.796222686767578\n", "544 12.682493209838867\n", "545 14.618804931640625\n", "546 13.567913055419922\n", "547 15.201546669006348\n", "548 14.514939308166504\n", "549 15.157068252563477\n", "550 14.894901275634766\n", "551 14.82116985321045\n", "552 14.973808288574219\n", "553 14.36568832397461\n", "554 14.864335060119629\n", "555 13.79199504852295\n", "556 14.384416580200195\n", "557 13.285518646240234\n", "558 13.8139009475708\n", "559 12.818103790283203\n", "560 13.309890747070312\n", "561 12.457561492919922\n", "562 12.768548011779785\n", "563 12.109518051147461\n", "564 12.345666885375977\n", "565 11.838518142700195\n", "566 11.98867416381836\n", "567 11.639802932739258\n", "568 11.519742965698242\n", "569 11.370664596557617\n", "570 11.15427017211914\n", "571 11.204188346862793\n", "572 10.994608879089355\n", "573 11.155523300170898\n", "574 10.941986083984375\n", "575 11.204839706420898\n", "576 10.84663200378418\n", "577 11.213382720947266\n", "578 10.813859939575195\n", "579 11.298030853271484\n", "580 10.721686363220215\n", "581 11.294883728027344\n", "582 10.642816543579102\n", "583 11.290977478027344\n", "584 10.621570587158203\n", "585 11.330678939819336\n", "586 10.58786392211914\n", "587 11.338900566101074\n", "588 10.599321365356445\n", "589 11.343742370605469\n", "590 10.636480331420898\n", "591 11.348445892333984\n", "592 10.661396980285645\n", "593 11.305547714233398\n", "594 10.69072437286377\n", "595 11.231929779052734\n", "596 10.729141235351562\n", "597 11.120464324951172\n", "598 10.836509704589844\n", "599 11.02585220336914\n", "600 11.147380828857422\n", "601 11.012079238891602\n", "602 11.700508117675781\n", "603 11.138029098510742\n", "604 12.44657039642334\n", "605 11.493819236755371\n", "606 13.279351234436035\n", "607 11.980254173278809\n", "608 13.828882217407227\n", "609 12.335530281066895\n", "610 13.890178680419922\n", "611 12.377365112304688\n", "612 13.672966003417969\n", "613 12.314437866210938\n", "614 13.351038932800293\n", "615 12.199349403381348\n", "616 13.001585960388184\n", "617 11.737051010131836\n", "618 12.477154731750488\n", "619 11.315387725830078\n", "620 12.10653305053711\n", "621 11.122674942016602\n", "622 11.923060417175293\n", "623 11.009233474731445\n", "624 11.797826766967773\n", "625 10.912956237792969\n", "626 11.667257308959961\n", "627 10.87093734741211\n", "628 11.612075805664062\n", "629 10.753063201904297\n", "630 11.500226020812988\n", "631 10.666036605834961\n", "632 11.389327049255371\n", "633 10.641620635986328\n", "634 11.34609603881836\n", "635 10.538703918457031\n", "636 11.240482330322266\n", "637 10.502617835998535\n", "638 11.186229705810547\n", "639 10.399940490722656\n", "640 11.07362174987793\n", "641 10.29592514038086\n", "642 11.001121520996094\n", "643 10.407917022705078\n", "644 11.066526412963867\n", "645 10.300006866455078\n", "646 10.924904823303223\n", "647 10.182451248168945\n", "648 10.847217559814453\n", "649 10.238099098205566\n", "650 10.836588859558105\n", "651 10.119577407836914\n", "652 10.745954513549805\n", "653 10.167619705200195\n", "654 10.740652084350586\n", "655 10.065475463867188\n", "656 10.654346466064453\n", "657 10.109355926513672\n", "658 10.62813949584961\n", "659 9.988624572753906\n", "660 10.55061149597168\n", "661 10.056044578552246\n", "662 10.53543472290039\n", "663 9.937051773071289\n", "664 10.44735050201416\n", "665 10.042400360107422\n", "666 10.486113548278809\n", "667 9.922344207763672\n", "668 10.333057403564453\n", "669 9.806156158447266\n", "670 10.256221771240234\n", "671 9.865896224975586\n", "672 10.259897232055664\n", "673 9.773486137390137\n", "674 10.196406364440918\n", "675 9.832138061523438\n", "676 10.19542121887207\n", "677 9.742813110351562\n", "678 10.134439468383789\n", "679 9.804479598999023\n", "680 10.11970329284668\n", "681 9.704248428344727\n", "682 10.07077407836914\n", "683 9.851264953613281\n", "684 10.099723815917969\n", "685 9.732878684997559\n", "686 9.96037483215332\n", "687 9.626123428344727\n", "688 9.859709739685059\n", "689 9.604263305664062\n", "690 9.819677352905273\n", "691 9.51850700378418\n", "692 9.78436279296875\n", "693 9.615999221801758\n", "694 9.804388999938965\n", "695 9.610206604003906\n", "696 9.761518478393555\n", "697 9.523723602294922\n", "698 9.675700187683105\n", "699 9.515082359313965\n", "700 9.639348983764648\n", "701 9.448809623718262\n", "702 9.563665390014648\n", "703 9.449939727783203\n", "704 9.554956436157227\n", "705 9.402902603149414\n", "706 9.48503303527832\n", "707 9.410470962524414\n", "708 9.466278076171875\n", "709 9.42030143737793\n", "710 9.462084770202637\n", "711 9.388456344604492\n", "712 9.390506744384766\n", "713 9.411766052246094\n", "714 9.380949020385742\n", "715 9.453214645385742\n", "716 9.358305931091309\n", "717 9.496848106384277\n", "718 9.340471267700195\n", "719 9.500185012817383\n", "720 9.268125534057617\n", "721 9.6624174118042\n", "722 9.285665512084961\n", "723 9.766561508178711\n", "724 9.265359878540039\n", "725 9.91065788269043\n", "726 9.266630172729492\n", "727 10.196619033813477\n", "728 9.33184814453125\n", "729 10.31056022644043\n", "730 9.4459228515625\n", "731 10.629362106323242\n", "732 9.821006774902344\n", "733 11.032565116882324\n", "734 10.545709609985352\n", "735 11.271682739257812\n", "736 11.432579040527344\n", "737 11.432348251342773\n", "738 12.460997581481934\n", "739 11.330101013183594\n", "740 13.059946060180664\n", "741 11.198677062988281\n", "742 13.333955764770508\n", "743 10.929497718811035\n", "744 13.135268211364746\n", "745 10.77116584777832\n", "746 12.882686614990234\n", "747 10.58820915222168\n", "748 12.530218124389648\n", "749 10.332746505737305\n", "750 12.048186302185059\n", "751 10.15908432006836\n", "752 11.673738479614258\n", "753 9.969121932983398\n", "754 11.312966346740723\n", "755 9.854571342468262\n", "756 11.113723754882812\n", "757 9.780363082885742\n", "758 10.999980926513672\n", "759 9.710268020629883\n", "760 10.88806438446045\n", "761 9.661212921142578\n", "762 10.785196304321289\n", "763 9.597941398620605\n", "764 10.69247055053711\n", "765 9.532906532287598\n", "766 10.603074073791504\n", "767 9.484548568725586\n", "768 10.532205581665039\n", "769 9.43923568725586\n", "770 10.460396766662598\n", "771 9.406749725341797\n", "772 10.359094619750977\n", "773 9.35378646850586\n", "774 10.313379287719727\n", "775 9.339227676391602\n", "776 10.270576477050781\n", "777 9.321992874145508\n", "778 10.17398452758789\n", "779 9.28928279876709\n", "780 10.135478973388672\n", "781 9.282571792602539\n", "782 10.083499908447266\n", "783 9.312825202941895\n", "784 10.074797630310059\n", "785 9.29830551147461\n", "786 9.953083038330078\n", "787 9.271284103393555\n", "788 9.887079238891602\n", "789 9.270550727844238\n", "790 9.776187896728516\n", "791 9.274053573608398\n", "792 9.742859840393066\n", "793 9.29629135131836\n", "794 9.691925048828125\n", "795 9.311891555786133\n", "796 9.628150939941406\n", "797 9.317649841308594\n", "798 9.54599666595459\n", "799 9.326900482177734\n", "800 9.46576976776123\n", "801 9.333112716674805\n", "802 9.316043853759766\n", "803 9.245628356933594\n", "804 9.174911499023438\n", "805 9.130691528320312\n", "806 9.026357650756836\n", "807 9.047698974609375\n", "808 8.901134490966797\n", "809 8.962682723999023\n", "810 8.791932106018066\n", "811 8.87315559387207\n", "812 8.687568664550781\n", "813 8.800554275512695\n", "814 8.623148918151855\n", "815 8.716794967651367\n", "816 8.543046951293945\n", "817 8.56474494934082\n", "818 8.397558212280273\n", "819 8.531536102294922\n", "820 8.37232780456543\n", "821 8.428328514099121\n", "822 8.274665832519531\n", "823 8.431962966918945\n", "824 8.270166397094727\n", "825 8.379289627075195\n", "826 8.207696914672852\n", "827 8.401199340820312\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "828 8.20716381072998\n", "829 8.469131469726562\n", "830 8.236584663391113\n", "831 8.424156188964844\n", "832 8.168313980102539\n", "833 8.484416961669922\n", "834 8.172074317932129\n", "835 8.536100387573242\n", "836 8.157014846801758\n", "837 8.581727027893066\n", "838 8.13553524017334\n", "839 8.696174621582031\n", "840 8.188529014587402\n", "841 8.957174301147461\n", "842 8.366853713989258\n", "843 9.087259292602539\n", "844 8.61898422241211\n", "845 9.600666046142578\n", "846 9.278848648071289\n", "847 9.91394329071045\n", "848 10.14834976196289\n", "849 10.423606872558594\n", "850 11.416866302490234\n", "851 10.418106079101562\n", "852 12.281963348388672\n", "853 10.481473922729492\n", "854 12.87403678894043\n", "855 10.396160125732422\n", "856 12.916685104370117\n", "857 10.285785675048828\n", "858 12.771074295043945\n", "859 10.352560997009277\n", "860 12.81513500213623\n", "861 10.118997573852539\n", "862 12.345061302185059\n", "863 10.030735969543457\n", "864 12.146442413330078\n", "865 9.814250946044922\n", "866 11.731513977050781\n", "867 9.791189193725586\n", "868 11.683649063110352\n", "869 9.596960067749023\n", "870 11.304178237915039\n", "871 9.476587295532227\n", "872 11.074976921081543\n", "873 9.46214485168457\n", "874 11.042204856872559\n", "875 9.287127494812012\n", "876 10.719624519348145\n", "877 9.23538589477539\n", "878 10.631698608398438\n", "879 9.11007308959961\n", "880 10.393648147583008\n", "881 9.10506820678711\n", "882 10.377655029296875\n", "883 8.979528427124023\n", "884 10.174203872680664\n", "885 8.9866943359375\n", "886 10.169689178466797\n", "887 8.958577156066895\n", "888 10.132844924926758\n", "889 8.853513717651367\n", "890 9.939226150512695\n", "891 8.794721603393555\n", "892 9.840280532836914\n", "893 8.682388305664062\n", "894 9.676387786865234\n", "895 8.69548225402832\n", "896 9.683582305908203\n", "897 8.684111595153809\n", "898 9.66401195526123\n", "899 8.594156265258789\n", "900 9.512399673461914\n", "901 8.573217391967773\n", "902 9.489818572998047\n", "903 8.463871002197266\n", "904 9.332828521728516\n", "905 8.482060432434082\n", "906 9.340717315673828\n", "907 8.466593742370605\n", "908 9.309955596923828\n", "909 8.376811981201172\n", "910 9.1652250289917\n", "911 8.364412307739258\n", "912 9.147921562194824\n", "913 8.264249801635742\n", "914 9.025073051452637\n", "915 8.291736602783203\n", "916 9.049306869506836\n", "917 8.298337936401367\n", "918 9.050772666931152\n", "919 8.218606948852539\n", "920 8.925788879394531\n", "921 8.221832275390625\n", "922 8.929916381835938\n", "923 8.125205993652344\n", "924 8.825401306152344\n", "925 8.168115615844727\n", "926 8.862433433532715\n", "927 8.189055442810059\n", "928 8.871030807495117\n", "929 8.194257736206055\n", "930 8.863895416259766\n", "931 8.089027404785156\n", "932 8.718398094177246\n", "933 8.125476837158203\n", "934 8.73470687866211\n", "935 8.16848373413086\n", "936 8.743915557861328\n", "937 8.072606086730957\n", "938 8.597414016723633\n", "939 8.140182495117188\n", "940 8.633056640625\n", "941 8.234619140625\n", "942 8.650079727172852\n", "943 8.373706817626953\n", "944 8.667745590209961\n", "945 8.419292449951172\n", "946 8.564188003540039\n", "947 8.71246337890625\n", "948 8.499603271484375\n", "949 8.754026412963867\n", "950 8.378808975219727\n", "951 8.707267761230469\n", "952 8.223587036132812\n", "953 8.973301887512207\n", "954 8.249604225158691\n", "955 9.120798110961914\n", "956 8.240376472473145\n", "957 9.567197799682617\n", "958 8.391382217407227\n", "959 9.934947967529297\n", "960 8.551895141601562\n", "961 9.747421264648438\n", "962 8.641094207763672\n", "963 9.514118194580078\n", "964 8.857434272766113\n", "965 9.758781433105469\n", "966 9.328777313232422\n", "967 9.747819900512695\n", "968 9.824708938598633\n", "969 9.374284744262695\n", "970 10.072290420532227\n", "971 9.379064559936523\n", "972 10.451343536376953\n", "973 9.188913345336914\n", "974 10.514688491821289\n", "975 9.137795448303223\n", "976 10.573251724243164\n", "977 8.986966133117676\n", "978 10.422307968139648\n", "979 8.891181945800781\n", "980 10.281803131103516\n", "981 8.728313446044922\n", "982 10.022692680358887\n", "983 8.668171882629395\n", "984 9.946191787719727\n", "985 8.618570327758789\n", "986 9.84320068359375\n", "987 8.564799308776855\n", "988 9.743132591247559\n", "989 8.508552551269531\n", "990 9.638229370117188\n", "991 8.444400787353516\n", "992 9.526444435119629\n", "993 8.40636157989502\n", "994 9.446041107177734\n", "995 8.32719898223877\n", "996 9.311468124389648\n", "997 8.256400108337402\n", "998 9.157243728637695\n", "999 8.188175201416016\n" ] } ], "source": [ "N, D_in, H, D_out = 64, 2, 100, 3\n", "\n", "better_net = torch.nn.Sequential(\n", " torch.nn.Linear(D_in, H),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(H, H),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(H, H),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(H, D_out),\n", ")\n", "\n", "loss_fn = torch.nn.CrossEntropyLoss(size_average=False)\n", "\n", "learning_rate = 1e-3\n", "optimizer = torch.optim.SGD(better_net.parameters(), lr=learning_rate)\n", "for t in range(1000):\n", " # forward\n", " y_pred = better_net(X)\n", "\n", " # loss\n", " loss = loss_fn(y_pred, y)\n", " print('{} {}'.format(t, loss.data))\n", "\n", " # зануляем градиенты (чтобы не было остатка с редыдущего шага)\n", " optimizer.zero_grad()\n", "\n", " # backward\n", " loss.backward()\n", "\n", " # обновляем\n", " optimizer.step()" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnIAAAH2CAYAAAD052HGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8nGd59v3fOaPd2mx535d4iZc4TuyQlYQlgVAggZYCYSl94AFKaVkKbXlLKQXKS/u2PKUNNISypQ+hQCkJtGFrCU2c1Y4TJ/Eir7K8yJJs7dZoNJr7fP+4R7KWGVm2Jc2MfHz56GNr5p7RNUbRHLqW8zR3R0RERETyTyTbAxARERGRC6MgJyIiIpKnFORERERE8pSCnIiIiEieUpATERERyVMKciIiIiJ5SkFORKYEM3ujmf3KzNrMLG5m+8zsc2Y2M9tjExGZKKY6ciKS78zs74APA98EHgQ6gLXA+4FD7v6GLA5PRGTCKMiJSF4zs9cBPwbe7e7fGHZfFLjN3X+alcGJiEwwLa2KSL77CLBjeIgDcPeku//UzJaamWf66L/ezG5J3bZ00G23p2779aDbPm1mp8zsBjPbYWY9Zvacmd04+Oub2TvNbKuZtZhZq5k9bGabh49z0Ncd/nFL6v7+8b92HP69RGQKUZATkbxlZoXA9cDPxviQjwHXDfr4u3M8vwH/L5BMc3cZ8H+Be4A3AW3AT81s7qBrlgL3pe6/CzgGPGJmyzN8ybelxvW2sb0cEbnUFWR7ACIiF6EGKAbqx3h9rbs/2f9JutmxYe4CFgE/AaYPu68U+DN3vz/1XA+nxvFh4E8B3P0zg75WBPglsAV4O/CZQc9VmPpzh7vvNbOuMb4eEbnEaUZORKaCcd/sa2ZFwGeBzwPtGS770cAA3LsIg9o1g57jcjP7kZk1Es7qJYDVwKphz1Oa+jN+jmFFzKwgNVMoIqIgJyJ57TRh+Fk8Ac/9PsJViy9nuL/L3WPDbmsC5gGYWQXwC8IZvY8CNxHOxu0ESoY9bkbqz9PnGNODhGGwx8x2m9nvjOF1iMgUpqVVEclb7p4ws8eAVwGfHMenLk8935+4e0+GCbByMysdFuZmAw2pv18HLARudfe9/ReYWVWa51oOtLp7xznG9RFgK+EM3juBb5rZC+6+YywvSkSmHs3IiUi++3tgc7rZKTOLmNmrL+A5PwY0Ex5UGM1AfTozKwduBZ5O3TRiudTMric8ADF4jAbcBjwxhnEdcPft7v4o4SyfARvG8DgRmaI0Iyciec3df2JmXwS+bmY3EC4/dgFrCAsC1xHOZJ2PdwB3unswyjUx4K9SAe4EYfgrAr6Uuv/J1Di+ZmZ/Qzg792ngeP8TmNkq4C8J99XdPoZxLTSzNYQh8R2EewN3jv1lichUoyAnInnP3f/IzB4HPgjcTxh06ggLBf8tI/ekncsT7v6Tc1zTTbi8+Y/A5cBe4DXu3pAaU6OZvSn19R8E9hMGyz8e9Bx3AsuAN7j7z8cwrn9K/RkDDgDvdPfnxvaSRGQqUmcHEZHzZGafBj7o7urjKiJZpT1yIiIiInlKQU5EREQkT2lpVURERCRPaUZOREREJE8pyImIiIjkqUum/MjMqipfOndOtochIrkg0YOXFtJdFMWiZdkejYjICHue2XnK3Wed67pLJsgtnTuH7ffene1hiEguaKglvm4uzy+qorByc7ZHIyIywqZozZGxXKelVREREZE8pSAnIiIikqcU5ERERETylIKciIiISJ5SkBMRERHJUwpyIiIiInlKQU5EREQkTynIiYiIiOQpBTkRERGRPKUgJyIiIpKnFORERERE8pSCnIiIiEieUpATERERyVMKciIiIiJ5SkFOREREJE8pyImIiIjkKQU5ERERkTylICciIiKSpxTkRERERPKUgpyIiIhInlKQE5FLUod3ZHsIIiIXTUFORC4tDbW0x+tpJUGsYlG2RyMiclEKsj0AEZHJkqjbSjcx9t1xK7GKRVTanGwPSUTkoijIicglpf3O1xErn6UQJyJTgpZWRURERPKUgpyIiIhInlKQExEREclTCnIiIiIieUpBTkRERCRPKciJiIiI5CkFOREREZE8pSAnIiIikqcU5ERERETylIKciIiISJ5SkBMRERHJUwpyIiIiInlKQU5EREQkTynIiYiIiOQpBTkRERGRPKUgJyIiIpKnFORERERE8pSCnIiIiEieUpATERERyVMKciIiIiJ5SkFOREREJE8pyInIpaGhFoBG78nyQERExk/OBTkz+4aZNZnZixnuv8XM2s3sudTHpyZ7jCKSZxpqaY/Xc3jTfAAqbU6WByQiMj4Ksj2ANL4F3A3cN8o1j7r7aydnOCKS11IhrnHTCk6uu1khTkSmlJwLcu7+iJktzfY4RGRqSMSbab/zdZwsn6UQJyJTTs4trY7RdWa208x+ambrMl1kZu81s+1mtr25vX0yxyciIiIy4fIxyO0Alrj7RuAfgQcyXeju97r7ZnffPKuqatIGKCIiIjIZ8i7IuXuHu3el/v4QUGhmM7M8LBEREZFJl3dBzszmmpml/n4N4Ws4nd1RiYiIiEy+nDvsYGbfBW4BZprZMeAvgEIAd78H+C3g98ysD4gBb3F3z9JwRURERLIm54Kcu7/1HPffTVieREREROSSlndLqyIiIiISUpATERERyVMKciIiIiJ5SkFOREREJE8pyImIiIjkKQU5ERERkTylICciIiKSpxTkRERERPKUgpyIiIhInlKQExEREclTCnIiIiIieUpBTkSmroZauonR6D3ZHomIyIQoyPYAREQmQqJuK93E2HfHrcQqFlFpc7I9JBGRcacgJyJTT0MtbZuWULd4kUKciExpWloVkSlLIU5EpjoFOREREZE8pSAnIiIikqcU5ERERETylIKciIiISJ5SkBMRERHJUwpyIiIiInlKQU5EREQkTynIiYiIiOQpBTkRERGRPKUgJyIiIpKnFORERERE8pSCnIiIiEieUpATkamloZb2eD2tJLI9EhGRCVeQ7QGIiIyXRN1Wuomx745biVUsotLmZHtIIiITSkFORKaGhlraNi2hbvEihTgRuWRoaVVEphSFOBG5lCjIiYiIiOQpBTkRERGRPKUgJyIiIpKnFORERERE8pSCnIiIiEieUpATERERyVMKciIiIiJ5SkFOREREJE8pyImIiIjkKQU5ERERkTylICciIiKSpxTkRERERPKUgpyI5L+GWtrj9bSSyPZIREQmVUG2ByAicjESdVvpJkbjphWcXHczlTYn20MSEZk0CnIikr8aagHYd8etxCoWKcSJyCVHS6sikteCqzYAKMSJyCVJQU5EREQkT+VckDOzb5hZk5m9mOF+M7N/MLMDZva8mV012WMUERERyQU5F+SAbwGvHuX+24GVqY/3Av80CWMSERERyTk5F+Tc/RGgZZRL7gDu89CTQLWZzZuc0YmIiIjkjpwLcmOwADg66PNjqdtGMLP3mtl2M9ve3N4+KYMTERERmSz5WH7E0tzm6S5093uBewE2r16V9hoRyREv7IKf/Cd0dMKN18OtL4fi4myPSkQkp+VjkDsGLBr0+ULgRJbGIiLj4Vv/At/9AfT2gjs89zz84N/hK38P06Zle3QiIjkrH5dWfwy8M3V69Vqg3d0bsj0oEblADSfh/u9DPB6GOICeHjjRAN/7t+yOTUQkx+VckDOz7wJPAKvN7JiZvdvM3m9m709d8hBwCDgAfA34QJaGKiLj4bEn0t+eSMAvfzW5YxERyTM5t7Tq7m89x/0O/P4kDUdEJpoZGba5pu4bXYd3AFXjOiQRkXyRczNyInKJueE60p5hKiqC216Z8WGJuq20x+tpJUGsYlHG60REpjIFORHJrrlz4J1vC0+o9s/AlZTAwgXw27+Z9iGJuq10E2PfHbdyct3N6rMqIpesnFtaFZFL0NvfAps3wX/8FDo7w1m6W14azsoN11BLcNUG2hfPobBiPYWTP1oRkZyhICciuWHN6vBDRETGTEurIiIiInlKQU5EREQkTynIiYiIiOQpBTkRERGRPKUgJyJ5qdF7sj0EEZGs06lVEckfDbW0x+tp9ELUzUFEREFORPJFf4jbtEJFgEVEUhTkRC5VBw/Bw/8DfUm46QZYu2ZMvU2zJRFvpv3O13GyfJZCnIhIioKcyKXo3m/ADx+ARALc4YEfwy03w598FI6fgK/cC9uegYICeMUt8L53Q0VFtkctIiLDKMiJTHW1++HxJ6GwIGx71dEZhrh4/Ow1PXH49SNw5Qa4+x440x0GvEQCfvZL2PkCfPOrYbATEZGcoZ/KIlOVO/z1F8Pl095eiETgvvvhsuXh58P19IT398TDx/br64NTp+HRx+BlN0/e+EVE5JxUfkRkqnr0sXCWLZ4KZslkGOD27hsa1AZrbQuD23CxGDz/4sSOV0REzpuCnMhU9eP/DGfZhnOHaHTk7SUlMH9e+ucqLIRZM8d3fCIictEU5ESmqlgs/e3uMGN6GNz6FRfDogXhoYbi4pGPiUTgVa+cmHGKiMgF0x45kanq5pvgwKGhhxoASkvgjz4Ep1vgoZ9Dsg9ufQW89vYwxL3nXfC1bw462ODwF38GNTWT/QpCDbUk4s0c3jSfdnVzEBEZQkFOZKp67WvgJw9BY9PZww0lxbBuLVyzOZxl+41Xj3zcm94Ir74Vnns+XFLddCUUF03u2PsNDnGLl1NYuZnC7IxERCQnKciJTFVlpXDPP8CPfgz//eswlP3Gq8OPyDl2VVRUhEWCsywRb6Zt0xJOrrteRYBFRNJQkBOZKo6fgK9/G57ZAaWlcMfr4E1vgLe/NfyA8FTqd78P+w/C8mXwutvPLpn29sIjW8OZuFkz4dW3wZzZ2Xs9KbElK7M9BBGRnKUgJzIVNJyE934wPOAQBNDeAd/6F9j5PHzhs+E1Bw7Chz4Gib4wtD35NHzv3+CLX4AF8+EDH4bTpyHWE87e3f99+NQn4IbrsvvaZIQ2umjjDCUUMptqIjq3JnLJUpATmQq+/X/Phrh+8Tg8uxP21sKa1fBXfxN2bOjXv2/uc38NV14BJxvP1pBLJMI/P/sFeOB7Q0+4StYkCXjK9tJKJ44TwYgQ4XpfRyVl2R6eiGSBfo0TmQq2Pzs0xPVLJsP2Wqdb4Njx9I89dTrcQ5euEHAkAs88O65DlQu3x+ppoYOkBQTm9FlAL308aXtwMhR5FpEpTUFOZCqoLE9/e0EBVFaEteMsw2ON9CEQAA+XYidbQy2Juq0c3jSfRu/RQYeUepoIbFhgM+ijjxY6szMoEckqBTmRqeCNd2Ze/rzpRphZA3MzhKGqKnjJlvQnWfuScPWV4zfOsWiopT1eP6TkiIT6SGa4x0iQhcAtIlmnICcyFbzmVXDLS6GoKKz5VloaBrvPfQrKp4XXfOLj4e39hX4LCsJr/uyP4QPvDa8rTFVpMwuLA//v3w1LkUyy3k1rOLnuZoW4YaqYlvb2gIBqMszKisiUpsMOIlNBJAJ/+kdw12+HBxzKp8F114a15Ppdvga+fS/86Cew/0BYfuQNr4d5c8P7v/U1+PcHwz1xs2bCb90JV2zIzuu5RCQJOMBx6q2JAGcu01ntiyghfQHm9b6UJ9lDkmBgqTzqERZQQwudlFDEdMqJEaeLHsopoQwdVBGZysz90tggu3n1Kt9+793ZHobI+AgC+OGD8K8/gLY2WLQQ3vu/4Pprsz2yi9dQS/PaMvZM8SLAjrPVdtFO18C+N3MopJCX+UaKM/SwaKWTPXaUdrooopBCorRzhihRAgIMS51ojRAQMJMqNvsqCohO5ssTkYu0KVrzjLufc1lCS6si+egrX4OvfzOs+5ZMQt0R+MvPhwV9JS8000YHZ4YcXvDUwYVD1pDxcdOp4Hpfy+1+DQt8Jh3EwsdZksCcJP0nWsPPT9HOTjs4GS9JRLJAQU4k37R3wIM/gZ740NvjcfjKveEJ1TyWiDdnewiTotnaSdrI08KBOY20juk5DttJguHPMex0cmBOAy06DCEyRWmPnEi+OXQoPNTQX7R3sKbmMNDlYwHfhloS8ebUadVFU3pZFaDIC4hgI8uJAEXDfjQ74cxajF6qmDZw6GGs4cww4iQo1I98kSlH/1WL5JsZMyCZ4Q28sDAMeflmSIjLTsmRgIATnKbZ2immiMU+i3JKz/3AC7SAWdRybMTtUY+wzOcNfH6GHh63XfSmQpsDNVRwhS+jgOgoJUmGKqV4XMYtIrlFQU4k29xhTy10dISttKqrRr9+yeLwcMPBw0ML+RYVwW+8On09uDzQtmkJJ7N0wCFBH4/ai8SIk7QAczhkDVzpy1nIrAn5mmUUc6Wv4DkODhxQAFjCbOYyHQhn4p60PcToHbJkesrb+bU9H55ePYeoR1jpC4hqJ43IlKQgJ5JNR+rhTz4Z7nuLWLhc+pt3wnvfHdZyy+Tzn4GPfQKamsAiYXutzVfB+94zeWOfQmrtKGfowVPLnG7gBDzHIeb49HFfkuwiRjdxaqjkNr+ak7SSJGA21UwbVC6kg256hoW4/vElPcjcrSPF3Fjji1jOvNEvFJG8pSAnki19SfjIH0Nr29ADCj/6CSxeDLfflvmxs2bCt+6FvbXQ2AwrloWzdPkotazayvysDeEozQMhbrAI0EgbC5k5Ll8nQR9PWy2tdIX74wiYw3Su8pVpZ8ziJLBMae0cIS6SCnErsvjvKiITT0FOJFue3haePB1+yrSnB777/ZFB7vgJ+Ma3YfuOsEPDHa+DN70hLPSbr1LtuBo3reDkupsnfFnVcdo5Qy99VDGNYgoJCEhk2GcW4OPajH677aeVTgLzgUXRRm/lRTvMRl8x4vpqphFkWj51MoY5cyiikCVM7QMjIqIgJ5I9p1I14NJpGVZ+ouEkvPeD0N0dBr/2DvjWfbDzefjCZyd+rBMhNRO3745biVVM/CnVLmI8ZXvpoRdLzYYtYy7TvQKDtHEtwJlNddrnc3zgJOhY9p/10Mtp2kecUg3MOeqnWM+yEc9TRCHLmMtBbzjnDFxqUBiwiFms8UW0cYY2uiihiHnMUFFgkSlIQU4kW1avyvzmvOqyoZ/fdz/EYkNn7+K98Nzz4fLqmtUTNsyJFFwVtgCbjJm4x233iP1mdd5IjHjGx0WwtB0W6mhkr9UPnBhdyCw2+FKiowSlHnpT3RbShXcnQR/RNK25LvP5HExXILg/fdrQ2yIeYZHP5imrpYsYAQERIrzAYa73terJKjLF6BiTSLasXhkuiw4vF1JcDO9+19Dbtj8z9IRqv0QiDHMCQDc97OMYu6yOZtoGlkVP0R6GrmHBOWkBp62TSLofhQ41VI64uZ4mXrTD9FpfuERqzlGa2G77Rx3bNEoyLpNGiWZsydVHkHnGL80vAo6zx47QSTdJC8KDERbQZ0mesr3nvVTsOF3EaOfMeT82QR9HaeYIowdmEblwmpETyaYvfBa++nV46OdhId8Vy+GD74d1lw+9rqICmk+NfHwymfedHMZLPU08b4dwwvBRZ41Mp5xr/XJi9GYMIQn6KKeULo8NWfaMEmG1Lxpx/W47MmJ51C3c6/aU7WGe17CAmSPCVyEFLGMudd44pKND1COs8YUZDzWUUkwEG2O1uLD4b9uw1l/9kiQ5TQczOUeJm5R2zrDd9oUzmYT/Jht9BfOYcc7HHucUz9rBgVflOCt8PpezeIyvRETGQjNyItlUXAx/+AH42YPw3w/BP38Frrxi5HW3vTLzc9SOPhN0Keihl+ftEIF5ePo0NQvVQheHOUkV0zLOJVUxjRt8HfOYQcQNc5jmJWzx1cygYsi1jg8U5k2n0dp4wQ7za9uZtuvCWl/CSl9IoUfBodgLWe9LWTZKeZAIxlpfQtSH/rg2D8uLpBNkfLU25gLCCfp4zHZxhh6SFpC0gF7rY4ftp50zoz62mzjP2gGC1OOSFvZ/PWQNNI2x/ZiIjI1m5ERyxWiFfCsrIBqBZJqlue3PTNyYJkrqoMMRL4Qxzg6N+nS0pL09sIAjNLHC51NDBae9Y8hMVcQjrPUlFFLA1b6KgIAAz3goIHNAYmCZM2kB3R6n1o6y3pcNu8RYxQJW+nwcT7+km8YS5lDkhdQS1rsro5jVvpCT1soJbwGcCIYDL/E17LWjtNCZZvzBiHCayTGaw1nM4cvRBByw41ztqzI+9ijNaf+lkhZwiJPM9ukDt/WR5ASn6bIYFV7GfGpUvFjkPCjIieSDkuJw9q47NvK+wvR7q3LWBJQcSY4Ssfro41F7gVa6gHAmy4FySlnvS4fsg4ucI1rVcnRM43FzjqVOoqZjqf+dj3nMYJ4PXdKc7zNZSTen6aCQAuYwnQKirPcoj7Er7PyQ+jJRj7DC51OUYS/ecF3WM2QJeNDg6fQ034eD9FoibV0+gBY6aaGTGVTQRYyt9iJJwlm7KBF2c4SbfD1lqcLIAU7veZwOFrnUKMiJ5INrr0k/G1dUNHrh4FyTmolrv/N1nCyfNW6nVWdTzV6OjtgHZ270kKDHEmdv9LA+202+4bzCVCudHLaTYysDMokqKKOCsiG3VVPOTb6BfXaMFu+khCIu8/nMp2bMz1vl04gSGRHmLPXvN5qZXkU9TWmDYB9JnrDdXOkr2G/Hw6XqQbOZSQ94xg5wo6/jICfYZ8cJCDCM2VTTR5Iz9FBBGat8IdN1Clcucfr1RiQflJXBJ/80nJXrn4ErLYWlS+Adb8vu2M5Tf8mR8VRJGQuZOWQfWcTTxzQ36CSWdulxNPXWlL63aZqJJ3PGdCBgIlVSxmZfxW1+NS/1DecV4oDUEmd0xOuLEGGFLxj1sZH+3rHpJuVS+xd32iG6iI0MxgZtdLGPY9TaMfosSWBO0gIaaKGZdrotTiOtPG67aNSeO7nE5dyMnJm9GvgSEAX+2d2/MOz+dwH/H3A8ddPd7v7PkzpIkWy46Xq4/5vwy1+Fbb2uvAKu2QxRFXkF2OjLmU01BzhON3FKKeYMPQQ2cnN/gNNK18Cyai8J6mmm3c5Q4WUsYfaIciB9ZOhtOriasIVLmIUUsMbz+3RmAVFu8vU8awdo9S6M8ATtlb6CCkozPu4U7WyzfWlPzQ4WlmLJVIrZOWgNI2f0htXMSxLwPId4pV913kvVIlNFTgU5M4sCXwZuBY4B28zsx+6+e9il33P3D076AEWyraYG3vKmbI8iJxlGnAQdxHDCE5aZNs5FMEpSxXc76OYxe5EkTmABEYwDHB9RPHeez+AkLSPCRdQjLPd5dFucuCeY7dUsYTaFufXj9YJMo4QbfT0J+gjwjLXuBttt9QTp9tYN40ABlrayXjGFo54OHixOgh56KaV4TNeLTDW5trR6DXDA3Q+5ey/wr8AdWR6TiIyzRu8Z9+eMEWeX1RGkiuACGSd8IhjzCE9OPmsHSJAcCB+BOX2W5BnbP2TP3VxmUMW0Icu3UY8wjRJWsZCrfSXX+1ouY/6UCHGDFVIwphAH0En3uS9yKKeEK/2y8N/Tz94e9QhX+Ioxz7A5jNpRQ2Sqy7WfNgtgyLGwY8BL0lz3m2b2UmAf8BF3H9tRMpFcEO+FH/wQ/vNn0JuAG6+Dd74NarK7p2rC9Z9WTZUcGe+2XJlKkPSHubCkiBMlykt8DVGixEmEwSNNZuihl27iTEudnoxgXOdrqaeR+lRpjoU+i6XM0WnKQYooHL2Lg0MRBWzx1ZRTyvW+jgN2nE7vppJprPQFVDGNZczlsJ8cOgM6rCWZOdRQQVHOvZWJTJ5c++5Puzd52Oc/Ab7r7nEzez/wbeDlaZ/M7L3AewEWz5k9nuMUuTBBAB/7BOzbFwY6gP/4KTz6GHzjq1B98TXVctIElBwZLhilgVQpRWzwZUSJUkEphakZnHO1nBp+f5QIy5jHMs9cwPdSd5nPZzdH0pcucZhFFdf46oFZtOmUs8VH9gpe44tJWJKj3kSECEmSYdEWN5KptmVFFLDJLxvxWJFLSa4FuWPA4J44C4ETgy9w99ODPv0a8NeZnszd7wXuBdi8epX6GEn2bd8BBw6cDXEQttnq7IIfPgDv/p3sjW0CXUzJEcfDQwsEVFCWccltDtXs5ki6J6CGSgKc5+0AvYSlSOYyg42+nDKK6WLkUm8hBQOzcTJ2S5nDGWIc9pPA2d/EIxjLmcflvnhMy6YRjHW+hApKabMupns5C5hFM2100UMFpcxl+piLKg92mg7q7CQ99DLLq1nKnDHX1xPJNbkW5LYBK81sGeGp1LcAdw2+wMzmuXtD6tPXA3smd4giY9CXhMceh6e2QWVlWOttyWJ4ZgfE0uwPSyTgiSenbJC7UOl6fV7plzGX6SOuHa3rQgfdNNjQgwoN3sIZ6+FKX8ET7Aln9MwxD0tsbPLLdBLyAhjGel/GShbSRheFRCmhiGKKzmsJupNuttouglSx4AZa2MdxbvINLGDmBY/vICfYa0cHiiW3ptq43exXDByAEcknORXk3L3PzD4I/Jyw/Mg33H2XmX0G2O7uPwb+0MxeD/QBLcC7sjZgkXR6euBDH4f6+jC0RaPwowfhA++HisqwDlwiMfJxFWNrnZR3GmrpJnbeBxx6SfCY7aJvUPmQJAHPsI8bfT1Vw4rSttJFBEaegjTo8JH74NycLo8Bxst8I4fsJO3eRQVlLPd5lI9SYkPOrZhC5qQJ3GO1zfaF/WoHFQsOPOBZO8ANvu6CnjNOgj1WP6Q0SmBOr4e3a5lW8lFOBTkAd38IeGjYbZ8a9PdPAJ+Y7HGJjNn3fgiH66A3tXyaTIYfX/4n+Mcvwn3fGfmYkhJ4452TOszJkKjbSjcx9t1xK7GKRee1rHqU5rSzbAEBB+0EV/nKIbcXU0iEKMEYm8L366SbGcxhvS89r8fJxOkiFh6YGBG+ocU7SdB3QSeDm2lLzbIO/b5yC2doN13EmEWyRUetRMbbz35xNsQNZhHYuw8+/uGwtVZJSfhnURG85tXh6dUpJFG3lbZNSy4oxAF0WSxtPTK38I1+uDlUZ14KHWWFtEz1x3JOkLa6XMggfYeNMRhtqVzL6JKvcm5GTiTvJTPMCLmH9936CtiyGbY+Hga+azbDwtFbHuWr2JKVxC6wp+povT6HL6s6t4v/AAAgAElEQVRCuK/tWr+cR3kh8/n3YaUriilkJlP0pHAeK6csdVJ1ZGAL99udPZjQR5LjnKLLYlR4GQv6W4ulMZvq9F3D3Fh4EfvuRLJJQU5kvN18E/zox9CXpjL9tdeEf1ZXwWtvn9xx5ZmFzGIvR0n60NZYESIs9Fm8YIdppJUCoiz1OSxhzqjN3MPSFeHjA5wKStniqzUTk4MiGFf4Mp7j4MChBDw87LJxULHg4QciokTYQz03+vq0J44LKeBKX85zHMIJC0dHPUIJRaz2RSOuF8kHCnIi4+3tb4VHtkJb29kyIyUl8MY7YP6lUX+sf2/cxXRwCHt9bmCHHaAt1euzjBLW+mK2WS0Jknhq0/oujtBMO1t8NbOpptnbznZ3GPR8r/RNdBGjiMKcLS2yMzZKMd1xsLE0P5aSFzCTMi9mv52gy2NUUcZKX0DloLC+Pc2BiKQH7LAD3OTr0z7vQmZR7eXUWxM9HpYfmU+NijpL3lKQExlvVZXwjXvgJw/BY09AVRXc+VrYfHW2RzbxGmpJxJs5vGk+7YuXU1i5eUzVuWLEaaCFgIDZTKeSMiDs9XmTr6eXPhyniAKesN1hH85BQS1pAU3eRhtdXOHLecReIOlJkhZgbkQwrvaVFFLAdHL3dPADx3toPeUUn66csK9xeHUny2ZYXgS66VRwTZpiwQBn6OFMmgMRGLR7F70kMtaGK6eUtb5knEcrkh0KciITYdq0sLn9Jdjgvm3TEk6uu37M++LqaORFOwyE29j2coxFzOQKXz6whNbfgul5O8QpOtLugQsIaKadlSzgFX4lR2mmhU6meQlLmJMzhxoyzbgdbnG6aytY313CluqJ+dG8ra2P/bWwo6YDVmWe+cuHkBcQjLooPlpdQZGpREFORLKmixgv2uEhdb0c55ifYhbhkle/GHHqacp4AtWIpPqphnuhljOP5TnUSmtnLD4Q1jJZOYEhDmBLdQFbKOf+07DrifTXlK3uhBnxnA9z5ZSGh2HGcCCiX0BAgiRFFGhvpEwZCnIiMj5Sy6ow9iWrY5xK2+80aQF1NDLfa+glwSk6aKMrbQ2ws3xI8Mslg0PcqGFtkhoL3FVUnvG++wfN2OVymDOMjb6CHewf9UAEhAFutx3hCE04TpQoq30By5inQCd5T0FORC7ekL1xY68Zl7C+jLEsQR8HOMFeq8ew0WuHOWzyy9LOwkyW/rCWTv++t7uKyictrF2ou4rK2Xa0hB00cXhmD3cuyM1DIQDzmMENvo79dpwuj1HJNJb5HCJE6KF3oOXWc3aIE5wamPkN6GMPRzE3lpE7s7YiF0JBTkQuTkMt7fF6Gjet4OS6m8+rZtwcn069NY0IaRE3Kr2MWjuaevPNvN/JHJb63Ivqv3mxdsbi7NgXUHy6kpXdI4PPApjQJdPxtqW6AI7OZv/pHh4gtw9HVFPOFl+N4+y2Izxuu1MlZgJmUcU6XzokxPVLWkAtx1jqczUrJ3ktf36yiEwF/UWBC9L8pxcEsPMFaGqClZfB8mWTP74L1LtpzXkdcOg3iyqqKafVuwa6OJgbxRQSs/iIYsADUu/JUSLMoIK157GcOx6Gz77l04zbWG2pLoC2EvbXhiddc33f3H6OU0cjgflAm7Ymb6fH9qeC3chC3QmS9JG8oHZfIrlC370iE+3AQbj7njCkeerNv6YG3ve/4LZXhp+fbISP/DG0tQMOgcP6tfD5v4Ti3H3zvFiGcZ1fzmFOcoQmAgLmU8NlPp/HbfdoD8TcuMHXUU3m/V4Tob9EyPqjswduy7cZt7EaOByR4/vmHOeAnRgR/N2cDj+TccYtig0ckBHJV1PvJ49ILjl2HP7gjyA2rDfo6dPwt1+C7hicOgX/+gPoGzZj8MIu+PK98NE/mLzxZkGECCuYzwqfP+T2mVTR4d0ZT6lGsEkJcYPLhRxu8YEQNxWDWya5vm8uSUAyzYwbQJQo0yihw7sHCkgDRDzCMtdhB8l/l85PIpFsuO87EM9Qr6u3F/7xK4Cl78/a2ws/+wV8+PchcmlUnU8ScIp2kgQs9tnUWyN9w1p09ZvomZRM5UImss5bLsvlfXNRIhRSEBaKHiYg4Cq/jF12hFPePrB/bjGzWIPackn+u/R+GolMphd3h3vfMkmOch+E/VoTidxdXk2dVm1l/rmvPYcm2thu+1KfOQHOTK+iydrSXh8nwbN2gBqvZD41ExLs0pYLmSJ74C7E4H1zXNeV7eEMMIzVvojdHBmyvBpxYy4zqKCMa/1yYsTpoZdplA4UmRbJd5fGr/ki2TLzIuuazZub8yGuvx3X+R50GKyHXrZZLX2WTH0EBOY0W/uojztqzbxgh/mVPUcPvRf89UdzKc6+jSZX/z2WMoc1vohCjxLxCBGPsIhZbPLLBq4ppZjpVKQNcXESnOA0TbQSjFbqRiTH5OZ/kSJTxZt/C2r3Z15eHU1xMXzw/eM/pvEwKMSdb8mRdI7SnLYwcLrbBgxqlB54Ly9aHZt91UWNo99oNeEkdLjF2bgg26M4yzBWMJ9lPo9eEhRSQHSMcxV7OcoBO04kdb0B1/gaapi4nrci40UzciIT6Ybr4PWvOf/HLV4Ef/VpuO4l4z6k8RJcteGiZ+L69VjviDpfQPiOOnh/XIaScm7QQMvowW+MhndhkJFWdpfQXVvBA8d7MvaOzZYIRglFYw5xx2hmvx0nMB+YEU5YkidtD4k0e+5Eco2CnMhE+1/vgsIMHQeiUSgsgKKisx/v+V24759h81WTOsxsqvFKoj6GH0ejHDAcrxC3Y19Ad20FdxWV5+wyYrZtqS4YCHM79gU5F+bGqo8kz9nBIadZBzvO6Ukekcj5008pkYlWWgKveRX87JdDl1iLi+CVL4d3vg0eS3Uwv+E6mDM7/fNMYXOZTilFnPH42TdVZ9TgNoSHxYXHo5TEQGFfGdVAjbnTsGNf7taYG80RGgky/AKQJCA+QfsuRcaTgpzIZPiD3wtLjPz8vyBi4WnVl90CH/5gOFv3xjvC61pa4ctfDYNdaSnc+bowBEandtHSCBFu9A3ssSMc99MEBBnfYCE8jRjgYOHfI0TY4PnTCWMqyfUac6NptNaMvywYMJ2K9HeK5BAFOZHJEARhSAMwC1t0vfAinG6Buak9Zi2t8O7fg87OsOwIwJfvgW3PwGf+PDvjnkRFFLDRV7CRFTjOz2wbiTRFXiMY1/la6qyRbu+hhkqW+7yBBukXSgccLlx/jbmOaXF2xjqzOjOXoI99dpzjnMJxFlDDKl9IESO3NxRRkH7m16GQAmZRNSljFrkY2iMnMhm+9k14ZkdYE64nDj090HAS/vTPz7bt+tcfDA1xEF779DbYW5udcWeJYWzwZUR86DtsxI01vogaKrnaV3KTb2CtLxmXEDd4b5zkpyQBj9qLHKaBHuslbgkO08gj9gJ9aX4pWOpzMx6KuMbXqOuD5AUFOZGJ5g4/eQjiw/bbBEHYY/XgofDzx54YGuL69SbCWbkckog30+EdF/TYDrrZbvv4L9vB47aLJkYW/O2hlx4SzKaaMi8m6hHKvYSNvpwFzCQ5jnW++mfi1h+drRB3kRrrPauzmic4TYz4kBPQbk6cBPU0jbh+JlWs8Hnh8rxbqv6csd6XMkPLqpIntLQqMtGSycx15KIRaE0FmbLS9NcUFEBZ2cSM7Xyl6se1bVpC3eJFxCoWpVmwyqyFTp6w3WEQM+gmTitdrPUlLGMuAI20st324TiBOVGPMI0S5ng1z9thAnzghGoFpaz3pcyi+qJeVkVM9cIu1uCuD9lq4dVorUM6O/RLWkAjrSz3eSPuW8NiFvtsGmnDgLnMGPMMbwfd1NlJYvQyy6tYxCwK9bYqk0wzciITraAAFmRoYZVIwMoV4d/veC2UZHjju+WmiRnb+RgW4gorN593Dbnn7VD4RjtoxSppAbvtCH0k6SPJdtsXFvlNzaokLaCTbvbbCZIWhKdaU/XlOi3GE7aHnRwal/IjcnEGlyU53OKTXpakmMK0dQZx0u6R61dGCcuYy1LmjjnEHaWJR+wF6mik0VrZY/U8bDsnrMOISCYKciKT4fffN7LVVkkxvPY1UJ2aTbr9VXDtNeHtkcjZunIf/wjUXGSrr3ESXLWB2JKVxCrOv9l4kiSddKe9zzDaOUMz7Wl3JfnwwsBDH8wRa6SOxvMeUza5Q7wVek5B0zY49SwEU6D+bH+Yy8Ys52KfPdCdYbAoEZb6xReu7pegL5wdHvRLSdLCciW77ci4fR2RsdAcsMhkuO4l8Lm/gK9+HeqOwPRqePOb4I2vD0+uHjoMs2bCpz8Je/eFByNKS+Hmm6BmRrZHPy5soE1D+lZcBUTpoffC5tUM9nGMZT73Ikc5OU49B8/9DZxpAAIgAtFiiBTA5r+A2VuyPcL8VMU01vlidnFk4KCC46z0hePabiv8hWPkbxZu0OAt4/Z1RMZCQU5ksmy5Ovzo15eEv/17+OWvwpm3vj5Yuhg+/xl421uyN84JEiHCXKZz0luHVtJPLXtVUkYJRenrx42hOHCcBI6f10nD/tOqxad90rpqdhyCJ/8YkvFB4wwgGYMk8PQnnZvuhsMPwolfh3fPvwUufw8UX9xWwEnVWO90ljrMmNxCwcuYxzyvoZFWHJhDNaVc+Nd3nAR9RImOue2XyGRSkBPJln/5Dvz3r8N9colEeNuBQ/CJT8G9d2d1aBPlCl9Oh71Ij/eSJCBKhAjGNb6aBMmB2l9DpD41Ty2xZlBM4QWGuFQnh4urYDKgv5qMZRjKvu9AMpH58UEfbP0QJOPgfeGT1D/kND0FL/82FOTIuZfRDHR9qIUdNZPf9aGEIpZw8UupxznFLjtCL+H/YfOoYaMvZxZVafdkmsM8psYMuuQPBTmRbHCHf3tg5GnWZBKO1MPhOli2NAsDm1jFFPIyv5ImWmmnm1KKmMcM9tpRjtAYvjkOD0Cp7g0zqKDFO8/O2A26LuLGSl8w5nH0lxwpPl3Jyu6ScQlxnXXwwj9C8w6IRGHezbDhD0bOorXvB4LMgdP7jKQ7njx7jSeN3g6n/uew/A3nP7bYqXB2r68bZl0N09dmDprj6a6isIXX4ZZONo79/56c0EQrz9nBIadgG/w03RZns6+kmmmc9s7wDoOIRygiylpfkqURy6VKQU4kG5IBdKff+E9BATSfmpJBDsLODHOZwdzUzEUtR6mnaUjtr+ECnBqvZAuraecMu+wIHX4GS/1vpc8fKF8yFhtLi2FGnMOrO9lfC7SVhOUzLlCsCR75QBiUcCMI4MTDTusuePl9EB0UFCsWQ1f9KNOLkaEhrl+yx2h6yocEub4eqHsQjv0XWBQWvwaWvCbca9fv6C/D/XgQzvbt/w7M2gzXfCZ8zERb2V3CcTon/guNsz1WP6KUSWBOu3fxsO0kSfLsLxMORUS52a+geLymdkXGSEFOJBsKojBndlgQeLhELyzPzb6hHd5Bo49f2yLHOWgNaWt/DRYlQiXTKKSAmVRxs19BLwniJCijmCjnn0g2lhazcUFY8+xiw9zB74dLoYPDmSeNeJtz4n9g0a1nr73sLmh6OnX9CJ7aD5gu6DkdddB1DMoXho9/9ANw5tjZ/Xadh50TD8P1fxuGtJ7TYYgLes8+V7IHmrc7df8Jy14fhrump+DMCahYGs7YmbaC0UVP2tsDnGBwiAMw6PEEv7Kd3OTrKSdDTUiRCaD/XEWy5X3vHlmSpLgYbrkZZuZGuZF+ibqttMfrB4oAn2/9uEySBOHMxijMwyXZOUwfcnsRhVRQdkEhbrBlM4w5iy9unfHUzrP72QZLxoyWF4feNmMtbPoERAqdkSd4Lbwp7eSkEWuA/3lveNr16C/gzPGhhyaSPUbrHmh8Kvz82H9BkGY/XrLHqHsQuhvgl2+BZz4Hu78KT/85/Op3ID6y2cYlZ9R6cum+XSwsS7LN9k3YmETSUZATyZaX3Qx/+kcwd064YamsDN70Rvjjj2Z7ZEMk6rbStmkJ++64dVxDHIQzbRkLtaYCzRymc5NvIJLDfS/L5hImzmEs6pRMH3n9gpdBwTTImAgGr9kNu68vBvu+Dcd/FQay4ZIxo+GR8O/Hf5XmKVL6zsDTnwpn7fq6jSBhJGPGmRPw7BfSP+ZiTHZx4Iu10hcQ9aFvkWEt6lG+Dw3O0EMXsQkenchZWloVyaaX3Rx+JBLh3rjJ2IF+AWJLVhIrnzWuIQ7CN8U1vogXqRuyvBrxCHOpZpOvzIuSDyt+O5wFC4ZlFU/CsV/ByrcP3bcG4enT3nPOfKX5fgiM5mecaQszPcY5cxIefjd0HMjwHDhl86BlFyMOXnif0bTNSXRD4TidkO2ureDw6s5JL0VyMRYxix7vZT/HMYwAp5JS5nkNtRzNuKczgtF3jllmkfGkICeSCwrPp2Pp1LKEOQTu1HKUPpIYxmJms86XpK3Sn4tqNoQHCBofG17wzog1OSe3wtwbww4OPc1QOidcYu0+MYYCeWkUVYUFhDMV2Du9k1FPxgJUrYa2vWE94uEsEta1G48gl+1SJGPVRYwGWgb6rZZTyioWstzn0Uk3RRQyjRIcp50uTnhLxv/rKgj/4U7TwT47RgfdTKOE1b7wovsCiwynICcymfqS8PD/wM9+EZYgue2V8MqXhbNxU1wrney2etrooohCVvg8ljEXw8I+lz6HXvooTFWX6yNJPY20WifTvIRFzKKZdvbbcWL0Uk4Jl/viEXvnssED6DoC6d7ZkzHj0L87O78Iyd7UIYfRz3YMfuYRzxktcZb/Vrj/LePS7Lme32DZ6+DIg+nvLqqG4nEuh3ZXUTnbjpZweGZzzpUi2W1HOETDwCr0Xo5ymS9gDYsoIMp0KgauNYzNvprDnORF6oaUzIl6hLW+hCgRTtLCdttPQNjGK06Cp6llgy9jMbMn/0XKlDX13z1EckU8Dh/4cFgnri/VVHP3XvjpL+DvvhCeZJ2iWujkCds9sHwaI84e6umgmyt9BRC+QRan9svFiPOIvUAfSZIWEMGo5RgGA0taHXSznX1s9OUsZNakvp4gAccfhpOPQWEFxBrDU5/peThDBpzX7Js5kUIwcywahkUPYNGrwo9YM5ze6UNOpPZ/vdG/jjN9HUxbAOt+D16824ccmIgWO1d8OGdX+cfdKdo5zMkhS6UOHOQEs72aGYNC3GDLmEuVT6PWjtLp3ZRRwipfyGyqcfxsL9ZBkhbwInUs9Jl5M9ssuU9BTmS8Pf8ifOtf4FAdzJsD77gLFi6AD3wIus4MvbanB2r3wSOPwstvycJgJ8cuqxtRYiRpAce8mdUsHNFC6Xk7RJzEQB7pf5MdvispaQG7OMICn3leXR0uRl8MHv1gquxHz+DesZm+/oWMy5l7Q9iWq2wuND4Z1qibuQmmzQ+vWPo6OPg9CBKDS5Wce6k2UgibP5V6jtdD6Wyo/bbT3RCWH1nzu1Cz8QKGnKeOWBPJNFOYSQLqrYkZPjTI9dLHMZo5Yz1U+TS2+GoKhp2c7qF3oBvEcI7TSYwqpo3fi5BLmoKcyHh68mn4i8+d7djQ1gaf+XxYVmR4iOvX0xO26nr5LZM0yMnXTvrXHsFooZMFg4JcgNNE25jzT4I+eukbmM2baAf/LVxGDRL9A5yYAHn1n51tx7XgZSPvL66Gl94DO/8OTj3no5QtIbwjAgWlsPkvoWzQmZU514Yfl6oEfZnLiXjfkJva6OJx243jJC1sMbeHem7yDZQN+h4eHuwGc5xCvfXKONLcrsh4cYe/v3tk262eOLR3jP7YKX7YYbQ3tuEBrJ2uzHkkjbBA65g3nV20cBZsgmf/ImPvutDbDlYAdo6GAhYNi//u/FtG1LabbLlUimS+14woMwLhfrd5fnajoONst330WXJgdjlpAb0keM4ODHlsIQXUUDmyIo2HByHKyL3DHpK/FORExkt7B5xuOf/HFRfDS7bAd78P37gPdu0523l9iljqc4kM71TgECVKDZUA9JHkMdvF47Z74P4RMvyzPGl7JiXMHf0FJCa625Q5s7f0n0rNLNkLW/8AOg6B9xoe768/lyY9YHjCCOJG7KTx+MfCPXb9OurCVl6ndk78t153bQWHWzxnwtwCZlJG8ZDvz4gb5ZQyn7OFubuIhcv9w7jBaTpHlBzZ5JdRSjFRj2AOBR6hhCI2+6qJezFySdL8rsh4KS4+/3dBs7Ad1xf/IXxsMgnf/yG8ZDN86v+B6NQ4ALGKBXRwhiZvHyh3GyHKdb52YG/bC3aYVjqH1ufq3/Ll4TJskC7JGXR6jF/aM2zxNRk3p1+MoC8skhueFD0fw/espcYfCUt8RArCWnMWCTs0REudwmlw5cfP/cwnHwt7rY5s5XV2v1y0NP0pWe+Duh/DqreH3RxOPReOwYCi6XDD/0kVOR5nuViKJEqEm3wDh2jgGM2AschnsYy5Qw4kJAlGXUQPf5E4+99rCUW8wjfRRBuddDONUuZQrUMOMu7OGeTMbI27752MwYjktdISuHYLPLnt7KlUCMPa9Gro7g6XWftFI/DWN8P3/i0sCNyvpwee2g6/+G+4/bbJG/8EihDhGl9DJ9200kUxhcyieqBbQ5KA45waWWQ1FeLmMp3VvohH7PlM3auI08cT7OblfuWIwxPn0lnaQWNZRcZ+qwe+S9gtIVOj+3QiTvVKiLc6PafCpc0Fr4ANfxjOtiW6oLA8PDxx7Jdw5oRTvQrm33zu2TgIT8kOL0A85MsXQclMOHN05JiDhNFxyNl1D5x6dmgv1r6Y8+v3wC1fH7qXbjzdVVTO/acBuibmC5ynAqKsYiGrPGOVZSqZRqb9kOWUpO1QYhhzmJ4TJXJk6hrLjNyzZnYv8Gl3b53oAYnktY9/FD70MWhsCsNcYUHYeusf/i48nfqd70FzM6y8DH73nbDz+fSzeD098OP/nDJBrl8FZQPFUgdLpp9rA8I32ct8AVVMo4wSzmRoZt7/PL+0HRQSZTGzWeOLztmLdWNpMRsXwAN08uKpDjg6e0SYO/TDoT1NMwtfRcUyWPf+s4cI+mLhadHB3R2KU3Vhiypg+RvH8NTDVC4Nw1oyQzeooNcoKHMihT5iT1+kyKlaCQfuJ035EiPR6Tz8Lrj5a1CeOdtcUppoDfsCD55k9fCXlI2pEjoi2TCWIHcN8H+A/Wb2l8BX3F39R0TSqaqEb9wDO56DuiMwby685JqwRtz8eWE7rsGeenro7N1gPZkDy1RTSJRiCumhd8R9YWukMPyt8UU8x8ERpUwGpN5gEyQ57CdpszNcP2j5djR3Lihh54w4u073sIXyIff1jmlfnFNUBS/7JpTUDL2noHQsjz8/s18CxdOhO5a+5Ei01FnyG7D7nrDu3eBxRgph8e1Q+61Mz270xZzdX4VrPjvuQx+wY1+QE8ur5xIQsMMOpJ2QraJ0QpbzRcbqnIv17v6Cu78SeA/wh8ALZnb7hI9MJF9FIrD5KvitN8AN141e6PeazVBSMvL2wkK46YaJG2OOMYx1vmTE6cGoR7jM5w+cel3ATNb6knBj+jm2IwbmtNFF2zgs31UuP/c1kcKwHMjwEDdRIgVw091QcyUMrz1iBU5JDSx5DdzwJahYFs7MRQqdyhVw4z+Ey6bTRuuw4EbTtokb/11F5RSfrmTHviBnDj5k0prpe8igjTOTempaZLgxH3Zw9wfM7CHgI8C/mtljwEfHe/+cmb0a+BLhrtF/dvcvDLu/GLgPuBo4DbzZ3evGcwwik2bdWthyNWx75uwMXGFhuKfuTW8IPz9+Ah59LFyCvf5aWLI4e+OdQAuYSYFH2UM9XfRQQiFlFHPEGjlKEwuZxUpfwDLmMsen87A9R9KDUcu4OU4LnUNaLPXfdoYeKimjetjsW7ymg21Hh+6Vm38LdPa2sPbKb7Hs1K8paA840nAbu0+9gx6qoRCq/yTJ7jKHtvN73en25I1VSQ3c+CVorYXdX4WWnY4VwIKXh10bIoVQvQpe/i3oOQ0YlAxqvbX+g7Dtk07Ql/4fMTrBVXH623btoCkvZuZEcpH5eZyyM7NqYDPweuD3gSRwD/Dn7t5+0YMxiwL7gFuBY8A24K3uvnvQNR8ArnD395vZW4A3uPubz/Xcm1ev8u333n2xQxQZf8kk/OK/4MH/CA9DvPTGMMSdbITPfgHqj4bXRSJhT9Y33gHvf8+kDS9Rt5WGO19JXfksKm2Cdr8P00eSX9tOYvTiqQMQETemUcJL/QqiRGiji6ds79mSEOmyiIfLtht9BfOpIUacx233kCXcSsq4ztcOzPo90NBJV/wMBS2VvLl7PomCPh5JPszvvO9/U1XXSFGsd8hkYEAhZ+bOoPXKFRS3d9Izo4r973gNjdefuz1CT7ez7+hLuauo/JzXTpQTW2HbJxlxkCNS5Cy7E9b//sSP4f7eLtZd15WzQS4g4Oe2nYQN21XkMOv/Z+++46Ou7weOvz43ssiCMIMsGWEjSkQEUXFvrdpabH+Wttr+rK1W/Vmtra2trXbZoR3aWqxVqpUK2uLCqiC4EBAZkgRIICRhBTIguf35/fG5y90ld8kld0nuwvvp4x6QG9/7ECF55/N5D/KYoyf3zsJEnzbTWrBeaz2ro+fFUrV6G1Dsv40FXMDHmF2zj4HrgW1Kqc9orT+Ia9UmH2+H1nqX/72fBa4AtoU85wrgh/7fLwUeVUop3ZmIVIhkYrXCRReYW8CmzfB/3wVXSM6Yz2c+Xv5v055k5kk9v9YeUskBHCFBHJij0mPaQSl7KeIEDlBvene1l/6mTL7cesrYzX6O4qAZZ9hr6vUxNqldnKzHsVVVYC3cRy4K3yjFi96t2JSPc+/8E/13VGNzewOXbWHFTe6+/eS8ur+li9uolR/gu2A2vqsjjGQIcYBGRo4pZ3HOVSxqLOz05yle5S/C9idM6xHt1eZXH1gzIXuEGdclTEHDKXo86yjFh0YrjVVbsGJhuo7h3F2IbqNjUqsAACAASURBVBTLnv4dwHvAH4H3gfVa69CM5KeUUt8B/gpMiXM9w4HKkI/3ArOjPUdr7VFK1QMFwKE431uI5PGbR8ODuFAOB7z0co8Fcvb0QeQt/zdDZ45l35Qzu31XzouPErW3bSsSTDC3gyoq2IcX3WYoORBx3KhWmoPUR3zMpzRV+hA+vNRwxP+4xmLVaIsPLzBtydstQVw0YQO7XG6sr32IdeGNpuAliuE1JVC+E/vE5bxQGMyJrG8c1O2B3Z5XYOvvW1XjKk2/ETDtFpM/17gHcsfE1g4lHuObMthQ2kD5QAdXDo+QM5oEBtOfs/QMKtR+julmBuhcRjKYNGnHKnpZh38DtdYjYrjOYuCn8S8n2uFIp59jnqjUTcBNACOHDI5vZUL0FIcDdu9p/zlHe7D/1rAi8moga2M1sIr6kSdiz+1wt7/Ltqs9uIhSyYs5AXQH2kBE0sEOXbT7g0FcyN3+jy3eLiSzaw3vvg9XXxn9OcOKGE4R2dtXcjIVLXfvdrpYumARI7ZNiSuHrj2f/qVtSxXtVTTtM/3ljlWDxWp26CZ9FYbNh6Z9ph1Jogs6ivNtUDmYsloHy2lkzACVlMes/chgih7V28sQIkyiWkwfBBYk4Dp7gdDA8QSgOtpzlFI2IA+IOBdJa/241nqW1nrWoLy8BCxPiB5gs5l8uGjS7HDmvJ5bD8CwIuzpgxilChiiMmjQ+7vtrXazP7Y59Iked9rO9UouPxVve/9PInG7Td7j1k87fGre6PPCbqPS07jmzcVUTt7KElfig3afBxxRzjC0BxorwOdUeJoUXodiy+/hjYXwwT2w8nOw/oHWLU3iV5xvY3xTBjnNuYm9sBB9XEICOW2sSsCl1gHjlVJjlFJpwHXAS62e8xJwg//31wBvSn6c6FNsNpg/1/waydChcN45PbumHqLReJKwlcObP72B5oE5nRrsAJhClVvvhLdXd+pleaPPY1J6Hl9YvZgJI1YnPJhTVrBHa32maTvBQiu0R+E5pvC5FdWrYcvvE7okIUQXJdXQN621B7gFeA34FPin1nqrUupHSqnL/U97AihQSu0Abgfu7p3VCtGNbrsFhheasV9KmZvFAlddDn/6nZnr2gcpFDl0Q/fcOB0tLOCP2/7IO/d8lmMFOa26tnXA44Ef/gTebOdnXY/XHMO+tAJKygCwj57HhKxxXPT+G5yb9jcW57Q+nOg6pWDc58Ga0epPYdEx7XT6nIqKFf45rgm2f4+m/LD8bC5ErDrVfiSVSfsRkXJ8PtNfbucuGDIY5s2F9LTeW09NCc4pQ9k/cki3tiI5QB0fqpLwQobAl6nYp2TF/vzO0DCiHr60HtTS5fDnxaZ9TKwuOBfuvjOYfAcmH/K2u8DpNNdSwMQi+NkDJmCvKaHKuYvSieN4q2BuwvLmtA+2/hHKl5t+cz435IyGo3vA64jtE33mn02fukRaV+ehLMtBVlHy5soJ0RNibT8igZwQIjY9FMgBHKKebWoPDTSRjo0RehC1qoFa/LOyWhclaIXd3wduEHnk6n5sV3siH4VG+pLX4fM0do/mtCorZ+62Ygm8YNt2eOiXwV5/sbjkIrj9m6btjNbw+RvMbN7Qr8VpaXDpRfCtm1vuqq9YyYaJo1medmVCK1rdR+FopSlgyBgEq26EhnLQUZoEB2kGTIczHknYUsIscR3FWdDAyRMsEsyJ41LC+sgJIURPG0ge8/W08Du1yaEro4pS9vqPNzVWLAxnIDP0iWEzVbU2z22Zy6rN0e2JeijjGE4aNio5yCdqF77W0Z2GkXoQ0zmRBr2fCdveZ9jWJtSwovDnTZ4If/sz3PN9eD/GeVYrXoGPN8EjD5umz/UN4UEcmNYzL78WFsjlpY9keKbVtGFPIHs29J8U/Pj0X8PGn8G+dyLPcA1S1H2qcR6BtHxo2AFeN+SPNzt88VqYls2SWpK+LYkQvU0COSFEylAoJnAChbqAGg7jw8dQBpBHvzbPncAJZOtMStmLAxd59GOiHhE2rmskgzlMA1W6tiXgU1qRSRpTGI3F/5+9vdQxpeChB+CJJ+HpZ9sGZZFU18CPfmpak1iiXNnhgLKdMH5sm4fW1Xm6rS1JWg5M+RocXAdeR/vPVVao/QQ2P2J29pQCFMy4E05IQB+DwAivLRxgOakTzPnQNOHAjo10unnOmTjuHTdHq6eMKdTrn3yyt5chREpzV6yhiWZKrziP5pwRPTayqztpNDUcpkLtw4OXYbqA0QzBjg13w0fk7dnFmI3V2NMHQesdudaamuDnv469StVuA6/P5ENGkp4O/3M9XG+mENZXrOyRHnOOw7Dys+Bzt3+8as/VaA94msKfZ03XzP0d9J+YmPWsq/NQNfNgSgRylRxgi9qNz8yAoD85nKLHk0Ev5reKlBTr0WpSVa12J6f2Ul+xsreXIURKs4+eR176SCa8uJKhW1d1az+5nqJQFFLA6XoK8/V0xjOcZl3b+SAOICsLfngv3HdPbG/u9jc+jtajzumEvz0Ne6uA8LYkquj1bukxB5AxAPIngrJG/0Hfmq4ZOgd8EY56vS7Y8Vy3LC2pVXOIj9Uu3MqDV/nwKc1hGlirtqJjr3MWolOOm6PVwzkF7Ha6GFWxkqwEtDewj+7hhqxCABypg7XvgdcDs0+Fob2wI9bDkx56yxCVwbCN1Z3/t+5wmEKJaVNh85aOn+/zgdVi+gZ6Iky08Plg1ZqWXTn76HmMqVgDa1+HubC46aRuGec16wew5pvgqtem+a8F0GDvZ6pbi26AqlWmFUkbWnGsMrGBy5FDOqmPV6s4xHpVFmE8HDi0i0PUM4j83lmc6NOOm0Au353FE8V3MS3rYzKy4utJMLy6nAnbX2F4+omx/ZQuRCL85xX43R/M7o32we8fg89dC1+5oePXJtqwIuw1MGZjNeVA/Uj6zFFrXLZshbu+B2hwdKLJWkYmZPcz1aut+XzgCR+jYB89jwk1JfR7/w1Gjilncc5VCQ/mMgfBuc/A/g/h2F7IHgGDi01eXEDTPqjM1Hibw7+mKqsOK6CIV7KP8GrCwUa1M2oipQaO4mBQj65KHC+Om0AOYFFjIesqB9Mc53VeyXKw87TVzF/7OmMqDppjl3hJQCjas6fSBHEuV/j9z78AM6bBrJPbvmb3Hti4CbIyYe4c6Ne2ICAuw4qwU8SYjWto2riT0ivOoyGHPhHMZTZWktmZliJg/t/cfZ/Jk+sstxvmzzMjvVr/P7bb4fQ5bV8TmNNavpIDCzaxpDaXhWnZnX/vdigrDI3w1gHDF5iZrT6XRnuDUYwlDcZdl9ClmGCuLoOyEigvaoQBzqQJ5io52O7RqQKySc6dRJH6jqtADkhIcnAx2SypnA9zoXSAYnBleVzXy/+4lCEVe8gbfV7caxN91KsrIzeedThg+b/DAzmfzyTcv/W22QqwWuHh38H934fZxQlfmn30PPJqSpjw4kr2zxzLvilnpmww16D3k9lYyYQXTQpGp45V162PXrTQHrsdpk6GRV+ED9aZXTmnfzcvIwPOPTti5WpAFpnk91KMYE2H+X+EjT+HQxs1aMg9EWbcAf2GJ/79ivNt5utvCWwoaIAJyRHMOZUbraIEchoySGMgMu9bdI/jLpBLlIVp2awrOZ/VWQ7gjLiulTF/J9e8uZhRFSvJSx+ZmAWC7PL1JXV10ScI1NWHf/zaSlM16Wy1s3Pfj2HpM5ATZchmYCcpK6vz62uVN5eKwVyD3s/QrasYsnFn54M4gMaj5sg7FkqZilSfF06eCd+723zeH3sUXn4VVr0DmZlw+SVw+mkdXm5wZTnOghnQ2PUducNbYNufob4M0vubEV6jLgkfQhFwrBqq3gRPMwyZDXN+AT6X+ePYu/DXp7MCbUk2cCApgrlBOo9KDgZ7FgZoyCSduXpKWI9DIRJJArk4BH46jFv1DJ4ovotLGpcxOoHtBPptfIVsbCY4lKAu9Rw9Biv/C7vKAWV2ZxytGnulp8Gc2eH3/Wt52+eB+Y686h249OLw+yt2w89+BaU7zMdF4+Gu22H0qM6t15831x87+zr3yl4XGsR1+d/L9KmmlUisfF648zY4/9zgfZkZprfc1VfGfBl7+iCKNlfwFefPu9yW5NDH8P5d4PUXLniOwZZHNEf3wNSbw5+7axls/YOJWbUHdi2FQbPglO+BxRrh4t2kON9GWW0u0D2Vu50xhAH0I4OjuhlfYGdOgx0b8/U06SUnupUEckliUWMhi3OuSmzX9umXcUnjMqaU72R4DRLMpZLyCvjWHSZ3yuE0uzdut+k7FmhZYbNBbq7ZtQnVEOUbm9ttdo1CHamDW74Nx5qCjWw/LTH3Pf1XyO/7VXbuho8Y2tJmJI4fegqHwXkL4L9vdVzooDW43PDYE3DuAlPAUlcHK9+EA4dg6iST12iL4Uv0sCLyKGJUxUqueXMxSxcsoqxibKfy5TY/EgziArwORfkyzfiFkO7/a3CsxgRxPpcKeR7sW6tZcbHJBes/xRyt5o6J+e1TngXFPD2VUrWXSn0QH5ph9GeiHilBnOh2Esglke5oIbDEdQN7/IUZQyr2JPTaXTp+ErH50YNmRy4QXDmdZkctJ8fkVHm8cNY8+OJCyGn1DfvUU+CV19sexdrtMGN6+H0vrTDJ9aGNwQNBxr9fNtfvpLSN2xmKO+mPV9vkw8XaK649d9xqBt4//4Lp/dZRztzRY2ZM17798N0fmPYjHg88jwnivnUzXHZx5PPNVvJGn8ekijWcmlfNaqLn1LWmtRmvFYnFDnXbYchp4D4G2x4HX4QOKWjlH6EGh7do3vkGnPN3M7+1u20o9SXFCC8bVibrUUymkzvZQsRJArk+bmGaKczYMz+xmccjvVX+qt01Eswl2v4DUFXddtST1mZH7dm/QUE73yG/uNDkyDU1BwOJ9HRT3TqpVaDy6XYTtLXmcpmduc5KkR5zcRU1tMdiMYHXZRfDTd8IHldH4/OZ19z347bH4R4P/Pb3Zqfuf65PzPoiUAqsmeCNUM6vfWDPhX3vwkf3m1mqdHR6rBU+l2bXCzD5xu5YcVCqjvASIpEkkDsOLEzLhuoZCb3muropPD1/eEuRRlebLCdkF6Svcbujz99UKni0Gs2QwfDnP8Dip+DD9ab9yOWXwDVXtd3ZGTUSPtrQthGtzQYjR3Rt/SE95kxbkvCHezuwS0g+XCwuvwQe/VP7x6wjToC9e6MXsni98Mxz5v9dV4pQYjTqEqh4SYcdmaI0aXmQNQzeva3t0Wt7fG7F4S09M8kg2XvMCdHdJJATXVKcb4NtU1i6YBET0qu7dA3TWHmH5O+1NrwQsnMiBwAD+ptArSPDhsJ37+r4eVdeBi/+O3Igd8Wlsa034vubHnNZFWs46cU1LXebOa291zw4YflwsbjwAlj7Prz3QfTnzDu9bXVxazYb7CyHaVMSu74Qk2+CxnJzLIoCZQFLOkz4Inxwj38nLipNm064Fk12F38O6IpkbUsiRE+QQE50WXG+jeLqGayr69o3mIQ1Vu5rQaBScPft8L37zbFn4PjNbofv3B5TvlSYnbvgky2mMGLuaab6NWDYUPjJ/fDAQ8HAMSMdvn+PeSxOrY8sW/ebC9UdgV3oLNiE58N1xGY1gXJ7gVxBf5g+zezCRuPxQF5u4tcXwpoOpz8MdaVQXwrpBVD5Gmx5xBQzRGXTKEz1atj17HDi1d254sgWpmWzpBY2lDYkRd6cED1BAjkRt642WQ5trFx3rKJL1+izzZRnnQJ/egSeXWoqWMeOgc9dY45CY+Xxwv0/gQ8/Mvl1Niv8SsFDPzatMgJOmQlLl/jbnABjT4w+xD1e/hw6Nu6kf0g135EEF0cEcuAmhUxnSNu4vecLdHbsbP/x0+fA/v3R56wqBScM7/oxdyfljYMjn8Kmh8FxAKLOnMKM4Rp9OQw7Az76oX8eK2Y3b+bdvVe1Knlz4ngjgZzoVYHGykuz2vuxPzrnzAZ/i5U+OPt29Ci4+46uv/75F0wQF5gSEDjBu+f78MKzpgAiwGqF8eO6/l6dEQjmtgVHWeU7DxIojmjOiT9oCWvsG9jp7Y1+ih0FxGlpcOd3I/f9s1ph0ED4yQ+7ZWmRbPgp1LxjWo+0y6KxZcH4hWYm64XL4EgJaC/0nwSWXv7OInlz4ngigZzodXE1Vm7MNv33xiyD8p1QsSvml2Zj63s7eaGWvxQM4kJpDe9/CGfGN5EkLq0CqkBxRB12oJMzTiNI6+5ChlidNtv0iovEZoWt29rOVg3olwVLnuy+3dFWGiugelV4j7hIlE0zfAFM+rIJ4sDMZB0wufvX2Bmhs1klb070ZRLIiZS3qLGQJa4bWDG9IebX5OUcDI5FS8ZgrqnZTHXYuMkUN1x2sTli64xjxyLf7/O1bQzc2/zFEYO2daHlSSS9FcR5PLDmPfhwHeTlwUXnm+Pw3a16OFqtphFwaK/A1pyuHgviwEx36GiKlDVDc/K9UDi/R5YUt5YiCH/enARzoi+SQE70CQvTsjs3Z7KxsGUs2pRyM8oskl4J8moPw9e/aYIth8PkT/1rmZmykJYGZ50B111rihfac9J0UzXZOlDwadNTLhn19g5aPJodZhrH3r3m91arGZf2pS/Aspfg8BHz/8JqNWPQvnWzmZMbrfVI655/HXBXrKHcW8++w11r+2HPNvlt0VgzNQNPgmFzu3T5XhU6m1WKIERfI4GcOG4FxqKtmXuQ7Ny2WxHF5WuYsL0Xcu/+8Jj5ph/4Bh9Igj9Ua359fhn892144k+Q3S/6dW78Mqz/2ASDgWAuIx3mzzP9y0Ri/eOfZuctcFTq9ZrbE38zfQG1NruhNpspYrDZzFivs86AVWvCj8HT0+HrX435resrVrLb6WqZtbqwCwVIQ0+P8oBFkzMSJi4yhQ0qxnmqzQegfoc5fs0b3+nlJJzkzYm+SgI5cVxb1FjIusrIfdmWjxjIJf7cu8EVBzu8VsKqId95N/ouDZhWFUfq4MX/wPWfi/48peDmG+EfS6Gmxtw3ZjR85YbErFOEe21l5Hw3rzd8hrLLBdtL4V8vwuevhe/cAePGwtLl0NhoduJu+gpMnBDT27or1lAybTTPqCtZVF0IXRyPa8uC2T+BD75reslpj9mhG1QMxffHXsDg88CGB6FmtRnxpb3QbzjM+UXPjOxqT3fkzWk0O6lhl6rBjYf+ZDNZjyK/q3m/QnSS0tHyM/qYyRNn6SWPr+vtZYgUs8R1lAkjVjN0QPvJQ4Mryxm+eUdidu/Ou7T9vmIBEyeYFiWtrX0Pfv4wNDS2PVZVyhzJPvk49O/id3wR2dULobY29ucXDjPFDHFyV6zhtYtPY/WmM0yKQbzXO2YqV92NUDAD8lvFk9WrofRp054kr8js1PWfGHx82+Owa2mrSRBKkzsWzo5S99EblriO4ixo4OQJlriCufWqjH0cxqv8s8s0WLFwup5CfwnmRBxmWgvWa607HIUjO3JCtCPQHmV1B+1RnAUzWnbv4p5UcdqpJhjraOB6doRvEn/6Czy3NHoCvdamCGLpMrhxUdfXKNo6ax68uCJyP7hIjiZZwYmfvR+MvBD2rYVNv4Lm/ZB7Ikz8MhzaBKVPBduTHPhAU7sRTvs5DDzJ/PUqXxZhnJdWNOzS1O8wveqSQSLy5o7STA21+FTIvzcFXnxsYzdzdfdN4xAiQAI5ISLQ2hwJWWwxtkdpzGaJ6wb2hE6q6OpR6y1fN5MYHM3RxzdlZJgRWhs+hrdWg9ViEuiXLosexAV4PKa/nARyifXZa+D1N6Gpqf2j8YARPTjDqpN2LYVtfw4GbAfXa2o3g/aB9oQEaVrhdcInv9EseNIcx3qaIl8TH+x9I3kCOWibNxcqlsDuCEdRZrZFm8fqSM5AXfQ9EsgJEcJ9zIwl2vuGyfXJGwfTboWCGIo8A7t3T88fHl9rkyGD4am/wH9eNoGawwElZaYVhc9ngrbzFpiCh/c/NI8rZR6PJYAAOVYN8Hhgxavw75fB7YIz58O1n4GcTh6Jle2E2+8Ct6fjndSAyy7u/Hp7gMcRHsQZCp8LUJF/SGjcDV4XWNPA1g88ETvfKA59nHypPIG8OTYGA7eyrNgKItLb+RZqJ8aqECHiJIGcEH5aw9rboKEctNt8E6svg/fu0Mx7tG2uUCTF+TbYNoWlCxZxdu1aJmwPb20Sc2CXlwvXX2duAAcOwjtrTe7c7GKoqoYHfhacCKB17EGcxQJXXxnbc/syreGe+2DzluCc2Wf/afr3/eUP0K+diuBQPh98977O9eYbMtj0kUtCDTvbaUMSJQ6zWM0NTF7d/nc1EZvSJV8cB7QdM1hMNktiKIgYSD5WLHi1L+yPa9UWxuj4ZxULEQsJ5ITwO7QRjlYGg7gArwtKnoTZP43tOsX5NoqrZ7A4ZxB5c4PVrvOq13R9lNjgQeHB11+ejDzWKRYLzjLB4PFuw8ewZWswiANwuU0fv18/ambcHjwIJ55oKn2nRcl32l5qGvvG6rRT4a7bzWSHJGTPBl97PxMoDTr4b0TZzaSHQFuS8dfBwXW0zF4NsKRrTjgn8evtLgvTgo2Eo+XQWVDM0ZN5T23Dp31oTBXrEPozlk428BaiiySQE8KvvqztNx8AtOLI9s5vJSxqLITGwpaPF+cMahklNjieHDpoP6E+cAQbdp8y2yzXXAX/e2PX37cvef8D07i3NZcL/vtWMNfw401w5z3wo+9HDoCbmszntyNpaXDXtxO6E+euWMMBGrvcBDiSnFFgTQefs/WumiZ7DLiOgM+l8bpNDmn2CJN+EDBgOgydB/vf0y3Hs5Z0Tb9hMOqyhC2zRwQKIrZwgOU4GBNSvR7YpcujH+frUzhIPU7cDCCHbDJ7a8niOCSBnBB+mYPAao8cIwVmSsYjMEosWBCxpuvB3LkLYNPmtrtyGRmmN9nOXWbM1+xTIT/PPHbKTBjQv3Pv43ab/K80O4w90eTi9RWZWWbKQqQj6dYFI04n/O4P8Mzi8Pura2Dbp+G7epHYbPDAfXBq4nZCA5McVs89n9LK+QlpPQLgagRvM7Q9GlU4D2nO+6fZcWveD3kTzFFq6F8LpWDWfVD9NlT8W+N1wPBzYNQlYEvB+Ca0IGJryP3lRcEcOgsWhtDJf1tCJIgEckL4DZ0Lyg40h+9EWDM0469PzHtELIhIHxl8QqxHrmfPh3+vgNIdwWAuI8OM5friwsTM6Fz5JvzmEZPX5POZvL0f3wcTQtr019XB++tMOePsUzsfKPam8xbAc8/Hnlu4b7/ZfcvKMh8//Sw89Yz53HRUKWy3Q1oCpwjUlHCARp6e3/VJDtE0loMlLfLutLsRXr8GZtwBYz8b/RrKAsMXmFtf0LpyfV2dh7ISE8wxQOa3it4lgZwQftZ0mPcbeP8ecDVolMV8M5vwhcQOCQ8URARmveZnm6Ax/+NShlTsia0gwmaDh38Gb7xlkvMtFrjwfDPuKRFB3NZP4Ze/CR8b5XDAt78Dz/3djAZ7cQX8/o9mV0sDvkfMWLBrPxP/+/eEESeYCQqPPWGCMZ/P/Fk87shJ+UqZgAygtMwEcZEmOURTezghyw44NrOI+sZBfCaBQRxAxsAoKQYAKDzH4OOfabKGwICpCX3rlBEI7GIpiBCiu0kgJ0SI3LFw3nNQtx3cR6H/JJP8nWjF+TaK/Uet+L+/O2c2cEnjstgLImw2uPA8c0u0Z5+PMm7KA/99E6ZOMTNhXW4g5Lv+X56EqZNhkr/Vf7MD3n0P6htgxjRzPNtbdu4yc2qrqmDKZFM8cvWVcPppsOod82eZOB6+830iRnKTikwg98lmuPeHnQvivF7T5y9B3M6D1HmGRCwMjVe/QsgvgiOf6vCecSG8Lih9Bk570MxUdR6B7JGpeXQaj0Q0FRYiXhLICdGKUiaA6wlheU2N2SzOuYo9p30cf1PheO2tinxc6HCavLDNWyMHMi6XmQE7aaIJeO6+z1zH4zHBjNYmAJ0/F77xdXMU6/HC4VozqSJwbBkPrc06M9KDyVur3oGf/sLk/Pl8ptL0pRXw6MNw4hi47lrzvOdfMOuLNCLN4zWVrHfd23FOXGsTxsEJialirK9YyW6ni//kfob0ilxIS8hlw5z6ALz/Hagr0+CN1EZE0VihWfMtOPKpf6aqB8Z/ASZ8sW+lUnYkkEMXKIgIJYGd6AkSyAmRRBY1FrKkNhfmAvEWRMRjYhHs3tO2+jUzAzZ+Ajt2Rg70tIYNm2DfPhPENUVo8+92w9vvmGDwus/C4qdMAOh2mwjA64XxY+Hmr8HMGbGvWWsz2eLv/4BjR01Q+PnPwTVXws9/HX5M7Hab28O/g0d/Hby/sTH6nNtjx0xenKudObiRKoYBdlWYP2NaHFFXTQn1zj3sdrpYusDkxrXuf9ZVWsPB9bB7hZnMUHgmzHsEdv0Ltv9V43O1isyUxtVgduO0x98sGCh7RpMxAEZdmpBlpYxAMEdl8L5YmwoLEa8EJNMIIRJpYVo2uuR8np6/iE+d9dRXrOz5RXz+WrMzFcpiAZTZlWpvesGhQ/DVm00BRDReLxypM8ezjY0myPL5goUHZTvh7u+bXb1YPfMsPPEkNDSA12ca9D71tAnWohUjbNsePgZt5gwTrLamFMw62eTGRfuzK9X+52Xz1uiPxWj/zLE8UXwXn6mekbAgDsw0kw/vheo34cD7is2/hXduhpGXQFoeYAn//FnsJo+u9dGr16Eo/XvClpVSivNtYbfxTRk0leRQflizqbmTO7hCdIIEckIkoeJ8G5+pnsETxXfxwZhRVFW8AjUlPfPm+w+YHDBF8IxMASf7p6JH27EK8PlMbly0ObEBbnf7/fCcTnjsr7CrHP7vXrjwcrjqc/DE39oe67pc8MxzbY88HU548+32g6/QHnAnzYCiCW0LRrQ2x7PRKAVzZkcvNFHEPrqrh9XvgN3/CYzkMp8Lr0NxdA9UZWmjYwAAIABJREFUvgLz/wADZ4KyaSx2Tb8TNOM+F5zk0JrjUM+tPZkV59tYmJZNU0kOG0p9EsyJbiOBnBBJbFFjIStyruKV086ltGkH7oo1LbduCey0NjthVdUmEAvsZNnTTC5ZrNMkvN7EBC5lO+Abt8G6j0xQdqQOnlsK3/1B+PMOHiLq/CerLVhtGspiMcFp6GNKRW/fUnvY5A5Gkt0Pvnc3FJ8SOUHM54Pp8ZV41jv3UNUcY6uUTqh5B7wRYnOvU1H5OmQOhrkPw0UvwflL4ZynTVsRHWUpWYWR7z9eLUzLZmrlYDaU+lhe1cVpLEK0QwI5IZLcosZCSivns3ru+bx28Wktt0Bgl1C7yqFmX9sgzOUyg+VHjoj9Wvl5puAgHkq13WVzucxore0hgWz//Oj94Nxu+OzVJj/N6t9GslohKxNu/1bb53+wLvq1Qo9oLRbTu69wGPzmF+Z6t34DcrKDuXAWC6Snw53fNr92RU0JVRWv8MGYUSxPu5KplYO7dp0olCV6cULo6bi9H6Tnm+fmngj5k8BiDw+eremaSTI4pI3ifBtTKwfTVJLD8iqH7M6JhJJATogUEMibW73pjOBt7vmUexOcQ3f4CFijfFlwOMx4r/QYEvYzMuCWr8PXbzT92rIyg4FPhj8HzdrBrNH0dPOaSPltXl94zllWFpx1ZuRiAp8Pnv2nOcYNBKher6lCfeJvba+fnhZbLz6l4JcPmWkPgbYqhcPgqSfgC583kzQuvgD++Fs456yOrxdFvXMPby04jxU5V7GosTChuXEAw+aDinhJTfM+cNRGft1pD0LhWSaYs6Rp0vI107+d2J6LfYnkzYnuIlWrQqSINt3lW0+IiKWRcEfGjY2eAze80OSB/eR+00S3YreZ9nDSdFi1JlixabHAgjPhnLNNsHOlf8BmUxN8/Ak8+bSpeo2066WUCfQ8HrjgXPhki3mf1my2tlMk7viWeY8P14HNHqyY9fngWITqWYcD1rwL739o/lwB55xtjm87mvhgtcKQQW23s/Lz4H8WAgvbf30nVKcPNztx+Ym5ntaw/z2oeAncx8yorSObW89WVXidmu2L4aQ7g/d6nVD5OlSvAnsWzLrf9J3LGGB290R0rRsJlw8MHrVKdavoKqU7Gi3TR0yeOEsveXxdby9DiIRbnFPtbyS8M7ZGwh35zaPwyuvh7TrS0+AH95rmuZEcqYPVa8xrTp0Fo0dFfp7HAxdcZnbUWsvMgP+9Cfr3N813BxbAy6+ZGaetc/OysuCFfwR390IdOABvvWPamsSS0ze8EM4+04zsGuUfl/bgL+C1N9p/3cACeP6Z7m2aVlNC6WQ7y4fOof/7kxO2G7fpYah8jZah9sqm0R6I1GE4LV9z0Yvm955mWH0zNFUHXquxZpg5qtMinFKL6NbVBQt9yrIcOAsaOHmCRYI50WKmtWC91npWR8+THTkhUtwi/4SIPaetTkwj4W/dDMOGml2p+gYYNQK+9lWY3c7A9/75cEUMzcO0jlqTgLLA6JEwfVrwvovONy0/Xn7VFC0oZXb8fvZjE8RV18Abb0JTs1nfSdNh8GDIzYl96kFVNfzjn/D8v+BLX4TPf9ZMfnhrdfTpDenp8O1vdnsQV+XcRUXdWKq8Azk/QUFc/U6ofNUUMwSYNiKR/8dYQmpBdr0Ax6rA1/JahdcBu/+jGXWpyZ0TsQkNyouRCRGi6ySQE6IPWJiWzZLK+S2NhIfEc9RqscDnrjG3rvB4zfzXl181O3DnLoBLLzLBj91uRnht3to2N02p4Giv0Ptuu8VMXvhkiykkmHWyuc6L/4HfPxbsP/fiv00Q+JP7TUNhXydOG7xec1v8d5h3Opw4OnqeXL9+8OCP4q5CbY+7Yg3l3npWzz2f0sr5LGpM3Jy4/e+CL2LXl7ZBqSVNM/Ki4Md7V4YGcUE+D9SslUAuHoGmwmW10khYdI4EckL0EQvTstvkzQVkkdkzEyJ8PvjufaaRb6DadFeFOar9/W/MEe0dt8LNt4VPc7Db4c5bI7cJARg6xNwC9h8wQVzojlmzw+TgrXjF7A5OngRbt3V+Juqbq0yO26gRporXHRL1pKfDj+/r1iAOoIlmll/6ZQ59cmL4GLcEUDZMmVukFEClUTbQboUlXdNvGIwLSfVrLwfueBrL1V2K821Ql0FZCZQXNcKA8IIICexEJJKaKkQfEtpI+Fdzr+exi77AYxd9ITghorubCq9bD5u3hLcMcTph71543Z9zNmokPPVnUwQRegT660fh1ddje59V70SuZnU6TZsUgAfvhwvPM5WsFovJhbvuGrOjlxllurvPZwI/peBXP4Mz5oLdZm6DB5lecSefFNsaE2B8U+KP2ArPjBx0WdI1E/4HhswxOXMATfvg7UWmaTDAyAtNi5HWlNVUv4r4hTYS3vpedstNmgqLaJJmR04pNQB4DhgNVACf1VofifA8LxCY27NHa315T61RiFSxqLGQdSH9xp4ovqulIGJwxcGW++3pg+Ivjgi1eo3ZGWstMGHhsovNxwMGwMZNJrfNp03w5HKZYC43N3pRRYDTGb2qNFCkkZFh+sTddovZ+Qvt4/bMs/C3Z9ru1qWnBd87ux/c911zPYfTH3R2/7aTu2IN+70ujjZo+nf89E7rVwgTvwwli7VpBOxTWDM1+eNh0EzY+azJmdP+jcimGs3aW+G852H0FVD1NjTs0nibFSiNNQ3GfR5yRnbDYo9jrXdil9TChtIGmCC7dCJc0gRywN3Af7XWDyml7vZ//J0Iz2vWWvfcj8RCxMFZB55jkDXU7Fr0pLBk6pCCiKEDTDAyuLKc4Zt3MDje4ohQgd5vkaY6hAZSW7eZKQmtR3Q5naa3W0eB3OxiM8De2WqHwm6HM88Ivy/QlDfUVZfDqytNhWtglFhGhnnfya3y9NLTu97Mt5NCc+McFWMT3jMuYPznYcipsOc18BzTDJ0LQ2bDB98z7UXCKbxuTfWbMOpSmPc72LcWqldr7P1g5EXQf1K3LFOEWJhmCiK21joYMtL8G27MbKAcKY443iVTIHcFcJb/938D3iZyICdE0nPUwvoH4PBmE8BZ02HqN2FEAlq9dVWgIKK0MnDPGUxoqXRdk5hg7vxzYMWrbQOsjAy45MLgx3v2Rr9GdXXH7zNhPJw5D1avDbYYSUsz1bOf/UzHr8/KgsceMcewb62CjEyzW3j2/F5L9goEcU/PX8SIbVNY2E1BnPaCqxGyR8HUm8MfO7YXIhU9+JyK+p3mSNViM8ezhWd2y/JEOwI5dGw3H+cyiC0jDrAchxRHHMeSKZAborWuAdBa1yilos2hyVBKfQR4gIe01st7bIVCxED7YM2tpteW9ipwg9cBm36hSc+Hwe108ehurY9rEt5UeGIRXHMVLF1mjjN9PhPEnTHXVIMGnFAYvT3I0KGxvdfdd8LsU021alMTzJ8HV11hKltjkZUVX3VuNyi97AIcm7pnJ05rKP8XbH/S/H1UVhh9OUz+mgnOAPLGw9E9rRsDA2jqujm9UsSmzd8Nf6VroDhCgrnjT48GckqpN4BIX6Xv7cRlRmqtq5VSJwJvKqU2a613Rnm/m4CbAIYNkQQO0TMOrgfHIX8QF8LrVGx/UvdqINdacb4Ntk1h6YJFnF27lgnbX4m/qfCNi8xkhzffNq1I5s81FaShO13TpsKQIVC5NzzXLSMdvvSF2N7HYjGjr+IYf3U8KV8G2/4cbAKMGype1Lga4eS7zV2DToWq/0Z6taKuRKO1VKcmm9bTIpggwdzxpkcDOa31udEeU0rtV0oN8+/GDQMORLlGtf/XXUqpt4GZQMRATmv9OPA4mMkOcS5fiJgcrTTHV5Ecq4x8f6ie/mZZnG+juHoGi3MGccmYZVC+k+yKPS2P56WP7HxgN/bE4PzRSJSCh38GP34QtmwzI7csCm78ctsctz7KXbGGJppbPt7tdPFhfaGpVI1hnG1naB9sXxwSxPl5nYqq/2qmfA3S+0NmAVgzzI5dm2t4MT2DJZBLSgvTslsKIsoHmqPWAAns+rZkOlp9CbgBeMj/64utn6CU6g80aa2dSqmBmPanP+/RVQrRgewR0Qsb+g2PfL/2Qdk/YOdz4KpXZA3TTP4aDD+7+9bZWmBCxJq5wZ+LRnqrEjMtIpIB/eHXP4faw9DYaNqDROsj18fUV6xkt9PF0gWLwu4fsW1Ktxyreo6BJ8K4WTCTG45WmkBuQLT2eEozcKbMUk12gYKIsloHW/33Of0zXaUgou9KpkDuIeCfSqmvAHuAawGUUrOAr2utvwpMAh5TSvkwPfAe0lpv660FCxHJoFPMAPGmfTrseNWarpm4KPJrtjwCu18O7pg01Sg2PqjxuWHE+T2xamNhWjZUz2j5eIlrbMu0iIQVRLRWMMDc+jB3xZqW3x+gka1jxrIi5yqmbhscHrjld8/727JMwOaNMNHB5zZV1QC2TDMzdfNvNV4XoBXKprGmw7Rbu2dtIrECR60BgcBOpkX0XUpHaqrZB02eOEsveXxdby9DHCeaD8H6H8GRbaaTvsUGU79B2LijAFc9vHY1+Nxtz6wyBmrOX9q7eUnr6jxUTt5qCiLS0+IviDie+Oellk4cR1XhGAAcTZrNTSexqLGwR5ey7XHYtTR8xqqyawbNhDm/CH/u4a2w83loqoGC6TD2WsiMVn4mkt66Og9lWQ6yiiSYSyUzrQXrtdazOnpeMu3ICdFnZA40/bYch/195IYFKwNbaygHS5rZGWnNeQQ8zWDP6t71tidQEBFsKuwviIgkkc2FU0WUaRlu58GweanjNwaPthZ1U2uR9kz8MriPwp5XNBa7+fs28GSYdV/b5w6YYm6ib5CCiL5NAjkhulHGAKCDU8OMgdGGmJvgz5YEX2+L820UNxayOOcqGLOMquzIW4RFm1d2rTgiRbkr1nCARqqmjYvw6BA+6Teb0sr55sg6wQUMnWWxwYzbYdJXYc/LUL0aju6F9T8xO8XDTu/5ptWiZwVy6DZwgPKBwYoW2aVLbRLICdHLsk+AvHGY9g6eYIBkSdeMuiy5vrkGCiI43PYxZ0EDl4wxY8CG19Dng7nQKQyllZEHjY7fm9FtjX2j8Tig5h1wHIS8CTDo5PAihfJlULYkJB9zr2b/WrDnwqk/hoEyN6dPK863QeVgCKmg3zDigBREpDDJkRMiCTiPwPvfgcbdJnDzuWHI6XDKvWDt5Z2czljiOsqEEWZaRD9rMHoYTE73FErEo6aEeucejhJlO7QDtU4fSxcs6rZK066oL4O13zY7vD6XObLvNxzm/Rbs2WZk3OvXRM7HBLBmaM5dAhkFPbxw0atCc+hCSWDXu2LNkZNATogk0rATmvZD7onBSsJUs8R1FGdBQ8vH07I+Ni1MrHnJE8z5ixAC1aNdNbVycNIEcdoHr38WHAfDgzRl15xwrmn6W70KNj4EnqbIgZwlTTPhi1D0Pz2xYpFM1tWF/0AjxRG9T4odhEhBuWPNLZUtTMuGxtD2B4PDxoAlg/1eV8uR6KLGGEd6RdJN7UK64sin4Dna9n7tVlS9oZn5HTPztz0+l+Jo5fHxw70I1+YHkroMyqQ4IiVIICeE6FahVa/Dhx/q7eUAcLRB46gY22b2bCrzNGG6a0bg8wA+0+Mw2nPAHK1KtaqAkEpX/7QICeaSlwRyQohuF6h6Xfd+cjQj60+EHYgU139S9Orn/Akm91JZYfYD8N7dGl9L0aI5ZlUWjS0TTujBBtQi+bWudJXRX8mnb30lE0Iktb4WPCUTezZMvAFKntLBmaqq7VSGgTPhgudhxz+h8lVw1GqUMvef9H+927NQJKdApWtZrYNDI83frcbMBspxSA5dEpCvqkII0UeMvx6yR0PZMxrHQcifCBO/1DbvMi0XJn/V3LxOQKVWdbToecX5NqjLgO3m41wGUZbloLyoEQbIsWtvkkBOCCH6kGFzzS1WHRVACBHQekc9dFqENBjuPRLICSFEBJ5mqFwJtZsgawiMugz6DevtVZlpDHXbIWMQFEwLb/YrRE8L5NAFGgyXZTmk0rWHSSAnRAryusDnBFs2qMgtwUQcHLWw6iYzm9TrUCibZtdSmPUDGNqJ3a5E8rnho/th/wdm3JbWkJ4Hp/8a+hX2zpqEgPCdumLCiyOkqXD3k5/lhEgh7qPw0Y/g5YvhlSth5XWw793eXlVieV2w/32oetNMvOgNmx8x7x0oGtAehdepWP+AP6esF3z8S6hZa3q9eZoU3mZF0z54+yuw7z3TEFiIZFCcb2Nq5WCaSnJYXuVgU3Mv/aM5TsiOnBApQmt49w6o32mavAI074OPfqg57Wem6jDVHdwAH34P8Pek9Xlg3Odh0pd7dh371oD2Rt7qrP0EBhe3/3pXgzmWPVYF/Yug8Kz4ctFqN5kK00CrkCCFpwnW/UAzcDrMfsjs1gnR2wLFEWUltBREhJJj18SRf/JCpIgjW6GxIhjEBXidik+f0JzxaO+sK1FcDfDBPcFdsICdz2nyizqXwB+vqLtbCrS3/dce3gbv3QE+L/icij2Zmm1/hvl/hMxBXVvPJ7/zv3kUPqeidrNm7xsw8sKuvYcQidbSVLgEtobcnyWVrgklR6tCpIj6nWZXLpLG8p5dS3eoejPy/V6HYudzPbuWwacClrafbO2FghnRX6d9sO77Zpapz2kCL2+zwlELm37ZtbX4PGYGb0e8DsXuFV17DyG608K07LBbU0kOG0p9cuSaILIjJ0SKyBwcfTcoMzkGJsTF5KRFfsxR27NrmfYtOPw18Do0PpcCi8Zqh+m3gy0z+uvqd4D7WIQHfIoD6zReV+f7tSmrOS71uTt+ro4y2UGIZNJ6WkSAtC3pGgnkhEgR+96N/I1a2TQTvtjz60m0/pPBmgne5vD7lVUz8KSeXUu/Qjjn71DxIhzcoMkaBmOvhrzx7b/O52qnilh3rSBBKTjhPNi7UuNzRz9etaZrTrig89cXojcEpkUE2pYAbBghla5dIYGcECnAeSRasjvYsmD4gviu72qAvSuhaZ+Z2TnsDLDY47tmKK8Ltj8BFf82w93zJsC0b4QfUw4uhuwR0FgeErD4R0yN/0Li1hKr9HwousHcYpU3IfpjuePAcQiaaiBnVOd2Uad+Exp2QeNubXbmdGB31nyerBma7JEw6qLYrylEb2szss8/Bmw5jbI71wkSyAmRAup3mMDK52r7mLvR5FF1tVrx8BZ4707w+UzSvDVTs+1xk5yf3j++dQd8+D049DEteWP1JfDe/2lO/zUMmGKeoyww77fw6RNQ+ao5hhx0Mkz53+RoxBsLaxpM/zZs+qVZP1qhrBplBzS89eXgMemweTDzntiOWu1ZMP9PpmK2vtQ0Awb/58kJJ5xrbjKlQaSy0EpXaSocO6WjZU/3MZMnztJLHl/X28sQoksadsLqm9tWdAJYMzWXvGKO4I7uhbIlpsI1qxDGX9dBcr4XXrsanEfCr6tsmmHzYMbtUPEfOLINckbC6Mshq5NBVcNOWP2/prq2tYEzNXN/07nrJSOvEw5tNL8OPAmOVcOOf8DRKrPD2VgOR0rCK44t6ZpRl8D0W9u5sBDHqSWuozgLGjh5QnhN5vEU2M20FqzXWs/q6HmyIydECsgdC/2GQ2OFDutvZknXjLnCBHFHtsPa2/wNa32KxgrNofUw7VYYdUnk6x75NHKDW+1R1LyjOfCR2T3yORX77ZpdL8DsB81OWayObCdq54y6ktiv0xs8TaYJc/qA6DueB9bBuh8Q1vtu4peg+Efm4+YD8Mb1bdvG+JyKPSs0U29O7DG2EH1BoCBia22wGMLpn+kqOXThJJATIkWc9pA5Am06oFEWE2ANmQOTvmoe/+TXptVFkMLrhM2P6KjHbl4XUYMs7QXPUVqeoN0KrxvW/0hzwQuxz/jMKIj+3LT82K7R0zxN/kkKq83aLWkw6UYYc0X485xHzLFx653Skqc0eRNM3l/zwejH4j6vqXJNT9LPgxC9KdCHLmBdZQZbOMByHIwZEPw3dzzt0kUigZwQKSJzMJz9NzMwvfkg5I0Lztj0uqCuNPLrlMXsvEWq/Ow/OVpLk0DKRdsoz+MwOXv57ST2hxo0C6wZ4GnWoIPXs2Zoxn0utmv0tA/uhcObaSm68Dph6x80tn4w4tzg8ypXRq5E9ToUO57VZBSALcMfMEfQUXNhIURQoNK1rNbBoZHBryXLMxuO6+IIaQgsRApRyuRcFc4PH5SuLNF3vbQvehK8LQOm3mJaVwSCN2XVWDOI+tVBqc610bDYzGD3zEFgy9TY+mksds3Ii2H0FR2/vqc1VsDhrbRp9eF1KLY/Ef5c5yEz+zSSgxtMXuOqrwcKGtrmIyuLaXEihIhNcb6NhWnZ5G5Pb7k1leRQflgftw2GZUdOiD7AYoOhc2Dfu7rNjFBbFuQXRX/t6MtMIcOOZ+FYjaZgKoxbCOsfgCPbwnfRwBwT5nfQT6213NFw3nMmQHLVm2A0o6Dt89xNZsfRnm16tkXtydaNGveAxQqRYtWmGvOruxH2vAK1WwCrhjZzWTX4VEtPPJ9LE2l3U3sV+9bqTrU4EUKEty4JjAE7XitdJZAToo+Yfrs58nTWabzNCmuGyaWb/ZOO89kKZrStbj3p/+Cdb4DPbaYbKJvGYoVTvmemDXSWskDBtOiPlz0LJX8FZTM7fun5prAid0zn3yse/YZH33HMGGgqUld/3UyhMJW4gd3MQKAWKWhTUe6HtLwELVyI41jraRHHU0GEBHJC9BEZA8w0gpp3oK5EkzXU9Baz53Tterlj4JynzdHfkW2m4eyYqyD7hMSuG6BmLZQs9gdG/tORpmbN2lvh/OfDj4a1F3Y8D7uWgrsB8opgytdgwNTErCVvLOSMgfoyjfaE5/SNvQY+vM80UA7uVAYDOGU3u4iRChsisWZoxlyVmHULcbwL5NAFCiIC+nr+nPSRE0L0utX/C0e2RZhakak56a7wyRUbHoTqt8L70lnTNac/nLhgztUAH90PtZv8Fadecxx8ZFsgSIs0YUMz55ew9U9w+JO2z7GkabTPHINr/ybemKtMw+PeOEIWoq9aVxc+y7Asy0FWUepNi5A+ckKIlNG8P/L9HqfpwxbQVANVb7YtMPA6FVv+oJn/h8SsJy0XTv8VOGpNi5G6Mtj8m+iFDQF7V5pgrzVlNRWsZ/we9r9nqliHzDbHuEKIxGoz+ss/LaK8qBEGhBdEpFJgF40EckKIhNJeM9HAlhH7PNG8InC8GyGHTENWSLBz5FOTQ0eEo8v6KO1X4pFRYG4bfhJ5qkYorWHPy4Qdx/ofIW88zP6pudaoSxO/TiFEdIF+dEtKYGvI/c4+UhwhgZwQIiGa9sP2J6DqbcxQd23y7Gb9oOO8uqIvwf53IzygoWYVFJ5hPkzLj9q/GHt2lAeicDdBQxnY80xVbXscte08qDTWNHOse3B9pMch98TIVbpCiJ6zMC38i8SSWthQmvrBnARyQoi47XkFNv3KTJsIDbXqSzVrbjGtRyxpUPmamUHqqDUtUSZ91eSeocGSDr42baAU1W9pTr7HVMoOnGHaqXiawnfvLOmRiwa8Ltj7BlT917z/qEth6OlQ+jSU/t2fr+Y182Nn/zS8N1+o/CI48GGEHUOLZuhcmPxV2PZnwBchzNQK55HjIxdZiFTSVypdJZATQsTFUQubHm7bQNdQeBya6lVmcPyuF4JHlAc/0hzeDHN+YYoJLLZIgZx5TPtMIKesMOeX8O7t4HVotDaB2NA5MP768Nd5nbDmm9C4O/iehzaYY866UjPrNPB+jRWmQva8ZyO3Vpn4Faj9xLQcCbDYNflFcOqPTbHC0NPNn6n1Eaw1QzPk9Fg/m0KInhQ6LWI5qVcQARLICSHiVP12+497mxV1ZZqKZa2DveAs2Lm/A+2J/PrcseFD5XPHwAVLzeQE52HoPyXy0e3uV8KDODC/P7ylbZNjtMJ9VHNwPQw+te21+k80Aefm32nqd5hJDSMuDK84PeFcs9vYtE+3/Dktdk3GQBhxfvufIyFE7ynOt7UURGwoaKB8YGq1LpFATggRF6+z/Zmh1kyNxT943hy9hqvfYQojir4MJYtDdrT8uWfTb2v7GmU1A+nbs/f1KAUKUU45tRea9kW/XsF0OOsv/mbBqm3LEGs6zP+TObbd+4Z5k+HnQNEXzJ9PCJG8AgUR6yozoNLcV5blSIlpERLICSHiMrgYSp4Eb8RgTmPLhCFzoDzKTFFrupn6MP46yBkBpc9omg+YvLSJXzKjurrC0tmvbsrs/nX4tHamZNizYcrXzU0IkXrajP7yF0SUD3QwZkDwp7dkCuwkkBNCxCVvPBSeBdWrQvPDzI5UXhEU3w9ZQ0xvtmZH+LGmJU0z8qLgtYbONbdEGHmxmXDRelfOYgeNRocc81rsmtwTof/kxLy3EKJvCBRElNU6WlqXOP3Hr8lSHCGBnBAibjPvNjtzu5ZrPEdh4EwztSBnVPA5p/0c3r0NvE6Nz2uOJvMnwuRu2r064VyoegNqt5jZs4Gj2tFXwqCT4ZPfapr3mx224efAtFtlwoIQoq3AsWtAILBLluIIGdElhOgxPg/s/wAcB83Raf9J3ft+2gcHPoSqt8wR7ogLYMCU4OOeJpO71+ljWCHEcW1dnafbR3/JiC4hRNKx2GBYgo5OY6EsMOQ0c4vEltVzaxFC9B2h0yJ6uyBCAjkhhBBCiC5o3VQ4oCePXCWQE0IIIYTookBT4UDbEoANI3puWoQEckIIIYQQcQhtWwKETYsI1R2BnQRyQgghhBAJFJgWwcZg4FaW1T2VrhLICSGEEEIkWJtdupAxYIksjminR7kQQgghhEiE4nwbC9OySa/NZUOpj03NzoRcN2kCOaXUtUqprUopn1Iqat8UpdSFSqkSpdQOpdTdPblGIYQQQoh4LEzLZmrlYDaU+lhe5WBTs7Pl1hXJdLSSpK06AAALRElEQVS6BfgM8Fi0JyilrMDvgfOAvcA6pdRLWuttPbNEIYQQQoj4BCpdy2odHBppRso0ZjZQjqPTOXRJE8hprT8FUO3PyDkV2KG13uV/7rPAFYAEckIIIYRIGS0FEdvNx7kMoizLQXlRIwyIfXcuaQK5GA0nrFMLe4HZvbQWIYQQQogua10QETYtIkY9Gsgppd4AhkZ46F6t9YuxXCLCfVGHxSqlbgJuAhg2ZGRMaxRCCCGE6C2BaRGx6tFATmt9bpyX2AuMCPn4BKC6nfd7HHgcYPLEWVEDPiGEEEKIZNGmdUk7kqZqNUbrgPFKqTFKqTTgOuClXl6TEEIIIUSvSJpATil1lVJqLzAHWKGUes1/f6FS6mUArbUHuAV4DfgU+KfWemtvrVkIIYQQojclTbGD1noZsCzC/dXAxSEfvwy83INLE0IIIYRISkmzIyeEEEIIITpHAjkhhBBCiBQlgZwQQgghRIqSQE4IIYQQIkVJICeEEEIIkaIkkBNCCCGESFESyAkhhBBCpCgJ5IQQQgghUpQEckIIIYQQKUoCOSGEEEKIFCWBnBBCCCFEipJATgghhBAiRUkgJ4QQQgiRoiSQE0IIIYRIURLICSGEEEKkKAnkhBBCCCFSlARyQgghhBApSgI5IYQQQogUJYGcEEIIIUSKkkBOCCGEECJFSSAnhBBCCJGiJJATQgghhEhREsgJIYQQQqQoCeSEEEIIIVKUBHJCCCGEEClKAjkhhBBCiBQlgZwQQgghRIqSQE4IIYQQIkVJICeEEEIIkaIkkBNCCCGESFESyAkhhBBCpCgJ5IQQQgghUpQEckIIIYQQKUoCOSGEEEKIFCWBnBBCCCFEipJATgghhBAiRUkgJ4QQQgiRoiSQE0IIIYRIURLICSGEEEKkKAnkhBBCCCFSlARyQgghhBApSgI5IYQQQogUJYGcEEIIIUSKkkBOCCGEECJFSSAnhBBCCJGiJJATQgghhEhREsgJIYQQQqQoCeSEEEIIIVJU0gRySqlrlVJblVI+pdSsdp5XoZTarJT6WCn1UU+uUQghhBAimdh6ewEhtgCfAR6L4blna60PdfN6hBBCCCGSWtIEclrrTwGUUr29FCGEEEKIlJA0R6udoIHXlVLrlVI39fZihBBCCCF6S4/uyCml3gCGRnjoXq31izFeZq7WulopNRhYqZTarrVeHeX9bgJuAhg2ZGSX1iyEEEIIkax6NJDTWp+bgGtU+389oJRaBpwKRAzktNaPA48DTJ44S8f73kIIIYQQySSljlaVUv2UUjmB3wPnY4okhBBCCCGOO0kTyCmlrlJK7QXmACuUUq/57y9USr3sf9oQYI1SahPwIbBCa/1q76xYCCGEEKJ3JVPV6jJgWYT7q4GL/b/fBczo4aUJIYQQQiSlpNmRE0IIIYQQnSOBnBBCCCFEipJATgghhBAiRUkgJ4QQQgiRoiSQE0IIIYRIURLICSGEEEKkKAnkhBBCCCFSlARyQgghhBApSgI5IYQQQogUJYGcEEIIIUSKkkBOCCGEECJFSSAnhBBCCJGiJJATQgghhEhREsgJIYQQQqQoCeSEEEIIIVKUBHJCCCGEEClKAjkhhBBCiBQlgZwQQgghRIqSQE4IIYQQIkVJICeEEEIIkaIkkBNCCCGESFESyAkhhBBCpCgJ5IQQQgghUpQEckIIIYQQKUoCuf9v715DLavrMI5/H5wyqChtdLSLVjQvskCLGJJ5Y1OkTjTTxYHxTRqaGUQvIkISCuaVvYqKLpaJFmQTgjWRVl6wiURxqslLJs5Il2HEMYuJSIzJXy/2mtjU2eesM2dc66y9vh8Yzj5nr7N55sd/H56zLmdJkiQNlEVOkiRpoCxykiRJA2WRkyRJGiiLnCRJ0kBZ5CRJkgbKIidJkjRQFjlJkqSBsshJkiQNlEVOkiRpoCxykiRJA2WRkyRJGiiLnCRJ0kClqvrO0IkkTwF/7DtHT9YCf+k7xCrlbGZzNrM5m9mczcKcy2zOZmFnVtUpS200miI3Zkn2VNXb+s6xGjmb2ZzNbM5mNmezMOcym7NZGQ+tSpIkDZRFTpIkaaAscuPwjb4DrGLOZjZnM5uzmc3ZLMy5zOZsVsBz5CRJkgbKPXKSJEkDZZGbQ0m2JXk4yXNJZl4JlOQPSR5MsjfJni4z9mUZs7kgyaNJ9iW5qsuMfUlycpLbkzzWfDxpxnb/btbM3iS7us7ZlaXWQJITk+xsnr8vyWu7T9mPFrO5NMlTU+vk8j5y9iHJ9UkOJXloxvNJ8qVmdg8keWvXGfvQYi7nJTk8tWY+23XGobLIzaeHgA8Au1ts+46qOmdEl34vOZskJwBfAS4EzgIuTnJWN/F6dRVwZ1WtB+5sPl/IM82aOaeqtnQXrzst18BlwN+q6g3AF4DPd5uyH8t4f+ycWifXdRqyXzcAFyzy/IXA+ubfFcDXOsi0GtzA4nMB+MXUmtnRQaa5YJGbQ1X1SFU92neO1ajlbDYA+6rq8ar6F/A9YOvzn653W4Ebm8c3Au/rMUvf2qyB6XndDLwzSTrM2Jexvj9aqardwF8X2WQr8O2auBd4eZLTu0nXnxZz0TGyyI1bAT9L8qskV/QdZhV5FfDnqc8PNF+bd+uq6gmA5uOpM7Z7UZI9Se5NMq9lr80a+O82VXUEOAy8opN0/Wr7/vhgc+jw5iSv6SbaIIz150sb5yb5bZLbkryp7zBDsabvADo2Se4ATlvgqaur6octX2ZjVR1Mcipwe5LfN781DdpxmM1Ce1Xm4vLuxWazjJc5o1k3rwfuSvJgVe0/PglXjTZrYG7XyRLa/L9/BNxUVc8muZLJnstNz3uyYRjrulnKr5nckuofSTYDP2By+FlLsMgNVFW96zi8xsHm46EktzA5ZDL4InccZnMAmN6D8Grg4Apfc1VYbDZJnkxyelU90RzqOTTjNY6um8eT3A28BZi3ItdmDRzd5kCSNcDLGMehoyVnU1VPT336TUZy/mBLc/vzZSWq6u9Tj29N8tUka6vKe7AuwUOrI5XkxUleevQx8G4mFwII7gfWJ3ldkhcC24G5vTpzyi7gkubxJcD/7b1MclKSE5vHa4GNwO86S9idNmtgel4XAXfVOP4w55Kz+Z9zvrYAj3SYb7XbBXyouXr17cDho6c0jFmS046eY5pkA5N+8vTi3yVwj9xcSvJ+4MvAKcCPk+ytqvOTvBK4rqo2A+uAW5r3zRrgu1X1k95Cd6TNbKrqSJKPAz8FTgCur6qHe4zdlWuA7ye5DPgTsA2g+TMtV1bV5cAbgWuTPMfkB+01VTV3RW7WGkiyA9hTVbuAbwHfSbKPyZ647f0l7k7L2XwiyRbgCJPZXNpb4I4luQk4D1ib5ADwOeAFAFX1deBWYDOwD/gn8OF+knarxVwuAj6W5AjwDLB9JL8YrZh3dpAkSRooD61KkiQNlEVOkiRpoCxykiRJA2WRkyRJGiiLnCRJ0kBZ5CRJkgbKIidJkjRQFjlJkqSBsshJ0jIk2Zbk2SRnTn3ti0n2J1nXZzZJ4+OdHSRpGZr7Qd4P/KaqPpLkU8CngY1V9Vi/6SSNjfdalaRlqKpK8hkm9+rdD1wNbLLESeqDe+Qk6RgkuQfYALy3qm7rO4+kcfIcOUlapiSbgLOBAE/2HEfSiLlHTpKWIcnZwM+BTwLvAV5SVef3m0rSWFnkJKml5krVe4Brq2pHkjcDDzA5R+7uXsNJGiWLnCS1kORk4JfA7qr66NTXdwJnVNW5vYWTNFoWOUmSpIHyYgdJkqSBsshJkiQNlEVOkiRpoCxykiRJA2WRkyRJGiiLnCRJ0kBZ5CRJkgbKIidJkjRQFjlJkqSB+g/1oVQhcpakCQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "h = 0.02\n", "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", "\n", "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n", " np.arange(y_min, y_max, h))\n", "grid_tensor = torch.FloatTensor(np.c_[xx.ravel(), yy.ravel()])\n", "\n", "Z = better_net(torch.autograd.Variable(grid_tensor))\n", "Z = Z.data.numpy()\n", "Z = np.argmax(Z, axis=1)\n", "Z = Z.reshape(xx.shape)\n", "\n", "plt.figure(figsize=(10, 8))\n", "\n", "plt.contourf(xx, yy, Z, cmap=plt.cm.rainbow, alpha=0.3)\n", "plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.rainbow)\n", "\n", "plt.xlim(xx.min(), xx.max())\n", "plt.ylim(yy.min(), yy.max())\n", "\n", "plt.title('Спираль', fontsize=15)\n", "plt.xlabel('$x$', fontsize=14)\n", "plt.ylabel('$y$', fontsize=14)\n", "plt.show();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Предлагается самостоятельно проанализировать то, что было изменено, чтобы улучшить качество модели (и *обратить на это внимание*)." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "6N_9wfvPYJeK" }, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "H_thmQJOYJeK" }, "source": [ "

Полезные ссылки

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "RpSrLf9FYJeL" }, "source": [ "1). *Примеры написания нейросетей на PyTorch (офийиальные туториалы) (на английском): https://pytorch.org/tutorials/beginner/pytorch_with_examples.html#examples \n", "https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html*\n", "\n", "2). Курс Стэнфордского Университета: http://cs231n.github.io/\n", "\n", "3). Практически исчерпывающая информация по основам нейросетей (из cs231n) (на английском): \n", "\n", "http://cs231n.github.io/neural-networks-1/, \n", "http://cs231n.github.io/neural-networks-2/, \n", "http://cs231n.github.io/neural-networks-3/, \n", "http://cs231n.github.io/neural-networks-case-study/#linear\n", "\n", "4). *Хорошие статьи по основам нейросетей (на английском): http://neuralnetworksanddeeplearning.com/chap1.html*\n", "\n", "5). *Наглядная демонстрация того, как обучаются нейросети: https://cs.stanford.edu/people/karpathy/convnetjs/*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "1Qldb1U5YJeM" }, "source": [ "6). *Подробнее про backpropagation -- статья на Medium: https://medium.com/autonomous-agents/backpropagation-how-neural-networks-learn-complex-behaviors-9572ac161670*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "g1HSslRhYJeN" }, "source": [ "7). *Статья из интернета по Backprop: http://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf*" ] } ], "metadata": { "colab": { "name": "[seminar]mlp_pytorch.ipynb", "provenance": [], "version": "0.3.2" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 1 }