{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zqPnyO9_YJ8t" }, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "vcnGdgw_YJ8u" }, "source": [ "

PyTorch. Основы: синтаксис, torch.cuda и torch.autograd

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zqPnyO9_YJ8t" }, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "-Iv1eX2mYJ8v" }, "source": [ "

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "JwiKExhuYJ8w" }, "source": [ "На этом занятии мы рассмотрим основы фреймворка глубокого обучения PyTorch. \n", "\n", "Когда хочется написать какую-нибудь нейросеть, решающую определённую задачу, будь то какая-нибудь простая классификация чего-либо или обнаружение лиц людей на видео. Всё, конечно, всегда начинается со **сбора данных**, а уже потом реализуются модели и проводятся эксперименты. \n", "\n", "Однако люди быстро поняли, что писать свои нейронные сети каждый раз с нуля ну очень уж долго и неразумно, поэтому придумали так называемые **фреймворки** - модули, в которых есть функционал, с помощью которого можно быстро и просто решать типовые задачи, и уже с помощью этих средств писать решения к более сложным задачам." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "tQoex-sZYJ8x" }, "source": [ "Есть много различных фремворков глубокого обучения. Разница между ними прежде всего в том, каков общий принцип вычислений. \n", "Например, в **Caffe и Caffe2** вы пишете код, по сути, составляя его из готовых \"кусочков\", как в Lego, в **TensorFlow и Theano** вы сначала объявляете вычислительный граф, потом компилируете его и запускаете (sees.run()), в то время как в **Torch и PyTorch** вы пишете почти точно так же, как на NumPy, а граф вычислений создаётся только при запуске (то есть существует только во время выполнения, потом он \"разрушается\"). **Keras** позволяет как строить блоки, так и компилировать свой граф:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "97MX7yrBYJ8y" }, "source": [ "

\n", "

Картинка взята из этой [статьи на Хабре](https://habr.com/post/334380/)

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "J74O-qdJYJ80" }, "source": [ "

Установка

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "WpXxZ4-4YJ83" }, "source": [ "Инструкция по установке PyTorch есть на [официальном сайте PyTorch](https://pytorch.org/)." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "uxdjjnqIYJ86" }, "source": [ "

Синтаксис

" ] }, { "cell_type": "code", "execution_count": 71, "metadata": { "colab": {}, "colab_type": "code", "id": "RtyozNmmYJ89" }, "outputs": [], "source": [ "import torch" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zjfLQzEPYJ9B" }, "source": [ "Сначала немного фактов про PyTorch: \n", "- динамический граф вычислений\n", "- удобные модули `torch.nn` и `torchvision` для написания нейросетей с минимальными усилиями\n", "- в некоторых задачах даже быстрее TensorFlow\n", "- легко проводить вычисления на GPU" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "0AL-T8TMYJ9D" }, "source": [ "Если PyTorch представить формулой, то она будет такой: \n", "\n", "$$PyTorch = NumPy + CUDA + Autograd$$" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "X_O59VXWYJ9E" }, "source": [ "(CUDA - [wiki](https://ru.wikipedia.org/wiki/CUDA))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "lWIeTFxSYJ9F" }, "source": [ "Посмотрим, как в PyTorch выполняются операции с векторами. \n", "\n", "Напоминание: **тензором** называется многомерный вектор, то есть: \n", "\n", "x = np.array([1,2,3]) - вектор = тензор размерности 1 (то есть (1,)) \n", "y = np.array([[1, 2, 3], [4, 5, 6]]) - матрица = тензор размерности 2 (в данном случае тензор (2, 3)) \n", "z = np.array([[[1, 2, 3], [4, 5, 6], [7, 8, 9]], \n", " [[1, 2, 3], [4, 5, 6], [7, 8, 9]], \n", " [[1, 2, 3], [4, 5, 6], [7, 8, 9]]]) - \"кубик\" (3, 3, 3) = тензор размерности 3 (в данном случае (3, 3, 3))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "hS97vo7mYJ9G" }, "source": [ "Простейшим примером 3-мерного тензора является **картинка** - это \"параллелепипед\" из чисел, у которого три размерности - высота, ширина и количество каналов, значит это тензор размерности 3." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "tEeGGtRiYJ9H" }, "source": [ "Понятие тензора нужно знать потому, что в PyTorch мы оперируем переменными типа `torch.Tensor` (`FloatTensor`, `IntTensor`, `ByteTensor`), и пугаться их названий совершенно не нужно - это просто векторы, у которых несколько размерностей." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "byu7xZtLYJ9J" }, "source": [ "Все типы тензоров:" ] }, { "cell_type": "code", "execution_count": 73, "metadata": { "colab": {}, "colab_type": "code", "id": "lPIaCLY3YJ9K", "outputId": "5e988a80-4c1f-499d-c99b-e6df2f1a25be" }, "outputs": [ { "data": { "text/plain": [ "torch.ByteTensor" ] }, "execution_count": 73, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.HalfTensor # 16 бит, с плавающей точкой\n", "torch.FloatTensor # 32 бита, с плавающей точкой\n", "torch.DoubleTensor # 64 бита, с плавающей точкой\n", "\n", "torch.ShortTensor # 16 бит, целочисленный, знаковый\n", "torch.IntTensor # 32 бита, целочисленный, знаковый\n", "torch.LongTensor # 64 бита, целочисленный, знаковый\n", "\n", "torch.CharTensor # 8 бит, целочисленный, знаковый\n", "torch.ByteTensor # 8 бит, целочисленный, беззнаковый" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "NWk2gidNYJ9P" }, "source": [ "Мы будем использовать только `torch.FloatTensor()` и `torch.IntTensor()`. " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "CllyEKvVYJ9Q" }, "source": [ "Перейдём к делу:" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "R4HEetsxYJ9Q" }, "source": [ "* Создание тензоров:" ] }, { "cell_type": "code", "execution_count": 74, "metadata": { "colab": {}, "colab_type": "code", "id": "MVawE9lRYJ9R", "outputId": "9b2eaa65-1723-4e73-a379-bec4fe9a665e" }, "outputs": [ { "data": { "text/plain": [ "tensor([1., 2.])" ] }, "execution_count": 74, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.FloatTensor([1, 2])\n", "a" ] }, { "cell_type": "code", "execution_count": 75, "metadata": { "colab": {}, "colab_type": "code", "id": "ARjaPS_GYJ9W", "outputId": "f6ede400-76f4-467c-8170-2b4f3f18b262" }, "outputs": [ { "data": { "text/plain": [ "torch.Size([2])" ] }, "execution_count": 75, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.shape" ] }, { "cell_type": "code", "execution_count": 76, "metadata": { "colab": {}, "colab_type": "code", "id": "4gXf0akDYJ9Z", "outputId": "00f41e92-4419-4f6b-da27-5a02ab8aebfc" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1., 2., 3.],\n", " [4., 5., 6.]])" ] }, "execution_count": 76, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b = torch.FloatTensor([[1,2,3], [4,5,6]])\n", "b" ] }, { "cell_type": "code", "execution_count": 77, "metadata": { "colab": {}, "colab_type": "code", "id": "0HTkoIzpYJ9e", "outputId": "3833f329-6757-44c9-847c-95a7e848f0c5" }, "outputs": [ { "data": { "text/plain": [ "torch.Size([2, 3])" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.shape" ] }, { "cell_type": "code", "execution_count": 78, "metadata": { "colab": {}, "colab_type": "code", "id": "WYJ9L-yFYJ9j" }, "outputs": [], "source": [ "x = torch.FloatTensor(2,3,4)" ] }, { "cell_type": "code", "execution_count": 79, "metadata": { "colab": {}, "colab_type": "code", "id": "qWAEP27IYJ9m", "outputId": "466bc5b2-d9aa-499b-b9ac-f680f5ba6cc2" }, "outputs": [ { "data": { "text/plain": [ "tensor([[[-7.5219e-19, 6.8804e-43, 7.0065e-44, 0.0000e+00],\n", " [ 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00],\n", " [ 0.0000e+00, 0.0000e+00, 1.8754e+28, 6.4978e-07]],\n", "\n", " [[ 8.4336e-07, 1.0424e-11, 2.1258e+20, 1.0681e-05],\n", " [ 6.7003e-10, 1.6132e-07, 8.5405e+20, 2.0431e+20],\n", " [ 4.0292e-11, 7.1450e+31, 1.6534e+19, 3.0854e+32]]])" ] }, "execution_count": 79, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x" ] }, { "cell_type": "code", "execution_count": 80, "metadata": { "colab": {}, "colab_type": "code", "id": "o5CsOJ_xYJ9s", "outputId": "e95a7efe-61c4-425f-9c16-10ad2530bfa0", "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "tensor([ nan, nan, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000])" ] }, "execution_count": 80, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = torch.FloatTensor(100)\n", "x" ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "colab": {}, "colab_type": "code", "id": "uIVhvmzZYJ91", "outputId": "a74de636-f085-4b05-b2e8-1081062985e2" }, "outputs": [ { "data": { "text/plain": [ "torch.Size([45, 57, 14, 2])" ] }, "execution_count": 81, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = torch.IntTensor(45, 57, 14, 2)\n", "x.shape" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "cUzl8DFRYJ95" }, "source": [ "**Обратите внимание** - если вы создаёте тензор через задание размерностей (как в примере выше), то он изначально заполняюстя случайным \"мусором\". Что инициализировать нулями, нужно написать .zero_() в конце:" ] }, { "cell_type": "code", "execution_count": 82, "metadata": { "colab": {}, "colab_type": "code", "id": "QX0QcIOzYJ96", "outputId": "0e8f43f9-dff1-4d69-ce85-dfb8789d78da" }, "outputs": [ { "data": { "text/plain": [ "tensor([[[-1587672904, 491, 64, 0],\n", " [ 0, 0, 0, 0]],\n", "\n", " [[ 0, 0, 1681143862, 926180914],\n", " [ 1630613554, 845504824, 912405604, 912668720]],\n", "\n", " [[ 959461687, 946092342, 929247537, 895694948],\n", " [ 842294628, 858863156, 845570404, 1667522869]]],\n", " dtype=torch.int32)" ] }, "execution_count": 82, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = torch.IntTensor(3, 2, 4)\n", "x" ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "colab": {}, "colab_type": "code", "id": "XV3w9lVwYJ-A", "outputId": "0cf6fea1-add6-443f-dc11-d399b589c95f" }, "outputs": [ { "data": { "text/plain": [ "tensor([[[0, 0, 0, 0],\n", " [0, 0, 0, 0]],\n", "\n", " [[0, 0, 0, 0],\n", " [0, 0, 0, 0]],\n", "\n", " [[0, 0, 0, 0],\n", " [0, 0, 0, 0]]], dtype=torch.int32)" ] }, "execution_count": 83, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = torch.IntTensor(3, 2, 4).zero_()\n", "x" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "SAAmLRwzYJ-E" }, "source": [ "Аналог функции `np.reshape()` == `torch.view()`:" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "colab": {}, "colab_type": "code", "id": "ePjL3X3GYJ-H", "outputId": "aabd638d-80af-40f4-e1a1-78d465d2e65f" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1., 2.],\n", " [3., 4.],\n", " [5., 6.]])" ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.view(3, 2)" ] }, { "cell_type": "code", "execution_count": 85, "metadata": { "colab": {}, "colab_type": "code", "id": "spMYG2xXYJ-L", "outputId": "64c72e62-3e41-4e4a-bc06-d5ffc2510039" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1., 2., 3.],\n", " [4., 5., 6.]])" ] }, "execution_count": 85, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "amVu0N1zYJ-O" }, "source": [ "**Обратите внимание** - torch.view() создаёт новый тензор, а не изменяет старый!" ] }, { "cell_type": "code", "execution_count": 86, "metadata": { "colab": {}, "colab_type": "code", "id": "hWqyugOSYJ-Q", "outputId": "ab6dce61-7e71-40c4-ab72-a47d37802b88" }, "outputs": [ { "data": { "text/plain": [ "tensor([1., 2., 3., 4., 5., 6.])" ] }, "execution_count": 86, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.view(-1)" ] }, { "cell_type": "code", "execution_count": 87, "metadata": { "colab": {}, "colab_type": "code", "id": "S9CvF56uYJ-U", "outputId": "e58092ee-bc2f-4b94-f430-2786c733d315" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1., 2., 3.],\n", " [4., 5., 6.]])" ] }, "execution_count": 87, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "t401j1E2YJ-Z" }, "source": [ "* Изменение типа тензора:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "colab": {}, "colab_type": "code", "id": "f_iDUEH4YJ-Z" }, "outputs": [], "source": [ "a = torch.FloatTensor([1.5, 3.2, -7])" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "colab": {}, "colab_type": "code", "id": "EzJOrXitYJ-e", "outputId": "ea9cd7e1-0e34-42a2-b89d-9e27bc43fdb3" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 1, 3, -7], dtype=torch.int32)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.type_as(torch.IntTensor())" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "colab": {}, "colab_type": "code", "id": "hAe3OWWUYJ-i", "outputId": "53dc463a-47eb-4531-f38b-5bdc901bf572" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 1, 3, 249], dtype=torch.uint8)" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.type_as(torch.ByteTensor())" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "4rkatrUeYJ-n" }, "source": [ "Обратите внимание, что при `.type_as()` создаётся новый тензор (старый не меняется), то есть это не in-place операция:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "colab": {}, "colab_type": "code", "id": "fCgDT14MYJ-o", "outputId": "c3828601-0218-4563-e47a-91817ffd253a" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 1.5000, 3.2000, -7.0000])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "hG1OwtFBYJ-r" }, "source": [ "* Индексация точная такая же, как и в NumPy:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "colab": {}, "colab_type": "code", "id": "prr0EGIYYJ-r", "outputId": "7228cf78-765f-483b-b4b7-ba032503e545" }, "outputs": [ { "data": { "text/plain": [ "tensor([[100., 20., 35.],\n", " [ 15., 163., 534.],\n", " [ 52., 90., 66.]])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.FloatTensor([[100, 20, 35], [15, 163, 534], [52, 90, 66]])\n", "a" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "colab": {}, "colab_type": "code", "id": "IBq7JiccYJ-w", "outputId": "e0dd266d-7d63-4251-99f2-dc465746a17f" }, "outputs": [ { "data": { "text/plain": [ "tensor(100.)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[0, 0]" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "colab": {}, "colab_type": "code", "id": "vE9cb0XkYJ-z", "outputId": "057de53a-7eec-4163-da79-c9a7e02afcce" }, "outputs": [ { "data": { "text/plain": [ "tensor(100.)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[0][0]" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "colab": {}, "colab_type": "code", "id": "UUZzx2F_YJ-2", "outputId": "fc1331a9-ffb5-4861-eed1-2cae074914d2" }, "outputs": [ { "data": { "text/plain": [ "tensor([[100., 20.],\n", " [ 15., 163.]])" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[0:2, 0:2]" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "KffQsYwWYJ_D" }, "source": [ "**Арифметика и булевы операции** работаю также, как и в NumPy, **НО** лучше использовать не опреаторы `+`, `-`, `*`, `/`, а их аналоги: \n", "\n", "| Оператор | Аналог |\n", "|:-:|:-:|\n", "|`+`| `torch.add()` |\n", "|`-`| `torch.sub()` |\n", "|`*`| `torch.mul()` |\n", "|`/`| `torch.div()` |" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "UMpR8T11YJ_D" }, "source": [ "* Сложение:" ] }, { "cell_type": "code", "execution_count": 88, "metadata": { "colab": {}, "colab_type": "code", "id": "1XHkSjEUYJ_D" }, "outputs": [], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])\n", "b = torch.FloatTensor([[-1, -2, -3], [-10, -20, -30], [100, 200, 300]])" ] }, { "cell_type": "code", "execution_count": 89, "metadata": { "colab": {}, "colab_type": "code", "id": "td25tuFKYJ_H", "outputId": "441d0d2a-7fd5-4c46-b684-c18f0aeacaca" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0., 0., 0.],\n", " [ 0., 0., 0.],\n", " [200., 400., 600.]])" ] }, "execution_count": 89, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a + b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "jWYjf1ENYJ_M" }, "source": [ "Лучше:" ] }, { "cell_type": "code", "execution_count": 90, "metadata": { "colab": {}, "colab_type": "code", "id": "wHka8PxFYJ_N", "outputId": "919e345c-1bce-45ee-d21c-097cafe07b4d" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0., 0., 0.],\n", " [ 0., 0., 0.],\n", " [200., 400., 600.]])" ] }, "execution_count": 90, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.add(b)" ] }, { "cell_type": "code", "execution_count": 91, "metadata": { "colab": {}, "colab_type": "code", "id": "FHQj4DbBYJ_Q", "outputId": "a416faf0-c0e8-4514-dc0d-09295c282176" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -2., -3.],\n", " [ -10., -20., -30.],\n", " [-100., -200., -300.]])" ] }, "execution_count": 91, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b = -a\n", "b" ] }, { "cell_type": "code", "execution_count": 92, "metadata": { "colab": {}, "colab_type": "code", "id": "xLCDzO7iYJ_V", "outputId": "6d3a65f0-17a1-46a6-f402-3b18d9fe05eb" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0., 0., 0.],\n", " [0., 0., 0.],\n", " [0., 0., 0.]])" ] }, "execution_count": 92, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a + b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "z_3gVLY3YJ_Z" }, "source": [ "* Вычитание:" ] }, { "cell_type": "code", "execution_count": 93, "metadata": { "colab": {}, "colab_type": "code", "id": "PANHq3eFYJ_a", "outputId": "c21fba72-6779-4cde-ab74-8fe8d0ec5438" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 2., 4., 6.],\n", " [ 20., 40., 60.],\n", " [200., 400., 600.]])" ] }, "execution_count": 93, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a - b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "tPL1W8ajYJ_i" }, "source": [ "Лучше:" ] }, { "cell_type": "code", "execution_count": 94, "metadata": { "colab": {}, "colab_type": "code", "id": "AH7xhdkRYJ_i", "outputId": "70ed9dba-43c7-42a9-bbe9-cc5bcfe3f924" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 2., 4., 6.],\n", " [ 20., 40., 60.],\n", " [200., 400., 600.]])" ] }, "execution_count": 94, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.sub(b)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "KOX3dTuZYJ_l" }, "source": [ "* Умножение (поэлементное):" ] }, { "cell_type": "code", "execution_count": 95, "metadata": { "colab": {}, "colab_type": "code", "id": "J380CvNGYJ_m", "outputId": "4d641233-d110-40ca-8f88-b546ac7df1c2" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -4., -9.],\n", " [ -100., -400., -900.],\n", " [-10000., -40000., -90000.]])" ] }, "execution_count": 95, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a * b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "UG7aOFSvYJ_o" }, "source": [ "Лучше:" ] }, { "cell_type": "code", "execution_count": 96, "metadata": { "colab": {}, "colab_type": "code", "id": "cixW0jgdYJ_o", "outputId": "91691a93-794e-4482-eec7-ab2830b59a25" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -4., -9.],\n", " [ -100., -400., -900.],\n", " [-10000., -40000., -90000.]])" ] }, "execution_count": 96, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mul(b)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "V-GYTERKYJ_q" }, "source": [ "* Деление (поэлементное):" ] }, { "cell_type": "code", "execution_count": 97, "metadata": { "colab": {}, "colab_type": "code", "id": "fUBQns2SYJ_r" }, "outputs": [], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])\n", "b = torch.FloatTensor([[-1, -2, -3], [-10, -20, -30], [100, 200, 300]])" ] }, { "cell_type": "code", "execution_count": 98, "metadata": { "colab": {}, "colab_type": "code", "id": "2hrKeMjAYJ_v", "outputId": "2a589e95-b308-4c0c-9290-60512494e1bd" }, "outputs": [ { "data": { "text/plain": [ "tensor([[-1., -1., -1.],\n", " [-1., -1., -1.],\n", " [ 1., 1., 1.]])" ] }, "execution_count": 98, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a / b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "KvcGLFVNYJ_0" }, "source": [ "Лучше:" ] }, { "cell_type": "code", "execution_count": 99, "metadata": { "colab": {}, "colab_type": "code", "id": "cpg3YFVPYJ_2", "outputId": "0cf1d61d-1834-41ea-96f0-ba8a24bb210b" }, "outputs": [ { "data": { "text/plain": [ "tensor([[-1., -1., -1.],\n", " [-1., -1., -1.],\n", " [ 1., 1., 1.]])" ] }, "execution_count": 99, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.div(b)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "VeLAbx1tYJ_7" }, "source": [ "Заметьте, все эти операции **не меняют исходные тензоры**, а **создают новые**:" ] }, { "cell_type": "code", "execution_count": 100, "metadata": { "colab": {}, "colab_type": "code", "id": "57BFtnlgYJ_8", "outputId": "6b0ab791-b96b-49ca-f339-484fb21c9a3f" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 2., 3.],\n", " [ 10., 20., 30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "code", "execution_count": 101, "metadata": { "colab": {}, "colab_type": "code", "id": "UziXyvtDYJ_-", "outputId": "8468055e-7708-4255-ebaf-1fa621ef970f" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -2., -3.],\n", " [-10., -20., -30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "vLpwZPc3YKAC" }, "source": [ "* **Операторы сравнения**:" ] }, { "cell_type": "code", "execution_count": 102, "metadata": { "colab": {}, "colab_type": "code", "id": "FfRzvw_UYKAC" }, "outputs": [], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])\n", "b = torch.FloatTensor([[-1, -2, -3], [-10, -20, -30], [100, 200, 300]])" ] }, { "cell_type": "code", "execution_count": 103, "metadata": { "colab": {}, "colab_type": "code", "id": "o1RUTP2BYKAD", "outputId": "3e2b13da-6b0b-4531-81bf-4d21298c9a16" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0, 0, 0],\n", " [0, 0, 0],\n", " [1, 1, 1]], dtype=torch.uint8)" ] }, "execution_count": 103, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a == b" ] }, { "cell_type": "code", "execution_count": 104, "metadata": { "colab": {}, "colab_type": "code", "id": "JO3HaVIAYKAF", "outputId": "d103a172-5fbc-42e5-ed8c-70604d54bfa9" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1, 1, 1],\n", " [1, 1, 1],\n", " [0, 0, 0]], dtype=torch.uint8)" ] }, "execution_count": 104, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a != b" ] }, { "cell_type": "code", "execution_count": 105, "metadata": { "colab": {}, "colab_type": "code", "id": "DExLBj4VYKAH", "outputId": "0bcd542d-47be-435f-8569-903e6b363a3d" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0, 0, 0],\n", " [0, 0, 0],\n", " [0, 0, 0]], dtype=torch.uint8)" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a < b" ] }, { "cell_type": "code", "execution_count": 106, "metadata": { "colab": {}, "colab_type": "code", "id": "0_IH_FUNYKAJ", "outputId": "94602ba6-9a82-4522-827d-1ec114ac9185" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1, 1, 1],\n", " [1, 1, 1],\n", " [0, 0, 0]], dtype=torch.uint8)" ] }, "execution_count": 106, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a > b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "24nzBJR8YKAM" }, "source": [ "* **Булевы маски**:" ] }, { "cell_type": "code", "execution_count": 107, "metadata": { "colab": {}, "colab_type": "code", "id": "C-xVDslwYKAN", "outputId": "0794a382-8fcf-4a66-d440-d39f76ea2439" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 1., 2., 3., 10., 20., 30.])" ] }, "execution_count": 107, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[a > b]" ] }, { "cell_type": "code", "execution_count": 108, "metadata": { "colab": {}, "colab_type": "code", "id": "XatDJF5YYKAQ", "outputId": "4e85dc39-c846-4b92-ac79-76b921cbe096" }, "outputs": [ { "data": { "text/plain": [ "tensor([100., 200., 300.])" ] }, "execution_count": 108, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b[a == b]" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "P-pLWPv2YKAU" }, "source": [ "Опять же, тензоры не меняются:" ] }, { "cell_type": "code", "execution_count": 109, "metadata": { "colab": {}, "colab_type": "code", "id": "AKsqvCD5YKAW", "outputId": "bb0b1877-35cb-4e0f-c72d-a2f68546dd73" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 2., 3.],\n", " [ 10., 20., 30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 109, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "code", "execution_count": 110, "metadata": { "colab": {}, "colab_type": "code", "id": "PYurJLVuYKAa", "outputId": "47a09648-6dd2-4ae2-cdf7-bdc509a443c0" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -2., -3.],\n", " [-10., -20., -30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 110, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "54bQky1bYKAf" }, "source": [ "Применение **стандартных функций** такое же, как и в numpy - поэлементное:" ] }, { "cell_type": "code", "execution_count": 111, "metadata": { "colab": {}, "colab_type": "code", "id": "lWnGSuUhYKAf" }, "outputs": [], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])" ] }, { "cell_type": "code", "execution_count": 112, "metadata": { "colab": {}, "colab_type": "code", "id": "J6ZyxZBNYKAl", "outputId": "1df429c0-60b7-4ce9-f0cd-00145923296a" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.8415, 0.9093, 0.1411],\n", " [-0.5440, 0.9129, -0.9880],\n", " [-0.5064, -0.8733, -0.9998]])" ] }, "execution_count": 112, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.sin()" ] }, { "cell_type": "code", "execution_count": 113, "metadata": { "colab": {}, "colab_type": "code", "id": "SScTSkJMYKAo", "outputId": "fdf7d1bf-efab-49ba-bc75-cc107d0b87bd" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.8415, 0.9093, 0.1411],\n", " [-0.5440, 0.9129, -0.9880],\n", " [-0.5064, -0.8733, -0.9998]])" ] }, "execution_count": 113, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.sin(a)" ] }, { "cell_type": "code", "execution_count": 114, "metadata": { "colab": {}, "colab_type": "code", "id": "0bYVYVV7YKAr", "outputId": "e0705a7e-5d5d-4fcf-9710-7a109d797fb7" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.5403, -0.4161, -0.9900],\n", " [-0.8391, 0.4081, 0.1543],\n", " [ 0.8623, 0.4872, -0.0221]])" ] }, "execution_count": 114, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.cos()" ] }, { "cell_type": "code", "execution_count": 115, "metadata": { "colab": {}, "colab_type": "code", "id": "c-rYL8HQYKAt", "outputId": "55051cd8-627a-4594-d9ac-82976a0c73b6" }, "outputs": [ { "data": { "text/plain": [ "tensor([[2.7183e+00, 7.3891e+00, 2.0086e+01],\n", " [2.2026e+04, 4.8517e+08, 1.0686e+13],\n", " [ inf, inf, inf]])" ] }, "execution_count": 115, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.exp()" ] }, { "cell_type": "code", "execution_count": 116, "metadata": { "colab": {}, "colab_type": "code", "id": "EbiCDYUzYKAw", "outputId": "ac1e3627-617e-4f37-875c-0adc20e8fbaa" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.0000, 0.6931, 1.0986],\n", " [2.3026, 2.9957, 3.4012],\n", " [4.6052, 5.2983, 5.7038]])" ] }, "execution_count": 116, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.log()" ] }, { "cell_type": "code", "execution_count": 117, "metadata": { "colab": {}, "colab_type": "code", "id": "yh1sySpiYKAy", "outputId": "6908104e-8250-4041-c7d1-de44ef13fc2f" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -2., -3.],\n", " [ -10., -20., -30.],\n", " [-100., -200., -300.]])" ] }, "execution_count": 117, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b = -a\n", "b" ] }, { "cell_type": "code", "execution_count": 118, "metadata": { "colab": {}, "colab_type": "code", "id": "PNU0UxqIYKA2", "outputId": "230053f8-ffc0-4d62-9ee0-73d24b8a6bc4" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 2., 3.],\n", " [ 10., 20., 30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 118, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.abs()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "eqERQjISYKA9" }, "source": [ "**Сумма, среднее, максимум, минимум**:" ] }, { "cell_type": "code", "execution_count": 119, "metadata": { "colab": {}, "colab_type": "code", "id": "HLmJGdl9YKA-", "outputId": "55275b53-d830-4530-f024-9603879269fd" }, "outputs": [ { "data": { "text/plain": [ "tensor(666.)" ] }, "execution_count": 119, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.sum()" ] }, { "cell_type": "code", "execution_count": 120, "metadata": { "colab": {}, "colab_type": "code", "id": "ervIuNGnYKBD", "outputId": "a21ff9f0-34da-4d7a-e95a-49239822d053" }, "outputs": [ { "data": { "text/plain": [ "tensor(74.)" ] }, "execution_count": 120, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mean()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "1qxkdUQ2YKBF" }, "source": [ "По осям:" ] }, { "cell_type": "code", "execution_count": 121, "metadata": { "colab": {}, "colab_type": "code", "id": "cc6RjJYeYKBG", "outputId": "3ab3407e-c63f-4a9e-9534-9244c953b671" }, "outputs": [ { "data": { "text/plain": [ "tensor([111., 222., 333.])" ] }, "execution_count": 121, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.sum(0)" ] }, { "cell_type": "code", "execution_count": 122, "metadata": { "colab": {}, "colab_type": "code", "id": "Dgi5BmhOYKBJ", "outputId": "7a57158b-ea66-46bf-ac15-5591ba6e6000" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 6., 60., 600.])" ] }, "execution_count": 122, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.sum(1)" ] }, { "cell_type": "code", "execution_count": 123, "metadata": { "colab": {}, "colab_type": "code", "id": "-tIkcTENYKBK", "outputId": "6292c5e3-da1e-4a1b-b803-015a91aaef6d" }, "outputs": [ { "data": { "text/plain": [ "tensor(300.)" ] }, "execution_count": 123, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.max()" ] }, { "cell_type": "code", "execution_count": 124, "metadata": { "colab": {}, "colab_type": "code", "id": "V4U_LyyPYKBL", "outputId": "9b22f063-7ec0-48e8-a65e-7de1e87ba796" }, "outputs": [ { "data": { "text/plain": [ "(tensor([100., 200., 300.]), tensor([2, 2, 2]))" ] }, "execution_count": 124, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.max(0)" ] }, { "cell_type": "code", "execution_count": 125, "metadata": { "colab": {}, "colab_type": "code", "id": "TSqmVtWjYKBO", "outputId": "f65b698c-826e-4b97-da8f-0153e3bae821" }, "outputs": [ { "data": { "text/plain": [ "tensor(1.)" ] }, "execution_count": 125, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.min()" ] }, { "cell_type": "code", "execution_count": 126, "metadata": { "colab": {}, "colab_type": "code", "id": "OeA2hlZ2YKBP", "outputId": "668880df-8589-4d5f-c92e-6142a84058db" }, "outputs": [ { "data": { "text/plain": [ "(tensor([1., 2., 3.]), tensor([0, 0, 0]))" ] }, "execution_count": 126, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.min(0)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Ys-9hnr4YKBS" }, "source": [ "Обратите внимание - второй тензор при вызове функций .max() и .min() - это индексы этих максимальных/минимальных элементов по указанной размерности (то есть в данном случае a.min() вернул (1, 2, 3) - минимумы по 0 оси (по столбцам), и их индексы по 0-ой оси (0,0,0) (номер каждого элемента в своём столбце))." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "kepFeTZ0YKBV" }, "source": [ "**Матричные операции:**" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "7RYm3mdrYKBX" }, "source": [ "* Транспонирование матрицы (тензора):" ] }, { "cell_type": "code", "execution_count": 127, "metadata": { "colab": {}, "colab_type": "code", "id": "T8-1TSkeYKBY", "outputId": "c2367178-b759-462d-9f22-08f13ca6c56e" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 2., 3.],\n", " [ 10., 20., 30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 127, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])\n", "a" ] }, { "cell_type": "code", "execution_count": 128, "metadata": { "colab": {}, "colab_type": "code", "id": "HSAZwai7YKBa", "outputId": "d8d0bbc2-1283-4dc2-cbd3-be45a9b3b6e9" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 10., 100.],\n", " [ 2., 20., 200.],\n", " [ 3., 30., 300.]])" ] }, "execution_count": 128, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.t()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "QqsPkleRYKBd" }, "source": [ "И снова - сам тензор не меняется (то есть при вызове создаётся новый):" ] }, { "cell_type": "code", "execution_count": 129, "metadata": { "colab": {}, "colab_type": "code", "id": "NRPamd2EYKBe", "outputId": "eedf4477-f1c3-4527-ab34-cdd914d556e9" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 2., 3.],\n", " [ 10., 20., 30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 129, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "q8YhoxHPYKBh" }, "source": [ "* Скалярное произведение векторов (1-мерных тензоров):" ] }, { "cell_type": "code", "execution_count": 130, "metadata": { "colab": {}, "colab_type": "code", "id": "i1pRtl7HYKBi" }, "outputs": [], "source": [ "a = torch.FloatTensor([1, 2, 3, 4, 5, 6])\n", "b = torch.FloatTensor([-1, -2, -4, -6, -8, -10])" ] }, { "cell_type": "code", "execution_count": 131, "metadata": { "colab": {}, "colab_type": "code", "id": "qcS-28dwYKBp", "outputId": "38a9040f-e59b-468c-c2e2-caee3d56b993" }, "outputs": [ { "data": { "text/plain": [ "tensor(-141.)" ] }, "execution_count": 131, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.dot(b)" ] }, { "cell_type": "code", "execution_count": 132, "metadata": { "colab": {}, "colab_type": "code", "id": "MXyhOelQYKBs", "outputId": "f0ec3908-d565-4eb6-cf46-fdaf4c20cc50" }, "outputs": [ { "data": { "text/plain": [ "tensor(-141.)" ] }, "execution_count": 132, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a @ b" ] }, { "cell_type": "code", "execution_count": 133, "metadata": { "colab": {}, "colab_type": "code", "id": "sxqg4xhoYKBt", "outputId": "66c3dec7-220f-420b-9677-b88f5a0db078" }, "outputs": [ { "data": { "text/plain": [ "torch.Tensor" ] }, "execution_count": 133, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(a)" ] }, { "cell_type": "code", "execution_count": 134, "metadata": { "colab": {}, "colab_type": "code", "id": "_v2ZjPqDYKBz", "outputId": "7c6d752f-d697-4bcb-b65f-6ca8db4d0cec" }, "outputs": [ { "data": { "text/plain": [ "torch.Tensor" ] }, "execution_count": 134, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(b)" ] }, { "cell_type": "code", "execution_count": 135, "metadata": { "colab": {}, "colab_type": "code", "id": "UsWB9vTuYKB0", "outputId": "e591fa1a-2ac6-43f1-d9a8-28bab50caaef" }, "outputs": [ { "data": { "text/plain": [ "torch.Tensor" ] }, "execution_count": 135, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(a @ b)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "KSrnLwkGYKB1" }, "source": [ "* Матричное умножение:" ] }, { "cell_type": "code", "execution_count": 136, "metadata": { "colab": {}, "colab_type": "code", "id": "ViurnbjPYKB2" }, "outputs": [], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])\n", "b = torch.FloatTensor([[-1, -2, -3], [-10, -20, -30], [100, 200, 300]])" ] }, { "cell_type": "code", "execution_count": 137, "metadata": { "colab": {}, "colab_type": "code", "id": "5TrCnfWiYKB3", "outputId": "38f50f16-6be9-4523-e0ca-0743a4b400a3" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 279., 558., 837.],\n", " [ 2790., 5580., 8370.],\n", " [27900., 55800., 83700.]])" ] }, "execution_count": 137, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mm(b)" ] }, { "cell_type": "code", "execution_count": 138, "metadata": { "colab": {}, "colab_type": "code", "id": "HMzIz-_MYKB4", "outputId": "9b149f62-fa77-4859-d6e9-41b8839d1bff" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 279., 558., 837.],\n", " [ 2790., 5580., 8370.],\n", " [27900., 55800., 83700.]])" ] }, "execution_count": 138, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a @ b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Ig2pr52xYKB5" }, "source": [ "Тензоры неизменны:" ] }, { "cell_type": "code", "execution_count": 139, "metadata": { "colab": {}, "colab_type": "code", "id": "22QOePSyYKB6", "outputId": "204fcd82-a75c-43cf-d670-bb04fac63cfe" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1., 2., 3.],\n", " [ 10., 20., 30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 139, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "code", "execution_count": 140, "metadata": { "colab": {}, "colab_type": "code", "id": "JHr6eKADYKB6", "outputId": "bc4df416-6bdb-4aaa-b93f-000b1219222b" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1., -2., -3.],\n", " [-10., -20., -30.],\n", " [100., 200., 300.]])" ] }, "execution_count": 140, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "code", "execution_count": 141, "metadata": { "colab": {}, "colab_type": "code", "id": "vtuvt20-YKB8" }, "outputs": [], "source": [ "a = torch.FloatTensor([[1, 2, 3], [10, 20, 30], [100, 200, 300]])\n", "b = torch.FloatTensor([[-1], [-10], [100]])" ] }, { "cell_type": "code", "execution_count": 142, "metadata": { "colab": {}, "colab_type": "code", "id": "fdDnu5XjYKB9", "outputId": "cc0a9d0c-d63f-468d-907a-05f836fa9828" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "torch.Size([3, 3]) torch.Size([3, 1])\n" ] } ], "source": [ "print(a.shape, b.shape)" ] }, { "cell_type": "code", "execution_count": 143, "metadata": { "colab": {}, "colab_type": "code", "id": "oD4N4WN8YKB_", "outputId": "babf6634-58db-49be-80f6-b949f074c83a" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 279.],\n", " [ 2790.],\n", " [27900.]])" ] }, "execution_count": 143, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a @ b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "va9w-BavYKCB" }, "source": [ "Если \"развернуть\" тензор b просто в массив элементов (`torch.view(-1)`), умножение будет как на столбец:" ] }, { "cell_type": "code", "execution_count": 144, "metadata": { "colab": {}, "colab_type": "code", "id": "V7KsdYcBYKCC", "outputId": "9e2f3ec8-1f56-4c9c-d0a0-a98fda07faeb" }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -1.],\n", " [-10.],\n", " [100.]])" ] }, "execution_count": 144, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "code", "execution_count": 145, "metadata": { "colab": {}, "colab_type": "code", "id": "t7k6spayYKCF", "outputId": "04926526-0040-4c8f-8e7c-de195bc0590e" }, "outputs": [ { "data": { "text/plain": [ "tensor([ -1., -10., 100.])" ] }, "execution_count": 145, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.view(-1)" ] }, { "cell_type": "code", "execution_count": 146, "metadata": { "colab": {}, "colab_type": "code", "id": "8u8xQ20bYKCH", "outputId": "02532845-68a0-4280-cd68-3a31aff29f48" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 279., 2790., 27900.])" ] }, "execution_count": 146, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a @ b.view(-1)" ] }, { "cell_type": "code", "execution_count": 147, "metadata": { "colab": {}, "colab_type": "code", "id": "QvCu1sJEYKCJ", "outputId": "9776bf5e-a880-4141-9340-9881da45066d" }, "outputs": [ { "data": { "text/plain": [ "tensor([ 279., 2790., 27900.])" ] }, "execution_count": 147, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.mv(b.view(-1))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "gZitrCN6YKCL" }, "source": [ "**Перевод из NumPy в PyTorch**:" ] }, { "cell_type": "code", "execution_count": 148, "metadata": { "colab": {}, "colab_type": "code", "id": "bnLS4CGXYKCL", "outputId": "12cdf1ad-af71-4ab5-95f0-bfa937664f88" }, "outputs": [ { "data": { "text/plain": [ "array([[0.73297029, 0.0647068 , 0.95213803],\n", " [0.01135127, 0.69825433, 0.96293269],\n", " [0.87771321, 0.64451539, 0.32162612]])" ] }, "execution_count": 148, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "\n", "a = np.random.rand(3, 3)\n", "a" ] }, { "cell_type": "code", "execution_count": 149, "metadata": { "colab": {}, "colab_type": "code", "id": "PXge57waYKCM", "outputId": "c9a6ab8f-f1d5-40bf-8196-e9ebc346d9ef" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.7330, 0.0647, 0.9521],\n", " [0.0114, 0.6983, 0.9629],\n", " [0.8777, 0.6445, 0.3216]], dtype=torch.float64)" ] }, "execution_count": 149, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b = torch.from_numpy(a)\n", "b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Nvcr3ymPYKCN" }, "source": [ "**НО!** Обратите внимание - a и b в этом случае будут использовать одно и то же хранилище данных, то есть измение одного тензора будет менять и другой:" ] }, { "cell_type": "code", "execution_count": 150, "metadata": { "colab": {}, "colab_type": "code", "id": "0N_ZW6TQYKCN", "outputId": "58717139-15fe-4481-9c7a-e2707076af15" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0., 0., 0.],\n", " [0., 0., 0.],\n", " [0., 0., 0.]], dtype=torch.float64)" ] }, "execution_count": 150, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b -= b\n", "b" ] }, { "cell_type": "code", "execution_count": 151, "metadata": { "colab": {}, "colab_type": "code", "id": "wzUnCKsiYKCP", "outputId": "a2a76868-1dff-4ba1-9631-6d51dd1d6523" }, "outputs": [ { "data": { "text/plain": [ "array([[0., 0., 0.],\n", " [0., 0., 0.],\n", " [0., 0., 0.]])" ] }, "execution_count": 151, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Al73RqKSYKCR" }, "source": [ "**Перевод из PyTorch в NumPy:**" ] }, { "cell_type": "code", "execution_count": 152, "metadata": { "colab": {}, "colab_type": "code", "id": "u5rngw4vYKCR", "outputId": "89bad9d3-3f9a-4047-e3ba-3b4c4f65fa2f" }, "outputs": [ { "data": { "text/plain": [ "tensor([[[0., 0., 0., 0.],\n", " [0., 0., 0., 0.],\n", " [0., 0., 0., 0.]],\n", "\n", " [[0., 0., 0., 0.],\n", " [0., 0., 0., 0.],\n", " [0., 0., 0., 0.]]])" ] }, "execution_count": 152, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.FloatTensor(2, 3, 4)\n", "a" ] }, { "cell_type": "code", "execution_count": 153, "metadata": { "colab": {}, "colab_type": "code", "id": "yjK68Kk-YKCS", "outputId": "d45da6e7-f087-4457-8ce7-c4dabd4fd4fa" }, "outputs": [ { "data": { "text/plain": [ "torch.Tensor" ] }, "execution_count": 153, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(a)" ] }, { "cell_type": "code", "execution_count": 154, "metadata": { "colab": {}, "colab_type": "code", "id": "viLUF7gQYKCU", "outputId": "11b68e23-6d5f-4e23-e5e3-4b3b1c7c408a" }, "outputs": [ { "data": { "text/plain": [ "array([[[0., 0., 0., 0.],\n", " [0., 0., 0., 0.],\n", " [0., 0., 0., 0.]],\n", "\n", " [[0., 0., 0., 0.],\n", " [0., 0., 0., 0.],\n", " [0., 0., 0., 0.]]], dtype=float32)" ] }, "execution_count": 154, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = a.numpy()\n", "x" ] }, { "cell_type": "code", "execution_count": 155, "metadata": { "colab": {}, "colab_type": "code", "id": "9HxgiLxKYKCV", "outputId": "fb84e251-e314-4828-ccc6-eb3cf7d40b1c" }, "outputs": [ { "data": { "text/plain": [ "(2, 3, 4)" ] }, "execution_count": 155, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.shape" ] }, { "cell_type": "code", "execution_count": 156, "metadata": { "colab": {}, "colab_type": "code", "id": "SdW5bhmeYKCX", "outputId": "fd819e54-8fc0-4389-92a9-41cfa7dc9786" }, "outputs": [ { "data": { "text/plain": [ "numpy.ndarray" ] }, "execution_count": 156, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(x)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "7-KS81rUYKCY" }, "source": [ "Напишем функцию `forward_pass(X, w)` ($w_0$ входит в $w$) для одного нейрона (с сигмоидой) с помощью PyTorch:" ] }, { "cell_type": "code", "execution_count": 157, "metadata": { "colab": {}, "colab_type": "code", "id": "vkaIwpn6YKCY" }, "outputs": [], "source": [ "def forward_pass(X, w):\n", " return torch.sigmoid(X @ w)" ] }, { "cell_type": "code", "execution_count": 159, "metadata": { "colab": {}, "colab_type": "code", "id": "VyqlDItxYKCZ", "outputId": "87ae60cb-715b-456e-b92e-992ea55db795" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "result: tensor([[1.0000],\n", " [0.9985],\n", " [0.0474]])\n" ] } ], "source": [ "X = torch.FloatTensor([[-5, 5], [2, 3], [1, -1]])\n", "w = torch.FloatTensor([[-0.5], [2.5]])\n", "result = forward_pass(X, w)\n", "print('result: {}'.format(result))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "MzLVLQ2GYKCa" }, "source": [ "

CUDA

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "PrIicAAiYKCa" }, "source": [ "[Краткое видео про то, как GPU используется в обучении нейросетей](https://www.youtube.com/watch?v=EobhK0UZm80)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "2nDkAG6NYKCa" }, "source": [ "Все вычисления в PyTorch можно проводить как на CPU, так и на GPU (Graphical Processing Unit) (если она у вас есть). В PyTorch переключение между ними делается очень просто, что является одной из ключевых его особенностей." ] }, { "cell_type": "code", "execution_count": 170, "metadata": { "colab": {}, "colab_type": "code", "id": "_0Nsl2IoYKCa", "outputId": "93510974-05a5-4a77-fe33-25208cca12cf" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.3411, 0.9390, 0.6315, ..., 0.5529, 0.9262, 0.4085],\n", " [0.7098, 0.3078, 0.2261, ..., 0.3477, 0.4198, 0.9744],\n", " [0.9441, 0.9007, 0.6284, ..., 0.3763, 0.2614, 0.0571],\n", " ...,\n", " [0.2367, 0.0427, 0.3351, ..., 0.7677, 0.6626, 0.9452],\n", " [0.2880, 0.0035, 0.8371, ..., 0.5273, 0.8663, 0.1853],\n", " [0.5813, 0.2115, 0.8499, ..., 0.2056, 0.5935, 0.6047]])" ] }, "execution_count": 170, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = torch.FloatTensor(1024, 1024).uniform_()\n", "x" ] }, { "cell_type": "code", "execution_count": 171, "metadata": { "colab": {}, "colab_type": "code", "id": "FVr5_SAdYKCb", "outputId": "e4f3d375-f02f-402a-a5c5-912ead39ca76" }, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 171, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.is_cuda" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "mAy8GGXAYKCd" }, "source": [ "Переместим на GPU:" ] }, { "cell_type": "code", "execution_count": 172, "metadata": { "colab": {}, "colab_type": "code", "id": "6UGKZbfyYKCd" }, "outputs": [], "source": [ "x = x.cuda()" ] }, { "cell_type": "code", "execution_count": 173, "metadata": { "colab": {}, "colab_type": "code", "id": "ue2SLd9nYKCd", "outputId": "50a73229-ba39-45b2-f4b1-19f790a20643" }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 173, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.is_cuda" ] }, { "cell_type": "code", "execution_count": 174, "metadata": { "colab": {}, "colab_type": "code", "id": "dqs1JNv3YKCe", "outputId": "53a4b10e-bc54-4fc6-cce8-f07723a29a40" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.3411, 0.9390, 0.6315, ..., 0.5529, 0.9262, 0.4085],\n", " [0.7098, 0.3078, 0.2261, ..., 0.3477, 0.4198, 0.9744],\n", " [0.9441, 0.9007, 0.6284, ..., 0.3763, 0.2614, 0.0571],\n", " ...,\n", " [0.2367, 0.0427, 0.3351, ..., 0.7677, 0.6626, 0.9452],\n", " [0.2880, 0.0035, 0.8371, ..., 0.5273, 0.8663, 0.1853],\n", " [0.5813, 0.2115, 0.8499, ..., 0.2056, 0.5935, 0.6047]],\n", " device='cuda:0')" ] }, "execution_count": 174, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "dI73B59PYKCg" }, "source": [ "Перемножим две тензора на GPu и вернём результат вычисления на CPU:" ] }, { "cell_type": "code", "execution_count": 176, "metadata": { "colab": {}, "colab_type": "code", "id": "KS4QxJJoYKCg" }, "outputs": [], "source": [ "a = torch.FloatTensor(10000, 10000).uniform_()\n", "b = torch.FloatTensor(10000, 10000).uniform_()\n", "c = a.cuda().mul(b.cuda()).cpu()" ] }, { "cell_type": "code", "execution_count": 177, "metadata": { "colab": {}, "colab_type": "code", "id": "1ImlgEbwYKCg", "outputId": "3d114438-ea49-4b20-f02e-e7e187be4e5b" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.2262, 0.1965, 0.4094, ..., 0.0736, 0.2933, 0.1275],\n", " [0.1296, 0.1271, 0.4009, ..., 0.0191, 0.0168, 0.4168],\n", " [0.8918, 0.0294, 0.1511, ..., 0.8481, 0.0005, 0.0947],\n", " ...,\n", " [0.2218, 0.2828, 0.3110, ..., 0.0599, 0.6793, 0.6230],\n", " [0.0466, 0.4787, 0.1043, ..., 0.2514, 0.1290, 0.6491],\n", " [0.0375, 0.0251, 0.5316, ..., 0.7669, 0.5638, 0.3338]])" ] }, "execution_count": 177, "metadata": {}, "output_type": "execute_result" } ], "source": [ "c" ] }, { "cell_type": "code", "execution_count": 178, "metadata": { "colab": {}, "colab_type": "code", "id": "4DAN9MhDYKCh", "outputId": "2ee2c90b-07e4-46c6-d7fe-e34b589e8ea1" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.8050, 0.4509, 0.5789, ..., 0.1817, 0.8211, 0.8840],\n", " [0.8009, 0.2267, 0.7719, ..., 0.8452, 0.0451, 0.5779],\n", " [0.9509, 0.0704, 0.7669, ..., 0.8987, 0.0028, 0.2946],\n", " ...,\n", " [0.2457, 0.8630, 0.5177, ..., 0.3907, 0.9143, 0.6533],\n", " [0.8565, 0.8996, 0.1657, ..., 0.9056, 0.6489, 0.7691],\n", " [0.1040, 0.4405, 0.8164, ..., 0.9914, 0.8730, 0.4389]])" ] }, "execution_count": 178, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "xWc-PiA9YKCi" }, "source": [ "Тензоры, лежащие на CPU, и тензоры, лежащие на GPU, недоступны друг для друга:" ] }, { "cell_type": "code", "execution_count": 180, "metadata": { "colab": {}, "colab_type": "code", "id": "1lwV1sgSYKCi" }, "outputs": [], "source": [ "a = torch.FloatTensor(10000, 10000).uniform_().cpu()\n", "b = torch.FloatTensor(10000, 10000).uniform_().cuda()" ] }, { "cell_type": "code", "execution_count": 181, "metadata": { "colab": {}, "colab_type": "code", "id": "G6tYdYtTYKCj", "outputId": "402a280c-4907-49df-92f4-3a5818f72478" }, "outputs": [ { "ename": "RuntimeError", "evalue": "Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #3 'other'", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mRuntimeError\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0ma\u001b[0m \u001b[1;33m+\u001b[0m \u001b[0mb\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;31mRuntimeError\u001b[0m: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #3 'other'" ] } ], "source": [ "a + b" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "QEtwaIxDYKCk" }, "source": [ "Вот ещё немного про то, как можно работать с GPU:" ] }, { "cell_type": "code", "execution_count": 182, "metadata": { "colab": {}, "colab_type": "code", "id": "bJgVfAbUYKCk", "outputId": "3d761ebc-89fc-4e2b-9a3f-54bb74d11ebe", "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[[1.9693, 1.1198, 1.0172, 1.2034, 1.2875],\n", " [1.4131, 1.3164, 1.9345, 1.9337, 1.9572],\n", " [1.7345, 1.2186, 1.9105, 1.6177, 1.1420],\n", " [1.6490, 1.1597, 1.5847, 1.6580, 1.8715],\n", " [1.0228, 1.0453, 1.0512, 1.1437, 1.1437]],\n", "\n", " [[1.7028, 1.4623, 1.1701, 1.0893, 1.0764],\n", " [1.2116, 1.5412, 1.0967, 1.8721, 1.9107],\n", " [1.1340, 1.1484, 1.4099, 1.4459, 1.7837],\n", " [1.3865, 1.0431, 1.3485, 1.6199, 1.0908],\n", " [1.4057, 1.8685, 1.2232, 1.4738, 1.3243]],\n", "\n", " [[1.2735, 1.7283, 1.4247, 1.9058, 1.8978],\n", " [1.2789, 1.7611, 1.8657, 1.1219, 1.0049],\n", " [1.2962, 1.6973, 1.7559, 1.2467, 1.7554],\n", " [1.9915, 1.2676, 1.8824, 1.6168, 1.6185],\n", " [1.8360, 1.0345, 1.2038, 1.7657, 1.8525]],\n", "\n", " [[1.5283, 1.6573, 1.5225, 1.4269, 1.2714],\n", " [1.0401, 1.5209, 1.4956, 1.9208, 1.8503],\n", " [1.2160, 1.6476, 1.2769, 1.7877, 1.7272],\n", " [1.3110, 1.1294, 1.3310, 1.3048, 1.4476],\n", " [1.2117, 1.9958, 1.4981, 1.4867, 1.8652]],\n", "\n", " [[1.2041, 1.9721, 1.5594, 1.8929, 1.7112],\n", " [1.1751, 1.3152, 1.7033, 1.0654, 1.1322],\n", " [1.6051, 1.0064, 1.7763, 1.3360, 1.9534],\n", " [1.7684, 1.1432, 1.1610, 1.6972, 1.7266],\n", " [1.5783, 1.3838, 1.8352, 1.8236, 1.3754]]], device='cuda:0')\n", "tensor([[[1.9693, 1.1198, 1.0172, 1.2034, 1.2875],\n", " [1.4131, 1.3164, 1.9345, 1.9337, 1.9572],\n", " [1.7345, 1.2186, 1.9105, 1.6177, 1.1420],\n", " [1.6490, 1.1597, 1.5847, 1.6580, 1.8715],\n", " [1.0228, 1.0453, 1.0512, 1.1437, 1.1437]],\n", "\n", " [[1.7028, 1.4623, 1.1701, 1.0893, 1.0764],\n", " [1.2116, 1.5412, 1.0967, 1.8721, 1.9107],\n", " [1.1340, 1.1484, 1.4099, 1.4459, 1.7837],\n", " [1.3865, 1.0431, 1.3485, 1.6199, 1.0908],\n", " [1.4057, 1.8685, 1.2232, 1.4738, 1.3243]],\n", "\n", " [[1.2735, 1.7283, 1.4247, 1.9058, 1.8978],\n", " [1.2789, 1.7611, 1.8657, 1.1219, 1.0049],\n", " [1.2962, 1.6973, 1.7559, 1.2467, 1.7554],\n", " [1.9915, 1.2676, 1.8824, 1.6168, 1.6185],\n", " [1.8360, 1.0345, 1.2038, 1.7657, 1.8525]],\n", "\n", " [[1.5283, 1.6573, 1.5225, 1.4269, 1.2714],\n", " [1.0401, 1.5209, 1.4956, 1.9208, 1.8503],\n", " [1.2160, 1.6476, 1.2769, 1.7877, 1.7272],\n", " [1.3110, 1.1294, 1.3310, 1.3048, 1.4476],\n", " [1.2117, 1.9958, 1.4981, 1.4867, 1.8652]],\n", "\n", " [[1.2041, 1.9721, 1.5594, 1.8929, 1.7112],\n", " [1.1751, 1.3152, 1.7033, 1.0654, 1.1322],\n", " [1.6051, 1.0064, 1.7763, 1.3360, 1.9534],\n", " [1.7684, 1.1432, 1.1610, 1.6972, 1.7266],\n", " [1.5783, 1.3838, 1.8352, 1.8236, 1.3754]]], dtype=torch.float64)\n" ] } ], "source": [ "x = torch.FloatTensor(5, 5, 5).uniform_()\n", "\n", "# проверяем, есть ли CUDA (то есть NVidia GPU)\n", "if torch.cuda.is_available():\n", " # так можно получить имя устройства, которое связано с CUDA\n", " # (полезно в случае с несколькими видеокартами)\n", " device = torch.device('cuda') # CUDA-device объект\n", " y = torch.ones_like(x, device=device) # создаём тензор на GPU\n", " x = x.to(device) # тут можно просто ``.to(\"cuda\")``\n", " z = x + y\n", " print(z)\n", " # с помощью``.to`` можно и изменить тип при перемещении\n", " print(z.to(\"cpu\", torch.double))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "iFg82MJIYKCl" }, "source": [ "

Autograd

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "udpOWYK-YKCm" }, "source": [ "Расшифровывается как Automatic Gradients (автоматическое взятие градиентов) - собственно, из названия понятно, что это модуль PyTorch, отвечающий за взятие производных. \n", "\n", "Возможно, для вас это бдет шок, но PyTorch (и любой фреймворк глубокого обучения) может продифференцировать функцию практически любой сложности." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "OAfNuSknYKCm" }, "source": [ "Импортируем нужный класс:" ] }, { "cell_type": "code", "execution_count": 197, "metadata": { "colab": {}, "colab_type": "code", "id": "PJdDG51vYKCm" }, "outputs": [], "source": [ "from torch.autograd import Variable" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "dZzZFE-0YKCm" }, "source": [ "Идея такая: оборачиваем тензор в класс Variable(), получаем тоже тензор, но он имеет способность вычислять себе градиенты. " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "VzMGFf1xYKCm" }, "source": [ "Если а - тензор, обёрнутый в Variable(), то при вызове a.backward() берутся градиенты по всем переменным, от которых зависит тензор a." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "bTVyleFWYKCn" }, "source": [ "**Примечание:** Если вы используете версию `pytorch 0.4.0` или более новую, то ***`torch.Tensor` позволяет брать по нему градиенты, класс `Variable()` использовать не нужно.*** (`torch.Variable()` - deprecated)." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AEDz971fYKCn" }, "source": [ "Примеры:" ] }, { "cell_type": "code", "execution_count": 198, "metadata": { "colab": {}, "colab_type": "code", "id": "rYWqXsAjYKCn" }, "outputs": [], "source": [ "x = torch.FloatTensor(3, 1).uniform_()\n", "y = torch.FloatTensor(3, 1).uniform_()\n", "w = torch.FloatTensor(3, 3).uniform_() \n", "b = torch.FloatTensor(3, 1).uniform_()\n", "\n", "x = Variable(x, requires_grad=True)\n", "y = Variable(x, requires_grad=False)\n", "w = Variable(w, requires_grad=True)\n", "b = Variable(b, requires_grad=True)\n", "\n", "y_pred = (w @ x).add_(b)\n", "\n", "loss = (y_pred - y).sum()\n", "\n", "# берём градиенты\n", "loss.backward()" ] }, { "cell_type": "code", "execution_count": 199, "metadata": { "colab": {}, "colab_type": "code", "id": "ZsePHOo0YKCp", "outputId": "652875bc-e5bd-4a73-abc5-7760e49810b1" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.6708],\n", " [1.5246],\n", " [1.5356]])" ] }, "execution_count": 199, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.grad" ] }, { "cell_type": "code", "execution_count": 200, "metadata": { "colab": {}, "colab_type": "code", "id": "qLkn5C7JYKCp", "outputId": "3bfd1927-4962-45ac-c8d3-053b1ebae784" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.0988, 0.3401, 0.6621],\n", " [0.0988, 0.3401, 0.6621],\n", " [0.0988, 0.3401, 0.6621]])" ] }, "execution_count": 200, "metadata": {}, "output_type": "execute_result" } ], "source": [ "w.grad" ] }, { "cell_type": "code", "execution_count": 201, "metadata": { "colab": {}, "colab_type": "code", "id": "CYkrFbD5YKCq" }, "outputs": [ { "data": { "text/plain": [ "tensor([[1.],\n", " [1.],\n", " [1.]])" ] }, "execution_count": 201, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.grad" ] }, { "cell_type": "code", "execution_count": 202, "metadata": { "colab": {}, "colab_type": "code", "id": "ouMNnbtXYKCr" }, "outputs": [], "source": [ "y.grad" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "DIsqxWIvYKCs" }, "source": [ "**Обратите внимание** - градиенты лежат в поле `.grad` у тех тензоров (Variable'ов), по которым брали эти градиенты. Градиенты **не лежат** в той Variable, от которой они брались!" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Nf2xLNDgYKCs" }, "source": [ "Получить тензор из `Variable()` можно с помощью поля `.data`:" ] }, { "cell_type": "code", "execution_count": 203, "metadata": { "colab": {}, "colab_type": "code", "id": "5wU_TFedYKCt", "outputId": "9b98a122-a26b-4f09-b3c6-933e550682a4" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.0988],\n", " [0.3401],\n", " [0.6621]], requires_grad=True)" ] }, "execution_count": 203, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x" ] }, { "cell_type": "code", "execution_count": 204, "metadata": { "colab": {}, "colab_type": "code", "id": "_NnBKF9kYKCt", "outputId": "f9e34f9d-6442-4a50-cbc4-7ac52a8a026b" }, "outputs": [ { "data": { "text/plain": [ "tensor([[0.0988],\n", " [0.3401],\n", " [0.6621]])" ] }, "execution_count": 204, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.data" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "J094WEHQYKCv" }, "source": [ "

Полезные ссылки:

" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "l1gNHZwOYKCv" }, "source": [ "*1). Статья по PyTorch (на русском): https://habr.com/post/334380/*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ftVUTfvUYKCv" }, "source": [ "*2). Туториалы от самих разработчиков фреймворка: https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html#sphx-glr-beginner-blitz-tensor-tutorial-py*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ovymy5F6YKCw" }, "source": [ "*3). Статья на arXiv о сравнении фреймворков глубокого обучения: https://arxiv.org/pdf/1511.06435.pdf*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "VXEEowQAYKCx" }, "source": [ "4). *Ещё туториалы: https://github.com/yunjey/pytorch-tutorial*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "9cROZcrYYKC0" }, "source": [ "*5). Сайт Facebook AI Research - отдела, который разрабатывает PyTorch и другие активно вкладывается в разработку инструментов для AI: https://facebook.ai/developers/tools*" ] } ], "metadata": { "colab": { "name": "[seminar]pytorch_basics.ipynb", "provenance": [], "version": "0.3.2" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 1 }