Python Language/04 – NumPy.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l-Y2NMDivEPB"
      },
      "source": [
        "# NumPy – Numerical Python Library\n",
        "NumPy is such a crucial library in Python that often NumPy isn't even regarded as a 3rd party library and has become pretty much a core component – if not of the Python itself, then surely of an engineer (especially data engineer's) arsenal. I won't waste any time in prologues and directly jump to the business. We can import NumPy as:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "7onLauBbuguF"
      },
      "outputs": [],
      "source": [
        "import numpy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I4xnP3SLCoHc"
      },
      "source": [
        "We will divide this tutorial in two parts:\n",
        "\n",
        "- Fundamentals – covered here\n",
        "- Advanced – in next notebook\n",
        "\n",
        "## Outline\n",
        "\n",
        "This notebook is highly influenced from the **[NumPy's own documentation](https://numpy.org/doc/stable/user/index.html)**.\n",
        "\n",
        "- NumPy Arrays\n",
        "- Linear Algebra\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EWzRSc2JEQza"
      },
      "source": [
        "## Arrays\n",
        "\n",
        "NumPy arrays are better than Python lists for a number of reasons, including efficiency and some other features we will cover in this notebook.\n",
        "> **Note:** If you are interested, please feel free to look up the differences between the NumPy vs Python arrays in detail over the internet. I would like to mention them one by one and steadily, hence not mentioning them all in a go.\n",
        "\n",
        "There can be a number of ways of making a NumPy array, like:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c7JIeFqwFMPV"
      },
      "source": [
        "### From Python Collections\n",
        "\n",
        "Python collections (lists, tuples or sets) can be simply converted to a NumPy array as:\n",
        "\n",
        "**`<np array> = NumPy.array(<Python collection>)`**\n",
        "\n",
        "For example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "UXjgFKLTFqd4"
      },
      "outputs": [],
      "source": [
        "listA = [34, 21, 45]\n",
        "setB = {2, 4, 6}\n",
        "tupleC = (2, 4, 8, 16, 24)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Arm65ITMF_fW"
      },
      "source": [
        "Now converting them into NumPy arrays respectively:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1kRDUkYsGClW",
        "outputId": "21077366-5d5a-4c80-f103-ecc896ae36eb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[34 21 45]\n",
            "{2, 4, 6}\n",
            "[ 2  4  8 16 24]\n"
          ]
        }
      ],
      "source": [
        "npA = numpy.array(listA)\n",
        "print(npA)\n",
        "\n",
        "npB = numpy.array(setB)\n",
        "print(npB)\n",
        "\n",
        "npC = numpy.array(tupleC)\n",
        "print(npC)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0DRJ_66iGdxD"
      },
      "source": [
        "Here we need to pause a little bit to reflect.\n",
        "\n",
        "- `npA` and `npC` have 3 and 5 members respectively, but `npB` has only 1.\n",
        "- It may sound like `2`, `4` and `6` are separate members of the NumPy array (`npB`), but they are actually part of a single set.\n",
        "- Which also means that NumPy arrays can have Python collections as its members as well.\n",
        "- Tuples are immutable and we haven't touched their immutability one bit here. `tupleC` is still intact and `npC` is a separate array having tuple's members copied into it.\n",
        "\n",
        "By the way, before we proceed. It sounds too much to specify `numpy` before every function call and can be abbreviated/aliased. Usually we follow the `np` alias.\n",
        "\n",
        "---\n",
        "\n",
        "Let's confirm the above points by using the NumPy array's **`size()`** method. Its one of a number of methods for the arrays (we will refer to NumPy arrays as arrays from now on)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_93Rw5DZHos_",
        "outputId": "8b665a6a-79dd-4fa0-ea95-69966ab0e3e2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "3\n",
            "1\n",
            "5\n"
          ]
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "print(npA.size)\n",
        "print(npB.size)\n",
        "print(npC.size)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jZRygB5DIVgm"
      },
      "source": [
        "### Intrinsic Creation\n",
        "\n",
        "We can also make an array from scratch by using the built-in intrinsic methods, like:\n",
        "\n",
        "- `arange()`\n",
        "- `linspace()`\n",
        "- `eye()`\n",
        "- `diag()`\n",
        "\n",
        "We will see the first two right now and matrices one soon in the respective section."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6UrBvXAkI2lx",
        "outputId": "349c607d-092e-4cc9-cc6f-d78cebc646ba"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "range(0, 10)"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "x = range(10)\n",
        "x"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GOkdyKwVIcLB"
      },
      "source": [
        "#### **`arange()`**\n",
        "\n",
        "Whenever we need an array with an arithmetic sequence, `arange()` is the function we are looking for.\n",
        "\n",
        "Its syntax is:\n",
        "\n",
        "`<arr> = np.arange(a,b)`\n",
        "\n",
        "Where $[a,b)$ is the range of the values in the given array."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "37lzouN_JAH5",
        "outputId": "2d9519ca-4caf-448d-8812-320267e9e7e0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "first10 = np.arange(1,11)\n",
        "first10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "csYumGkAJKBz"
      },
      "source": [
        "We can also use it for sequences other than the default (`1`) step as:\n",
        "\n",
        "`<arr> = np.arange(a,b,d)`\n",
        "\n",
        "**Note:** We can check the nth element as:\n",
        "\n",
        "$$a_n = a+(n-1)d$$\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wZXInoswKAEp",
        "outputId": "585f4ad9-31e8-40be-9080-633e6caca69a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  3,  5,  7,  9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33,\n",
              "       35, 37, 39])"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "odd20 = np.arange(1,40,2)\n",
        "odd20"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1xtVHEHCKOm9"
      },
      "source": [
        "#### **`linspace()`**\n",
        "\n",
        "Often while plotting data in 2D, we need to divide the domain and/or range evenly. For example, we have the range of $[1,10]$ and want it to divide in 50 equidisant points.\n",
        "\n",
        "For such a cases, `linspace()` comes to the rescue. Its format is:\n",
        "\n",
        "`<arr> = np.linspace(a,b,n)`\n",
        "\n",
        "where `n` is the number of total values we need (like `50` in the above example).\n",
        "\n",
        "**Important:** Note the closing square braces for the range above. `linspace()` does **include** the `b` as well."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ts278O11LDC1",
        "outputId": "510316db-91a1-42b6-dc3b-18b3bc666b4d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 0.        ,  0.20408163,  0.40816327,  0.6122449 ,  0.81632653,\n",
              "        1.02040816,  1.2244898 ,  1.42857143,  1.63265306,  1.83673469,\n",
              "        2.04081633,  2.24489796,  2.44897959,  2.65306122,  2.85714286,\n",
              "        3.06122449,  3.26530612,  3.46938776,  3.67346939,  3.87755102,\n",
              "        4.08163265,  4.28571429,  4.48979592,  4.69387755,  4.89795918,\n",
              "        5.10204082,  5.30612245,  5.51020408,  5.71428571,  5.91836735,\n",
              "        6.12244898,  6.32653061,  6.53061224,  6.73469388,  6.93877551,\n",
              "        7.14285714,  7.34693878,  7.55102041,  7.75510204,  7.95918367,\n",
              "        8.16326531,  8.36734694,  8.57142857,  8.7755102 ,  8.97959184,\n",
              "        9.18367347,  9.3877551 ,  9.59183673,  9.79591837, 10.        ])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "x = np.linspace(0,10,50)\n",
        "x"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EPFdCMq8MgAw"
      },
      "source": [
        "**Question:** Why don't we get exact 0,0.2,0.4,... and instead are getting these values?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iT4sL5ffKhb5",
        "outputId": "6c568f69-6fdf-4dc1-bc20-8058b111afa0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "3.14\n"
          ]
        }
      ],
      "source": [
        "import math\n",
        "print(round(math.pi,2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "45EXerbkPAAZ"
      },
      "source": [
        "### From Existing Arrays\n",
        "\n",
        "Similarly, we can use slicing (as we saw earlier for the primitive Python collections) and other operations. For example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TSdGK3VGdK70",
        "outputId": "4eb6058d-2b30-4a15-b0d9-dab25f36a588"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[ 2  4  8 16 24]\n",
            "[ 4  8 16]\n"
          ]
        }
      ],
      "source": [
        "npD = npC[1:4]\n",
        "print(npC)\n",
        "print(npD)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mk6JSf62frAx"
      },
      "source": [
        "Nothing is exceptional here. But try to change `npD` values:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5sgeX01hgaYQ",
        "outputId": "acfd6ed3-2f70-49e3-b164-8ca9a576bde5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[ 2 -2  8 16 24]\n",
            "[-2  8 16]\n"
          ]
        }
      ],
      "source": [
        "npD[0] = -2\n",
        "\n",
        "print(npC)\n",
        "print(npD)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ohU8ZXjxgims"
      },
      "source": [
        "**Ouch!** The change has reflected in the original array (`npC`) as well.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ltCofJtiHTVm"
      },
      "source": [
        "In case you forgot, try to do the same for the Python lists:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "BmTOE4GnhbMV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "049cb5ad-96d9-45e0-9aba-5232fd957e63"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Before changing listB's first value [1, 2, 3, 4, 5]\n",
            "Before changing listB's value, listB is: [1, 2]\n",
            "After changing listB's first value, listA is: [1, 2, 3, 4, 5]\n",
            "After changing listB's value, listB is: [-3, 2]\n"
          ]
        }
      ],
      "source": [
        "listA = [1, 2, 3, 4, 5]\n",
        "listB = listA[0:2]\n",
        "\n",
        "print(\"Before changing listB's first value\",listA)\n",
        "print(\"Before changing listB's value, listB is:\", listB)\n",
        "\n",
        "listB[0] = -3\n",
        "\n",
        "print(\"After changing listB's first value, listA is:\",listA)\n",
        "print(\"After changing listB's value, listB is:\", listB)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WaUwUoYFh-Mv"
      },
      "source": [
        "It requires some attention. Whenever we make a new (NumPy) array from an existing one, three types of copies can happen:\n",
        "\n",
        "- **Aliasing** – The same array gets a new alias. Both names are valid.\n",
        "- **Shallow Copy** – The new array refers to the original array and any change in the new array will reflect in the original one too (as we already saw).\n",
        "- **Deep Copy** – New copy is truly a copy (an independent one) and any changes in it won't reflect in the existing array. It's achieved using the **`copy()`** method.\n",
        "\n",
        "Let's illustrate it a bit with the example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VRTitedxHSxl",
        "outputId": "ec50809c-60e8-4aff-d84a-9faf4a2b6c52"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original array's id is: 132042939353296\n",
            "Aliased array's id is: 132042939353296\n",
            "Shallow copied array's id is: 132042939358288\n",
            "Deep copied array's id is: 132042939358000\n"
          ]
        }
      ],
      "source": [
        "npD = npC\n",
        "npE = npC[0:3]\n",
        "npF = npC.copy()\n",
        "\n",
        "print(\"Original array's id is:\",id(npC))\n",
        "print(\"Aliased array's id is:\",id(npD))\n",
        "print(\"Shallow copied array's id is:\",id(npE))\n",
        "print(\"Deep copied array's id is:\",id(npF))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7LWPKlNuItpD"
      },
      "source": [
        "We have already seen how change in the shallow copied array reflects in the original array. Let's wind it up by checking it for the deep copy."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "Ana9MApQIshk",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "36f32e81-0356-487a-d9cd-8a3f1a08cf9e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "npC before changing the deep copy [ 2 -2  8 16 24]\n",
            "npC after changing the deep copy [ 2 -2  8 16 24]\n"
          ]
        }
      ],
      "source": [
        "print(\"npC before changing the deep copy\",npC)\n",
        "\n",
        "npF[2] = 6\n",
        "\n",
        "print(\"npC after changing the deep copy\",npC)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4c-ssL3nKDnO"
      },
      "source": [
        "### From Files\n",
        "\n",
        "CSV etc are common file formats and used for platform independent data sharing. We can easily import a CSV using [**`numpy.loadtxt()`**](https://numpy.org/doc/stable/reference/generated/numpy.loadtxt.html#numpy.loadtxt) as:\n",
        "\n",
        "`np.loadtxt(<csv file name>, delimiter = <>, skiprows = x)`\n",
        "\n",
        "where `delimiter` is `,` for CSV, tab for TSV and so on. While `x` is number of rows (header) to be skipped. If you are unsure about it, set it to 1.\n",
        "\n",
        "> **Note:** Usually, these file handling processes are managed in a better way using the **Pandas** library, something we will see in the following notebook.\n",
        "\n",
        "---\n",
        "\n",
        "Now we will see NumPy's functionalities for a number of applications."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Za32jIiSLJsJ"
      },
      "source": [
        "## Linear Algebra\n",
        "\n",
        "One of the hallmark benefits of NumPy is ease/efficiency of dealing with the matrix (or vector) data. Actually, NumPy has a dedicated module (**`linalg`**) having a number of linear algebra features.\n",
        "\n",
        "Let's cover some of them."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vrZ9IW4ZUeEY"
      },
      "source": [
        "### Vectors\n",
        "\n",
        "Since NumPy one-dimensional arrays are vectors, they can have some linear algebra functionalities as well. In order to keep things simple, I will talk about vectors first and will only later on generalize it for the matrices.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-1F4h0qdX98e"
      },
      "source": [
        "#### Inner Product\n",
        "\n",
        "The inner product of two vectors is:\n",
        "\n",
        "$$a.b = a^Tb$$\n",
        "\n",
        "It can be calculated in NumPy using 3 different syntax:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "7xxPW42fWS-k"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import numpy.linalg as la #LA as an alias of linear algebra"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "ZjbYZSNpV1Sh"
      },
      "outputs": [],
      "source": [
        "a = np.array([1,2,3])\n",
        "\n",
        "b = np.array([4,5,6])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6WCCutckWW5o"
      },
      "source": [
        "**Using `inner()`**\n",
        "\n",
        "We can take the inner product using...`inner()` function (no prizes for guessing that, unfortunately).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yLzh4j_eWk4I",
        "outputId": "6556b907-9306-45c6-eb51-63b942b8f29c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "32"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "c = np.inner(a,b)\n",
        "c"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t16-tDkeWxM2"
      },
      "source": [
        "**Using `dot()`**\n",
        "\n",
        "We can also take it using the `dot()`. Its just an alias for the same function. Since inner product is commutative, so lets try it for $b.a$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "gAZ3AxhQXC2X",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "aaa1425d-a1a4-475a-96ec-6f34247ab663"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "32"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "d = np.dot(b,a)\n",
        "d"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q2yFRwiEXGqM"
      },
      "source": [
        "**`@`**\n",
        "\n",
        "If you ask my favourite syntax (as you will see onwards), its neither `inner()`, nor `dot()` but the closest to mathematics one:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "Vcdz7a_VXZu9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "751e941f-4841-42ca-97bf-bfdc91e99b2b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "32"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "e = a@b\n",
        "e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K0NQogWsjhun"
      },
      "source": [
        "#### Cross Product\n",
        "\n",
        "Cross product is pretty interesting as it results in a vector itself.\n",
        "\n",
        "$$a \\times b = a \\times b \\sin(\\theta)$$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "Mj-RF57Ok6T8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a1baf9f6-08ad-4d5e-f3bc-a1a0864a762e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-3,  6, -3])"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "crossProd = np.cross(a,b)\n",
        "\n",
        "crossProd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eT_9jfSglH17"
      },
      "source": [
        "As we can say, its a vector itself. Its not commutative, as we can see:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ecCLSNbjlMlP",
        "outputId": "8becd17f-b04d-4dcd-fbfc-fa286bdc70ed"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 3, -6,  3])"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "revCrossProd = np.cross(b,a)\n",
        "\n",
        "revCrossProd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cDUG1bHulWUT"
      },
      "source": [
        "They sound like complement of each other (i.e. each element being additive inverse of other)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yp9hLusMYAIn"
      },
      "source": [
        "#### Linear Functions\n",
        "\n",
        "A linear function can be written in the form of:\n",
        "\n",
        "$$f(x) = a_0x_0+a_1x_1+a_2x_2+\\dots$$\n",
        "\n",
        "There can be a number of ways of defining a linear function.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t1opXOnHZhK_"
      },
      "source": [
        "**1. Specifying the co-efficient variables**\n",
        "\n",
        "It can be helpful if we have a linear function with just a few terms. For example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "d87RgxpeZG4N"
      },
      "outputs": [],
      "source": [
        "linearFunc = lambda x: a*x[0]+b*x[1]+c*x[2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_DpFJMJUajIr"
      },
      "source": [
        "Let's test it with some co-efficients and a vector:\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "d_VgGyKVaz93",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a39b7cb1-0e31-4842-da7f-8cc64eb2144d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "70"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "a = 2\n",
        "b = 12\n",
        "c = 3\n",
        "\n",
        "z = np.array([2,4, 6])\n",
        "\n",
        "linearFunc(z)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ePP4nByTa47A"
      },
      "source": [
        "**2.Specifying it as an inner product**\n",
        "\n",
        "Previous method is just ok, but I don't find it neat. A better way would be to have it like:\n",
        "\n",
        "$$f(x)=a^Tx = a.b$$\n",
        "\n",
        ">**Side Note:** Usually, we assume vectors in programming as column vectors, hence $a^Tx$. It would be $ax^T$ if it were other way around, i.e, row vectors.\n",
        "\n",
        "Trying the example above, we get the same answer:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "Vn_N83W7dH0L",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c719f496-8057-485d-a3ee-2828700ac574"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "70"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "linearFunc2 = lambda x:a@x\n",
        "\n",
        "a = np.array([2,12,3])\n",
        "\n",
        "linearFunc2(z)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W5TGwedQd1p0"
      },
      "source": [
        "#### Norm\n",
        "\n",
        "A vector contains a number of elements. To get a \"summary\" of the vector, we have norm. It is defined as:\n",
        "\n",
        "$$||x||_p = \\sqrt{x_1^p+x_2^p+x_3^p+…} = ({x_1^p+x_2^p+x_3^p+…})^\\frac{1}{p}$$\n",
        "\n",
        "It has some special instances, like:\n",
        "\n",
        "- **$L_1$ norm** – Putting $p$ as 1, we get:\n",
        "\n",
        "$$|x| = |x_1+x_2+x_3+…|$$\n",
        "\n",
        "- **$L_2$ norm** – The most common form of norm is Euclidean norm and defined as:\n",
        "\n",
        "$$||x|| = \\sqrt{x_1+x_2+x_3+…}$$\n",
        "\n",
        ">**Note:** Norms beyond 1 are convex.\n",
        "\n",
        "For norm calculation in NumPy, we will use **`linalg.norm()`**:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "uT0oa4m_hvP5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5e1314bc-2655-418e-d7bf-476ccc77598b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "16.881943016134134"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "x = np.arange(1,10)\n",
        "\n",
        "la.norm(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nMd9icJLiE76"
      },
      "source": [
        "Lets verify it. I mean why not?!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "EoPImC51iNYx"
      },
      "outputs": [],
      "source": [
        "import math\n",
        "normLambda = lambda x: math.sqrt(sum(x**2))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "VeOqfFevibSI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "df504514-14b0-409a-badc-a01fc572858b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "16.881943016134134"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "normLambda(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gZcfpfsSixOe"
      },
      "source": [
        "### Matrices\n",
        "\n",
        "Now we can use these bulding blocks to learn how to implement and use matrices in NumPy.\n",
        "\n",
        "There can be a number of ways of making a matrix."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "radPoH_ifxtk"
      },
      "source": [
        "#### From Vectors\n",
        "\n",
        "If we revisit a matrix:\n",
        "\n",
        "$$ A = \\begin{bmatrix}\n",
        "\ta & b & c \\\\\n",
        "\td & e & f\\\\\n",
        "\tg & h & i \\\\\n",
        "  j & k & l \\\\\n",
        "\t\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "It is nothing but a collection of rows or can even be treated as a collection of columns.\n",
        "\n",
        "If we treat it as a collection of rows, it can be written as:\n",
        "\n",
        "$$ A = \\begin{bmatrix}\n",
        "\ta_1 \\\\\n",
        "\ta_2\\\\\n",
        "\ta_3 \\\\\n",
        "  a_4 \\\\\n",
        "\t\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "Where,\n",
        "\n",
        "$$ a_1 = \\begin{bmatrix}\n",
        "\ta & b & c \\\\\n",
        "  \\end {bmatrix}\n",
        "  $$\n",
        "$$ a_2 = \\begin{bmatrix}\n",
        "\td & e & f \\\\\n",
        "  \\end {bmatrix}\n",
        "  $$\n",
        "\n",
        "And so on.\n",
        "\n",
        "---\n",
        "\n",
        "This is known as **stacking**. We can do it for collection of rows using **`vstack()`** as:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "pnMRI0jTlEFl"
      },
      "outputs": [],
      "source": [
        "import numpy as np"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "PnWPxynaizO4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d22b0ee4-6d92-4954-ab43-3818e9381def"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  2,  3],\n",
              "       [ 4,  5,  6],\n",
              "       [ 7,  8,  9],\n",
              "       [10, 11, 12]])"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "a1 = np.array([1,2,3])\n",
        "a2 = np.array([4, 5, 6])\n",
        "a3 = np.array([7,8,9])\n",
        "a4 = np.array([10,11,12])\n",
        "\n",
        "A = np.vstack((a1,a2,a3,a4))\n",
        "\n",
        "A"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rHp3djpnkHra"
      },
      "source": [
        "Similarly, we can also treat a matrix as a stacking of columns (in horizontal order, obviously).\n",
        "\n",
        "The above example will become:\n",
        "\n",
        "If we treat it as a collection of rows, it can be written as:\n",
        "\n",
        "$$ A = \\begin{bmatrix}\n",
        "\ta_1 & a_2 & a_3 \\\\\n",
        "\t\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "Where,\n",
        "\n",
        "$$ a_1 = \\begin{bmatrix}\n",
        "\ta \\\\\n",
        "  d \\\\\n",
        "  g \\\\\n",
        "  j \\\\\n",
        "  \\end {bmatrix}\n",
        "  $$\n",
        "\n",
        "and\n",
        "\n",
        "$$ a_2 = \\begin{bmatrix}\n",
        "\tb \\\\\n",
        "  e \\\\\n",
        "  h \\\\\n",
        "  k \\\\\n",
        "  \\end {bmatrix}\n",
        "  $$\n",
        "\n",
        "And so on.\n",
        "\n",
        "We can do it using **`hstack()`**:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "XAFRe-8OlH6z",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "187180ae-404e-4148-b21d-cb3b8d4485d7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  2,  3],\n",
              "       [ 4,  5,  6],\n",
              "       [ 7,  8,  9],\n",
              "       [10, 11, 12]])"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "a1 = np.array([[1],[4],[7],[10]]) #Making a column vector here.\n",
        "\n",
        "a2 = np.array([[2],[5],[8],[11]])\n",
        "\n",
        "a3 = np.array([[3],[6],[9],[12]])\n",
        "\n",
        "B = np.hstack((a1,a2,a3))\n",
        "\n",
        "B"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Drt2PsbNoS_O"
      },
      "source": [
        "#### From builtin functions\n",
        "\n",
        "As we saw above that vectors can be made using the built-in functions (like `arange()`). Similarly, we can make matrices as well using some intrinsic functions, like:\n",
        "\n",
        "- `ones()`\n",
        "- `zeros()`\n",
        "- `identity()`\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Broadcasting\n",
        "\n",
        "One of the confusing aspects of NumPy is broadcasting and hence requires a special attention.\n",
        "\n",
        "Try it with an example first. What happens when we perform"
      ],
      "metadata": {
        "id": "-6RhnaEKJEJh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([1,2,3])\n",
        "\n",
        "B = A + 1\n",
        "\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5YCwQzuyWZ39",
        "outputId": "887bfca3-3091-40f4-ae89-6e4f3ed11525"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([2, 3, 4])"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "I am sure any linear algebra student/expert will find it bizarre (which it is). But lets see what happens behind the scenes:\n",
        "\n",
        "$1$ which is a scalar or a NumPy array of size $(1,1)$ will be _stretched_ to the same size as $A$ to make them consistent.\n",
        "\n",
        "In other words,\n",
        "\n",
        "  $$A + 1 =\n",
        "  \\left[ {\\begin{array}{cc}\n",
        "    a_{1} \\\\\n",
        "    a_{2}\\\\\n",
        "    \\vdots \\\\\n",
        "    a_n\n",
        "  \\end{array}} \\right] + \\left[ {\\begin{array}{cc}\n",
        "    {1} \\\\\n",
        "    1\\\\\n",
        "    \\vdots \\\\\n",
        "    1\n",
        "  \\end{array}} \\right] = \\left[ {\\begin{array}{cc}\n",
        "    a_1+1 \\\\\n",
        "    a_2+1\\\\\n",
        "    \\vdots \\\\\n",
        "    a_n+1\n",
        "  \\end{array}} \\right]$$\n",
        "\n",
        "  Actually, we can write the LHS of above equation as:\n",
        "\n",
        "  $$A + \\textbf 1$$\n",
        "\n",
        "  This is totally consistent with the Linear Algebra (for example, check _Convex Optimization, Boyd and Vandenberghe, Cambridge (2004)_) where a vector with all components one is represented as $\\textbf 1$ (notice the boldface here).\n",
        "  \n",
        "\n"
      ],
      "metadata": {
        "id": "KNecs4JEWoox"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Why Useful?**\n",
        "\n",
        "Broadcasting makes sure to use the vectorizing operations of NumPy (i.e. using C loops rather than Python).\n",
        "\n",
        "**Why Its Not a Good Idea**\n",
        "\n",
        "Its inconsistent with linear algebra and unnecessary wastage of memory. So before proceeding further, its necessary to know the bottomline: **Don't use Broadcasting if you are unsure.**"
      ],
      "metadata": {
        "id": "8_MITmyPWVCX"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n7FxB6PBVfEY"
      },
      "source": [
        "## Acknowledgements\n",
        "\n",
        "This notebook wouldn't have been possible without:\n",
        "\n",
        "- NumPy Documentation\n",
        "- Applied Linear Algebra, Stephen Boyd (2018)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNGoBhvZo/sQB9C6hkvoaf8"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}