codes/course2/Tensorflow Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial\n",
    "\n",
    "Welcome to this week's programming assignment. Until now, you've always used numpy to build neural networks. Now we will step you through a deep learning framework that will allow you to build neural networks more easily. Machine learning frameworks like TensorFlow, PaddlePaddle, Torch, Caffe, Keras, and many others can speed up your machine learning development significantly. All of these frameworks also have a lot of documentation, which you should feel free to read. In this assignment, you will learn to do the following in TensorFlow: \n",
    "\n",
    "- Initialize variables\n",
    "- Start your own session\n",
    "- Train algorithms \n",
    "- Implement a Neural Network\n",
    "\n",
    "Programing frameworks can not only shorten your coding time, but sometimes also perform optimizations that speed up your code. \n",
    "\n",
    "## 1 - Exploring the Tensorflow Library\n",
    "\n",
    "To start, you will import the library:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Aduan\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\framework\\dtypes.py:493: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Users\\Aduan\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\framework\\dtypes.py:494: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Users\\Aduan\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\framework\\dtypes.py:495: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Users\\Aduan\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\framework\\dtypes.py:496: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Users\\Aduan\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\framework\\dtypes.py:497: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Users\\Aduan\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\framework\\dtypes.py:502: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.5.0\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "import numpy as np\n",
    "import h5py\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "#我电脑里装的是tensorflow v2，以下两行代码可以把tf当v1版本的tensorflow使用\n",
    "#import tensorflow.compat.v1 as tf\n",
    "#tf.disable_eager_execution()\n",
    "#后来用xavier初始化的时候出现版本冲突，我又装了个python3.6+tf1.5的环境，所以import的代码由换回来了\n",
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "\n",
    "from tensorflow.python.framework import ops\n",
    "from tf_utils import load_dataset, random_mini_batches, convert_to_one_hot, predict\n",
    "\n",
    "%matplotlib inline\n",
    "np.random.seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have imported the library, we will walk you through its different applications. You will start with an example, where we compute for you the loss of one training example. \n",
    "$$loss = \\mathcal{L}(\\hat{y}, y) = (\\hat y^{(i)} - y^{(i)})^2 \\tag{1}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n"
     ]
    }
   ],
   "source": [
    "y_hat = tf.constant(36, name='y_hat')            # Define y_hat constant. Set to 36.\n",
    "y = tf.constant(39, name='y')                    # Define y. Set to 39\n",
    "\n",
    "loss = tf.Variable((y - y_hat)**2, name='loss')  # Create a variable for the loss\n",
    "\n",
    "init = tf.global_variables_initializer()         # When init is run later (session.run(init)),\n",
    "                                                 # the loss variable will be initialized and ready to be computed\n",
    "with tf.Session() as session:                    # Create a session and print the output\n",
    "    session.run(init)                            # Initializes the variables\n",
    "    print(session.run(loss))                     # Prints the loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Writing and running programs in TensorFlow has the following steps:\n",
    "\n",
    "1. Create Tensors (variables) that are not yet executed/evaluated. \n",
    "2. Write operations between those Tensors.\n",
    "3. Initialize your Tensors. \n",
    "4. Create a Session. \n",
    "5. Run the Session. This will run the operations you'd written above. \n",
    "\n",
    "Therefore, when we created a variable for the loss, we simply defined the loss as a function of other quantities, but did not evaluate its value. To evaluate it, we had to run `init=tf.global_variables_initializer()`. That initialized the loss variable, and in the last line we were finally able to evaluate the value of `loss` and print its value.\n",
    "\n",
    "Now let us look at an easy example. Run the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Mul:0\", shape=(), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "a = tf.constant(2)\n",
    "b = tf.constant(10)\n",
    "c = tf.multiply(a,b)\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, you will not see 20! You got a tensor saying that the result is a tensor that does not have the shape attribute, and is of type \"int32\". All you did was put in the 'computation graph', but you have not run this computation yet. In order to actually multiply the two numbers, you will have to create a session and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20\n"
     ]
    }
   ],
   "source": [
    "sess = tf.Session()\n",
    "print(sess.run(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! To summarize, **remember to initialize your variables, create a session and run the operations inside the session**. \n",
    "\n",
    "Next, you'll also have to know about placeholders. A placeholder is an object whose value you can specify only later. \n",
    "To specify values for a placeholder, you can pass in values by using a \"feed dictionary\" (`feed_dict` variable). Below, we created a placeholder for x. This allows us to pass in a number later when we run the session. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n"
     ]
    }
   ],
   "source": [
    "# Change the value of x in the feed_dict\n",
    "\n",
    "x = tf.placeholder(tf.int64, name = 'x')\n",
    "print(sess.run(2 * x, feed_dict = {x: 3}))\n",
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you first defined `x` you did not have to specify a value for it. A placeholder is simply a variable that you will assign data to only later, when running the session. We say that you **feed data** to these placeholders when running the session. \n",
    "\n",
    "Here's what's happening: When you specify the operations needed for a computation, you are telling TensorFlow how to construct a computation graph. The computation graph can have some placeholders whose values you will specify only later. Finally, when you run the session, you are telling TensorFlow to execute the computation graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 - Linear function\n",
    "\n",
    "Lets start this programming exercise by computing the following equation: $Y = WX + b$, where $W$ and $X$ are random matrices and b is a random vector. \n",
    "\n",
    "**Exercise**: Compute $WX + b$ where $W, X$, and $b$ are drawn from a random normal distribution. W is of shape (4, 3), X is (3,1) and b is (4,1). As an example, here is how you would define a constant X that has shape (3,1):\n",
    "```python\n",
    "X = tf.constant(np.random.randn(3,1), name = \"X\")\n",
    "\n",
    "```\n",
    "You might find the following functions helpful: \n",
    "- tf.matmul(..., ...) to do a matrix multiplication\n",
    "- tf.add(..., ...) to do an addition\n",
    "- np.random.randn(...) to initialize randomly\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: linear_function\n",
    "\n",
    "def linear_function():\n",
    "    \"\"\"\n",
    "    Implements a linear function: \n",
    "            Initializes W to be a random tensor of shape (4,3)\n",
    "            Initializes X to be a random tensor of shape (3,1)\n",
    "            Initializes b to be a random tensor of shape (4,1)\n",
    "    Returns: \n",
    "    result -- runs the session for Y = WX + b \n",
    "    \"\"\"\n",
    "    \n",
    "    np.random.seed(1)\n",
    "    \n",
    "    ### START CODE HERE ### (4 lines of code)\n",
    "    X = np.random.randn(3,1)\n",
    "    W = np.random.randn(4,3)\n",
    "    b = np.random.randn(4,1)\n",
    "    Y = tf.add(tf.matmul(W,X), b)\n",
    "    ### END CODE HERE ### \n",
    "    \n",
    "    # Create the session using tf.Session() and run it with sess.run(...) on the variable you want to calculate\n",
    "    \n",
    "    ### START CODE HERE ###\n",
    "    sess = tf.Session()\n",
    "    result = sess.run(Y)\n",
    "    ### END CODE HERE ### \n",
    "    \n",
    "    # close the session \n",
    "    sess.close()\n",
    "\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result = [[-2.15657382]\n",
      " [ 2.95891446]\n",
      " [-1.08926781]\n",
      " [-0.84538042]]\n"
     ]
    }
   ],
   "source": [
    "print( \"result = \" + str(linear_function()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*** Expected Output ***: \n",
    "\n",
    "<table> \n",
    "<tr> \n",
    "<td>\n",
    "**result**\n",
    "</td>\n",
    "<td>\n",
    "[[-2.15657382]\n",
    " [ 2.95891446]\n",
    " [-1.08926781]\n",
    " [-0.84538042]]\n",
    "</td>\n",
    "</tr> \n",
    "\n",
    "</table> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 - Computing the sigmoid \n",
    "Great! You just implemented a linear function. Tensorflow offers a variety of commonly used neural network functions like `tf.sigmoid` and `tf.softmax`. For this exercise lets compute the sigmoid function of an input. \n",
    "\n",
    "You will do this exercise using a placeholder variable `x`. When running the session, you should use the feed dictionary to pass in the input `z`. In this exercise, you will have to (i) create a placeholder `x`, (ii) define the operations needed to compute the sigmoid using `tf.sigmoid`, and then (iii) run the session. \n",
    "\n",
    "** Exercise **: Implement the sigmoid function below. You should use the following: \n",
    "\n",
    "- `tf.placeholder(tf.float32, name = \"...\")`\n",
    "- `tf.sigmoid(...)`\n",
    "- `sess.run(..., feed_dict = {x: z})`\n",
    "\n",
    "\n",
    "Note that there are two typical ways to create and use sessions in tensorflow: \n",
    "\n",
    "**Method 1:**\n",
    "```python\n",
    "sess = tf.Session()\n",
    "# Run the variables initialization (if needed), run the operations\n",
    "result = sess.run(..., feed_dict = {...})\n",
    "sess.close() # Close the session\n",
    "```\n",
    "**Method 2:**\n",
    "```python\n",
    "with tf.Session() as sess: \n",
    "    # run the variables initialization (if needed), run the operations\n",
    "    result = sess.run(..., feed_dict = {...})\n",
    "    # This takes care of closing the session for you :)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: sigmoid\n",
    "\n",
    "def sigmoid(z):\n",
    "    \"\"\"\n",
    "    Computes the sigmoid of z\n",
    "    \n",
    "    Arguments:\n",
    "    z -- input value, scalar or vector\n",
    "    \n",
    "    Returns: \n",
    "    results -- the sigmoid of z\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE ### ( approx. 4 lines of code)\n",
    "    # Create a placeholder for x. Name it 'x'.\n",
    "    x = tf.placeholder(tf.float32, name = \"x\")\n",
    "\n",
    "    # compute sigmoid(x)\n",
    "    sigmoid = tf.sigmoid(x)\n",
    "\n",
    "    # Create a session, and run it. Please use the method 2 explained above. \n",
    "    # You should use a feed_dict to pass z's value to x. \n",
    "    with tf.Session() as sess:\n",
    "        # Run session and call the output \"result\"\n",
    "        result = sess.run(sigmoid, feed_dict = {x: z})\n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sigmoid(0) = 0.5\n",
      "sigmoid(12) = 0.9999938\n"
     ]
    }
   ],
   "source": [
    "print (\"sigmoid(0) = \" + str(sigmoid(0)))\n",
    "print (\"sigmoid(12) = \" + str(sigmoid(12)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*** Expected Output ***: \n",
    "\n",
    "<table> \n",
    "<tr> \n",
    "<td>\n",
    "**sigmoid(0)**\n",
    "</td>\n",
    "<td>\n",
    "0.5\n",
    "</td>\n",
    "</tr>\n",
    "<tr> \n",
    "<td>\n",
    "**sigmoid(12)**\n",
    "</td>\n",
    "<td>\n",
    "0.999994\n",
    "</td>\n",
    "</tr> \n",
    "\n",
    "</table> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color='blue'>\n",
    "**To summarize, you how know how to**:\n",
    "1. Create placeholders\n",
    "2. Specify the computation graph corresponding to operations you want to compute\n",
    "3. Create the session\n",
    "4. Run the session, using a feed dictionary if necessary to specify placeholder variables' values. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 -  Computing the Cost\n",
    "\n",
    "You can also use a built-in function to compute the cost of your neural network. So instead of needing to write code to compute this as a function of $a^{[2](i)}$ and $y^{(i)}$ for i=1...m: \n",
    "$$ J = - \\frac{1}{m}  \\sum_{i = 1}^m  \\large ( \\small y^{(i)} \\log a^{ [2] (i)} + (1-y^{(i)})\\log (1-a^{ [2] (i)} )\\large )\\small\\tag{2}$$\n",
    "\n",
    "you can do it in one line of code in tensorflow!\n",
    "\n",
    "**Exercise**: Implement the cross entropy loss. The function you will use is: \n",
    "\n",
    "\n",
    "- `tf.nn.sigmoid_cross_entropy_with_logits(logits = ...,  labels = ...)`\n",
    "\n",
    "Your code should input `z`, compute the sigmoid (to get `a`) and then compute the cross entropy cost $J$. All this can be done using one call to `tf.nn.sigmoid_cross_entropy_with_logits`, which computes\n",
    "\n",
    "$$- \\frac{1}{m}  \\sum_{i = 1}^m  \\large ( \\small y^{(i)} \\log \\sigma(z^{[2](i)}) + (1-y^{(i)})\\log (1-\\sigma(z^{[2](i)})\\large )\\small\\tag{2}$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: cost\n",
    "\n",
    "def cost(logits, labels):\n",
    "    \"\"\"\n",
    "    Computes the cost using the sigmoid cross entropy\n",
    "    \n",
    "    Arguments:\n",
    "    logits -- vector containing z, output of the last linear unit (before the final sigmoid activation)\n",
    "    labels -- vector of labels y (1 or 0) \n",
    "    \n",
    "    Note: What we've been calling \"z\" and \"y\" in this class are respectively called \"logits\" and \"labels\" \n",
    "    in the TensorFlow documentation. So logits will feed into z, and labels into y. \n",
    "    \n",
    "    Returns:\n",
    "    cost -- runs the session of the cost (formula (2))\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE ### \n",
    "    \n",
    "    # Create the placeholders for \"logits\" (z) and \"labels\" (y) (approx. 2 lines)\n",
    "    z = tf.placeholder(tf.float32, name = \"z\")\n",
    "    y = tf.placeholder(tf.float32, name = \"z\")\n",
    "    \n",
    "    # Use the loss function (approx. 1 line)\n",
    "    cost = tf.nn.sigmoid_cross_entropy_with_logits(logits = z, labels = y)\n",
    "    \n",
    "    # Create a session (approx. 1 line). See method 1 above.\n",
    "    sess = tf.Session()\n",
    "    \n",
    "    # Run the session (approx. 1 line).\n",
    "    cost = sess.run(cost, feed_dict = {z: logits, y: labels})\n",
    "    \n",
    "    # Close the session (approx. 1 line). See method 1 above.\n",
    "    session.close()\n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cost = [1.0053872  1.0366408  0.41385433 0.39956617]\n"
     ]
    }
   ],
   "source": [
    "logits = sigmoid(np.array([0.2,0.4,0.7,0.9]))\n",
    "cost = cost(logits, np.array([0,0,1,1]))\n",
    "print (\"cost = \" + str(cost))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Expected Output** : \n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **cost**\n",
    "        </td>\n",
    "        <td>\n",
    "        [ 1.00538719  1.03664088  0.41385433  0.39956614]\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 - Using One Hot encodings\n",
    "\n",
    "Many times in deep learning you will have a y vector with numbers ranging from 0 to C-1, where C is the number of classes. If C is for example 4, then you might have the following y vector which you will need to convert as follows:\n",
    "\n",
    "\n",
    "<img src=\"images/onehot.png\" style=\"width:600px;height:150px;\">\n",
    "\n",
    "This is called a \"one hot\" encoding, because in the converted representation exactly one element of each column is \"hot\" (meaning set to 1). To do this conversion in numpy, you might have to write a few lines of code. In tensorflow, you can use one line of code: \n",
    "\n",
    "- tf.one_hot(labels, depth, axis) \n",
    "\n",
    "**Exercise:** Implement the function below to take one vector of labels and the total number of classes $C$, and return the one hot encoding. Use `tf.one_hot()` to do this. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: one_hot_matrix\n",
    "\n",
    "def one_hot_matrix(labels, C):\n",
    "    \"\"\"\n",
    "    Creates a matrix where the i-th row corresponds to the ith class number and the jth column\n",
    "                     corresponds to the jth training example. So if example j had a label i. Then entry (i,j) \n",
    "                     will be 1. \n",
    "                     \n",
    "    Arguments:\n",
    "    labels -- vector containing the labels \n",
    "    C -- number of classes, the depth of the one hot dimension\n",
    "    \n",
    "    Returns: \n",
    "    one_hot -- one hot matrix\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE ###\n",
    "    \n",
    "    # Create a tf.constant equal to C (depth), name it 'C'. (approx. 1 line)\n",
    "    C = tf.constant(C, name = 'C')\n",
    "    \n",
    "    # Use tf.one_hot, be careful with the axis (approx. 1 line)\n",
    "    one_hot_matrix = tf.one_hot(labels, C, axis = 0)\n",
    "    \n",
    "    # Create the session (approx. 1 line)\n",
    "    sess = tf.Session()\n",
    "    \n",
    "    # Run the session (approx. 1 line)\n",
    "    one_hot = sess.run(one_hot_matrix)\n",
    "    \n",
    "    # Close the session (approx. 1 line). See method 1 above.\n",
    "    sess.close()\n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return one_hot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "one_hot = [[0. 0. 0. 1. 0. 0.]\n",
      " [1. 0. 0. 0. 0. 1.]\n",
      " [0. 1. 0. 0. 1. 0.]\n",
      " [0. 0. 1. 0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "labels = np.array([1,2,3,0,2,1])\n",
    "one_hot = one_hot_matrix(labels, C = 4)\n",
    "print (\"one_hot = \" + str(one_hot))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output**: \n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **one_hot**\n",
    "        </td>\n",
    "        <td>\n",
    "        [[ 0.  0.  0.  1.  0.  0.]\n",
    " [ 1.  0.  0.  0.  0.  1.]\n",
    " [ 0.  1.  0.  0.  1.  0.]\n",
    " [ 0.  0.  1.  0.  0.  0.]]\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5 - Initialize with zeros and ones\n",
    "\n",
    "Now you will learn how to initialize a vector of zeros and ones. The function you will be calling is `tf.ones()`. To initialize with zeros you could use tf.zeros() instead. These functions take in a shape and return an array of dimension shape full of zeros and ones respectively. \n",
    "\n",
    "**Exercise:** Implement the function below to take in a shape and to return an array (of the shape's dimension of ones). \n",
    "\n",
    " - tf.ones(shape)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: ones\n",
    "\n",
    "def ones(shape):\n",
    "    \"\"\"\n",
    "    Creates an array of ones of dimension shape\n",
    "    \n",
    "    Arguments:\n",
    "    shape -- shape of the array you want to create\n",
    "        \n",
    "    Returns: \n",
    "    ones -- array containing only ones\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE ###\n",
    "    \n",
    "    # Create \"ones\" tensor using tf.ones(...). (approx. 1 line)\n",
    "    ones = tf.ones(shape)\n",
    "    \n",
    "    # Create the session (approx. 1 line)\n",
    "    sess = tf.Session()\n",
    "    \n",
    "    # Run the session to compute 'ones' (approx. 1 line)\n",
    "    ones = sess.run(ones)\n",
    "    \n",
    "    # Close the session (approx. 1 line). See method 1 above.\n",
    "    sess.close()\n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "    return ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ones = [1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "print (\"ones = \" + str(ones([3])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output:**\n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **ones**\n",
    "        </td>\n",
    "        <td>\n",
    "        [ 1.  1.  1.]\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 - Building your first neural network in tensorflow\n",
    "\n",
    "In this part of the assignment you will build a neural network using tensorflow. Remember that there are two parts to implement a tensorflow model:\n",
    "\n",
    "- Create the computation graph\n",
    "- Run the graph\n",
    "\n",
    "Let's delve into the problem you'd like to solve!\n",
    "\n",
    "### 2.0 - Problem statement: SIGNS Dataset\n",
    "\n",
    "One afternoon, with some friends we decided to teach our computers to decipher sign language. We spent a few hours taking pictures in front of a white wall and came up with the following dataset. It's now your job to build an algorithm that would facilitate communications from a speech-impaired person to someone who doesn't understand sign language.\n",
    "\n",
    "- **Training set**: 1080 pictures (64 by 64 pixels) of signs representing numbers from 0 to 5 (180 pictures per number).\n",
    "- **Test set**: 120 pictures (64 by 64 pixels) of signs representing numbers from 0 to 5 (20 pictures per number).\n",
    "\n",
    "Note that this is a subset of the SIGNS dataset. The complete dataset contains many more signs.\n",
    "\n",
    "Here are examples for each number, and how an explanation of how we represent the labels. These are the original pictures, before we lowered the image resolutoion to 64 by 64 pixels.\n",
    "<img src=\"images/hands.png\" style=\"width:800px;height:350px;\"><caption><center> <u><font color='purple'> **Figure 1**</u><font color='purple'>: SIGNS dataset <br> <font color='black'> </center>\n",
    "\n",
    "\n",
    "Run the following code to load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Loading the dataset\n",
    "X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Change the index below and run the cell to visualize some examples in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y = 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD7CAYAAACscuKmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO19a4xd13Xet+5zHpwhZ/gSKVIvS5Yt6xkzsl27rmJZhuwkForCRQKkUAsD+pMWDpoillugQAoUUFEgSH8UBYQmjYCkcd0kjgQjSKIyNho3flGWZOthmpJMiZSGHJJDcl73fXZ/zJ2711rnnj373hneS+SsDyDvPnfvs88++549Z6291voWOedgMBj+/qMw7gEYDIbRwBa7wZAT2GI3GHICW+wGQ05gi91gyAlssRsMOcG2FjsRPUpEJ4noDSJ6cqcGZTAYdh40rJ2diIoAfgrgEQBnAfwAwK86517bueEZDIadQmkb5z4I4A3n3FsAQERfBfAYgMzFPj8/547ceOM2LgkQRbfc1nV2rsv4k3jL0J/gULv46bkG83PdY0gHsmvsd8a7j30GsmrPnHkXS0tLfX/c7Sz2GwGcYcdnAXwkdMKRG2/EX/z5/9440A9b5BNMwYeUotoRMe0lNQzKrmN9uuxm8sdLjYMyynJK5BSE2lGgZTbkuLJ/C8r4PtVf4NiFBhV4okVVpASaauWCtXF9xkq/0e30YdIrhxc7e/5cImpct/Wjn30s87Lb0dn7/YSp8RHRE0R0gohOLC0tbeNyBoNhO9jOm/0sgKPs+AiA93Qj59zTAJ4GgHvvuVu/iHog9mfCBV7t/I9n6O2d7oH1wWpJ/b2TIhVl1vGD9Jsr9v0qR+myXoHq652XKgPzzb8dQGfgTcVvNsAoYuECR+NCcKpSk+CfwaH30CLabOfN/gMAdxDRrURUAfArAJ7bRn8Gg+EaYug3u3OuTUT/EsBfASgC+H3n3Ks7NjKDwbCj2I4YD+fcXwD4ix0ai8FguIbY1mLfHrSmlb07LPVo1/d7fR4FNDkK7nlyfVXueJLYgs/cOhdfpHqP3Wcgfi+hhqoqTu0Pm/YylOz0bQ6wPR9x3TD677mkO4290+z9krRlIc4glmTWhO87q07vGWUb6frtc6Vh7rIGQ05gi91gyAnGKMYHxJCgqJstV8Y6hYlrDWAjcegv6oUvq/pwcfcprxvuM6YmfZshkbD/QXq4AZUqKFr3bziI2Lp1b+mzwv41IfUw9nrZ4n7oTijjcQyJ5sOoQ/ZmNxhyAlvsBkNOYIvdYMgJRq6zb6qsIXVVuwxKc9UQuj0CuizFtQshNkItfeKQmw4hDTDbjhO4dCjsJtQ1N0XqPgafybBJzWVVIDTLWcE0ab9t17fdRvchM2is9syuPVy8TLA2pkt7sxsMOYEtdoMhJxiD6c2x/zmyA8Szvd8CnnZaIhTdx4l96SgvLorx2PnM7tIiWyAOPkvUS403GG2WIdAF5Dzt1RepCQRrsxQNPb6saER9oujdhcarPeP6m/bS9xL3/IVEcKfuNKt/l3o4+186bLAcXP20N7vBkBPYYjcYcoIxiPERAocOsMjyaguIyKEuQp5UMr5l+zvM6VP6E0MASnQPBaCI3mJ3gwch+hgcKaKPzAnP9pxM748HRPysPlKaQAYRR2ZvW+zUb9k2AkNGA4XVhK1hb3aDISewxW4w5AS22A2GnGBsUW9BjSOWYHEg7oRhNNHscQjtKTrUSn0xlMcctB1xuD6CZqgsZG+EhLzfBvcxC5+Z4sxgX4TMWhFfb9l42Ki6+DmI/TUGoSPZgL3ZDYacwBa7wZATjD4QZlPcCNjGQmKxNLMM4lnGmg3OiZDqhIuLg5nk4mxqLkC2TsG5ihXPs4NCso5SpshIdWJIS1NgTPGIp5DIPiu6pQi0Cfc51LCCIPWZhr3ZDYacwBa7wZAT2GI3GHKC0ZveNvWTlHltCHNSvMqOLB01zS+frfNmmlYGsYKkQukymoXcW0OklVxvjJzSdIRghkEpEAUYTtW6fbihJzyDNCLSRLdjiLa9xd1n2tTpUm00tnyzE9HvE9EiEb3CvpsnoueJ6FT3c26rfgwGw3gRI8b/AYBH1XdPAjjunLsDwPHuscFguI6xpRjvnPu/RHSL+voxAA91y88A+BaALw926ZCYqlpGkqLFO5Zl84FJATZOTwgSVGRL2X3uK+sGQmLr9j3jQt5vsemqroUQn6027YAIHjxHE2AM0f0AJ2U/3iETYHb0YBaG3aA76JxbAIDu54Eh+zEYDCPCNd+NJ6IniOgEEZ1YWlq61pczGAwZGHY3/jwRHXLOLRDRIQCLWQ2dc08DeBoA7r3n7p5wE0+6AMH3FiLpkjxlCFQGrhUSg4eQW0PinEt5AGbsbgd28KP59DJbDYCQ2hGJAaTnQLswnUf2cdyuffrBivR+GxLDBAplB3rtvAfdcwAe75YfB/DskP0YDIYRIcb09scAvgPgTiI6S0RfBPAUgEeI6BSAR7rHBoPhOkbMbvyvZlQ9vMNjMRgM1xBjJK/I1q1ChqZYvTwcoRWna8Y6wmnChBC3PaJq1AWG1bhFF/FEjwFbUFy74GkhZpJEVmVOgTZCZXuWxUckRur2wUjFyEtFewCGPOgGh/nGGww5gS12gyEnGLEY73piFoVMUkG5MtLzLmQ3C7naxfaROT7tJZct+wZjRwJyGvdqS2W8zfDCC3p0pdz8eDHb5U+oMqk+I/rTzdLkcn0Gm0ampx2AhKkGIQ+0YZFNjqEQ+1sE+guqsBGwN7vBkBPYYjcYcgJb7AZDTjBSnd2B6StBF8TBI3o0grnHhNkpoCcGlOqAVUvUhl1iVVVkeF+QuyJ4b/27TM8uM4MGxhsilJC6eNQwwnWRem62G6km/dgp8orIE0MbC5ld6HUQ6sPJzz6wN7vBkBPYYjcYcoLRmt4cF6VC7ljZjBJShB3AfJJhbgsST0S6jKVMKbEmtaClKeR5F/Ckysz1rG1S3iTlnPRcKxTLvi6Shz5NgDGMaSs2mi1gtk194frXXgPOvHBE3yCResNfO3QVe7MbDDmBLXaDISe4bqik+zWJ7muzy6ECM1QfwXb9t8FT0nhAfJYszYHgFCGNx4t5cvPcH3QaNdFu5ZXv9srtq5dF3dT77+uVp4/ezjqP4+TbvHoMwpvsWZXxW+dZPmgDOU5G9x9XE992EDV16yb2ZjcYcgJb7AZDTmCL3WDICcZGXhEibE/rUzvApiD631mzix5FyIstFJkX7XUmuCizO0narV758qvfF83qp3oJftBaXxd1Sxc8f+htv3ywV67s2q0G0v+6qfFmVw2J7XtYhkyF4d524m7ixh8kZU3bGLe8qr3ZDYacwBa7wZATjEGMd+x/D05mEZtyaFjRMUBntgX6G+aGDfRIc3TEkXSErpC0273y0k9e6pVX3zwp2nVa/rzVtbqoW19Z65UPLV/plcu7ZjNHodWJeJNUnLthNN1DJJlHWIeKvFiqaTC6aOALDGeWzIa92Q2GnMAWu8GQE9hiNxhygpHr7D3uiiCZYyQ1YMrsFEnWEAqwC3YRq5NlXHjjCtktA6QUWV12mHkNAC6+/kKvvHryZd9fW0a2ra1599m1ZlPUrXNCy0Kx73VTQwqRkYSgIu4EKOOHCiiz6Wnrv7eyY8bXIckj4xA/ypjeY9I/HSWibxLR60T0KhF9qfv9PBE9T0Snup9z0SMzGAwjR4wY3wbwm865DwL4KIBfJ6K7ADwJ4Lhz7g4Ax7vHBoPhOkVMrrcFAAvd8goRvQ7gRgCPAXio2+wZAN8C8OUt++sKHANRpmfKtyFWroAqEJDnQhFm8XzfvPtBIsX4OLKv1Wk2euVzP/o7Ubf21qu9coX9vPWGFNVrdS/G15pSFSjuP9QrT8zu8WMaSHyOqw3zqmUcBCnc4kxvw1u1hlNXQpGQ2dF4AQPmEMMYaIOOiG4B8ACA7wE42P1DsPkH4cDglzcYDKNC9GInol0A/hTAbzjnlgc47wkiOkFEJ5YuX976BIPBcE0QtdiJqIyNhf5Hzrk/6359nogOdesPAVjsd65z7mnn3DHn3LH5OdvDMxjGhS11dtrwXf09AK87536HVT0H4HEAT3U/n425YE8nGSTRmaBtibNPBVKshd1sI5WhkAomhjtAZFvWdkSztiravffi3/bKy6d+LOqmKxP+vMTr4utra6Jdg+npq822qLvtjrt75RLrLzQ38ZFig+Rzy9LTBzBnBsfVH6los2iLa+S9BOpcKA+hgI6m3PpOY+zsHwfwzwD8mIg2na3/LTYW+deI6IsA3gHwhYi+DAbDmBCzG/9tZP+NeXhnh2MwGK4VRh/1timrDu3CFJDHs7zkdGVs2uchIa6kOpTEE9lGo8baSq98+rt/Ldqtvn2qVy53iqKu1vEieZ15xq0ur4h2aw0f6ZbMyL2Uox+6v++YwoQJgfRPAS7+oSK70iGT2R1mkZYMGakY3zr7tw0RlMoU39nP92DRgxsw33iDISewxW4w5AQjF+Mzd+N3PhtPNpjcFOZkD+0cx8mBIW9A7SXXWPXuC2d+8De98uqZU6Jdq+bF87Wa3ElvtPxxg3vJ1SRvfKvof/r7P/3Lom56zzwbbuA+Y00LgfkYxisxhVjPvliWEW3KYamykqYk+uDHLun0ysXqlBzHxGTfS+lRhr0UB/I7TcHe7AZDTmCL3WDICWyxGww5wRhzvQ1uOhjmMtlfZFVw/vCUhhnZRxzRQv3qRXG8+PK3e+XO+bO9ctnJv8kt8sdXFef7lcueILLTZpFuLA0zANz84Ed65fc98BFkwQV02RDFSKbuqZ3ThCky1Dib1z02lTQfk1OkH82rl3rlxvl3RF374nl/sC5NmI6ZNwv8BpiODgBT9/g5rt5wS2qUmwgEdarnMZCPIAP2ZjcYcgJb7AZDTjC29E8hM0K0o1bQ3BOKMpEjiR5IhqdWWmRldYk0ja0tevF8+eQJUVdY9SI4MVG9Vpd9XF3xontdmYJWmYmtxe7tro98TLT7+V/6J71yRYmcsQgSMmTkuQqJ4NFBJvpSbI47dRnw01pm4vnFhV65efE90W5t0dcV65Loo1rwy6QA6bHIGfRcwQ+ssXhBtFtperPckc8cFnWFslexwuJ4aLbM9GYwGLqwxW4w5AS22A2GnGB8ud5SykmASDKrSuvhAWKLaMpDMS5Vl2SYcVxHtGutebfXldOvibrG+dO9cqElzT+1hu9ndZW5ujYaol2HRbZ11DSWZ2Z65Xs/+Zle+cOf/pxoV52U7pwcUhX3Wmlan3SBujjwvH6pHH+s0+byUq+8vvAz0ax5/u1eub2iqM8afh6p4+e3QPI9V2j7uvV1Od/NxOvwibrPJvuizZ6DWkO6J09PeeLOw4orPyvQLZWOOwAzvRkMhh5ssRsMOcHo0z9tSjABcolUJFoWz3tKExg8725a/OFia7Y5qdP2ot76e2+JZmunPXd7qSVNQZNsyldbUvxfZimZanXff6sjTW8tJi4WpqQ4/vHPP9Yrv++BB/04StKDzrFIrnQkWtK3XaLUFT4/acsmF8/9O6VQkO+XhInPzSvSo3D5Z14Fqp/zonvtyhXRrsQi+MpFaRoTvyG7lh5vk3nCLSu+vqTF50Ced5Wn0WJqwq4bDol2tx37RK9cLFeQCWFh1B6L2/MytTe7wZAT2GI3GHKC0YrxziHpiohpBze+DRlIghNoJ0+KDPQPeb+pGk5UcOXUD3vlxruSXEJMakmKbDUmEq7W5G58i4n1LbZTX69LL7lm4u/t/s/+Y1F38z0f9gdsCloq8EPsskPuDktLA6tTu8hyE1m+N/jv6Vi6qubSOdGudvbNXjm5Kr3OymBiccn3365Kj7/lZUa13ZaqRrHgx9Fhc6pVkgbj5Lt0VYrx63XftjwxLepmj9zSK3/ofj/3Rz94t2g3PbvbH7js53t79BRh2JvdYMgJbLEbDDmBLXaDIScYqc7uACRd84QLkVdoXVxQxUfmcQqRcwu9PBA9lEi97vKbP+qVGwte1yxqSyGxVMlNqeeuMM+41RWZ1qnBItYaTE9XXeCuh3+xVz5y132irsMiwFwnW98Ok2cyfbvD9Ny69ApzLMJM19Uve/27veTJHwoNqQ9XmEmwrExSBF/XYF6EidrDaNR8FODamhwHR6vt52ZdeSXSpNfF97xfzultt93ZKx+86VZRN3fwhl65VKlmXpsjPdv9zc6DGNqc+uyHLd/sRDRBRN8nopeJ6FUi+u3u9/NE9DwRnep+WtZGg+E6RowY3wDwKefcfQDuB/AoEX0UwJMAjjvn7gBwvHtsMBiuU8TkenMANuXNcvefA/AYgIe63z8D4FsAvrxFZz6Ig7Spxh+7gBgvxJyU6S2W152LtzroxtetXVgQdcvv/LRXLvA0Sx0pIheZF1eipOdmw3tqNVSwRJMd15lYfOsnPiPacdG93ZIirfB+YyK4a0tCBs7B5jQXOgvkaV9kpjIVZFLmJruC/C2IzUmBTUJ1Qnr8cY+6Vkt6CtZW+Xx48b+uRPBVJtZfXJOcfB1GNjF70JNG3H7Pz4l2Rz94T6+8e99BOUbllceR6dWWeq6yn8cMno8gUupnhBwfm5+92M3gugjgeefc9wAcdM4tbAzWLQA4EDdMg8EwDkQtdudcxzl3P4AjAB4koru3OmcTRPQEEZ0gohNLl69sfYLBYLgmGMj05py7gg1x/VEA54noEAB0PxczznnaOXfMOXdsfm5PvyYGg2EE2FJnJ6L9AFrOuStENAng0wD+E4DnADwO4Knu57Nb9eXgfD4skspsItxg9d+g/ma5kNkslRo460jpVq2aN4ddPvWyqGszE0+zyUxcStGamPA6Xkm5yxZLvu30pKzbO+P12aurnp+8flHymJ/5/v/x/SmTWqHjdfMOcwGF0tnLzP2UkzoAwEzFm7wmyN9LWf0u5ZI3NRWqMqqu2fbjWmERfOsN6babsGu3lC5eZ5z4DR6VVpPtaiVvNjv8YamL337fsV5576Ebe+UUySYNa/Jiz1zI7uVCzy0bBidIGWAgoT43EWNnPwTgGSIqYkMS+Jpz7htE9B0AXyOiLwJ4B8AX4odmMBhGjZjd+B8BeKDP95cAPHwtBmUwGHYeI456A5KuSYaUqcbx4wChlkgXlOKPUxfTF+/Xn/KSW3rjlV65eXVJ1LVb/TnMNJ8ZJzvoKDF7knG/VZXcV2WueOv1q/779fOiXfsd752mvfCqTASvsOiwsuKcK5Y8V11bcaGXJ2d75dlJ30dJpaEqMtNbJ5Hiecex47ZXJzS/2/IyU41aUtVosCi1yl5PBnH4Afnu2X/r+/145/eJOkGWIR65bIIUrZaFfDaFsZf1kSi1iauAHZVCqrPMNq7XfLlYlqpR5ZD33ivNS/NgjO5hvvEGQ05gi91gyAlGzEHn4DneCumqTQT46aDjOcRpoQCX/lg5J3e61895WmKnPLqaYieZ7WaTFIM7zDutWFSphMr+5kqKQKE84cW2g/Pz/pyq8uBit9aamRVV6+ssOIXxTJerE6JdkYn4jYYc42rTnzfFLAtptYlRPavgFL6zzq0CPNgHkOmqKvtvFHVHP+A9BQ+87wO98uSu3aKdoKPW6iHnFGRefSnPN6ZuJYroo9P0Y2zXZCBP44pPL7W66FNKtVlWWAAoMRWlrNTUEiMjKYpAGPmwL7/pPTjnP/lZUVeek+pLP9ib3WDICWyxGww5gS12gyEnGDlvvFevss0bmq9d6N8BXm1RmQqc81/UV71Z69KpH4t2PJqto6LZhKWMl9WfTGmqkWMsMP2yXJFeXIWC148rpQnWTvVRYnp0Xer9VPTncSKLYkn+1MLUqUyHTU6+eMn3UVX3wk1vTpveOIkGm8aS4o3fd+vtvfKRf/iLoq465c2DPJJQE3E01735cfX8GVG3uuA5/TnZRklFspXZuEiZzcByBFBb7uOIoTCvQWrK32WC7ZG4lpzHOpvvEvstOk6l+274qMPS2bdF3d49+7EV7M1uMOQEttgNhpxg9OmfuhIM6agB4RqXnXYpxOsuAmFUFEHCiBwWTzIuuWVJyFBkqYTaLWXn4x5kXJ1QaZy415ZTqkCjwc1VUlxssuCUChO7lSOVEOMLRfn3mmd1LTA1oaNMUjUeZKLG0Wn3DzxaV/fCUy2VSnIc3OSVME4+TdNW2bu3V9YUEQ3GMd+84r0G6xffle0uM467piSvqLD5abPxl6rao9BPMqkcTwXOPU9yyXTYs9lo+mfsytJV0W4p8cdF9eBygpOEPVgJ5HNVZw3nI/nuOOzNbjDkBLbYDYacwBa7wZATjFxn7yGV7C1QJyKSQm6w2QQBS2c8z/vyu76s9bMGy7+mVFQQ09cKbE9AkzoQi6TrKMbJJhtXUblDtpo8isz3ofkOK5xcYkLqbhNT3vRWY2ai1XXpzrq84iOvSh05B9OMlCIpcndZBXbtpJUdDyaISZRu3zjnTWPnL56VXTAX0yKbjyl1zzwyr7hbUiFS2bddWfH6/EWlU5Pz19JuqiIysiB/jBbT09fXvVvthQsyYrLI3quJerBqjAhlnf0W0/ukC+ydxz7aK++5+Q45xkCkaG/oW7YwGAx/L2CL3WDICcYnxiu4kKjO6kL8A1x0XLtyUdQsvO5TLCdtniJJ9sAjxXQa4oR51xVYqqIkNdxsHrEm41LTF+dppLhJR6eXQuJFSdeUnmt1dm88PfRaXZJGdFr+eFJxuZeYuYqbgvS9JJysQakCRdZHgfHu6aBFzi9fglQ1JqY8txzXhrTEmrCosbbmp1v2XnOLi/6ZuHBZEUgwLrySSkPFn0f9WzfZeXXm8FZzcmlN7/LegFO750Xdbbfc1ivfwDwK51lqKQCYmt7VK1OKp9HEeIPB0IUtdoMhJ7huxHhJA51VE+a44KQRC6+9IOq4p1zCfbUS7fnlUVBCZ5GLTkyubKqRcPIDp4I2+C57pyNF8DLz4hIph0qy/yYTfRs6rRMrt5iI2VEBHDMsa+nUlErJxO6Hi6ltReZRZuOoluWjVKn444R5RJKaj4QRf6wplWSt4QNceFCSU+oPD67Ru9LrjJjj8ooX6ZdWFQlF2/dZnZBjrOxinHz7pGh9iGV13Xv4aK88vVvmOZ1gHIDlqrQmFAts7oTzog4Wo77tAETxTtub3WDICWyxGww5gS12gyEnGKPOnpFyFluldeKQRyuXfPTT4umfijpubisWvM5UKsopSJhtRXvQOaZHF5lu2FYNudlM68oJ9feS2zhmUWTM3sYJCTcG4ouKCwJtpmNz3XaPyrNXrXjzUlHNQZOZr+oNP/6SsgESOy6W5UDazExJbM9BRzs6Nt7VNUlGWWMEn22mU+vng++LtFQEYp2RSKyyPYdGQRJw3nyvT9n8vrvvE3X7Dh3plSeZCQ1Ik4JkjTFsGRsqZ/NWX6QQ/Wbvpm1+kYi+0T2eJ6LniehU93Nuqz4MBsP4MIgY/yUAr7PjJwEcd87dAeB499hgMFyniBLjiegIgF8E8B8B/Ovu148BeKhbfgYbqZy/vHVvG+KGS5kKsjOrZloVlBnn8vmFXnlleVnUcbNWkWeCLepoF+adptyl2vyYiW9OBbtwTzunxVanfcjYeYxzLGE33VZplwrINjVxs9ks454vFSUDRqPOuNChM6sy4omEBaOUZB/cdLhWl3045lFX4B6QikSjzVSNujLtXWLmsZbzv8uE4o2fnPHHs3tk8MjhvZ6bbRfjVp87IE1oM3N+rgoFTaPBEJCyY4JRtmwnzJS6LvviXm3I7jv2zf67AH4L0tvxoHNuAQC6nwf6nWgwGK4PbLnYieiXACw6517Yqm3G+U8Q0QkiOnH5ytWtTzAYDNcEMW/2jwP4PBGdBvBVAJ8ioj8EcJ6IDgFA93Ox38nOuaedc8ecc8fm9uzu18RgMIwAMfnZvwLgKwBARA8B+DfOuV8jov8M4HEAT3U/n426ouuvW4SOMrtSOvWVy94ldmVNRlBNMg/FcoFFMRWlqabCIp6c0pVbwtU12xTE22kNnRMxav21yPR0bgJEQenlzOSlusDMrP+DWmKkhKsrcg+jw8xa5YJ8DCbYZBWImehUO07M0ViXbruc/7zC3YALUu9vc775gpyto/f8vC/f+7FeeXqPjBorMzNiQTF9UAahSUr75fs4SqcOqMqZT2raMjY4UWp6rypuXWRhO041TwF4hIhOAXike2wwGK5TDORU45z7FjZ23eGcuwTg4Z0fksFguBYYecrmTT6vkECixRdureKiWG1VEhC888bJXvnqsopqmmRRTUy8nShqcxIXs9X0UH8Rq6PuptnyommiTG2cGKKkCeGFaxxPNaxSPLFxTKiINW6mq7Eor0ZNitmclz6VGoqJ4AU2xml1rU6DeyWqqDfmlVdlnHEd5fLnWOqjGcWBf/j9d/XKc4d4OudILzMFQXOo64IenLxdyPyVfRR84vm15UCyxzHEHJhvvMGQE9hiNxhygpGK8c4BSdLfgywJyFiCvIKdf/bkK6LdBZbZck1RJzfbvpdSyYvZPCAEAKYnJlg7Tcjg23LBWpMp8J16LcYXmRir/9KW2UYyJ3+YmJRBG9zDK0VwwK5XZWmiKlOSMGFywtMva861dsJJKZjnWlW245aMXZOKx46bCVi5qVSShI9R79T/zKfpWql7a0Jhala042muoNQyx+aqxDKpliZUBl0eDJTid8smjRCPKt/FT3nJxYnn8rLxonpMl/ZmNxhyAlvsBkNOYIvdYMgJxkBesaldaF2Tt1BGDKZDXjh7ulf+6Qv/T7bjBI5K3akz4kFOlJiKsGPHJaXPl5i+WWR6aLksvbZ4GuWSiqCaYqmbZnZJPXeapTHinOklRebIx1gsyf5L3DzIyTFI6qjFIuO9V3/zi4zEchcbk74Xx6L0KsqkJiIL2Q6HSxR/PTFCS/Wjtde8abXB9Peimm9i42q25R7JKidzZ3r55IzU+yuMgFPsAajjluLHT9gzwdN+kYrgE/2pZ7PAnvfqjCcZqR55n2hXmvGUETqa0njjDQZDD7bYDYacYKRiPIGbikI2DAmeyuknf3e8V165LDNlChG8oINYWPALUwt0Rk3OGacDbTol7u5HbycAABRjSURBVIXHpi7RwTS+bqoqzWb75r2YtndeMnlxr7YCG78OmOFqTUF5nXETEve8Kxa1CO7LHfU3n3vQTTJuOc353iHO5S7r2kzc5SQX9TXp9dhgnHG1llKpal7l2TXp1Y7pKalecUdE0qZdlgm2w1JgcRUEAJL1K74Ppa502HOwurou6losoGgXS6NVVJ52xDVH/YplgU1tppa1zp0RzWYe9N7pxWmphmxeLiTM25vdYMgJbLEbDDmBLXaDIScYueltUzsJce61WzJC62cvfa9XXrl0LvO8ROfTZeg4ztfuvy8qHa/I2hWV7sZ508tML59RrqhzjFt8Zkrq7LuY62tVkVKU+T4DM61oU2SB6XWFFO89Gz9Pm6z1UE6moLjWHSfMZGa4VlPqufWad0nWU18kPy6eg29dRd/x/G4Xl6U+nLB9nbkZbxqbbcr5rjClXZvGlleY27SIWpTtJplJtFJR+yDsmdBpvHmuQB4RpwlBWuw+W4rSxFX8b8PzCqwvnBXtiovv9cq7bpU6e4y/rL3ZDYacwBa7wZATjFyMd5uiWYqgwsshF999R9Qtnj7l2zFRrKKisKZY3eWrq6KOm+W4Z5kmUygyzrXpCSku7mIeb7NMPN/DREwAqFZ83ZQeIxPjCyVl8uLpkZkJsKPMOB0wz7UJpQowVYPfc1NK6gCPMFPjICZmFhh/HDU0EYe/t1ZL8sbXmdjaYOV6W15radWL2eeWroi6hNmoVta92WxmVapG1QnGG6hTgrH+q1Um7ifZpCLanElsHtdWZTQlmAq0Vs0mRWm3+W+hvOs4Vzy7dEepXrNcZdOen5tyfECctze7wZAT2GI3GHKCMWZxlaive760N34s81GsrnmRPOGphJTINsXEuVpNilGtlhe/ikxWKiq5h581qcS5GSYG7p705QmlkhDbwXYq82mzxbKzFhQZBBPveBZU7eHGufHW16S6wqmweYZazcfASTvaSk1IOiw7KxPVC1KrQYWLxQ0VnMLIQ1odnq5Ktpvc77nl7j32iKhrNv08ri/7BCNry9Jz8jKbg0ZN7uivrLEstHUvuk821O/OLBw6qy1PlVuvSfG/seaf22rivQOLOtCLieStttSpGk2vonDV9IMf/YRoN33QZ5MN8eRlwd7sBkNOYIvdYMgJbLEbDDnB6HX2rs7dUcSDJ195sVd+63VJJDlRZumOOGEjKfMJ02X37N4l6trMbY6nNd6lotLmZr0Zbe+sJJeYZXr6FPPaqlZ1FBYjOVTc8I6beBRRJU+75BjZRFOZtbiXnCvJaxM/Znp/KgqL6cOkwrASQQDhdUNOgglIT8dEvTZKLMquVGK6ckmOY/cNR3vlW/7BP5JjFBsNLIpORRm22fy0mtJDjx9zoomCMrkWWGQhqQ0OTuqZqGs3aj71dZ3p7+22/M3EXacIVf0XU8z7cv/hI6JdUZGpyE6yU4FvIjY/+2kAK9ggVW07544R0TyA/wXgFgCnAfxT59zlrD4MBsN4MYgY/wvOufudc8e6x08COO6cuwPA8e6xwWC4TrEdMf4xAA91y89gIwfcl0MnODgkyYZo+c7pt0Tdj77zt73y+ppM3VSY9qL2JDOvac50HuBSVpzvczNeJE9avuGs4mSfY8dzk9LWNMU86gTPnOKBq7D0UunUSv7alQl57TIX+ZmoXlIBMy3mXVeqSO+9hBgfGxMxO4qkI2EeXYVEeXQxEbTDPMQ6bU30wTzjaoqnnwXNJExdSb1dGGlEXaXzKjIyCPlby/koMm+14pRUmyanpTrXv7/0sajLrBmI2j0TWUFhg6R40p6D/RD7ZncA/pqIXiCiJ7rfHXTOLQBA9/NA9MgMBsPIEftm/7hz7j0iOgDgeSL6SewFun8cngCAGw7uH2KIBoNhJxD1ZnfOvdf9XATwdQAPAjhPRIcAoPu5mHHu0865Y865Y3N7du/MqA0Gw8DY8s1ORNMACs65lW75MwD+A4DnADwO4Knu57Nb9dVutXDh/AIA4Iff/qaoW1nyfyuKimBR6Kgt//eprcwnHWZq4mQEgIw+qzvmzqpMKYxfEVVlNuOkjQWmi2t9r804w51SyIqcqECZHzlvekGTizMQ08t1fmvuItsp82gwlfaZnZco8sU249ivk98/abakzt5ibp6NhtTZOx3fR5u5D0/P7hHtpjrevXXl5HdEXfmm+3vlwoTfm9AEnNz9OUXAydO0cbNtQbsg8xx82v8ZmeA/r9hV0F0EUr1x0xvX01MusfwwtVmwtftsjBh/EMDXuw90CcD/dM79JRH9AMDXiOiLAN4B8IWIvgwGw5iw5WJ3zr0F4L4+318C8HD6DIPBcD1ipB5062urePG73wYAnH/7DVFXYHKONmXxVEJNZgqiguLyYgQYTSXmFFiEVpOZoZKmFD9bu724qAkOHBPPC6yPpuLMk15+UlwsMXWiUpX9U4GZqxiBRLmiws2Yp2BTmbzAVBRiIq12sOKqxpricq+ve5NdrcbMcGo+1mpejG8pjzGX+OPpCT/3k1XplVhhY0wu/kyNw6dpTg7c2StX9x4W7UplZhJVrnyClIJ9r81anLDDFfRWVpxZTqZvlu1io9SE1hfSJhTpn6VsNhgMPdhiNxhyAlvsBkNOMFKdvbG+jrd+1GWhUTpekZEeVpTOzjnPpd6YrdQ0VMpcHvXG876trNVEu7k1rwNPTUpdnCfy5VFdTvGAFwR5ocqx1mHnpcbor9dse314ckpyhHMuw7ZiPVlf9fcjePSVLajN5vHyFRm/1GD7ABNsv6CuIsrWmc7e0SZG5uJ70+EbeuWWGkeRueCS4nJvL/kcASsLnjN98tCtot30TR/slat7Doo6HmXoAuYv/vAM4gGbrYvrPYH+RKD9euxX3DjkdkRVZ7zxBoNhE7bYDYacYLTkFS7xKXS1FxQTd6sqSF9IKIwMkbQXFDe3KbmGe+E1mOi73JTqxJvnLvXKiRI59895cXqCeeiREtWJR6kp8yBYit9SWd5ni3nzXVnx5rByVaoaBeLee3IO6iwtcZ2lWtJRbx02qxeuLIu6NhOtJ6Z81FiL5H2iwKL21O/ZWfOkkMWLPrKto4gydk975YiUOrTOovZ4iqfiJZnKuLHivS8bs1KMr+y/qVee3ufVidKMct1mupGWiCnOaia92jSv+xDRcalxBET8GOObvdkNhpzAFrvBkBOMVIwnIhS7POo6qyjPOKrFcy4SVVj21ESJSlxU1cEpfCezxkTdptoRf/sqC8xYa4i6I/t93SwTP3UaqiKzJrRUoI1Ii6REWr5T/e4FLwaXlYg8zQg2dKZZvgHP00ZRSRJlTM/7cOPdd31I1O2Z2+fL+3y7iSlJlFFmaogOTll6z4vaZ1/7Ya/8k3ffE+1mJ30fkyrwaIqpSnur/tp655lYQE7tzClRt3Lae2quM9G9vHuvvNYBz/c2dVByv5V3z/trKTISyS2XLUqHA2Ey+kg9w9lHlsXVYDD0YIvdYMgJbLEbDDnBiHV2H4WkCSEhuLlVtBnTY7hHWqIif9oi5bEihkj682o3FYFEjR2fviyjwS6ytMGz0z56a9eUjOQilt+tVZD3Wdk11yvPHLxF1E0z3fDAAZ+/rLYqTWOcwFFziU9Pe97xGdbfzJ550W5mtyeRmJiYFHVlti9SCuRA416EReX1ePT9d/XKN97uI9aWFs6Kdqdfe7lXfvON10Tdnop/F3WYiXF3IsfLI+faDfl7Vtn8V3je6sUF0W590Xvr1d+QeQuqhzy3/ezdD4q6giDF9N/rqLqgSp2l68e4xfX6d+KzH+zNbjDkBLbYDYacYLRiPKiXGreqUgkJwgDN6cZE62abi+rSbCbMTspjjIvxCRN1NFEGT93bUCL+Mks9PH/otl55/613yHGwgRRVeqbJXbOZdY797a3OMLFbeckV2Pxo8Zl7IpaYWY5Uu0aDk1JI0a/EVCyubum5ymqnx8XVgn2HbxLt+PHSvcdE3dssJdiZhdO98sKlK6LdJHuWJpQpcn6Smeya7HdXXHVVlhOAGtJjcfkUE+tn5kTdnvff6/sXortWG7Nd6DIDYzTPXAbfXSzszW4w5AS22A2GnMAWu8GQE4zc9Lbp7qp1TWFSc9qk1j9lrlZ1EqGXqzqh7/j+JpSra52p6SXF0nj4tg/0yrd/6AHfThFCdjr+Yto82GLc642WdMeV5ASh6L5s/nN+zPX3otJRyyXviloq632LMitz85oyvXG9XNfxtNXMZFcqK92ejXFmXmYMuueTn+mV6zXPX3/lnHS5Pc9cYt97501Rt7Lq3Y53M728pOeNTX21qurYvVQUgYfUtwOmsuxUddnQkXPBpm7LRvZmNxhyAlvsBkNOMFryCiImZiped24mUrJIocBS/jIRNiEt5vCURv095jbHsYmmMtE1mMowp6Kfbr3zbt8FG2+zIUU7zneXIo1gYr1z2suqf+RfSIzXUW8F4dXG0j4rgg3uUNhRc8UPuRpS1lzlrKE2H3GuwA4bR0dFAXIRP0mUiM/urcLSPx2+/QOi3dE7PAddbUV6G1569+1e+Srz3lteviraNZiaUFSpqY/efHuvPHvT7aIuk+g9YDbTCHPSZZwzhO0t6s1ORHuI6E+I6CdE9DoRfYyI5onoeSI61f2c27ong8EwLsSK8f8FwF865z6AjVRQrwN4EsBx59wdAI53jw0Gw3WKmCyuswA+CeCfA4BzrgmgSUSPAXio2+wZAN8C8OUtr9gVQTXZgRAJ1SncY4xnN+209a59tvjMUznVGGHFigqcoKrnXLuJiYeA5Jqr1T3dMt991+NI1DiEJKw848A43oosgENTv/Gd9ZQAyHfxxbzJa3EROaUK8PRVAVpsfqzJSIQ6EWonUmUp1a7ILQtcdcm+l91794m6+QMHfH8f5kEsSiVh3pKa0pqTdpRKkmDD8fnm34cjX6KqUhx0gR39nUr/dBuACwD+BxG9SET/vZu6+aBzbgEAup8HQp0YDIbxImaxlwD8HID/5px7AMAaBhDZiegJIjpBRCfWG+2tTzAYDNcEMYv9LICzzrnvdY//BBuL/zwRHQKA7udiv5Odc0875445545NVUe7+W8wGDxi8rOfI6IzRHSnc+4kNnKyv9b99ziAp7qfz8ZccNPMU1Bki1zp0KmBhUmNfZ8E9Ja2ilirs1TP66zccnIK9h/yUVilqiRpXFv3hJN8v0Cp5cIDUI9RmBjVFHDzGN/T0BFaglxCkTTKKDUWlabaVRiZYynlGcfPy456E951Sp/nHpK8P20CFO10HY/gK/b3DASkN1zKo5DvORT67/3o8/R+EmXo5QAyCSJjUzSn+gg24/sxgyvtsa/afwXgj4ioAuAtAP8CG1LB14joiwDeAfCFyL4MBsMYELXYnXMvATjWp+rhnR2OwWC4Vhhx+id4OVxLIQn3xpJ13IyWJNleW/xICvGSiIJb2yrTMkPqBOMWX1lZFXWS5IGJ3Eoe5951KXMVN3kp8ZmL2hXGyV5WPHOiXUWK59w0VKlwcVz2wdMp6eAUEbiSQUKh67QqkBWEk24X8gZk5xX6i+Mbx9l1lGGK1HkFxGEqOXC2zSuT811DpIbKbhaSxrlnqb5UjBee+cYbDDmBLXaDISewxW4w5AQjjnpD78+LVjE4IaTmeOcc8EJ/V320WDtuagOABjOVOWLpfyd3iXbrNUYoQTKajViq5GKR67xKb+Y6u3btZHqoJnzgeeyqTBcvK72c6+y6riL0+Urfc/RxrOmtnCKcZPMYIL4sBsxmYTfYOLNZWBfP0LeDTBD6MBBBmXVaKuot+9pZZrow97yOQNx6fPZmNxhyAlvsBkNOQMMEzg99MaILAN4GsA/AxZFdOBs2Dgkbh8T1MI5Bx3Czc25/v4qRLvbeRYlOOOf6OenYOGwcNo5rNAYT4w2GnMAWu8GQE4xrsT89putq2DgkbBwS18M4dmwMY9HZDQbD6GFivMGQE4x0sRPRo0R0kojeIKKRsdES0e8T0SIRvcK+GzkVNhEdJaJvdum4XyWiL41jLEQ0QUTfJ6KXu+P47XGMg42n2OU3/Ma4xkFEp4nox0T0EhGdGOM4rhlt+8gWOxEVAfxXAJ8FcBeAXyWiu0Z0+T8A8Kj6bhxU2G0Av+mc+yCAjwL49e4cjHosDQCfcs7dB+B+AI8S0UfHMI5NfAkb9OSbGNc4fsE5dz8zdY1jHNeOtt05N5J/AD4G4K/Y8VcAfGWE178FwCvs+CSAQ93yIQAnRzUWNoZnATwyzrEAmALwQwAfGcc4ABzpPsCfAvCNcf02AE4D2Ke+G+k4AMwC+Bm6e2k7PY5RivE3AjjDjs92vxsXxkqFTUS3AHgAwPfGMZau6PwSNohCn3cbhKLjmJPfBfBbkPSC4xiHA/DXRPQCET0xpnFcU9r2US72ftmpcmkKIKJdAP4UwG8455a3an8t4JzrOOfux8ab9UEiunurc3YaRPRLABadcy+M+tp98HHn3M9hQ838dSL65BjGsC3a9q0wysV+FsBRdnwEwHsZbUeBKCrsnQYRlbGx0P/IOfdn4xwLADjnrmAjm8+jYxjHxwF8nohOA/gqgE8R0R+OYRxwzr3X/VwE8HUAD45hHNuibd8Ko1zsPwBwBxHd2mWp/RUAz43w+hrPYYMCGxiACns7oI3g6t8D8Lpz7nfGNRYi2k9Ee7rlSQCfBvCTUY/DOfcV59wR59wt2Hge/sY592ujHgcRTRPRzGYZwGcAvDLqcTjnzgE4Q0R3dr/apG3fmXFc640PtdHwOQA/BfAmgH83wuv+MYAFAC1s/PX8IoC92NgYOtX9nB/BOD6BDdXlRwBe6v773KjHAuBeAC92x/EKgH/f/X7kc8LG9BD8Bt2o5+M2AC93/726+WyO6Rm5H8CJ7m/z5wDmdmoc5kFnMOQE5kFnMOQEttgNhpzAFrvBkBPYYjcYcgJb7AZDTmCL3WDICWyxGww5gS12gyEn+P919noTUFQIngAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Example of a picture\n",
    "index = 0\n",
    "plt.imshow(X_train_orig[index])\n",
    "print (\"y = \" + str(np.squeeze(Y_train_orig[:, index])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual you flatten the image dataset, then normalize it by dividing by 255. On top of that, you will convert each label to a one-hot vector as shown in Figure 1. Run the cell below to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[[227 220 214]\n",
      "   [227 221 215]\n",
      "   [227 222 215]\n",
      "   ...\n",
      "   [232 230 224]\n",
      "   [231 229 222]\n",
      "   [230 229 221]]\n",
      "\n",
      "  [[227 221 214]\n",
      "   [227 221 215]\n",
      "   [228 221 215]\n",
      "   ...\n",
      "   [232 230 224]\n",
      "   [231 229 222]\n",
      "   [231 229 221]]\n",
      "\n",
      "  [[227 221 214]\n",
      "   [227 221 214]\n",
      "   [227 221 215]\n",
      "   ...\n",
      "   [232 230 224]\n",
      "   [231 229 223]\n",
      "   [230 229 221]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[119  81  51]\n",
      "   [124  85  55]\n",
      "   [127  87  58]\n",
      "   ...\n",
      "   [210 211 211]\n",
      "   [211 212 210]\n",
      "   [210 211 210]]\n",
      "\n",
      "  [[119  79  51]\n",
      "   [124  84  55]\n",
      "   [126  85  56]\n",
      "   ...\n",
      "   [210 211 210]\n",
      "   [210 211 210]\n",
      "   [209 210 209]]\n",
      "\n",
      "  [[119  81  51]\n",
      "   [123  83  55]\n",
      "   [122  82  54]\n",
      "   ...\n",
      "   [209 210 210]\n",
      "   [209 210 209]\n",
      "   [208 209 209]]]\n",
      "\n",
      "\n",
      " [[[238 232 223]\n",
      "   [238 232 223]\n",
      "   [238 232 223]\n",
      "   ...\n",
      "   [222 216 209]\n",
      "   [221 216 207]\n",
      "   [221 216 206]]\n",
      "\n",
      "  [[237 232 223]\n",
      "   [238 232 223]\n",
      "   [238 232 223]\n",
      "   ...\n",
      "   [222 216 209]\n",
      "   [222 216 208]\n",
      "   [223 217 207]]\n",
      "\n",
      "  [[236 232 222]\n",
      "   [237 232 223]\n",
      "   [238 232 223]\n",
      "   ...\n",
      "   [222 216 209]\n",
      "   [222 216 208]\n",
      "   [221 216 207]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[218 212 204]\n",
      "   [217 212 204]\n",
      "   [217 211 205]\n",
      "   ...\n",
      "   [214 203 194]\n",
      "   [214 203 195]\n",
      "   [214 204 194]]\n",
      "\n",
      "  [[217 211 203]\n",
      "   [217 211 203]\n",
      "   [216 210 203]\n",
      "   ...\n",
      "   [214 203 194]\n",
      "   [215 203 194]\n",
      "   [215 204 193]]\n",
      "\n",
      "  [[216 210 202]\n",
      "   [216 210 203]\n",
      "   [215 209 203]\n",
      "   ...\n",
      "   [214 203 194]\n",
      "   [215 203 194]\n",
      "   [215 204 192]]]\n",
      "\n",
      "\n",
      " [[[228 220 208]\n",
      "   [228 220 208]\n",
      "   [227 219 208]\n",
      "   ...\n",
      "   [231 228 221]\n",
      "   [232 228 221]\n",
      "   [231 227 220]]\n",
      "\n",
      "  [[228 219 208]\n",
      "   [228 219 208]\n",
      "   [227 219 207]\n",
      "   ...\n",
      "   [231 227 221]\n",
      "   [231 227 221]\n",
      "   [231 227 220]]\n",
      "\n",
      "  [[227 219 208]\n",
      "   [227 219 208]\n",
      "   [227 219 208]\n",
      "   ...\n",
      "   [231 227 222]\n",
      "   [231 227 222]\n",
      "   [231 227 220]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[214 200 185]\n",
      "   [214 199 183]\n",
      "   [184 167 150]\n",
      "   ...\n",
      "   [209 205 201]\n",
      "   [212 207 204]\n",
      "   [212 208 204]]\n",
      "\n",
      "  [[213 198 183]\n",
      "   [201 185 169]\n",
      "   [139 115  95]\n",
      "   ...\n",
      "   [209 204 201]\n",
      "   [210 206 202]\n",
      "   [212 207 203]]\n",
      "\n",
      "  [[209 194 178]\n",
      "   [162 142 124]\n",
      "   [122  88  64]\n",
      "   ...\n",
      "   [208 204 200]\n",
      "   [210 206 201]\n",
      "   [211 207 202]]]\n",
      "\n",
      "\n",
      " ...\n",
      "\n",
      "\n",
      " [[[235 226 216]\n",
      "   [235 227 216]\n",
      "   [234 227 216]\n",
      "   ...\n",
      "   [228 222 213]\n",
      "   [228 222 213]\n",
      "   [228 223 213]]\n",
      "\n",
      "  [[235 226 216]\n",
      "   [234 227 216]\n",
      "   [234 227 216]\n",
      "   ...\n",
      "   [228 222 214]\n",
      "   [228 222 214]\n",
      "   [228 222 214]]\n",
      "\n",
      "  [[234 226 217]\n",
      "   [234 226 216]\n",
      "   [234 226 216]\n",
      "   ...\n",
      "   [228 222 214]\n",
      "   [228 223 214]\n",
      "   [228 223 214]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[209 197 185]\n",
      "   [209 198 185]\n",
      "   [210 199 186]\n",
      "   ...\n",
      "   [201 194 187]\n",
      "   [202 195 187]\n",
      "   [202 195 187]]\n",
      "\n",
      "  [[208 197 183]\n",
      "   [209 198 184]\n",
      "   [210 198 185]\n",
      "   ...\n",
      "   [200 192 184]\n",
      "   [201 193 185]\n",
      "   [201 193 184]]\n",
      "\n",
      "  [[207 196 182]\n",
      "   [209 197 184]\n",
      "   [210 198 185]\n",
      "   ...\n",
      "   [199 191 182]\n",
      "   [200 191 183]\n",
      "   [200 192 182]]]\n",
      "\n",
      "\n",
      " [[[233 226 217]\n",
      "   [233 227 218]\n",
      "   [232 228 218]\n",
      "   ...\n",
      "   [225 221 211]\n",
      "   [225 220 210]\n",
      "   [224 219 209]]\n",
      "\n",
      "  [[232 226 217]\n",
      "   [232 227 218]\n",
      "   [233 228 219]\n",
      "   ...\n",
      "   [224 220 210]\n",
      "   [224 220 210]\n",
      "   [223 219 208]]\n",
      "\n",
      "  [[232 227 217]\n",
      "   [231 227 217]\n",
      "   [231 227 218]\n",
      "   ...\n",
      "   [224 221 211]\n",
      "   [224 220 210]\n",
      "   [222 219 208]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[192 184 173]\n",
      "   [193 185 173]\n",
      "   [192 185 173]\n",
      "   ...\n",
      "   [208 205 203]\n",
      "   [208 204 202]\n",
      "   [207 204 201]]\n",
      "\n",
      "  [[191 184 173]\n",
      "   [191 184 173]\n",
      "   [191 184 173]\n",
      "   ...\n",
      "   [207 204 202]\n",
      "   [206 204 201]\n",
      "   [206 203 200]]\n",
      "\n",
      "  [[190 183 171]\n",
      "   [191 183 172]\n",
      "   [190 183 172]\n",
      "   ...\n",
      "   [206 202 200]\n",
      "   [205 202 199]\n",
      "   [204 201 198]]]\n",
      "\n",
      "\n",
      " [[[230 220 209]\n",
      "   [230 221 210]\n",
      "   [230 221 210]\n",
      "   ...\n",
      "   [232 225 215]\n",
      "   [232 225 215]\n",
      "   [231 224 214]]\n",
      "\n",
      "  [[230 221 209]\n",
      "   [230 221 210]\n",
      "   [230 221 210]\n",
      "   ...\n",
      "   [231 224 214]\n",
      "   [231 225 215]\n",
      "   [231 224 214]]\n",
      "\n",
      "  [[229 220 209]\n",
      "   [229 221 210]\n",
      "   [229 221 210]\n",
      "   ...\n",
      "   [231 225 215]\n",
      "   [230 225 215]\n",
      "   [229 224 213]]\n",
      "\n",
      "  ...\n",
      "\n",
      "  [[216 207 200]\n",
      "   [217 207 200]\n",
      "   [218 208 200]\n",
      "   ...\n",
      "   [202 198 198]\n",
      "   [202 199 198]\n",
      "   [203 200 197]]\n",
      "\n",
      "  [[215 206 199]\n",
      "   [217 207 200]\n",
      "   [218 208 200]\n",
      "   ...\n",
      "   [201 199 198]\n",
      "   [202 199 197]\n",
      "   [202 199 197]]\n",
      "\n",
      "  [[214 205 198]\n",
      "   [216 206 199]\n",
      "   [217 207 199]\n",
      "   ...\n",
      "   [201 198 197]\n",
      "   [202 199 198]\n",
      "   [202 199 197]]]]\n"
     ]
    }
   ],
   "source": [
    "print(str(X_train_orig))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[227 238 228 ... 235 233 230]\n",
      " [220 232 220 ... 226 226 220]\n",
      " [214 223 208 ... 216 217 209]\n",
      " ...\n",
      " [208 215 211 ... 200 204 202]\n",
      " [209 204 207 ... 192 201 199]\n",
      " [209 192 202 ... 182 198 197]]\n"
     ]
    }
   ],
   "source": [
    "print(str(X_train_orig.reshape(X_train_orig.shape[0], -1).T))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of training examples = 1080\n",
      "number of test examples = 120\n",
      "X_train shape: (12288, 1080)\n",
      "Y_train shape: (6, 1080)\n",
      "X_test shape: (12288, 120)\n",
      "Y_test shape: (6, 120)\n"
     ]
    }
   ],
   "source": [
    "# Flatten the training and test images\n",
    "X_train_flatten = X_train_orig.reshape(X_train_orig.shape[0], -1).T#原来是四维矩阵shape[0]是样本的个数，这一维上每个点上放着一个样本，因此这样reshape\n",
    "                                                                #结果得到每一列一个样本，共m列的二维矩阵\n",
    "X_test_flatten = X_test_orig.reshape(X_test_orig.shape[0], -1).T\n",
    "# Normalize image vectors\n",
    "X_train = X_train_flatten/255.\n",
    "X_test = X_test_flatten/255.\n",
    "# Convert training and test labels to one hot matrices\n",
    "Y_train = convert_to_one_hot(Y_train_orig, 6)\n",
    "Y_test = convert_to_one_hot(Y_test_orig, 6)\n",
    "\n",
    "print (\"number of training examples = \" + str(X_train.shape[1]))\n",
    "print (\"number of test examples = \" + str(X_test.shape[1]))\n",
    "print (\"X_train shape: \" + str(X_train.shape))\n",
    "print (\"Y_train shape: \" + str(Y_train.shape))\n",
    "print (\"X_test shape: \" + str(X_test.shape))\n",
    "print (\"Y_test shape: \" + str(Y_test.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note** that 12288 comes from $64 \\times 64 \\times 3$. Each image is square, 64 by 64 pixels, and 3 is for the RGB colors. Please make sure all these shapes make sense to you before continuing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Your goal** is to build an algorithm capable of recognizing a sign with high accuracy. To do so, you are going to build a tensorflow model that is almost the same as one you have previously built in numpy for cat recognition (but now using a softmax output). It is a great occasion to compare your numpy implementation to the tensorflow one. \n",
    "\n",
    "**The model** is *LINEAR -> RELU -> LINEAR -> RELU -> LINEAR -> SOFTMAX*. The SIGMOID output layer has been converted to a SOFTMAX. A SOFTMAX layer generalizes SIGMOID to when there are more than two classes. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 - Create placeholders\n",
    "\n",
    "Your first task is to create placeholders for `X` and `Y`. This will allow you to later pass your training data in when you run your session. \n",
    "\n",
    "**Exercise:** Implement the function below to create the placeholders in tensorflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: create_placeholders\n",
    "\n",
    "def create_placeholders(n_x, n_y):\n",
    "    \"\"\"\n",
    "    Creates the placeholders for the tensorflow session.\n",
    "    \n",
    "    Arguments:\n",
    "    n_x -- scalar, size of an image vector (num_px * num_px = 64 * 64 * 3 = 12288)\n",
    "    n_y -- scalar, number of classes (from 0 to 5, so -> 6)\n",
    "    \n",
    "    Returns:\n",
    "    X -- placeholder for the data input, of shape [n_x, None] and dtype \"float\"\n",
    "    Y -- placeholder for the input labels, of shape [n_y, None] and dtype \"float\"\n",
    "    \n",
    "    Tips:\n",
    "    - You will use None because it let's us be flexible on the number of examples you will for the placeholders.\n",
    "      In fact, the number of examples during test/train is different.\n",
    "    \"\"\"\n",
    "\n",
    "    ### START CODE HERE ### (approx. 2 lines)\n",
    "    X = tf.placeholder(tf.float32, shape = [n_x, None], name = \"X\")\n",
    "    Y = tf.placeholder(tf.float32, shape = [n_y, None], name = \"Y\")\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return X, Y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X = Tensor(\"X:0\", shape=(12288, ?), dtype=float32)\n",
      "Y = Tensor(\"Y:0\", shape=(6, ?), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "X, Y = create_placeholders(12288, 6)\n",
    "print (\"X = \" + str(X))\n",
    "print (\"Y = \" + str(Y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output**: \n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **X**\n",
    "        </td>\n",
    "        <td>\n",
    "        Tensor(\"Placeholder_1:0\", shape=(12288, ?), dtype=float32) (not necessarily Placeholder_1)\n",
    "        </td>\n",
    "    </tr>\n",
    "    <tr> \n",
    "        <td>\n",
    "            **Y**\n",
    "        </td>\n",
    "        <td>\n",
    "        Tensor(\"Placeholder_2:0\", shape=(10, ?), dtype=float32) (not necessarily Placeholder_2)\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 - Initializing the parameters\n",
    "\n",
    "Your second task is to initialize the parameters in tensorflow.\n",
    "\n",
    "**Exercise:** Implement the function below to initialize the parameters in tensorflow. You are going use Xavier Initialization for weights and Zero Initialization for biases. The shapes are given below. As an example, to help you, for W1 and b1 you could use: \n",
    "\n",
    "```python\n",
    "W1 = tf.get_variable(\"W1\", [25,12288], initializer = tf.contrib.layers.xavier_initializer(seed = 1))\n",
    "b1 = tf.get_variable(\"b1\", [25,1], initializer = tf.zeros_initializer())\n",
    "```\n",
    "Please use `seed = 1` to make sure your results match ours."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: initialize_parameters\n",
    "\n",
    "def initialize_parameters():\n",
    "    \"\"\"\n",
    "    Initializes parameters to build a neural network with tensorflow. The shapes are:\n",
    "                        W1 : [25, 12288]\n",
    "                        b1 : [25, 1]\n",
    "                        W2 : [12, 25]\n",
    "                        b2 : [12, 1]\n",
    "                        W3 : [6, 12]\n",
    "                        b3 : [6, 1]\n",
    "    \n",
    "    Returns:\n",
    "    parameters -- a dictionary of tensors containing W1, b1, W2, b2, W3, b3\n",
    "    \"\"\"\n",
    "    \n",
    "    tf.set_random_seed(1)                   # so that your \"random\" numbers match ours\n",
    "        \n",
    "    ### START CODE HERE ### (approx. 6 lines of code)\n",
    "    W1 = tf.get_variable(\"W1\", [25,12288], initializer = tf.contrib.layers.xavier_initializer())\n",
    "    b1 = tf.get_variable(\"b1\", [25,1], initializer = tf.contrib.layers.xavier_initializer())\n",
    "    W2 = tf.get_variable(\"W2\", [12,25], initializer = tf.contrib.layers.xavier_initializer())\n",
    "    b2 = tf.get_variable(\"b2\", [12,1], initializer = tf.contrib.layers.xavier_initializer())\n",
    "    W3 = tf.get_variable(\"W3\", [6,12], initializer = tf.contrib.layers.xavier_initializer())\n",
    "    b3 = tf.get_variable(\"b3\", [6,1], initializer = tf.contrib.layers.xavier_initializer())\n",
    "    ### END CODE HERE ###\n",
    "\n",
    "    parameters = {\"W1\": W1,\n",
    "                  \"b1\": b1,\n",
    "                  \"W2\": W2,\n",
    "                  \"b2\": b2,\n",
    "                  \"W3\": W3,\n",
    "                  \"b3\": b3}\n",
    "    \n",
    "    return parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W1 = <tf.Variable 'W1:0' shape=(25, 12288) dtype=float32_ref>\n",
      "b1 = <tf.Variable 'b1:0' shape=(25, 1) dtype=float32_ref>\n",
      "W2 = <tf.Variable 'W2:0' shape=(12, 25) dtype=float32_ref>\n",
      "b2 = <tf.Variable 'b2:0' shape=(12, 1) dtype=float32_ref>\n"
     ]
    }
   ],
   "source": [
    "tf.reset_default_graph()\n",
    "with tf.Session() as sess:\n",
    "    parameters = initialize_parameters()\n",
    "    print(\"W1 = \" + str(parameters[\"W1\"]))\n",
    "    print(\"b1 = \" + str(parameters[\"b1\"]))\n",
    "    print(\"W2 = \" + str(parameters[\"W2\"]))\n",
    "    print(\"b2 = \" + str(parameters[\"b2\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output**: \n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **W1**\n",
    "        </td>\n",
    "        <td>\n",
    "         < tf.Variable 'W1:0' shape=(25, 12288) dtype=float32_ref >\n",
    "        </td>\n",
    "    </tr>\n",
    "    <tr> \n",
    "        <td>\n",
    "            **b1**\n",
    "        </td>\n",
    "        <td>\n",
    "        < tf.Variable 'b1:0' shape=(25, 1) dtype=float32_ref >\n",
    "        </td>\n",
    "    </tr>\n",
    "    <tr> \n",
    "        <td>\n",
    "            **W2**\n",
    "        </td>\n",
    "        <td>\n",
    "        < tf.Variable 'W2:0' shape=(12, 25) dtype=float32_ref >\n",
    "        </td>\n",
    "    </tr>\n",
    "    <tr> \n",
    "        <td>\n",
    "            **b2**\n",
    "        </td>\n",
    "        <td>\n",
    "        < tf.Variable 'b2:0' shape=(12, 1) dtype=float32_ref >\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, the parameters haven't been evaluated yet."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 - Forward propagation in tensorflow \n",
    "\n",
    "You will now implement the forward propagation module in tensorflow. The function will take in a dictionary of parameters and it will complete the forward pass. The functions you will be using are: \n",
    "\n",
    "- `tf.add(...,...)` to do an addition\n",
    "- `tf.matmul(...,...)` to do a matrix multiplication\n",
    "- `tf.nn.relu(...)` to apply the ReLU activation\n",
    "\n",
    "**Question:** Implement the forward pass of the neural network. We commented for you the numpy equivalents so that you can compare the tensorflow implementation to numpy. It is important to note that the forward propagation stops at `z3`. The reason is that in tensorflow the last linear layer output is given as input to the function computing the loss. Therefore, you don't need `a3`!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: forward_propagation\n",
    "\n",
    "def forward_propagation(X, parameters):\n",
    "    \"\"\"\n",
    "    Implements the forward propagation for the model: LINEAR -> RELU -> LINEAR -> RELU -> LINEAR -> SOFTMAX\n",
    "    \n",
    "    Arguments:\n",
    "    X -- input dataset placeholder, of shape (input size, number of examples)\n",
    "    parameters -- python dictionary containing your parameters \"W1\", \"b1\", \"W2\", \"b2\", \"W3\", \"b3\"\n",
    "                  the shapes are given in initialize_parameters\n",
    "\n",
    "    Returns:\n",
    "    Z3 -- the output of the last LINEAR unit\n",
    "    \"\"\"\n",
    "    \n",
    "    # Retrieve the parameters from the dictionary \"parameters\" \n",
    "    W1 = parameters['W1']\n",
    "    b1 = parameters['b1']\n",
    "    W2 = parameters['W2']\n",
    "    b2 = parameters['b2']\n",
    "    W3 = parameters['W3']\n",
    "    b3 = parameters['b3']\n",
    "    \n",
    "    ### START CODE HERE ### (approx. 5 lines)              # Numpy Equivalents:\n",
    "    Z1 = tf.add(tf.matmul(W1,X), b1)                                              # Z1 = np.dot(W1, X) + b1\n",
    "    A1 = tf.nn.relu(Z1)                                              # A1 = relu(Z1)\n",
    "    Z2 = tf.add(tf.matmul(W2,A1), b2)                                              # Z2 = np.dot(W2, a1) + b2\n",
    "    A2 = tf.nn.relu(Z2)                                              # A2 = relu(Z2)\n",
    "    Z3 = tf.add(tf.matmul(W3,A2), b3)                                              # Z3 = np.dot(W3,Z2) + b3\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return Z3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Z3 = Tensor(\"Add_2:0\", shape=(6, ?), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "tf.reset_default_graph()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    X, Y = create_placeholders(12288, 6)\n",
    "    parameters = initialize_parameters()\n",
    "    Z3 = forward_propagation(X, parameters)\n",
    "    print(\"Z3 = \" + str(Z3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output**: \n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **Z3**\n",
    "        </td>\n",
    "        <td>\n",
    "        Tensor(\"Add_2:0\", shape=(6, ?), dtype=float32)\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may have noticed that the forward propagation doesn't output any cache. You will understand why below, when we get to brackpropagation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Compute cost\n",
    "\n",
    "As seen before, it is very easy to compute the cost using:\n",
    "```python\n",
    "tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = ..., labels = ...))\n",
    "```\n",
    "**Question**: Implement the cost function below. \n",
    "- It is important to know that the \"`logits`\" and \"`labels`\" inputs of `tf.nn.softmax_cross_entropy_with_logits` are expected to be of shape (number of examples, num_classes). We have thus transposed Z3 and Y for you.\n",
    "- Besides, `tf.reduce_mean` basically does the summation over the examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: compute_cost \n",
    "\n",
    "def compute_cost(Z3, Y):\n",
    "    \"\"\"\n",
    "    Computes the cost\n",
    "    \n",
    "    Arguments:\n",
    "    Z3 -- output of forward propagation (output of the last LINEAR unit), of shape (6, number of examples)\n",
    "    Y -- \"true\" labels vector placeholder, same shape as Z3\n",
    "    \n",
    "    Returns:\n",
    "    cost - Tensor of the cost function\n",
    "    \"\"\"\n",
    "    \n",
    "    # to fit the tensorflow requirement for tf.nn.softmax_cross_entropy_with_logits(...,...)\n",
    "    logits = tf.transpose(Z3)\n",
    "    labels = tf.transpose(Y)\n",
    "    \n",
    "    ### START CODE HERE ### (1 line of code)\n",
    "    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = logits, labels = labels))\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-32-412488f44149>:20: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See tf.nn.softmax_cross_entropy_with_logits_v2.\n",
      "\n",
      "cost = Tensor(\"Mean:0\", shape=(), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "tf.reset_default_graph()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    X, Y = create_placeholders(12288, 6)\n",
    "    parameters = initialize_parameters()\n",
    "    Z3 = forward_propagation(X, parameters)\n",
    "    cost = compute_cost(Z3, Y)\n",
    "    print(\"cost = \" + str(cost))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output**: \n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **cost**\n",
    "        </td>\n",
    "        <td>\n",
    "        Tensor(\"Mean:0\", shape=(), dtype=float32)\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 - Backward propagation & parameter updates\n",
    "\n",
    "This is where you become grateful to programming frameworks. All the backpropagation and the parameters update is taken care of in 1 line of code. It is very easy to incorporate this line in the model.\n",
    "\n",
    "After you compute the cost function. You will create an \"`optimizer`\" object. You have to call this object along with the cost when running the tf.session. When called, it will perform an optimization on the given cost with the chosen method and learning rate.\n",
    "\n",
    "For instance, for gradient descent the optimizer would be:\n",
    "```python\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate = learning_rate).minimize(cost)\n",
    "```\n",
    "\n",
    "To make the optimization you would do:\n",
    "```python\n",
    "_ , c = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})\n",
    "```\n",
    "\n",
    "This computes the backpropagation by passing through the tensorflow graph in the reverse order. From cost to inputs.\n",
    "\n",
    "**Note** When coding, we often use `_` as a \"throwaway\" variable to store values that we won't need to use later. Here, `_` takes on the evaluated value of `optimizer`, which we don't need (and `c` takes the value of the `cost` variable). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6 - Building the model\n",
    "\n",
    "Now, you will bring it all together! \n",
    "\n",
    "**Exercise:** Implement the model. You will be calling the functions you had previously implemented."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def model(X_train, Y_train, X_test, Y_test, learning_rate = 0.0001,\n",
    "          num_epochs = 1500, minibatch_size = 32, print_cost = True):\n",
    "    \"\"\"\n",
    "    Implements a three-layer tensorflow neural network: LINEAR->RELU->LINEAR->RELU->LINEAR->SOFTMAX.\n",
    "    \n",
    "    Arguments:\n",
    "    X_train -- training set, of shape (input size = 12288, number of training examples = 1080)\n",
    "    Y_train -- test set, of shape (output size = 6, number of training examples = 1080)\n",
    "    X_test -- training set, of shape (input size = 12288, number of training examples = 120)\n",
    "    Y_test -- test set, of shape (output size = 6, number of test examples = 120)\n",
    "    learning_rate -- learning rate of the optimization\n",
    "    num_epochs -- number of epochs of the optimization loop\n",
    "    minibatch_size -- size of a minibatch\n",
    "    print_cost -- True to print the cost every 100 epochs\n",
    "    \n",
    "    Returns:\n",
    "    parameters -- parameters learnt by the model. They can then be used to predict.\n",
    "    \"\"\"\n",
    "    \n",
    "    ops.reset_default_graph()                         # to be able to rerun the model without overwriting tf variables\n",
    "    tf.set_random_seed(1)                             # to keep consistent results\n",
    "    seed = 3                                          # to keep consistent results\n",
    "    (n_x, m) = X_train.shape                          # (n_x: input size, m : number of examples in the train set)\n",
    "    n_y = Y_train.shape[0]                            # n_y : output size\n",
    "    costs = []                                        # To keep track of the cost\n",
    "    \n",
    "    # Create Placeholders of shape (n_x, n_y)\n",
    "    ### START CODE HERE ### (1 line)\n",
    "    X, Y = create_placeholders(n_x, n_y)\n",
    "    ### END CODE HERE ###\n",
    "\n",
    "    # Initialize parameters\n",
    "    ### START CODE HERE ### (1 line)\n",
    "    parameters = initialize_parameters()\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    # Forward propagation: Build the forward propagation in the tensorflow graph\n",
    "    ### START CODE HERE ### (1 line)\n",
    "    Z3 = forward_propagation(X, parameters)\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    # Cost function: Add cost function to tensorflow graph\n",
    "    ### START CODE HERE ### (1 line)\n",
    "    cost = compute_cost(Z3, Y)\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    # Backpropagation: Define the tensorflow optimizer. Use an AdamOptimizer.\n",
    "    ### START CODE HERE ### (1 line)\n",
    "    optimizer = tf.train.GradientDescentOptimizer(learning_rate = learning_rate).minimize(cost)\n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    # Initialize all the variables\n",
    "    init = tf.global_variables_initializer()\n",
    "\n",
    "    # Start the session to compute the tensorflow graph\n",
    "    with tf.Session() as sess:\n",
    "        \n",
    "        # Run the initialization\n",
    "        sess.run(init)\n",
    "        \n",
    "        # Do the training loop\n",
    "        for epoch in range(num_epochs):\n",
    "\n",
    "            epoch_cost = 0.                       # Defines a cost related to an epoch\n",
    "            num_minibatches = int(m / minibatch_size) # number of minibatches of size minibatch_size in the train set\n",
    "            seed = seed + 1\n",
    "            minibatches = random_mini_batches(X_train, Y_train, minibatch_size, seed)\n",
    "\n",
    "            for minibatch in minibatches:\n",
    "\n",
    "                # Select a minibatch\n",
    "                (minibatch_X, minibatch_Y) = minibatch\n",
    "                \n",
    "                # IMPORTANT: The line that runs the graph on a minibatch.\n",
    "                # Run the session to execute the \"optimizer\" and the \"cost\", the feedict should contain a minibatch for (X,Y).\n",
    "                ### START CODE HERE ### (1 line)\n",
    "                _ , minibatch_cost = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})\n",
    "                ### END CODE HERE ###\n",
    "                \n",
    "                epoch_cost += minibatch_cost / num_minibatches\n",
    "\n",
    "            # Print the cost every epoch\n",
    "            if print_cost == True and epoch % 100 == 0:\n",
    "                print (\"Cost after epoch %i: %f\" % (epoch, epoch_cost))\n",
    "            if print_cost == True and epoch % 5 == 0:\n",
    "                costs.append(epoch_cost)\n",
    "                \n",
    "        # plot the cost\n",
    "        plt.plot(np.squeeze(costs))\n",
    "        plt.ylabel('cost')\n",
    "        plt.xlabel('iterations (per tens)')\n",
    "        plt.title(\"Learning rate =\" + str(learning_rate))\n",
    "        plt.show()\n",
    "\n",
    "        # lets save the parameters in a variable\n",
    "        parameters = sess.run(parameters)\n",
    "        print (\"Parameters have been trained!\")\n",
    "\n",
    "        # Calculate the correct predictions\n",
    "        correct_prediction = tf.equal(tf.argmax(Z3), tf.argmax(Y))\n",
    "\n",
    "        # Calculate accuracy on the test set\n",
    "        accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
    "\n",
    "        print (\"Train Accuracy:\", accuracy.eval({X: X_train, Y: Y_train}))\n",
    "        print (\"Test Accuracy:\", accuracy.eval({X: X_test, Y: Y_test}))\n",
    "        \n",
    "        return parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Run the following cell to train your model! On our machine it takes about 5 minutes. Your \"Cost after epoch 100\" should be 1.016458. If it's not, don't waste time; interrupt the training by clicking on the square (⬛) in the upper bar of the notebook, and try to correct your code. If it is the correct cost, take a break and come back in 5 minutes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost after epoch 0: 2.067601\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-36-10ae8ff94657>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mparameters\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mY_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mX_test\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mY_test\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mparameters\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-34-d6a175c1b89a>\u001b[0m in \u001b[0;36mmodel\u001b[1;34m(X_train, Y_train, X_test, Y_test, learning_rate, num_epochs, minibatch_size, print_cost)\u001b[0m\n\u001b[0;32m     75\u001b[0m                 \u001b[1;31m# Run the session to execute the \"optimizer\" and the \"cost\", the feedict should contain a minibatch for (X,Y).\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     76\u001b[0m                 \u001b[1;31m### START CODE HERE ### (1 line)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 77\u001b[1;33m                 \u001b[0m_\u001b[0m \u001b[1;33m,\u001b[0m \u001b[0mminibatch_cost\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msess\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0moptimizer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcost\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m{\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mminibatch_X\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mY\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mminibatch_Y\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     78\u001b[0m                 \u001b[1;31m### END CODE HERE ###\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     79\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36mrun\u001b[1;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m    893\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    894\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[1;32m--> 895\u001b[1;33m                          run_metadata_ptr)\n\u001b[0m\u001b[0;32m    896\u001b[0m       \u001b[1;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    897\u001b[0m         \u001b[0mproto_data\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run\u001b[1;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m   1126\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[1;32mor\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[1;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1127\u001b[0m       results = self._do_run(handle, final_targets, final_fetches,\n\u001b[1;32m-> 1128\u001b[1;33m                              feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[0;32m   1129\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1130\u001b[0m       \u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_run\u001b[1;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m   1342\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1343\u001b[0m       return self._do_call(_run_fn, self._session, feeds, fetches, targets,\n\u001b[1;32m-> 1344\u001b[1;33m                            options, run_metadata)\n\u001b[0m\u001b[0;32m   1345\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1346\u001b[0m       \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m   1348\u001b[0m   \u001b[1;32mdef\u001b[0m \u001b[0m_do_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1349\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1350\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1351\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1352\u001b[0m       \u001b[0mmessage\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcompat\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0me\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[1;34m(session, feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[0;32m   1327\u001b[0m           return tf_session.TF_Run(session, options,\n\u001b[0;32m   1328\u001b[0m                                    \u001b[0mfeed_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1329\u001b[1;33m                                    status, run_metadata)\n\u001b[0m\u001b[0;32m   1330\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1331\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m_prun_fn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msession\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "parameters = model(X_train, Y_train, X_test, Y_test)\n",
    "print(str(parameters))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output**:\n",
    "\n",
    "<table> \n",
    "    <tr> \n",
    "        <td>\n",
    "            **Train Accuracy**\n",
    "        </td>\n",
    "        <td>\n",
    "        0.999074\n",
    "        </td>\n",
    "    </tr>\n",
    "    <tr> \n",
    "        <td>\n",
    "            **Test Accuracy**\n",
    "        </td>\n",
    "        <td>\n",
    "        0.716667\n",
    "        </td>\n",
    "    </tr>\n",
    "\n",
    "</table>\n",
    "\n",
    "Amazing, your algorithm can recognize a sign representing a figure between 0 and 5 with 71.7% accuracy.\n",
    "\n",
    "**Insights**:\n",
    "- Your model seems big enough to fit the training set well. However, given the difference between train and test accuracy, you could try to add L2 or dropout regularization to reduce overfitting. \n",
    "- Think about the session as a block of code to train the model. Each time you run the session on a minibatch, it trains the parameters. In total you have run the session a large number of times (1500 epochs) until you obtained well trained parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.7 - Test with your own image (optional / ungraded exercise)\n",
    "\n",
    "Congratulations on finishing this assignment. You can now take a picture of your hand and see the output of your model. To do that:\n",
    "    1. Click on \"File\" in the upper bar of this notebook, then click \"Open\" to go on your Coursera Hub.\n",
    "    2. Add your image to this Jupyter Notebook's directory, in the \"images\" folder\n",
    "    3. Write your image's name in the following code\n",
    "    4. Run the code and check if the algorithm is right!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'W1': array([[-0.01343878, -0.02075631, -0.00596321, ..., -0.0004043 ,\n",
      "         0.00758993, -0.00867865],\n",
      "       [ 0.00085004, -0.01427163,  0.01513397, ...,  0.02166441,\n",
      "         0.01887944,  0.0360632 ],\n",
      "       [ 0.0011263 , -0.01836148, -0.02808653, ..., -0.01702802,\n",
      "        -0.02742018, -0.02204648],\n",
      "       ...,\n",
      "       [-0.0069792 , -0.00095845, -0.01996959, ..., -0.01146619,\n",
      "        -0.01573055,  0.00724103],\n",
      "       [ 0.0051751 ,  0.00756894,  0.00589242, ...,  0.02110844,\n",
      "         0.0301815 ,  0.0208276 ],\n",
      "       [-0.00715603,  0.01143322,  0.02355248, ...,  0.01028908,\n",
      "        -0.02346217, -0.02073191]], dtype=float32), 'b1': array([[ 0.3596701 ],\n",
      "       [ 0.42009637],\n",
      "       [ 0.25785315],\n",
      "       [ 0.38105217],\n",
      "       [ 0.22752796],\n",
      "       [-0.3454218 ],\n",
      "       [-0.25278348],\n",
      "       [ 0.32718888],\n",
      "       [-0.14853111],\n",
      "       [-0.04504555],\n",
      "       [-0.3692636 ],\n",
      "       [-0.03000381],\n",
      "       [ 0.36112535],\n",
      "       [-0.4449353 ],\n",
      "       [-0.38764653],\n",
      "       [-0.17379539],\n",
      "       [-0.08586532],\n",
      "       [-0.13920827],\n",
      "       [-0.15635473],\n",
      "       [-0.09308108],\n",
      "       [-0.44851694],\n",
      "       [ 0.11364038],\n",
      "       [-0.08220884],\n",
      "       [-0.3483802 ],\n",
      "       [-0.14142033]], dtype=float32), 'W2': array([[ 0.23247677,  0.24080446,  0.20833407,  0.33279136,  0.2741984 ,\n",
      "         0.2654103 , -0.15672566,  0.29706693, -0.23542683,  0.00928864,\n",
      "        -0.29801413,  0.09539895, -0.3362421 , -0.28991732, -0.32380077,\n",
      "        -0.09394761, -0.23517284,  0.14627934, -0.22278145, -0.10849907,\n",
      "        -0.35281894, -0.11065003, -0.34082097,  0.1150704 ,  0.5281267 ],\n",
      "       [ 0.17717409, -0.17795216, -0.03587674, -0.1151945 , -0.38952985,\n",
      "        -0.17618608, -0.30563834,  0.13730499,  0.20558363, -0.24122192,\n",
      "        -0.02134246, -0.27324632,  0.20760392, -0.33499542,  0.07035059,\n",
      "         0.362903  ,  0.07499573,  0.32218376,  0.13179272, -0.10325703,\n",
      "        -0.36129934,  0.05090765,  0.2827509 , -0.3770507 , -0.17316338],\n",
      "       [-0.08864293,  0.1932335 ,  0.35975286, -0.35875556, -0.15941252,\n",
      "         0.17805047,  0.39398813,  0.13399194,  0.283458  , -0.1285735 ,\n",
      "         0.3132373 , -0.15472719, -0.38067928, -0.42793074, -0.3690721 ,\n",
      "         0.08057961,  0.11961657,  0.21317586, -0.3614453 , -0.35485688,\n",
      "         0.02060416,  0.15560555,  0.16977614,  0.03135381,  0.41971406],\n",
      "       [ 0.02297646, -0.32938036,  0.34878343,  0.21076642,  0.16188271,\n",
      "         0.31973365,  0.3422404 ,  0.1917098 ,  0.04604359, -0.38544637,\n",
      "         0.00891718, -0.07502517,  0.23432074,  0.07413824, -0.32115474,\n",
      "         0.16513869,  0.23847508,  0.03589145, -0.366423  ,  0.35193267,\n",
      "         0.06494147, -0.37683672, -0.21997547,  0.0120966 , -0.11283064],\n",
      "       [-0.3702829 ,  0.4810007 , -0.5701801 , -0.2592078 ,  0.22767442,\n",
      "         0.04259907, -0.05944753,  0.24731833,  0.1718035 ,  0.1878581 ,\n",
      "        -0.4005049 , -0.5370982 ,  0.03561528, -0.21371825, -0.3066067 ,\n",
      "        -0.10791274, -0.24193451, -0.19025144,  0.21594048, -0.22509621,\n",
      "         0.38103518,  0.1604235 , -0.38832492,  0.1265063 ,  0.06865361],\n",
      "       [-0.37339735, -0.11608142, -0.11754031, -0.15395044,  0.09204111,\n",
      "         0.3446988 ,  0.18578362, -0.28369337, -0.32281637,  0.34595424,\n",
      "        -0.31705812,  0.18668792, -0.324432  ,  0.5997158 ,  0.2838915 ,\n",
      "        -0.23345698,  0.3461421 , -0.05064401,  0.20559561,  0.34717092,\n",
      "         0.1895484 ,  0.3776349 ,  0.40085977,  0.25604773, -0.21001926],\n",
      "       [-0.32848692,  0.3271294 , -0.26522475, -0.12992586,  0.07698329,\n",
      "         0.29247513, -0.16121852, -0.3382312 , -0.34701306,  0.37175012,\n",
      "        -0.21785174,  0.45147792, -0.22428496,  0.19699426,  0.06337309,\n",
      "         0.04496982,  0.37615138, -0.28122482, -0.26910073, -0.2535243 ,\n",
      "        -0.30289218, -0.20757397,  0.1698348 , -0.10360412,  0.01755369],\n",
      "       [-0.36681905, -0.26637673,  0.5510911 ,  0.30575877, -0.35979417,\n",
      "        -0.36984307, -0.18717013,  0.1803598 , -0.3976276 , -0.07308245,\n",
      "         0.355106  , -0.4272023 , -0.2332663 ,  0.41435704,  0.25854176,\n",
      "        -0.17286843, -0.01294613, -0.02924248, -0.29194075,  0.00749174,\n",
      "         0.2799348 , -0.32191467, -0.00768414,  0.193215  ,  0.06058299],\n",
      "       [ 0.24228811,  0.03880498,  0.4859484 ,  0.13088758,  0.02300688,\n",
      "        -0.02598984,  0.3324806 , -0.52470595,  0.32710558, -0.39310572,\n",
      "        -0.25948796, -0.04460017, -0.13091296,  0.41845742, -0.03838292,\n",
      "        -0.07070552, -0.1768829 ,  0.10873217,  0.22090256, -0.3645029 ,\n",
      "         0.10230166, -0.11181808,  0.25109994,  0.4471103 , -0.20799395],\n",
      "       [-0.3121432 ,  0.4941726 , -0.39772686,  0.08834032, -0.4910658 ,\n",
      "         0.41363004,  0.12465096, -0.03811213, -0.21105312,  0.04100505,\n",
      "        -0.01871046,  0.31483412, -0.13729788,  0.2522919 ,  0.16173643,\n",
      "        -0.1855646 , -0.05515784,  0.17775925,  0.3385365 ,  0.14982632,\n",
      "         0.05722458,  0.4086055 , -0.3728933 ,  0.24264866,  0.01275589],\n",
      "       [ 0.21431333,  0.01158223,  0.268233  , -0.37130722, -0.3432984 ,\n",
      "         0.049413  , -0.21408807, -0.3683384 , -0.39216915,  0.26335877,\n",
      "         0.05878422, -0.19279255, -0.3232784 , -0.1815625 ,  0.35032815,\n",
      "        -0.11411446,  0.35753638, -0.12135255, -0.12958872, -0.09443411,\n",
      "        -0.06903899, -0.1484409 , -0.17794792, -0.09517676,  0.17769301],\n",
      "       [-0.11751994, -0.36113834,  0.13337201,  0.44009122, -0.28091604,\n",
      "        -0.20598133, -0.31311876, -0.11728039, -0.3735446 , -0.18768512,\n",
      "         0.116171  , -0.38801178,  0.30156675,  0.44114223, -0.38170582,\n",
      "         0.21556541, -0.24904382,  0.2192254 ,  0.19415021,  0.02172633,\n",
      "         0.26295662,  0.09014592, -0.07095411, -0.36523253,  0.05984877]],\n",
      "      dtype=float32), 'b2': array([[ 0.37248054],\n",
      "       [ 0.12522711],\n",
      "       [ 0.65195423],\n",
      "       [ 0.300747  ],\n",
      "       [ 0.14158587],\n",
      "       [ 0.52045095],\n",
      "       [ 0.27930504],\n",
      "       [-0.29972395],\n",
      "       [-0.2617952 ],\n",
      "       [-0.1779277 ],\n",
      "       [ 0.04490942],\n",
      "       [-0.3860655 ]], dtype=float32), 'W3': array([[ 0.01146445, -0.26800796, -0.41656396, -0.26731214,  0.5964118 ,\n",
      "         0.32172167,  0.04824822, -0.49940985, -0.2770642 ,  0.1422092 ,\n",
      "        -0.4047982 , -0.40868974],\n",
      "       [-0.39405602,  0.07153295, -0.4013819 , -0.24177794, -0.12971725,\n",
      "         0.18982475, -0.4434105 ,  0.7686867 ,  0.34095004, -0.51934654,\n",
      "         0.05453527, -0.17821358],\n",
      "       [-0.3373952 ,  0.330204  , -0.3554252 , -0.21417758, -0.53564346,\n",
      "         0.6423686 ,  0.32730582, -0.41244426,  0.3699392 ,  0.23354885,\n",
      "         0.39526904,  0.5977236 ],\n",
      "       [-0.17298976, -0.2678668 ,  0.5719386 , -0.22141424,  0.57674974,\n",
      "        -0.4564764 , -0.14004345,  0.04744806, -0.24751656,  0.41804886,\n",
      "        -0.05860412,  0.4880895 ],\n",
      "       [ 0.00443996,  0.08604974,  0.39148447, -0.1641174 , -0.2949224 ,\n",
      "         0.35175672,  0.06861744, -0.40552115,  0.19369824,  0.572874  ,\n",
      "         0.50335765, -0.60378766],\n",
      "       [ 0.5194874 , -0.33538705, -0.49392366,  0.13989702, -0.33541286,\n",
      "        -0.34156546, -0.01548425, -0.06700143, -0.65489215, -0.38660407,\n",
      "        -0.3065352 ,  0.29666826]], dtype=float32), 'b3': array([[ 0.3313116 ],\n",
      "       [-0.05992505],\n",
      "       [ 0.09282223],\n",
      "       [-0.44930893],\n",
      "       [ 0.69034183],\n",
      "       [ 0.28413412]], dtype=float32)}\n"
     ]
    }
   ],
   "source": [
    "print(str(parameters))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Your algorithm predicts: y = 1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAANEAAAD8CAYAAADpCEEHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9aaxkyXUm9p2Iu2S+rbaupas3dpPdZHezRcqkKGlkW5JljwayAdkwBp4xYMvAADKMGcMGDMOy/9h/BtAfL5gfM4DskSzJ0lAcUxQpiRS35r52s5u9k83e96qu6trey8y7RBz/OHFvxr0Zd8lXxXY9oU4h62XeJSJu3DhxzvnOiRPEzLhBN+gG7Z/U/98NuEE36KDTDSa6QTfoKukGE92gG3SVdIOJbtANukq6wUQ36AZdJd1goht0g66S3nUmIqK/R0Q/JqLniOh33u36b9ANutZE76afiIg0gGcB/HsAXgPwEIB/yMxPv2uNuEE36BrTuy2JPgbgOWZ+gZlzAB8H8Jvvchtu0A26phS9y/XdAuBV7/drAH6+fRER/TaA3waAyXTykTvuuENONIQmt28CXWXj9n2/f2OvYF9T6jMDdLVPhZUHC/UUu7ZdTS9y6Pn2qeg0yuopY1TxfeOmqwxefmH3+8zZs7h06fJKB73bTBR6QyvPwMy/B+D3AOAD936Af/8P/iWYGcwMMEORAjNDKQVrLUAWRASlwoKViADLAFi+t8+5lhFbkH9spZxmc6lxjsDMK39D5+Rj3f0MsJRl29UyoBSBlIKvdlff2/WEnrv6tJ+3uvdqacgcaJ8f87satEwAmMHg5XVMqL8OlFUd8/vIWtt5T/u7f4+1Fv/4v/7vg8/4bjPRawBu837fCuCNsTcTyYAiyIxJRCCo+ntjgLcGl5yn4MS+ZCRq/kZzsC3rqGbtZl0VE/tM439vll0xvHAQY3WGIbVsbxdThp61/QzuyAozyU8Shu5gKL+e0CDtu88/3/Xbb5/MZAAxg+UrGAwCt8pYrafdlnZ9Y9obaqv/HruE9LvNRA8BuJuI7gTwOoB/AOA/HXtzJW0qiQQGlGMOoOqI1UFWnVNqtReaA0uhLRhXB7/HjK6+djmr5fbN+M1zfRKiYuK2VBsiea7VCaJiXcF7qt/L432SewxztQd4SFJXkxtX/VCpsI2BvKrVht7v6vP1X9tuV7ucLgZt07vKRMxcEtE/AfB5ABrA7zPzU0P3+SpJW+Q2XqrCyjV+GV0DY3neMQX7x8LtcV/QkEwdEqFLSoTUsVXpEtaA+1QXv6yKgcLP0mbg1cnCr7NrYqgkWle7Qvf4dYTu6FOHQ8fGqqZdjBeSlF2M16Z3WxKBmT8L4LNjrycA5MYS+985JMqHZmUePbt0M1qjuKUK6F0TGsxdundnSzsGRvvW0ADwzsJnhLaK2aYmA7XLVj1tb0u3ZnltGy583+rzjKWhwT6mzP3UW9GBiFggotr4BiDfR8w8Ib27V4d3jLpi4HeVR751RM4SHsmACNs2fbSsf9ys274uJM19Kd+uf4zh3jy/ZLbuayrpquT5e6/uJ7/t7faNGR9dqmq73KHyDgQTAa6DggiqAUgQui4Eq7pveKZpAgY+9en/qO9yjMT9LzA0cNsvsI8RpQlLybCuRPWpLcW66h6jMlX2i7StTxUeN+uPnYxCNAy0hI+Hfg/VeWCYyKfaoCb5W0GQwBKOrMEHtJSNjncnZbofPbNbsEMbMyp5n2tHPogwBrQYq5qsM7jWU3fCNhh70roPmh8sPdAHQ/0yVuKPva6iA8FEXbp8e1ZjZhhjUOQ5iix3ulnbDgrXQURBH9HYDmfnlPMBcH+IrGP8rkNt27BqU9VXDYnstWUdSbB/iSDvp3pPvqrHUkhwYhqC0ocYcL9qmV9/8Jk7rj8QTBQmdraR++UGBgFgK53gO9ak07sHDxE1emldXb26nz2/T59as18jtkkKPny9LBv1jF/3S2sQDquny2vbTBcaxGHGrCa5FsAx7uEak+cQjVVp15Hm/rkDbxMFH5gAhl2RUEopJGmCOI6940sG6kfJxhjF/cdWZtWgf6Zp+/QZyEN1+oZ6lyo5pIpdG4bupsazr2lvdUnNIem4ru3Upx7+rbGJ2kiW77cgWob8hNSEvlm0DUBUp7jx3sO+iLGzZFVbZ73oZxb/bz/TLtWnLqbsvjd8bUga7Ec1BNxU1nHtWEYZoi5G6rKb+hiv23XQpAPDRBUxW4izqPrdZCYicsyw7IChF904x7Qc9fuY2cJEnq3ULG+dGbWLabuOhaRcsHWB+tr913fvUPnrQs+h+9Y559O6EivU30P27IFhIl8t6wMH5Ev7xHLG9D/hOlAzTzAqGfthKFG3KgN79F0jjeCuQdonQYaYI9SOIcYPzeSVZhCaQNZVI8f2exdjtEGHEK0jWSs6EEy0NNZ5ZQx2dUpjMEEFmafT9uDlH18w9dFYtIpr+2X1/j71akhN6SrLv38du6uP2jNzp0o6UgKua7+sc49/fd8YWfc6nw4EEykFEK2CA116NAWceX2DtG1z+BJIZtJmOf5fv5xhqiRcBTiMR8jGnF9nph66p+94W8VZagBUS50heH9Ijeqjaw2EEJF7FV7LaXxdB4KJgOaLGDT4PK1p7OzbaRf5xfa83PVfLC2D/KuBN8I/4n8fMpxDZa0zWPdDXc/Q298d7emzAYfspbE2p7tDJun6Y91vC1Ki/fSp4geGiXyAoK9z684LzPJjQQbmptpYq3UBm2osglNdu3qdQNNS59J24NBnHzbEGHBhP4zVmJwwrPKOUZGulYTxmWzIBVEzTy81pVObDgwTtaVIe4Vi+5o2jVG/GgzCzQV2vlrnqzTrUPc94s2vX3ygnRWT+efbJV0LKXO1fehfOwa5u5Y09D5W7GJqTpbdgAREInXQgWEipQTdqv4CA7PMCF2/a7byGcn3tttWufsZIN0DaxkW02aGhipbtdH7W6mE7oZetc+/Z91532fyEOrWVZ9/ro/2c34dxNFnCng29ljqat2BYaJOEKE14zH7Jvs4wzk0A9X1VSLI/bQdK1+H6uh6luagW8/gX1EPPUarGcs39AODfh3aD+NJteORxXWv67uvTQ0JNBQCFrq/4+kPBBOFBl0XwkTewOmbmcY48lYYCeNe6NCLHKtahlSiEMw7Rm3qHhjDzLFf5hlD69lgXDNA/VlHGXBgwTq+oDHtO3BMVNFYpGmswd/+HgQusLSP1jWkQ+fDTN63nLsfLKiO+Spv1/1jaR3m2Y9kaU8q3e9UGKhSxSTuliG6QTvQONSW5lsbC2z0OecruiomIqKXiOgJIvohET3sjh0loi8S0U/c3yPe9f8jSfrgHxPRr4+tZ8g35JXv/UAd/ybH+52ZfhlhdMdL1URNFHwd+PZq7KjQABt7vKs9jd/u439ft21dDN5Xfxi1bJYfumdJSwbpqrZLFRui+nl6rrkWkuhXmfnDzPxR9/t3AHyZme8G8GX3G0R0HyS7z/0A/h6Af07LNDO9VM2s7c7s8pw3yHv6rllqXWlVH6PVpeShQb2+fj/M7EPQ+hBjd6mLQDfqt+5EMKafx/RN2+UQvJ8YDIsqR93q+xpmot76e9SPn4Y695sA/tB9/0MA/6F3/OPMnDHziwCeg6QVHiR/MVdfp4dmRW5c2w9NdzHkUioBYALbpRiqoWdava+v/L5j7szKkdouDgzqIakzNKCH1OF1GWiMzdF4Tx0q05KBxqFu7jV73Sd20FhamVDcZNNaEtWgq2UiBvAFIvoBSepfADjJzG+6BrwJ4IQ7HkohfMtwFf0vtysKIfTSxwyA9otdvdfV62fEIYG/Wa2+AP/7en4lgrwe+YjQHraHQt+7rutq69DxtWDhNZk1+H2gulWVvIpAaEb8t8vvY97QtV1K3dWmzPolZn6DiE4A+CIR/ajn2lALgq0nLxf3qVOnABAIGu1kJO7aZoEh/ZpoOY27/GghCqlI9UzUNoAD5TAzoFyywUDbuqRg6HhokpA1U14uiAFDuWtgNvxOvBpONUR914f6sE/699XRp5J2qfTrtnnsvT81YIGZ33B/zwL4FEQ9O0NEN7vKbwZw1l0+OoUwM/8eM3+UmT96+PDh5QzA/apL98tdv+NCM9RKuS0Zv9TPmyr0mBc9JDmWz1afQReS12eHdaFhXb6r/TDYWJWv3YZ179vvPeuomnKDCLRr7iciok0i2q6+A/i7AJ4E8BkAv+Uu+y0An3bfPwPgHxBRSkR3ArgbwPfXq7VpGw2pNetSn+0QVpGoZiRqdyVVCF54ph9jHw05mOV3pfKtGAPBMvsmoDETlN+2q+nnrrLGqN9DE02I/Emxrw7fN1jHow48z9WocycBfMo1IgLwp8z8N0T0EIBPENE/AvAKgL/vGvcUEX0CwNMASgD/mJnN2MpqkcpOjVrn/ZFTvhoq3Wr5Xcfa6tzKy2CxaAVjCAAS4BV9us8GaasP4xy0S+O6+k0DBvkQDakx69pGITdFSO1r2ypEVD9Yu+9D6lkfcDSkTtdtQLcLo037ZiJmfgHAhwLHzwP4tY57/imAf7rfOoWq2dauzGDr6d0Mn5lCas4YGLmpmwMVWLdkoMo4Xs/e6GIkH5oO3dM+djVM9G5RaJJoMElHlPXVAhw+U3fZpQ2GP+hLIYBh1aRXnK+c7762rcL1qVWto6LWMa1cwx1qVqjNPvMaY4K2WRcquVrmalDrOnQ16rFPQ6ps/4S1nho8dLx9fmUcrVnOgWKiiuShhnXnxj0rZfSVHR48fS+6zaA+M1X1j5WafZIvdE2fjl/1ld/2dZmiS1KEbJGx5Xe1OTQhtAd7++/VSKRQvQwJJhpb7ru+K8TVUEOnptXZufdeat8iWm+fRKivHHFN4C6EVKmQmtZHWo8K6uisR8pv20vrl9Uud+i6itaBoMNqVX//rNufY8lXwX3bNkQHRhKF4eWOcwjPdJVKF5rJ1h0Y42gpMQdy3K/FqEPP275GntlH8fZHYyaa0PGQHddF67yHq7luqAxfc6iPH2SbKIzsNFW6NgXtBGpf451bsz19v5uVNL8OSsx1fRgd94VULXcGXa99nYE61t5Yp9xgGdSeDPpAlPWoC1oHc/3exjzrgWAiYD2xHQIG+iDlPtqPh7vVGtTSCN0SaagdvaDJSMnYvO7qX/3QAFvX/hqyNcfakWMZrPvcenbegWEiAKvSiIc7a0WVoNXrxzBnlxE7Ci1ir24iMNFo9W6oXe1rQr6Y7vLDbd+PZO4DZK5GaoQeYwwaty/QwSlshHFjoqIDwUTV4CCizi3Uu/TuVUlEnddfK6N0ULpVJurA4FrXDhgarF1903VuiKHWke6htnX1f7PObni7fW9o4HddU31ftbWDj9RLBwqdA0IzjIJs9mWD163eI95Q7+xadV8rRgOajtghx25vOV26/QjqYrY2MDEGzLja/ll5t2SDg7otbbukRqMt/tdQG8Vg3Ve7D4wk8snfDW9tH8Gahml7tuy7b2xblmihLCODV0efn2SobWNpeU94VWrzmvXqG2rvum30+6UtRarvFYWZg8GWwdYuJyt/3FTMEyi7s8wWHQhJ1Ejry5JzrtpKRY6NLKdS4dCvclTn/FwFYxC5/RrRxMP3rqN2jjGmK6khfxuaJdDyK42xA331yH83Ieprf8NXtIKmdkuc3gmtYhDrMQoBsKuTc9tXNca2OhCSqE4Q4h5wNRGHqHL+g4ekVH08UIdMRv0v92oGb7W4bqwdMUY6+n3SVdbwuXYEePNj3dcGExOW+XYILv6WwGAs5vOq8E6p1GdvDamEXZJiqP/a11XIeVVexUDWWhhjam3H13i6nK3XvyQigBQBdn9QbnuQNQYDpDO7rm/PRkPSord9LZ+UX48lQDHg4moB2r+/qKtN/uAMMRl70rBiEvKgeRl1hFovaFdNBFIK6eZGc4snbl/WneK3eWH3NV2TZPi6+gk6r6nAqlB5jXo6irn+mQiAvxjPWruikoAJ1daToZkZWGUmvz+WL5YaZ8bYTW3Rv1/Duk54oqomMNSaoEdX28YwG5EHu1dgh/xoMcxSHabGdXKfqt4NVpl1nX5qvonuZwRWB3+XVOqqsw+l7CrTp4PBRE4S+Tor0IS+/Te9wjCB3/5fwqpBHTJs/d8hGsVArCC7DvQMbJL/LJy+PaZY/xk7AJRGFa0JoiE90B5YTbvTFiUWly8hu3wJ1hgkW5vYOHYcpPWqGkVLiRZqQ9e7qlm4Ybs1jf+2NAo+q2t7JT1D67qq+9pmwso46ejL65+JnLrVBhcal9QdsXqNbz90ggNOhZL7qkETzq2w0rz9SJ7VaNhOqhlpbNHA0qRZtc0BLJmGvHsYywFbAzD1b6fGMePiq6/gtR88hPzCRSgAcZIgShJMjx3HkXvvxfaJk4BubQYQEisDRFBgLNds+naL/91/ps6yFKGhg1ZaR0BSttXtMZPn9c9ElX1AVOkIHca0u5zCHdJFFQpE3pDy7YO6GSNVtXHX+cN3RDlrDEKuu0n6zLdNyPVhreK3+qfX/rCMlx76Ll5/+PuAtdBKI4oSKFIgKGTvvIPzj/4Q+NCHsH3L6ZXyeGVFVeA5V34TmJcGfxfj+CBF1ztvHK9Fa/c7XmcMXf9M5EhE/NIADEoJN8N3DYbq3vDxqkurZdVybb0r+YhZby2baE1pVM0h+12o2ti3jBp/aupD8l597Af4ydceBDFhkiRQSQRmhjEWpAwSEgvu8rPPYuP4TVBx3CwL1SNz4xm67JmljSozSIXK9r3b0LEQaFQzklu2HxoTbUnUR9c/xF3Nmq3Z0n9IXzfueuam/dQ8LtWsQpjrMMWQ8R48X+HDI66tVK7uClBHia+rYnbBzRVdPvM6fvTgF1HkOVBBwVW7XJ8q11eaFHZffW1Uvd1q+bJNhOWWOlprRFGEKIqgta4/fbnHB558pS1dDNlX/iATEdHvE9FZInrSO7Z2vm0i+ghJ3u7niOif0RpPTSBxlPHqzOJTjdRx+LzXltX7sBykQ/e3yxrzKOGy3EzLzXU+PiM0DF10M5IFViaBofYPqrkATFngkU9/EvPdPcAus4FWtyqtoJSGYYkKYGZkFy6sPmn9HE1Aow1wtI8LI2kQqZVrQ797baN2+YKg9N435t2OkUT/NyR3tk/7ybf9LyAJGe92n3aZ3cRcv7XQJl8rs2/Plt9DM3U1ELuuad8fQorGvMjVgitmar6SlXb0vNNQi/cDfPByFsFPvv0NXHj9TZnElJIsr6QaNorSCkRKllVbA1uWnfYPiAAVDjfqYip57O4ce+3vY/q/YW8G7hn67dMgEzHz1wG80zq8Vr5tkiSOO8z8HZbW/5F3zyB1MgtWB8mqVOkfRNcaeesaBKPL9FGkjtNBmdah9vZ9H6LZpYt49htfgzWmYZPULgEikNIgpR0jCRCASDdm+AaDuNm/a/CGqL6myvE3YoAPMVTj+AAjDbVxvzbRuvm2b3Hf28eDRES/TUQPE9HDFy9e8I/3oiWNAdQjjfpIxvBw542RVF1gRN9LB3yksIMCp2tlKdD2obqDxIxHP/tp5IsMwDLUSitRrZRS0FEMpbX48UhBOWg72d7urKs+FpBEwUddafswc4SYof+5u9XEMXStgYVQ7atY4vJ4kLiRRljMrS4jv08946GVb+hihrBKt04H9w2gPjTItx3Wqa+KX/M//pY0XZ8uOvP8T/D2Cy8CLCp0HMeIdASlI+gogopjRHEMHcVirygClAII2Dh5opdBiKjhmwr1U7dat2Skvnva9/Wheu0mrKuB7JeJ1s23/Zr73j4+mvow+y4GW0tV8Osace/Q+bGght/2huTqKxvLGcif2ffTzpWymWHKEj/83F+hzDIAFkmSINIJtI4FGYsixHEC5aQSOTeAUgpMhMnOoUZ5fjsaA7vVtNF2CK+W5dcVqq9v0vA16DaCO2Yi2y8TrZVv26l8V4joF0ha9J979wxS5Wzr9ykE72z8CnVIpxTDSH26g9YZvEHJQM3zofYxkeRHG1AP16UXfvA9XD57FmVZINIacRxBO5tHIGUNRQSlxKdGRKLWEUFtbNThP70ShQaVVndh1/P0q9ptV0jjzh71tvedd1Q56Gwlon8F4FcA3ERErwH4nwH8LtbPt/1fQZC+KYDPuc8oUkqhLMuGb6hzVvElFrgOiPTP+bN/W4rtRwL5bQpJya5yhiDm9stuXO+0mtpv6NXV9wxd0rw6VuY5nv7aV1AWObRT48QfE0Ep5SIUlvUorZZ2kQJ2br+jv92NYxIjt/I+UPnt5EFphHoV8iOuRdS9zmzISzfIRMz8DztOrZVvm5kfBvDBofq6iKgZwb1WJ9WBYsOOSObVRWUhVbJPLRtqX0garpS3DOduXHM1aGNXXcuTwLPf/w5mFy6CrYWqnJtxAq10Xb5WBB3pmolIKVCkAKWwffLUYP3L6hjtkCaRsEA9Q7jfVdhS3Q+tzY6ZGdYYmKKo20lR1Bnl4FNVbjjGz98PKkwHKuxnzCBqUO17gQdvjJvVaGUJwLg2tqVa36DtlqZL6bJfeLqvje22Vt+NKfDMN76KsiyhwHU0gNYyEBU5iFsRFAFRLPaRTiKQUtCbm1Bxc0j1oZj19+pZO9pc2SxdUVKzS5dw7sXnsfvWmzCzBYgJSZoi3d7G1qlT2L7tVuh0UquevRMSCfM2z/f3+4Fgoi5byJcObSTGZ7r6nCeRusquj5O7qm/mbtXnt6mi/dhP8nskswcM6LH1NI4x8NLjj2Hv4iWwtaLCKe26S3a8IMWIYg2lgCiJoeMIpFXdrYfvvAtwsYf7Uae6pnwiAtlV9duYEq89+RReffRhFLu7kFBYhSiKUU6mKHf3UFy4hNnrb+LI/fdh4/ixlXLbE8m4nm/SAWAiXnnQ+kwPc3WpXSHAvUtaMNCwqSoKXttV375JuJjQVF362hGaSNpt9CeW5nnGk1/7Coxb5UlKwbIslS7LElEUifpGgIoUiAClCeQABYpjbB0/idpYk/iFzhm/31YKP6sPGNiyxNNffRCvP/kEzGJRQ/GRjpAmLDC8UtBlCTtf4NIzP4aK7sPkyOFem7iv7i66/gNQHfU5V9e5l4jQdsT2Dfwu1WpMXX0+rHEkA7IKwuxFuvYhAf12XnjrTVx8+21Yyw55c/F8zhatDXbXLgYAVaF1hJ3b7gBoNeI91Nax7e56BmsNnvjKg3jpkUewd+kKskWOPMtQFgWMKWHZSEsdSGGsBTFj74WX62Ql/rM3fvf0U1ePHgAmaqop7YfuWx/vUzWg2XcKjCA/2HI/oMa1kUzjyhhre4Wue/gLn8NiNpckHUxgC2gd187VOEkBIhhrYaxBaSxIKTAxLDMO3XZHoObhnN9j+rIhL5nxk+99B6889iiyRQZjDIw1sK4cpVSNGEZRBO2FJGlmZBcu9qOx3s+xr+4AMFGT+lSp9rE+ScCBdSSdL3RdQAMDKNg+iD0H49BsHVLjuq4FgNnFC3jj2WeRzTMURQFjDLLSoDAGlgHSWtY0KQ3SEYxlqDiCAcMYg53b7wRFPx3LQDSH5ft84ZGH8Ox3vo1snsMa8Z4QqJaeURTJats4gooikJYEKgKlA/Nz53r7ghrnqP70vcsDYBNhZb19eybpGjT9ZBs+il6Vzr2Aau/VsWDBtbSTQiX4YEKoH0LXtomZ8fjXvgJTFJhMElhrXFycBpRCaSzyvJQWxLEM1CQBlNvFjzSO3vW+lbqWfarA3O2aGGsbMVu8+OgjePprX8Vib46yKAG2IMuwBBfTJ30QJylUFCFOYymLGEQskRV53ttH3buEH2gmcgNXUe1wrajyG4UNaskvKtCRrh2ESz9Dq5YOaSZlUWXnd3ZlaJDsh3m6jdgmbuQ/97oQuk9FtsBLjz8CgsVkkiCOBK5mW00zQpYF3lCOuYwxsAQcveO2xirWapJb9sdq2/ufs9l2ZgaY8eJjj+LJrzyIxWxe2z4KjMoPSk4SKaUQJwniWBhJKefHcuuRqiBZvw1+W+p9tdfQPg4AE7mZIQjptxij6ggHycpFgfLYsUMHFFsdaztd24wYuq8Ldu+jdtu7SYE6nCU1Qzmjf0zVzIwnv/4g8vkCigQkmGxMEMcJSGvEcYokjkEs8XTMYhNpsIQEKcKpex8ItqP5bNKyZUaYMPAQUq+ZGa88+Tie/upXkM3mMEUpdrBhkHY2EAQsIgBxFENFEXTsIixcCJKFRWkKpIeOrajDfX6jMcx03TMRw3Wm7U8aseLkhLz4OI4b4RzNGc4NNuof9D7TAat8GZII66JjY4i8ARi0hRom7nCZZbbAj77zLZSmREQKSRJDawWdRIiiGOlkgiRJoHQEpRWSNEXknKlFkWPnltugW7kUhp5A2r468bWpkkCvPvkEHv/SF7HY3XOMLJqJAAdUhyFpx/RRmjpgQd6X0vKCmRmlKXH41MlhxiAPTYLPSOF3et0zERiSkLw1U4fUOsAbXAxYy5DEjl5xNdTm7q39MauMFGQsCp9rS6FGWwZoHdTPDwdqPnP1gRsAjqEakqtiQjn2xNcfRLZY1GUsg0wVolgjijVIyUBM0wRRKnF0IEKUxLjl/g+Ph7Bc/UTNMJqw/SPv+7VnnsbjX/oCFru7KLO8fk9aKWilEEUSQaGVQjqZIN3cQDSJQVrJ6llddYO0cePW26AcABLyFa2jwvl0/TORRz6o0HYctq8DgDhK3Avz35r8t8IEWDJSewn6SvnoRwnXZZ7Q8dDz1deTjN3m/R1SsLURWtUTxWKGp7/zLbAxgFN/iUjCeCKNJE2gNUFrQhwpxLGG0gQda1hrEU2mSLd2goOxj5ouhm476dzLL+GJL38Biyu7sEUpi8MVoEjWLMWRrm0gpRQ2traQTlLESSJ2D1HN4AzG5PgJbJ2+ea0JaywdKCaqqBpYlQQKzSjVdQ2i6r9VaSanViXJqsRxg9QBDQrddbYZIDTjhvTz6nvor7S1SjDsyqviAwM8ucKo7pbHv/YgFrM9WGOgq8GotbMlNLRSIHA94yst0QqaANKEQ7fc3tn29jOGKcxIzIzzr76Chz/7l5hfugKbFYAtAQCRkjAjcvabqHCxhCBFGjrW0G6xYMVEDML0+Akcuef99aUrOHQAACAASURBVLqnQaZfE1w4UEzU9v0MwaMr6lU14LGcoOtrlpU0ylgBF2iZl84vu/3dLzt0rouCtk5rsC5V1lUJ3CWV/eP5fIanvv0NGGNEPaock5EMTCIGGwOlY4nYdlEJKlJQSoCCo7feubQReyRv/0BcZaDLZ8/g+3/1aexduIhsNgPKUkCPCsIGpE3OJzTZmCJKEk+KRgBJ7geAkN50HMfef6+sug20ab8qnE8HgonaUQldM/rKcWuw985ZzN8+C5PPoEhBpxOk24eQHjqCZGsHKopR24wMQIUlS4g5x7Shj4bUvhBQMTRgh8omAp745lexmO0BLLGBkVISka01GBaKJfBUwaVPILGjZBKyiDY2odNJZ/29YIH3vQIYqt/zS5fw3c98CrsXLiLbmwPGAmxhDCNKNCp7TimNSBOSNHHqm4aOIlAUgYlk0aDWSI8cxU0fuE/i+1r1976r1uEDj84BYRvB90eEZvwim+PFbz2I2dk3kSaSCyBOxDDec4vMoo1NbJw8jc3TdyDe2AJAyx1EWlTZSvJ9aU+3A8PH2jlDz9v3vUsCh763qcgzPP2tb8AWFjpSiOo1QxFUJCtWZXAbEMUASz45BQYxoygNTr/nA52AQh9MvCoBALgA43w+x3f+4l/jnTffQjabwxYFYi0oIVvrgmDdGialEMUxkjSF0loYKdJQOkLkQpWSQ4dw4v4HZMFgqx1DtFT4x913IJjIJ3/m6rJdyjLHI5/5BIrz57CxtQk4XwdZQF5aCWMt7OUC+ZWL2H3lOWzdehcO3fUBUJIMRqqtCyBUNDTQx0iZECONh9YZP/7et7HYm8Eai1grxFGEKImgtHY2jyz7Blwf13cu+3zzxM297eyyC4MtYqDMc3z/L/8c77zxBvYuX0FZlNBEiCIAijBJJ7BG/DyJTuqIclHhNKJUfEORjhFFMeKNTZz44AMra5vG0lDuwTYdgNi5ZeBoSI1r/7bW4rVnHsNbz/4EZVGASwM2DFgCG3bfIVsNuo9ZLHDlxR/j3ENfx/z82w0GbVNb8lXw8joMtR8G6rqnywYKkSlLPP71r6IsCmjHKDWgoDViLbkUquOVl5+BeklEsn0ESocHZ6jfhvrFmhKPfO4vcebFF5AtMljLWGQ5SrdbHYgArRFPUsRJClMaFEUBa2W5QxTLuqaoAhimE5z40IehkxRBlaKjn1bdBeNpv2mE/xciep2Ifug+v+Gdu8ZphEn8PQjbRBXjVN+ZGRffPov5YoEyL2EKA1sasLHCNMxQqLzcEOlkAS4NisuXcPGx7+PKyy/UixB82q8Buo7tU/32P/7xdVRBnxiM53/4A+xdvCj2jlomHYnjCFGd01rSAislKB1Iu2ynYpueuOeBYPl+u8e2z5oSj37+r/Haj55GtligyAuQaxdDljBYMMqyhAUQTSZINzdRlAZMClGaIE4Tl7pLbKLjH3wA8XTqEMwqo+x6k5Zc0LTxRCqH+36/aYQB4H9n5g+7z2ddw34qaYQr3wLRskPaRqw/uE7dcy9KCywWGYq8QFmUMKV8iCE5pa1LgMEsBqxh+ZQlZs//GBd+9BSsg1abbanqoaV0pBVbtOOeVepC3trfQ9evRZbx6Je/gDzP3fqgZZJ4TTKpiOQRVEt0O8knp1UEIgVLCpvHjg9WNaaNtjT44Rf+Gq88+TgW8znKvABbWZI+mU6QpAmSJEFZWmR5US93UFGEnaNHMdnaRGkN4JY9qCjGsQ/ch3TnMBqMU/ulhoEZoP99duUn2m8a4S76qaQRVpF2s8BSdWpsSOt09yrxxbGbb8f7fvEXsTubYTabIc+WIf7+BrdkSRhIssGD3OyjiJC/+SreefIxsDGdagp5s1WXjyY0Ow8xRxcD7Yeq+/cuXcTld87XObSNKWvJzE7KWGMlKb1lsHG7EiqXY5stdm6+rRNQ8Klv0mAwbFni0c9/Bi899gjmsxlMaQAXupNOUmwf2kYURbJWyBgYB2iU1oAijWiaYrK1genWplP3FI7e8wFsHj8p0qddvfe7D+ENtXUMXY1N9E+I6HGn7lW7QvwU0ghfrI4uE1q0kDDGMuKY3fH7/+1fQ7JzCPP5HPP5HHkuerYJMQQgszEpKCY3KxOKc2dw4ZknwHa5PLvJFIR1Nby2StZW3a4lA/n08tNPoMyLegUoG0G8CISyKJHNF5jt7mI+m2MxW6BY5KICA/WO2qfe/8CoNvWpdbYo8fBnP4WXnngU8/kcRV4I8hkpREmEOIkRxRG2D+8gThNAKVhmGMtuV75YwpG0ArSCTmLc9P77sH3qFrRnssZ7HkgrHWwvrUb7h2i/TPQvALwXwIcBvAngf62qDVzbARp3t48baYQPr5S8AjI4mNk/Nt3axsf+4/8Ee3mJvfkci/kcRZ6jLEsYXu79qrWkhKoyejKLqkEsMVrF22/h8k9+hBC3jLEB2teH7h0CDYZspiGJBwDP/fAHYLYOjmekSYLNzQ0oDUHo4gTTjU1JFUwSvJstFigyUf+gE8SbW4PP2HaI+1RmGb77F5/AK089jizLUeSlDA7t9h5y8XpxrBEnEbYOb2O6McXGxgY2NjYQJxGi2PmLCCClcPSuu7F96raR72Ifw52cT5uoUzLtCwNk5jN1HUT/J4C/cj9/ammEXb1OoxO1jtk21It2J5688y7cfP8H8fYzzyCpc0dr6Ei7GCyqE1rI/S4lrm+XAMjPvI7djQ1s33FXwze1hG+dZ8FzMPQN6DFqXPu8f3wINm6fM0WOS2fPQJHCJE1hyxKRVkjjGBtbW4jjBAAjjiIUeQ5SEZJJiihJxBYpLG79mY+KTboyJYZRrjb4k89m+M5ffBxvvfAciqJAWYiNU0VJSGpiiRaPIl3ba1vbmyCngkSRlnfutng5dNsdOPqe9630TT+1vUCrfRcso4c/9yWJnI1T0X8EoELurnka4co/4epdoiQkcVF9mD6Rws/++m8gg8XubI4sE6ChyDIZHK3UxNbKTC0bV3k7tBFh8fLzKC5fatTVHqw15NDDQP6xIUnSReswEAC88dyzMHku623iCJubmxKNDXGkgi0m0yniJMFkc4rNQ9uYbm1ic3sLSmuUlnHyffdVUzLqHf78vH5+P7SYPJvt4et/9od484XnRAIVBkVZ1giYxMFJ9HgcCzMlSYQ0jTGdTrC5vYFDx45i56Zj2LnpJmwdPoJjt9+O0/f+jAObxvWDvLewlBxiPmbuBBb2m0b4V4jow5Bx8xKA/9JV9FNJIxxSD2rQBb6xWA3EyjoCto4cxb3/1i/j6Qe/jDSORXXRGpGOoMmhU0761Ksgvd0Uqihhay12f/wkko/+ElQUdUqDoWOh++rJYUDC7IeYGU99++t1nFykFDamE0zSFGUpWXImkwkAhrEWUaRFYmsNthamKHH49veC1HpKS/VOFnu7+Oqf/gEuvvUGijyHMRZlUYLBXhQ2Sc7vSEv9WtdaQ5wk4g9yf3UcId3YwC33f8TFxy2pDUn7/TrUztDvxmR5NRELHE4j/C97rr/maYTbD+bDzHLM/SL/wZcRu/f9m7+MZ779bcwXC0ySBGmSSOQyZFBF3t6fgITbq4qxHN6plAKyBbJzb2EjYMT67RuSFOtC1yEVqe86nxZXLuHcqy+DjXVAlqr9RFuHdhDHMZgIRV5AJJOB1hq2NLAwABHu+sgv9tYrCs1q1qUiy/C1j/8Rzr32KowxKIoSpbFgRu2XEtWNRAI5VTt2wbBRkiBKImglcX1JmiCeTHDi3g8jnkwH+yQscbrPd5U11O8HImKhS88Wag7I1WsJcTrBz/7dX8el+RyLxQJ5lsMa63xIBWxpKrSilj5EGmCqHbna6ejZ6y/Xx5rkVJ1ONaJ1dUt9azNXSMVbV40DgEcf/LwgbWwl9a8z4OM0QZqmmG5tYmtnW9bjTCdIp1MkkxTTzQ1sbG5h6/gpJJvbnfXKA2LFFrTG4Ft//md468UXkOclsqxAXhgYUyWHdDF7WiOqMq26pRdKEyInhWJny0apJB45eufdmG4f7m1Orz0aON5WzzuZqaO+A8BEq+qcj/V3zSbtwfe+j3wMamOK3cUceZHDOHvIFDlMUdTLz4kIoErNEMejcr4SIgIVOfKL7zTK9mpvtCFEIcYZyyjrUj7fw4uP/RDWGhAYkZL1N3Ec17nYqucT/8whbB8+jMnGBqI4hjEWt37o50fW1hyAP/zy5/HyU08gzwREYCw3HQOonpS0UtCR5I6oVDtB6mRxnY4iJJME6WSCzRM3Y+um071gkk/hCJdudbvL1nUHO+u5/pnIg7MrCdDWfUOwarvj4iTFA7/8K7iyWGC2mGEx30NZ5qJm5JmkYAJBqSrhn1tboyNUA8QywxiL7K3X6mmpyTBij42BpJeI4DrI0npM9thXv4wiywHL0FRFbOsGaFINAL9dRC4dltLYOXF6ZG3Ltr/53LN46ptfFwTOy6FNpBBFiSClWnsqXbTcukW5bVyiSCRVEiOepJhsH8Lh2+6Wskb3wGqfin3TXJw3xi7qo+ufidCUOL7R2IdutVUjZuCej/4iMra4Mp9jkS0kisFaGGNhykIc+FYy0hBpgCRZu2HJOFcWBfI8R3bhHYFa0ZIsgaxEXWqa/2whG2ksdaF62d4unn/0IdiyANjWMz8phSiJvUV2aqnGVgGnLqLjxD0fHNkmFyrDjMXuFXzzk3+GxXyGsiwb+0oJg2ikaYokid1Wr8KwVcyerGtyNlEcy7YuyQTH3/8hRHEiERT7nGya42jEYwGNKJcuhe66Z6IK4g4tzBtCXvzrAWC6uYNjd7wHsyzHIpdQoNIaWGIYZli2gNKwVnKsWYj0my8WmC8yGAaU1rDGoNi9XJfbGMgd+niXytbHQF0MMgYS/8EXPotsV5Y8EFCvG1LtHe+Uqvce8vd4NWxx6p4Pjgx8qcUyvvnJj+PS+XMoi1L2C3IfIkKkFaaTFJNJgiiSDcSW3SaOb62dDRTFDlhIceIDHwbFyXoiqK+1zKjCX7qkUUU+4NTVgOueiXxcP2QLVd/bPp8uBrv3F/4OdrMceWlQGFlXxARYcuH+1sKy+1iLLCsktI4AHcdgJhhjcfnVlwJt7fIktK5qDf6utgZVkRF05Z238fKTj9UppoDlkofamexSTUm206ih4jEzpkdOQMXJiNoqJ7OsVTrzwvOwxqAsDIq8QJJIGYoISRJLvgYSAIFoualynCSI0xQ6jqGTGFGaQCcJbrr7g27BZJ+LNExDroWVTWMHyuiiA7Mobx34sW92v+OeezEvDXazDFtlicLIJzIlirKANgV0FMEsMlhBtx3AAMz29sCWZdHaubMwWQadpk0p5LWnC4HrepYQc/nH/YHg+0Ha9NBnP4N8vpBc1ZahY10zUOVIriICZAtJEinljpvS4LYP/hvBNq60zzHQ7vlzeOzLn0eR5wDEr5ZOJrXUs8aCSCD00oE4URQhiSIkqWTp0XEMHSeIJlPoOMHh296HyeFjK/0a6qMxUPfqNQRAubKHxle3vLn+mYjDndDnnGwf939PN3dw6NRpnD9/FtvTieRTiyJJO2tK6CIHSHwXhoDI5Z/O8wxgdokxUkARZm++hu33vLeup2be9u8B1auLup6tj86//ireeuF5hzhamen1EkzQ2hns/laM7CIXtIYpDazS2Dp2orMO3yZlWNiywDc+8SdYzGawlhFHMWhDEslL6i2NONGAZRSLDNligTiJsXVoC8kkRTKZIHafaDJBnE6wc+pW7Jx+T0uDGh+uw63JrPN3R5H+BMbczcTAgVDnEFTVgG5/SnVN8B6l8Ev//n+A3UWOS7M5FnPZniPL5iiyBbLFHop8AVKEyWQiWUDTFFvbO9jc2UE8nUDHgnKVF87Vndu2ifqAjy70boiGQBS2Ft/9zKeQzxeyvIABTbzM5uPsoOqe5bIIkVhsLYo8x7E77wE6wmka9TkG/MHn/hLvvPlGbbdGTrpMpxNEkcbW9gYOHz6EyXQizAxCmbuNlZMEyWSCKJ1AJymiZILpkZtw9M73u74UUGidSWisndz0NYZtWR6I/gYOgCSq2t9WZXwa8hU0BjAI77v/ZzDZ3sKVeY6djQWmiwXiOAbAiLRTL9KJqECQQSHragyMm90l9syiuHwRyeGjK+3pAwXGtr06P9bv9OLjj+CdN16X2MBWYo+6zQ5yLsuyvr+yAcnt9XPintXAkq52PPv97+D5Rx6W1afWAiz1pHEEY0ps72whnSTyHkCYbG4ALEsf4GBvHUvGnihOkG5uyerZFhNXEejiROgeD33wdJev0T1hvZhGjjko/G8HxM2NhXQhdK69QG+lhNaxKIpx693vQ2YMdhcL5GVZL0c2RnZZY7aNQFTSSja80loGgYOEszOv13DCknG6nYFtKTr49CNeIjOjzHM89Lm/wnx3F3mWiypHTXSpkhTVSt8iz1GUBcpSdlkAGIduuzMIKIQG5Fsv/ASPP/g3srzEyMK6aqPkjY0JNjY3ALetSrHIYN0EtH30MA4dO4p0MoUxJdgy4jiFTlOc/uBHoKK41TfuvfJ4JgkdH+xvF2ArviSMuwcHQBK1baKuh2p3oJ90cXX2V/jQL/wSXnnyKWTGYJFnkhiDGaU1KE0JVRYS9KiXs3m1zw6xBbOBUhF4vgtbZFBJOv6Rep5jv/TYV76EC2fOIs8WYJboBPYyxApAIhOCNRYFWxeJwS4ho8D8px/4uTFPgMtvn8H3Pv0JLGZzYVpIaFSSxkhSgam3phMYUyKbzzHfm0nasomob9PpFFWuscqhevuHP4YobcbE1e4Na8EtYGasBPKPDwJUgYSYQ3QAJNGyI31pE4K122BCF+TNzLjlzrvBWmFRlJhlCywWC5lNiYSZXHYbJgC62mmtKl8WrYENiID5GX8xLyQ/wQDYPebljCFmxoW33sCPv/dtKLWM9QO5PYWc3eNfby2jLAzyvJAtG0vx5Rx6z91QSTrI4HsXL+Jbn/wTzC5fRp4VsFaCdnXkljIkCZI0RpzG2NzaQpJOJDOpm4SSREKPNra2sHn4MOI0xS0PfATJ5k7ttPWJQKJs8aq/cN2+bI+RcJ82zw+Vf/1LIrSRklXY15/ZQ5Cy/7uidLqBU++5E2dfeB6zrMAsm2Oj2EBsLBLLiLXbKY4Zyr1YiUiQF1kWBcCMOE1RvvM2cMtdgJNa7br6nutqJRIbg2/++Z8hm89ALrG7IoImGXh+0vfmIBS7qMitZPxJU5y872frdnXRYvcKvvn//jEunjmD+UxSXFXRDnEcSzRC7JZ5u8lnsrEBHUcostztPAFAE6I0QZSmOH3fhzE9clOwfwCX75xkQZ5xOS+qZ+q6p+/7Wv3rM1JHEQeCiXwK6cEho7cPHiZn0P6dX/8NfPyf/R/YXRTYWyywWWSYWIOSjeRsUJLXoXD5oMEWMAZFtgA7PwgTAGuxePtNbNwsi3qbLtdrq7a1n+eZ730T77z+Ooo8BzMjSRJMJhPEcSyhPkQuqylBR8rlq44lGUsh64miNEG6sw1dRQV0DJbF3hV841//Mc6+/AqyubNxqFoLJPsbpWksfinXx0pVGUsVNre3kKQpoBSSyQQ6jnHinvuxc+pWoA9AIaDaUoaZsVgskLgI75VLWxPtWJsyWG0HkNWm656J2AELXQa6/z0kBUKzfXXslrvuRjydYC/PcWWeYXO2wPYhZxuxIFWKtRjORS5gAhjJdII4iRso0fyNVzA9cbreAFjwHUKV/6FN18ImWuxexmNf+ZIs5/CCcyvn6ebmJtJJKgG1pGFKyesGReKINQWiVKKljdtEWMSEN2jc18VsF1/9+B/jzAvPYzFfyJYskCgOpeB26nZ7GzlUEA7liuNY8sbFMaI4RTyRBXbH7voAtm++DeMnGoUoirG5uYTpx4IL6/iRhspq03XPRNVL9PdnbTNGH4N1DVYiQhwluP9jP49Hv/417C0KzBcL7O5eQbqxCZ3HAEG2KHS7r8VRhDiWBX2kUOv4AEC2xPzsGyKNap/R8hHGhgONfXHMjO//9aex2NsDg2s/UBxF9daRSinEUez8RDGIpiI9tIJlCxVpzPb2UJYGexcvwRQZdDJZLnN3Yml++SIe/Fd/iLMvvlTD55VKBQdMaLcGSHImqFptqzZKJkUSjeC2shQGuj34zvr7aLnJ2Vh/0NC1V2ufHghgoZ3h1P8eAhCWtCqO207Pn//VfxfMwN4iw5W9BeaLOfI8Q1kWYDYgLVHPk80NJJNUkuJHEaoMm0TaRR1HmL/+EtiaRlvWeT3jX6bAy689+4zUYRlaK0xSUaeIAFsWKLIF8lzgbmtNPfnIhKQQxykm6QSTyRRJFOOFb38FbI3H8IyLZ9/C5//g9/DWc88jm89dCl8/gr3aFM1JQQXoqBVapDWiRNIA6yjCTS0GGuiUFWg7NGz7kLcQraMJ1GOm45YxaYRvI6KvENEzRPQUEf037vhRIvoiEf3E/T3i3XONUwl3M011rvorszlQJ9Lo8UYDwJGbTuLIiZNY5CV2FwssshxFmQmc7da61EYsSUqtaoaVzbC0YybZBGvx5nKt0U+LikWG7/31p1BkOQiMOIkQJ7J8QJJ8xA6pM8gXkrxSQvpLWGskSSOzZDwCwZQltI4wO/smXnn4WxK9UJZ48bEf4PP/1z/H2y+/LCnHiqKxg7v0g3MsK0lmKcleZJduyREXO7QuRZTEOHz7+7B9+vZBP1ldxwh/TWg8tCfbrmtDZa8DDgHj1LkSwH/HzI8Q0TaAHxDRFwH8FwC+zMy/S0S/A+B3APwP1EwlfBrAl4joHpaEJVUq4e8C+CwklXBvwpLK7mircG1EbtX2ccf8bDS0KpGIND72q/8O/ubjf4qsKDHPcmSZGM22NCjzAqWKAZZZVjQcVe8uVyFeOhLjOT//FqYnTkMlSS8vjVE5faoclWwtvvNXn8SV8+fBlmUZdRxjMplgc3MbWmtJzmhNvSVJUeQiPZ0UiqIImgjsVpHmiwx5toDWU5x//scosgWee+YZvPncc7IytSyRZ0Wjj5Vsi+qkkJvcXJ9VjurIRWZHSQodR9i6+XYcuu3O3udsD3hqHV/+FSk41iYaOhe6bgXi7rh9TBrhN5n5Eff9CoBnINlLfxPAH7rL/hDLtMDXOJVwk4FCvh//I6EnIdSukkzNewHgvp/9OSitsShL7M0WmM8XyOZzWVRWFOLZL0oZJIZlMwkADBmUpTGwpRXVjhT23nh5pcO7Xt/Y2U4pBWsMHvny5/DCY49ititbRQKEJEmQpBNZAbq5gZ0jR7GxvYM4nSCKYlhrGmmUJURHEpLIUgUWyJ5kgdzbzz+L/OI57F25gtlshiIvAVIojUVZGolkrywmz39nLbtN0gCQgo7cuqAowvbJW3H49vcF+98/5v9tD851bKD27766upDdMT6lUDt7iYjeA+BnAXwPwEmWfHJwf6uw36tOJUxeGuHLly7XzOEzSq96VznmuJVQhBp11H+3dg7h1G23oSwZsyzD3t5eneLWGPFNmLIUB6y1ktfGJXokZw9ZLFdBFhfOoZjt1oKvah2jxUzk2lod7+AnZgtTFHjkC3+Jx776ZVw89w7mezNYY5xRH4m9oaPaPovixC0vSADSsMw1IxVFUds2UZJCRzHKUnakYxapd+jwEdxyx61gtiiNBZTsGG7Y1nFy1jKssQ2nt8DZiWRSVRqkNDZP3orD77nHk/7dYE/zPVbPv+obbL/QfpWu3Z/jmHHluv3aRPX9RFsAPgngv2Xmy32XhtrVc3z1oJdGeGdnp76KmVd7pHlfbQ8xkaxMJb+Sjrg2Inz0l38VxlpkucF8niHPJNljtTKzLCXlk7EyG1t2y8YZIJJd2qwFjLGwxuDcU4/CmrLJSBVa54Rig3m83/7gYWZcPv82vvnJP8ILjz2EMs/rros0IY4TJNMJoiQGg2qmB4Asz6G0QrqxAVIKeZ6jKEQtk1WnUlMymTrHa1Fvv0mkcOjoERw/dRLpVLaWjKIIE5eqShhSpI9xW9dUeeIilycuShJs33w7jtz5fhdQ2u0I96nLRln9TWgP4SXztMsfl+QxzGCEriSR3SWvVhRDGOhPmPnP3eEzTkWD+3vWHb/2qYSZ6w252IXsV9/l9KpU8nGxKtk9uX8hfr7rA/cDIBTGSjhMlqEsc5RlUecKqJYNVAGX1TJyJgJcjgBjGLPdPcCUuPTCs6jzX9cRjQPP6p1nU+K1Hz2O7/75/4NLZ95AmiY4evwwbjpxDJvbm6BI10lVlJZIc9m/jJHlOfI8h2GLKBb7rHTby5R5DuOeh4jkPDOyeQY2jCwrsMgKxHGKU7ecxsbmVJynQJ2V1DJcH5naXktTycqTbm4i2dzEzuk7cPSue0Ek4EudMZXbEiWs1o5bJwwACsttd/xPe3+i7v2KQu1oMmv3PWPQOYIka3yGmf8379RnAPyW+/5bWKYFvvaphNtqm1M54IxZX8Wrjc22ceiFuLcZiYhw6MgxbB3agbGMsrQoCwOTS3TzcuMWgeYss+wy4Q1EY62wCQtT6ThBeeUSLjz/Y7BbdjAGGQLEzjj36ot46LOfxJNf+zzKbFE7UOM4xmRjisTtmM0kAziZTJFMJgARskKis4kk2LSWgNbCliVMUUruPQd7q0gyGuVZhjzLMZ/JBmlgwtbmFm697Rbs7Gwj0gpaEdKJJMNnEHZ33UK8JJZ8dVtbmG7tYOf07Tj63vuAVpZS99RN7aKDLHhlBXe3DSnvtGKmfshbrmUQbA9/yO3DcmYMOvdLAP4zAE8Q0Q/dsf8JwO8C+AQR/SMArwD4+1LxtU8l7OvEK9+dZ7z6zeykkAVAHmJHy95idrMcOdlE4ue47b3vxXOPP47SSaPSgQkgLPO0VXYMrbYNDJAiJOm0hsLzS+/grSd/gOTIMUx2DiPZ2ESUJGJTeagiM4ONwaUzb+L1Hz2J86+/CGLG5uaWg4+dmx9RegAAIABJREFUA1VHINLIFgtcOHcWpihBbr1TlKQijdjWfRJFkVs7JP4cNrJ2CGWJIssQT1IoiqGUxnw+l0nCWpCSrKhaaWxvH8LJ0xZn33gTWZaBmaHjCNPNDcx2Zcl8mibYPHQIGztHsH3L7Th21/2onc7BsUxdJ5ZXVP0TGAtDtBwPXWADeRMrlhMNA6jcJF7z+6odk0b4m+jkVfxaxz3XNJVwb8dZ1KH8lfNPVKhlbrGKkaxT6iTI2mMg93jve+ABPPfYY7CWxf4xRvwqbpe2SuL5jFlFBliX4CRJU1BZOtvDIEpiWFNi7/WXce6FH4EZsiFvFGM63YKOE8eAjCvn38bswnkQMQ7tHEJZFgIhay12l6rWN2lsbO1g58hRnHvrTVTLl6uIba00VKow2diELQ3me1eA0kARISsKAMBkMhGkLi+gE9c/pUGe526XhiopizDIzuHDmO/NUJ4/jzzPRW2bpEiSBGmqMdnYwOaRY9g6dSuO3XV/I7WVSOgWdF134aq9swIx94yDdaI86iFT8S/JZMqVCgFvPyJq1d+jWV73YT8ym/ixc019WmBWuHwTDg5XDD9mrTouDtPKhqmLqiXMHXe9H8yMwkHA7NAntgbw9jRaznDciJ+r8roprUFu9odlF+EsUHKRZyjnM2TZAmX0DuIoWeY/YMbO9g5UtZKWUS+Ykz5QAKladSVSOHbyZuxevoxFNkdCQD6fe8ymQFqAgyvnz0OTv1SCYEqLLMugrYVlAQssWyTTqTx3xHXUdJpOcPzm0xK0eu48iEhUuDTF8eNHcPzWW5FuHsKx9woDrZAneKrJzi1wGDMIesdH6G9VT/2uKuCmli7sMZA0sAsSH2LU656JAEgaXKrULrXUEvzn4mZHVssW0EKBGtg/kUPvpEN3Dh8FmCXpurEwVhyH7LZnjKJENgBTJLuOlwaKFIwrt5JIYEkJxbxcTKagEOkIlnJYKCRRgiRJG6ExMuj1EslSCnHl2zGiWjJVz+CWdVtGXhRYzOeYz/ZQzDNEcYx0kjqbBwCoXpG7DGEhZFmGvCwQpynYWuTzBXQSQ8Ux2BYwRYHtQ4dcfypMNzZx04lTmO/NJIGL1phOJzhy8iS2j53EoTve6+yrZV9XAxjcVt/qKa5+Z/5fn9pqb6PsgXvdxU4LaVJXmSFmOvBMBFQPzGA2jjd8xIWWULKv47J/71KCLXNCLwc/iJBMN5BMJg7WtihNgSLLYTYM4moNEZWIowSsm4vEqugFcYoKE5JySVaspIoyZQ7ODaIoQpouV8JWCRRJiYMSehkJzjqSzbCcFKvQCyaAoIHSYOfIMeRZjsXeriSXLEuUeQRKxahna6GiCGWWOSRPwTJkFa/bDa8sSyyyDLosncNUVK7pdAKbCnrHljGZbOD4qVPY291FFGkcPXkcOzedBCUJtk82Uw43Bl6tOrUZyUmlAKMsywiALwFJ0fYjVZKn1tbabUKYSdZVEQ8MEwHycJVNspw9gFrBRYth3NExUQEEgLTG4RPHcfH111G4aIU8z5DNZhJuqo2ktSUFU2qXrAQg0rAMEAuT29Igm82QpIkMEbaYX7kkM/fWltgtOkJR5AAYkRKwAA4qX0oLVauzihlRnEiUQrWcu1zmup5ubWO2e6V+HuvCfsCAKQqB450KSkq2uNc6wizfq31HeVFAWQOeS/mxJixme4gnE+gocXafwdbODrYPH0Y8SXHkxAnodIKd07cvgYQACWK6ztt274V8lat9vHvg11muWsDTGBojfXw6AEy0fJD20mDpGAtABw1VUdOWx30HX9DZR8Ct77kL5195FUVZOsh3gYXW4NIiTlNgukS9ANQb87pCZalAniPPMyRpgrIsoRUwcaic2DqlW8wWyRofeBuM1U69ahQAxKgN9SrAE0QgJf1hjAEphenmDq68c15i5kyVU5xkQihL5Hku0LhLGm9sWQMJxloYtuI4zaSPTaQxn+1hY+cQGFY+DjyZbG1h5/gJJNMplNbYOn5y2W6MH4B9VKtXnubHJP0RUusqqqUPhdvhT7Rj6h+iA7EUwvcTtUW9/F7N9sPMjc72z/kM1exMwqk77pTIhcJgPl9I6E9ZoChyGFO6SOhStquE0/fdQrfMLT0wxjgbh5CkElaTTqeNjcUUEWIXFFo5ZFXlOCXVUGAUibpao4xsHTwr5621YMPQOgKRcsiigAZlLtvGLPvB3SOPi0WeIyvEKctAHaFRFAVmszkWWYaiEHWPIc9JWmOydQjxxibidILJ4SNu17rhgdl+r+17uga9r5px4PrqGuudHwIKQmNmP8x/ACTRqgEIhFU06nj+rpknxEy33HkXDDMWRYmyMMgWC6RJ4qIRShR5BlIRbCoGvykNtJYdI8qyQBxHKAFEUeLUMwJcuGbl7/G3VYkiWeJcaaW++V0Z5RUEXg06ZhnsFWLIDOSLBYw1SDY2sXtlF0pZkTxR7AJnjYT0kDABQ1KEWWYssgxpmrrZWySXMQZFabFVGGR5jjjPoeMUzArRZAN6sgGQRlGUOHriZvjenHZPdzFGiEKAwcq9y0Lq3xVWOsRA69DfGnWueo6Q+G7D3s1jYQpLoCWjHb7pOEBAYQwWeY75fIHULaEWwEzB8gxxmiI2ppZQlarEAGJnCxlmgC10beestjFJEm+RGwPWuqTzIoGqwYHaphDJJYzkFFZSMG5LmGQyxeahw3jn7BlhLpUhIlmuUNVbli4quyxBSiHb23XqJMFYCRsq3I521lrk8wzFpIRlcvmyUxhjMd+bYXbxLezcchrRdLN+YcaII7eYO0fs9jbidCKTRPv9OGG0zkBnN9u0J5tQEUPM1Mmoofs6mnjdMxGwykBtJqi0gr4k/0EbKECT6RbiJIG1JRaFBJ1mmQR9MiDh/aRQ5Lms22FGWZTQWtVrddipRgrKjX2GYYImrLzpOom8kS0v2RqH9KH2dTEqlc1AKQ0LhoJjBqWgNLtEkxpKRaBjEXZ3r2D34iXMywIRKUSRQpokIFKYLyTqIM9z2fkizzHZmCCKIhhrYAqD0hjEOgJbSQ8mzyrPXxQligvncOncW4gixo+++SUcv+s+7J4/h8vnzmJ++RLKbA4iIElTbB46jCM334rT934YG4cCW0V68HcfM42FnkPnhpg0hO7Jd0A2Beu+/wAxUQUILI/5++k0LBsS63NV4x4mUgqHjh3FxbNnUVhGUYptpKMIZVGiLA10CpSmxGK2B2axZdLJBKYopU0Mpz5FYKtgtSB3lqlhhPpqndbahdvIwjulNUq3lMM6yN0Y43xFVEdqy3HrJJZkaiWjcejYccwXGeYXZigZULnE0cXxcnlIlmWYzfZQlLJEQiIVIpjMLQHXMnBMUWIxmyF22Y2uXLqI3SvvIIo0LFuYV17B8489LhmRgHqT4unmBjQBxXyGd159AbvnzuDm99+PE++919v5u23ljHxPQSi8+69PXaheEFqvg2W723IAmIg9JmKsRuv2JF5foxZfSt1y1924cOasREMXJdI4rlUIW6lSbHH58iXEcYLpdHOZHL4o5cUY2V2BIsAahtIMZgKT2+KxZcARCYNZpw6pKgyFlm0TiJ/BCrBGyi1c4sXSWmguoeMYURxhurmFrcNHsXvpkqxaJWHOLMtr5s3zHJcvX4Gud6urBhSjtBYpZDAVeQYGI1vMYa3FbD5DupGgLAnMwnyVekUE6EgjmaSyZWccIYo0Iq2QRAqXX38R2e4V3PLAR6EjPS5iAd3+nXXuaR/rt718BlraXiG67tE5Zh81WTJPxUzLOLbqej+ocH8z3H0f+TmxJ0qDwkGsZWlgnK+lNCUWiwVKa+p5tL1MgiEMZ6yFhez1WpbVpmIEXyZV0rRKWUxwfh52Wx1WfQHI6tLcRTHYpd1SFMZB2bK4DqSQpBNMtg/J+bJAVsgSj/lc1kpV220yoU6hDMiWKGBJyFiWBlkumZBme3u4fOkiSKEOB9IuwoLZRUWzSHMdS0IXQGIbqzxxxMDs3Bm8/PC3UG3ZebU0JH2CksbBeP6l4kKozo2H6q97JgLCEPcKUhcM7WkDEM3y2scquvWuu+t1OaWxyCqnphIDfj6T5B/ard5ktiiKQhjL+WMq9KtGFl1rGhNCx/tpJ+63DoWzDORF4YJiHYO65d6LxRzz+RxlUSBbzJHNZ7CFSB1WhJKdH8gYANbF5MlyCYnOcAG3NYAhA6w0YhfmWYHFIkNRmtpPJiCLk46uraQ0jGWY0tb+qbrPjQEbA1OU2H37DM785EfBd9Bny4TOhWzdbkYSJpHJ2UkbJu97e+IOjxefDoA6Vxl9gj4tf/tO1WVkQq36uJRW4bKa5bTL29o5jHQ6hc0XLtE9UEI8+pYZUZIiihKwRe3tT5IUBKDIc8lrYCwUXCYdWm49z+ychagAkmrwyYK6uj0OiVNKGNCQSEFyKmVZGrGVrOx+DieZrly+CDBQ5jnyIgexQRTHyLMMzMapic01PnlRIssyTDZk8V211aZ1S8OZAXLxg1uHtjDZmNSTlnW2GalltIUlwiLLkCAFoUCmMk/aSvZVw4yzP34ax+64E1E6Cb6nISnQ9vf00jLyNHBtHd6A9sw2BLcDB0IS+R3UTFriq3IrPh+qT4RLbUgyP1JAgIKb77gdzIBlQmEM8rzEYr6AjhOkkymIlhEERSHBmtYtk5aFb7JTXQUyMMuSbEGxq7wEACBxe1XUQ6UKwTkvKz+QbO2iRDW0BsYuoxCy2Qx5vkCRLbDY28XelUuYz/ewmMuGZRXleY7CeDkSmKEjjbwosDfPYdyWkP6yD2NdYn8wdo7uYHNnUxzCylltSklbVSTR51GMKgzKuPx2ZSGQdzafoyxyQe3c/q1vP//syvvoe1f+79Dx8LXLCWwsdWk9IbruJRFz2DdUS5OKXzyq1gn5DFR1yqrk8e2s5aE77n4/Xnv2JzBskDt1TraZVHXSj0hrqESScoCXC/eqjzGmjuyuHKZWLzPlSMOEQQhwMXFusVgFZBh2NgucL8jZX2WJsixQLBbI5ntY7O2CIIxtjOSWK4ocsAJSaK2xqJLBx3o5aZBIwEUuy+BVrF3cmctq9P+192axlmzJed4Xa63M3OOZap7u3M0e2GRTpFukIAiyLJgyHyTLhg35weYDAetFsPwmEnqwBL/IhmQZGkCgxaaaolocLFHiTIpskSIFNjg0h+4m2fPt6Q5V91bVqTPtvXNafoi1MnOP59Stou6pRi3cc+uc3LlzXLEi4o+IP7ynP+oz3h6RphoYrj1U1CGYrNfunGtouWIzY7xXP7KqSJxjWmnblzRTCi2XJMwO7q+dpOvSbh4uQXQ59rHp+2cJ9i6OJ0ATLa4KnTYhKzRQYzYFZCuOs8SIuvu99J5vVGaf2iNGyQhVm3h8pfY93iPea/eDZvJWTXnD4srYdKuLPlLzeaz9B7EW42xAAiOQUTXsPHVVK9ReFJSzKbPJCWU+o5jNlDuhKCjznLLMKfIJRZEjUpMmll4/U5PQWHy4BrGas1dUFUUZiB1DmpAHxltDdna3sM42QdjSe0TaNB8R0dKIxIVkVQfWkPZ7KqzGMJ3lgY+hoqxyJXZMtTvh8rsRukvjpne30ZTzNMvV2YVudUB+07fPvSaClrgQgnIR5rTMnBChhWtdEw/W+T+LpBUaVBMR9q5cDb6/fqesWgjbuwRCXKcsS6aTCUYMWdZrULpu6494/ca5AAOrmSmidS4SPq8qLfcgwMQSJkgkA6nKUFNUK5tpmecU+ZTJ8RFVkeONCm5ZFFRlqaUX4XzWaRPiqqwajVmHxyjWaslHVVOV6nsmacKly3uMRgPVSLG0g3nPQY+tRYRpmgGa1SCiKF+SOmYnEy2ELFv6YemY29130/ZIbZmbFlHWs/tA8wBT9/sbv7oIKDTuwerxKDTCf0dEXhGR3w8/39X5zmOlEY58bnF1XHdT3gv1QmwlnDd8vvjwVj1MFazheAcxpoG1Yz5ZFJK6g4yVRRkme9H4ETEYutgqM0LW3ofs5CBMcbJHXyoyGYkJLa4q1T5VkVMWuabVzGZMT9SUq6LmyaeUZU5Vl60AWRsSXi1J6kKJRGBNCo9TkTgV0NGwz4sv3GQw6Ov91zUVPlCEtT8SGheLNYg1gffOhswJNenU73LUddUx+QxlVTKbTjFpr3l3ygQUX400guAbBG35fa3eBtGfXJwDq8Y6NLCL+m6Su0ehEQb4h977v9/dWR4zjTDBTIo+hscHso7WgdHf1WwQs6yh4kq3/CD90r76o7U7adajmE4pSu0rWtctKND1cypPY76JoJWwdYWpWqrh+HLi39HsjJqv0VDQdOyLMSPNe4O6LBWRK0rK2ZR8dkJ+ckSVz9qFolaTV+VEsKZrdkHibMjsJjAlQeqcBkadYWdnzGAw0Ilehs6BeKIVLQaMbX06sVpQ6JzDOMEap3x7Ypr3ZpzF+QTjlGjFWEtZqDa9eP0ZfXe+fQ9LghGEAm+IzEs6NTqJyR6a9JzO18/iU606pypBfT+nKb1HoRFeNx4rjXCcXNFhNcY2D6+pCEW72q0b6zTREsrXnFFRp60Le42JWNWR17pocs7ifs310YF9o+CaDsAhUItpygZaQWoVq4TKWE31KUNbS9UagmYzlMWMqswp8ylFiA3Fc+qE8lD7hqehrto6JGdtiDNF07Cml2U8e+MSz9y40hKYlBozUmAglBgIoRYpUZ7t0CGj1+81rV0k9CpqS/p1gch6oRt7YI2ta10MRxevBiDojD4rUWN1tFTz77LGWGf6rT1fowDj3DCnXtuj0AgD/A0R+YSI/KC0XSEeK43w0dFRh3hDTRsbBMZIiJabVhiicM05hWuEZ5NtLCJs7ew0Dn4EDqKZNvdyGnte/RboWpttO5OyrIIZJ01QsjGPopx5bZNSVSVa6lAHwCAHX1NXJcVsQj6dkJ8cU+Yz9eR8p74o2vGehidCEMR7bdpca/m6tZa9vW2uXt7jws62splWqsVmZUFVa7BV66Yk0HKlmICsWedwAZGz1iKeoPloePFEhCzTPrBZvwdi9Rzec/Xd7yeGCh4WEYuzXTZkHq+2Plh7nsZVYH5x3XQceDQa4e8HXgTeD7wG/IO466pr3rB9eWOHRng8Hs2ZWlFzC0YRpuD/dAWtc5yNv+u+3TKu+ZXn0tXrYSLWc+ZW1y9ohCfkr4lpAWwRwddaaRN7+EQNVtN1Wk1T5h0fS8wK135HkT7fq280OSE/PmZ2ckJRFM39+7qb7aAAhSKL8bo1YGqMYTwec+HCHv1e1sS6ZgFcyMsCvAZ1G7PSBLMuPKuqqpt4UbexMvimd2skzLfWKhLnTUPFtXPtGbau3Oy8h9VjXYzoLGPd+998jrNpxO44EzonK2iEvfe3O5//M+Bnwp+PmUZYVzl1WhOlpDIxzrGM/8cRnfjFbeu00qptV27cpOow6uRlSdopUdfrsq0ZFUZZFE35+KIWpa6oSqPtKr3X2JBoXCva+xKFKBTeWSNYA4WvqIsZdVlQ5lOqSrMlrHMqiF5Zi4woN533KkTWCFLV1F5Ik4x+fxjy8OapkYvASUdA76wNDEtWfc2Ypa3PlUY7J0nS8O9ZYxuh7nbv8943vYp6Wztce/e3qIByNkF5GOE5baw8fkQLz7h/d7xlGuHg48TxV4FPhd8fO41wRHziiigBnYu/d9Vt99+WSKQ18YyZC3WGz83ScQCu3nyGJugYtrXIXNXwdBOuC9q2mL6qKIu82b85F9JoLhFRXy74SZqvh/rGtcfg8VolhxGwii5QhbhQXVbYUCyovAt6DoOg7Nceo7Q+mOAPGWPJ85I8Lxr/Lg8k+VGLJUkSrllbScZwQZI4RPS6ksRhrVHEz1lc0DoxFmatJUtTsiTBGUNiLdYYkv6Am9/8AaUG2zDmrYXT99s0Fo/RBaUWTbizaq/ueBQa4f9JRN6P2kJfAv56OPFjpxFuZkdYzQWzNPG7ptYqwRLRFTpm1K16Nd2HLSL0BmpKxhyyOvIQhDyyqtJeqCpoIbMgrNINFF5VlLToXfSRmvuJLVpEMOLDsT3S5Y2gVmHxNdSVphWVlRYDZhqbiTVFQngj1AT4AvGestJYV1lXwTyNyFabjR2TSr339Ho9irLQQsJGQ8ZjexJnSZ3Tql9jtOlXEKCogSwE9lg9RzmtML2qBVo672uT9ln3b3xPb01zRbbT1QSSD6P5HoVG+Oc2fOex0QiLxJ957dMNpuqkbEsjugvPnCCFG+miYpvGcDxuygS89w0jThSYJt7jW/CgosIkhrwsyIyhMDmpzQJNlm3Qs+hPxO9FLVlbC4F9FcATc/PB+0o1UZlTh4zrtMmACLGloJBs0Lc+UCLnRUlR1XP349G1zTmFhrMsJc9z+oljOB5ineXgwSHUkKSueWK9XqpClHbMOGtJEtsEXMUrkjidTChmORhhMBxTnxzzuY/9Gi99+59T7vA1QrBubBKodeDE2uPL5uOcdTwBGQsqII3AEInlu+bbokDEbSE2E+DnaH9HX6T7LVlxvN5w0FDiVgHubR5v4DUoyhLrXKAmFsq6wtZqzlSBL05RryrEYmxzXbXXtKE4vO+kLYkB65FSK1JrXyO+RuoyZCzUDVWwh0bo6qom0YIfZnkR6oxqZmUJAiYUwimo0fqM1qogDYc9DbmJPrdBv8fJyQkGT5okWKNmnZpxrlkMnHO44B/WVcHhgwOODo7IZzn9QY/R1hjnNKXJzyZ8+fd+ixc+8Gfn3trZnf/5bZvQvdXfgy5b0io/eX6b2XhtT4AQMe8LRW3TPABQE6mzr6gANXZ1DITSTftZNgUXzkqS9EmSlKLIlfwwsOZon58aZ5NQU+MxXuMy4j1FVSJWZ6KiVxWVNxhf4ytIjKGqCgxO6bC65/dgrKUq645WDaZUaC5swktOMyVGmePjqz1lVZCfzMjzktpYvDFYlzIrZlTTadAe4d6tPrOyLOn3+6SpI+tl5NNJaFymlMhGwFnBBQ2UJtpcuaVBNuSznPtv3uPeG/dALGmWMBoPQ5MwoSxyzWZIEmYP7jE7OSYbDNe+9z85MEFWGHCrzxmBzk3Jc0+EEAEN1Kq/08neVk+gFY64+7w/pHuF/YzphBbWfwdjSHs9ZtMJlbFaQtA5j/pLnbKBqsJGuNm3pQSY+YRTNaUUGo8LRFwMlL0nvGQfjhWITIwIiG9qjKx1RGfTe82WpqrZ3z9ia2ub0ShpYz21LibHJ4fkeU6vlzVC6r2S7jtrcAFN29oa89qrr2OswuFauqA+U5olGBvQOg9HB0e8eecuJ0cnCMJgNGawNaQXukb0spS8yEMMqsYJuCzh+O5t0t5zcwT4Z9FGZxGu9fucHcI+qwyfeyHSudySGapAdE2xUOi25CO1WQ0N2EBMEfEBBj7NNxL2Ll7kcH8fUJTO120aUmwwFuNARVEgAY2KGdh5WeIgmD2amIpp4zZlWSocHgTDWNvEYHwNRhzYUPimd4tNHL1eijhHXYa8vaqEytPLMoZbKvx17ZHaY8QiAQwb9EYcnRyEZ6ZCJATgo66wRqmKi7zgwoU9ZrOZmm9JK0QAJ8cTDvZPuHf3PlVZk/Uytna2cYlT0y1x2nrS2gDh63szxiiRZZIwOzpotHh3dJG5VWGJ7j5vVaBaY3/zfhsppMI490IEKiBxJUdirnX7WTTNCPvoi5Z5wWqgK5oWK/r9eWh8/rzCtWee48tf+DyIruhlqUQkVVFQheyJmIgaNV4MzFahhDsKclWVoVs3zbXqidrzeSEkaUJZxKpRi4Q0HmM0E3ww7FPUihSausYZoShr+oMhtZ9q7KpWeNwazZBIQqc9MZ5ZPsE5C1Jr8+Tg62jbSAVCXJoon3hYjIqi5MGDI+7fO+D4eEKa9egP+oy2BmT9jOF4qGT9vV4TaBVjqMoSEwTQZSk2/F7MpmcShocFH7rf2egjyfJ55/72ZnnbinHuhSgojuA7dBp30an67AjLnOYRo3EOWlQukoGsY5lZ9JGee8c7+c1f/WhDZxUfvMLaGuk31uLCsWMGgbW2oaGKQqblDiUmAA8igrHRICWYcYrcWYG6KtRcE8EE/mwTIPWsn0GuUHdVRuDf4xLXMAWphgwFc1mKMU7hbVOTF1OsM42/Q9BMaZbhkqSJe5VlyeHhMQcPjrl79wHWOnqDPqOtLbJ+Rr/fY9Dv0xsNNTYUul3YADqo6YouAM7hEc3LKypqtx6q7pq+8/PhbODDog+09PmKY3UF+mEE99wLkdrsIdmz6wX5CHcvx3fimEsDktaE65oL64EFHc+9412AxlGKstRS8aIgcU6DizZ2WLANhB0DrIoJeCprqa2hLI1O8lBvlCQJVVWFrhCaVqNa1Ieb1NQmqljpqtfa7/fAJapxak9VlFQh0yHydBsRrS4FrEtxmV5rUVS43Db4pTZO1qpcE1q6lFXJ8eGM12/f5e6b9zFi2N3bYXtvF2ctLnVkaUoSklCTXkqWpRCyS1ySzE1A5xRA0eC3trGsPAxG23MWRPffhxlvDYBY/Z3565Cl7avGuRciQoQ/ao/Gi1mIBXWJHOO2eRSus+LTEid2t64Spu0LV1SD1B7nhOPJFBtMI5doNnPMPYuTVtE1zcA2omQltbN4qxWqxliSMNGi5sSI+mpBiLzXBmLGaNa1r8pAAFnTH4+CoKZI7SkSS1kYCmoOH+wzGG5R5SVJmmnZiLVNO5VqNmNyfESWJmQBLIiV9JNJzu039rlz74CDgxNSZxgNe4zGA4ajPrsXL3J8cKC+jrPqK2XKc2eMkCRZQ+Qfn2eT/hRAGGstsUfeaPdC5z28VS1z2n6rtdCq7yz6YsvHXn3OJ0CINNkRaOZ7V5iiiVfXdZM5DMwJVfvjG9OQ+BPGOp/IuYTR9jaH+/uqNQLftkb+9UeMNLlnaZoqKWHgl9PanpK6dqFtJU2uWnxZxisNsECbDkSE9lsTVjndLP3BkNlsRh3guFG9AAAgAElEQVTyhMpAaWWt5fjggP5oxGh3RF0JxjpskmBswuG9+xy9eZvECOlghLGK+h0fTbhz9wH39g/xImRZwmjcJ3OOfi8lzRLNo/MVFy9d4O7dN8l6I1xilS+hMU0byLTdFq7fBpNXrQeDE8PwwqXm3rrjrJplk2Ct+yz61X7FZ2cBMlaNJ0KIrHWaVBYQJk1l0RuryrJpsJVlWfPSFoWo+WaneFKDrzR+E3QgcR8zEQx/8S//9/zkv/xBJXef5WSh1WI05SJkXde1svIYi3U+dlwB1CT1IeggEtJh6piYSlPIJh5wprk2IXJsq6nYH4418BoaNouH2WSqIAG6qLz+1Ve4dP0qW9u7KiT3D5geHFGcnDAcjnC9jMrX3L5zl9tv3OV4miNGgYCsn5ImIR/OGZJEMxG0zCPHuiEXL11kcjLROJFEBFUdK+cMRtrrbdBVE0pYPFinnc6T3mDpXT9MJsK6sVGAIFjLnQrdBRM/arCz+kZPgBCpiaOt3wllyBZnkzZPy1mc04ZasXZl0dfpBi7pCEpXgLqjqw2++U//WX76X32YLE3xVc1kOmU6y5XlB5BeD4xQVm3TrF6/H5JQZyE4qkmrdVlQhk53ev00sDvUoaCtLUEwojxuer0w2Nolz6cQyfLrGmuTsBgYksRRebj9ym3evH2X0XCI+Bqqiv5oiBjDvTfvcefOXaZ5gTfad9U6h0scaeqwVrWKDXQVvqqVB0KgKrVzeD7LwXvqumKQ9rFWey5pHp1oXq3ExUDjV1Wtmsh6w/YzL85ZAqeZUhtnyBoha2RmlZYKGmnt+XwbQzttnHshqmtPnk8xRuMO1qkDb03SOPOROFD3r5vsbVg20+LENCZQ3q4BFeIqVNc1w61dxts7HD/YJ3EaqT85OQavTEDWWkigqgy+0r5ARiy9vrapL/MZbZMUnXgQ+Ohqj9juqlfPxTAizuY9gTRSe7jWpZZ253nRABrWOZwHG9KTqqpk/8EDjNe0HjPVity7b+6DGHr9PuIcXgy1xERXizHqtThrcMYol7YLYISvKUI/o6osSV3SFkka9XVMFP6qVq6HulJQyBiMGLZvPc/wwpW5571qMm9C507TTovAwNJY8KkfZZx7IdKsgJpemmqNfsgXa9JWRLOz6bDrLD7YuKJorEgDTb72jUqP54nfbSHy9sV+03/x7Xzso/8eYw1FWSKF0Mu0kjVyLuR5jjXaESFJE6w1+Fr5GmIxhe8AHQJNAy4tZWgbKTdBYU2Ppqpq0qyvyFtVYlyCKUvS3gCbHClQIUrp66zy5EUfy3tdjMRBnpfYNGE4HGNCIzIRQ03N0ckRUGOMg5DQKoQSDLTGKU0zJRixATqPPk9EQr3H1xV5PqPIZ2qmhvsxiWP32rNcfO4blt5zVyjO6pO8NVQuDFldy7R4/K8LTSRCYzaBrsrGtDEj3Uca3d0Vgi7yZgImFC26qMp1cq+ubWmPAy+86338p1/6RZ2EVtScc1N6WdaABGmSUJQFeT6lX/XxXtEvJPo8NPGpqOVABUkzCloT0nvNWfNeqBBtkmVUUxnjcK6mMgWJ09KL/nAAkwlVWZJUDlBilxgMBg2mejG4fp8s6yOB785YB+Lx4pnlJyTO4qtYwxQ0kXUYhCLP6fX7VHUdSsMBrxW4vhZtehZK0n1da/m6MfT6A/q7F3jm/d+uwasVptZZEbh17+os2870fb8MMDzRwEK09YE5+rE5DSLKoWb8smZpUDnapNWYGRDHKpOvu817z8c++kuUVUW/l2kGgGhbxv3DA8qqZGs81qKzRMk46rpUxM1qQLM2BlCqKkXgNKsiClLkqpNGwMOCEDSVDfC59zW1LanLoEGqiiTL6HvNiPC1x5gCV1bK1gMUIfMhSVMoKmqviaTGqFnsQhtKj6csJiTOUYv6aNapnxXLvW0AU5IkaZJri1z9TB8Aj8FoSNrrY12KtS5kKSRcePG9mCRdCxUvv/vNsZzTPluMQTX7nHaMhzgfPAFCJDGtJ+ScaSS9hbEV+vVrq1zb39usBcIR9NGcTjPx8qf/kM/90acoi4rpdMb21pAksU1FaJ7nHB8fU5UF3vcRD4lzWDdhMByGdB1B0Py0Mi/I61lTOh7vI5ZnO6PVQLWvqOsCY5OmgrUsAv9BCOZWZUWaZIFI30FPswOkLLGxg7gtNHG014NJTj4rgmD0FQJ3iTYvy4O/KL5BBK1tC+2SLA15fqrdijwH8fR6A9K0Fx6lkIdOfDYpSdI+rt9n97l30tuKXDan+z6Ln3f/Xbdtfiz7OWcRSjWj56mE289Xf//cCxG0whBAXWJaTSRV13T8DutM3D9qMAlZAIbm+3gJXPLrA63ee/LphH/74Q8qrzVKcDg5mYbYidWmwCGbwVolh0+sVVrfNCWfucBB4AKJfIUXSJMklEjUWO+xDeRaa3xHBAK/g3VJAzr4UFUbX258wcZoQZy2u5yG/D3VQMZmuEAYYpOEo4MJaZZpi8kkRYzl8MED8uMD+llKL02xVov1XKKMPs6FIKrVv4/293nt81/i4s2rZGmfrD/ApRneV4EVqcR7sEnC1Xd/C+lgvPL5nrZt3cR/WBRvPtj6cGBE16xfNZ4QIYpZCiEfTmxjxi1qpm72gn6lzVxoEw4VKVoMxsb9u+OX/s2PcPeNOwH1M5RFgfS0m3hR5NRVTV5VgYMt8FXXyjk9nZwEM00Ybm3jEuWe1pYoNf1+PywE+lqdtcrxLR4f+sK6JJ0rrWgQPa9Z5C5JqKoSZx3eVeR5jRFD4hyIkBdFQx7f7/epEbZ399i/f6DX7z1VPmN6/02Ggx5pv6exHieajeASXJKQ9vr0R9sMRtvYJKM3uEsxLTi4/Tq9Xo/ecEzaH2KTHsY6vOj9XHzx3XMCdBYErrvfqpy6Vcd5mG2rPm9jSMvCeRp6d6oQiUgP+DUgC/v/a+/9/yEie8CPAc+hHAv/o/f+fvjO9wHfA1TA/+a9/8Ww/VtpORZ+Dvib/gzLiaJxJuS+dSZ/B9o+LQ+u2a4BjI37xvHKFz/P733s16nKQovSAqnI4eExSZrQ62nV53SWk6Xa6j7rDcjSTH0KAagpi4Lp8TFJL2tJQALVcJpK00GOECuqK09V5RibdHqbQu2rptmyr2O/ouBbeeWM896r2eUhweOKApemOsnTHmWh/Yr6gxH7d++x/8YdMmPYvbirEHovw2UZdVWS9FTDpFkfG1qnWJeQ9Xpk2Q2S3oD7W1vsv/YK2WBA2u/TT8cYp0K/dePZM0HZsBlgeFiYu5GHdVpMLbbVZiKnmYnL4yyaaAb8Be/9kSh11n8SkZ8H/jvgo977vyci3wt8L/C35HHTCBMTFxf8nsiSI/N5c5t+YFnY1glTVVb89L/8EGWeK6GiCHVVMRr1mU48+/sHDeXuZDLlweExWZLQK0t8mjLLC6x1So0lmsnggx+RpillPsP7mrKYkbhg7lWlMpQGXyYSw4O+1GIyoyrVxJucnFCUGvD0QJL1SHt9gA5C5nF5QdofkPVH9Hp98uksaHPDcLzFtWdukU8m2sGurnFpRjYccHJ0pJ3Sgykas7IjEYtLE8Z2h8HofWxduMTdr77MUaLl7zbZQpxj59ozc8+0axadFT4+62gFYvX2Tb/PbfPzx3gsEHfQFEfhzyT8eJQu+M+H7T8E/Crwt+jQCAMvi0ikEf4SgUYYQEQijfBGIZImrWReCwEBsG6FKJo864QqnHfJdJs7XmgH//Ff+2XeeP3V0CpF2VbrWqtLL+xucXB0wuR4RtWrGAz65LMZx5Opst8gDIdDjo6P1PwzFiOW/mhE2uupxuoPqPIp1DXF5BiT9XHOBkrgmsHW1lzeXFUWVHkeGFC1YVba75H2+6ShF6sNgdi0Vt/KA+7kGOcykqyPcymmb7X5FhIah2VKqFhV5KE9pYjQGwxa4KNB5WIOox7fuYQkSUlvPMvWxcu88ZXPcXJwj6TXY/faM3Na9Cxm2OkaZrNZt86nWRz1Gm2jc6iFnB4rxC3ajObjwEvAP/Xe/6aIXPHKJYf3/jURuRx2v4FqmjgiXXDBQ9AIoxqLCxcuYIxttM6c5uiAB+F7K4VG6YaXoe/udwAi7r3/5hv82s/+W3xV4r3GWnzt6fV71EVJOhxw+cIO+wdHeITJ8USL5IqCIpDAz2YKPmgGdhBsJJhF6s8kaUY1m1ADZZljTKYmXq8fVkW95qosmR4dMZucBGIUy3B7W4ni0ySuMuEpRJYRGxYdiw2T3TqnNFuhIteFruh1VWmwOmiaWMEqEkzN4JT6WgOsgpqjBGvAOoN1Q66/4z08uHeb/t4l9m4+rxWtzdxb7/t0TbxVhJuL/677vfvv4ufzn62GFrwPAXg83RjIaVrzTEIUTLH3i8gO8G9FZBPt1aqlYN0SsfLKvPcfBD4I8MILL/iuBorEGDXzQjDHNCpaomZCkqb+ZzqBplbztM81CpLnZz7yA8xOjhA0y8E5S01FljhKtOnWuJ9x6cI29/YPEJcwOTlhPB4xnU7pZynZ1pDUJYh4ZtMTzUczWvSWDoYYhLosqGbTwN/tKYuK8c5u03VbvKcqK20jOTlWLdPLsM5SVpWaWM5RlhUucdReA6zOOfWnRHBJ1tQu2cQpKCExPuWbPD+A6WQChoDERdL5IFjWNlaBavxKy7q9B2tC79qa3WffwfV3fXOMTTSxhCgY85TDzfveKBDrhG/9/qtmVffLzRRYo91qIjvhYzHnFi58X0R+FfVlbovItaCFrgF3wm6PlUa4xQOcBu5CfxsrLdlI186OSNycVoqwdkeO4zbt1dNyv/3hb/8GX/7MH4LXsmpvILFKmu+9Z3t7i+PjY7zvMx5rC8Y37x2QDgecHB2zvT1mNp2SOsOglzHo90lcgi8rJoeH+KqiLkqGoxEuSSldSjk9ARHSVIkcrTGhXWTBbDqlyKf0huNQIZtjk4RaREuurcEmjqIoG8i/9h4XFpQ01eRU57SENklUOGJX7+6zy3q9Bso3ITisSk6Ds1WkaA2LUVWXGB84wPGMrz/DhWeeD1ZDXNlZmV6zOg6z/Pmq/TeNVf7MSm3km/8tnUfnRfv5acJ5FhrhS0EDISJ94C8Cn0bpgr877PbdtJTAj5lGWJpIeatpTJP42f1sZTc9kYZosbst1in4kIrjvef48AG/+OP/QjMEAhoXqXGVf0An13g0JJ/lCIatrTGXL+4g3rO1NeLg4EjJNwSmJyfkM+0dlGU90iyjKgomRwdMj09U2PpDRnuX6A3HTYGfUnIFASoKXNbDphlJ1iPJ+nhUo2EMLknVVOs+hyAYkX7ZWkMsxdD9TMhYaBeaOFG6rTKBJjNbjAQNpwHjqtKesbN8ireGG+/7Vi4++6Ka3t2cRN/6KmfRMIvmdhfqfitjrV+zRkjbZ/F40blrwA8Fv8gAP+69/xkR+Rjw4yLyPcBXgP8hXMxjpxGOE6M13ST0uZkPrEYBa0YMD3V8qFXHjuSKP/uRH+Dk8IGacaJ1L6lJKPLAlyZCVZWMxyOODg/JZzP6g4yLly5S1zXTacHWeMjhwTH91DEaDXFOIe3pbKKmU69PWeScHD1gKGOlqOr3cclIWU3Lknw2Yzo5UeFI2haVNk0wicanNEVIIe6sN2g6gotI0+5EEBWgqiRU8IFJ5p5ZhPvjsLbbhEzPW1VV06bSo+QnZVlS+prdG89y+aV3YZOsfewLMHY0H1eZYIvjLP7OOoChK6zr/Kq5a/PL3lF77NUpQ6vGWdC5T6A9iRa33wX+qzXfeWw0wkDbHKproi2UgwMxn0G7GMwJTHe/1iD2PhDGA5/9xO/wuU/+nmYtx64IoQygyAm/a8JlVVXs7u1xuL/PaGvEME25eu0Kr792h7IocWPHwZH6QYlLGI7HJEnKycEDjoqc0dYOXjwnx4dN6YYxaqKWdU2RzwKnQugKaA3OJQ0hpIt8CqG/kTEyBz/rcwFfK5NPWYZMiDqmAbk5IUKkWV8EmpaYoB/XIXhsQ6JuWZZgDDfe9S1sX705tzgtTWC15da+201Cs/j52cbZzUS91vU656znPvcZCyowmkQq0Twz85WrEf6OWQntd5e1k/ftO/eBcPHk6JBf+NEfIrYxSZzTis1QGzMYDKjrOnAoBOYea9ja2WH/7n16/R6D0YjnXnqBz//RpxkMB9RlymyWc+fObRC4ePEKo+09To4eUJUFo/EWadKjrksmRweUsynOafQg6WWY0lDkBSaB1PVCqYdFvMEkyqBKXTYZ5BLBEu+D1vH6nKyauVWpHSqE2HpFmkkktNpDfwKfnkRqMTVvQnU7kqY8+y1/hmy0nMqjz3hh0i3+eUbgYNO29fCzIhmrjt3VQnO+kaw6X2zxfPp4qE55b8toVtZom8f+RF3wwDQrSueRtYLFwjEiOuVVq/zcv/oQJ4cPtADNKoWUsteowCWBV0EddeVUK/KC4daYrNfjwf4DQOgPRrzjPe9mdnzEeHuLnb09bJJw7803eP2VLzOZHtEfjQFPMZtixDAYbGGMZXK4z+G9OxS5ll0nWYaxRpsc5zOtOercg3Oprgi+oq6K4K9JgOMDTFwrZVeSJsGcqvHUaMeJIGh1bEUZNTC63av5F7UZAmVVkG7t8MKf/vNkw9UCtNl3Wf/ZIkK3+Nm635eFZZ1wrT5e2LB87IfIXDj/migIgzGuKVCThUwFE1dM7ztC1aYDNcdqYFaPrwUx8Nk/+Dif/eTvYqHRQiIsVMd6+v1eAw/H1J08z7lw5TJvvP4aD+7dwxphvLXNMy++yJ2vfZXLN24w3t6iKmaU0wl3X/saW7sXGI93mE1PEGBQbWGtoz/eoSymIEJZFqGUwJHPpuRVhTFWSxnC9VtrSbJ+0+irKktlO7UWCbmFdV3jqwqbBEYfr9W0tQkwszUB5tfYiPh5VDOia3UgZNm59QJXv+F9ATxYPVZqgDX7bDrGWfygdedcROgWfbRHOfeqce6FCDpBVGNC4HReCxG00KIvtOgzxWwG5TCA48MDfuHHPgx1jU20UZVzLTo1Go04Pj5GUG00nUzUVwrnyacz/HDItVvPcPurX6E/6OOs4/K1G1BWPLh9m+svPE+6vQ2+YnZ8xOz4AHzFYDCkMELuEsbbF+gNB9T1iGJ6QlXmTI/LprK0mE4RRJsNh7oigCRJ8dmAophSFjPEC9ZmDYqIeIo8R6wK/qQoIGghYywGCa1iVCuL2KaBmRqzAJ5a4Mq7vondWy9u1DTrJl23gji+l8XJeppv9DAjUqwtXtcZv9zuLx7/9UEjHDIOYCF5NGRzB7qZGD6F1ZkL8cG0KFHNz37kB5geH2JFAruNDWn/roGae70eeZ6TJAn9gWYSSICB8Z7jwyMuDoZcuXmLu6+9QpZqzc31557D+Ir7r7zCtReeZzDaZjgaUxU5s5MjLbara3xdqH8jkPb7GGfJjx4wnRzjqxrvhbquyCfHGoQdjgMllwIoaa9PWWkOXVWVuNqFMvG2m3hdVhiXhGJAhai9xIrX7gSr8V5BBE9YtHoZz733z3RqgR5ydGzsroB0g68PM8kXBXDpdKf4Qqv2hcaQ0cUk/t74g5sF6dwLUUSbGiAhtPNoCOnn4goRnl3dNt0YwyRok8998uN8/hMfB7QMOwpRrNiM5QexPqmqKrIkQQTSLNX0mbDP8cEDLly9SjE94cG9e2RJSl3k3HzHO7n98ue4/+orpM89z3C0hRmOGO/sUeUzynxKXkxJ8iOysk/a65NmfeqqxB8dkE+naPVqjRdDWcwoZwlWemFuhuTTJAspPQVlWWDxOJcG5E6rX12ijKU+wNPW0+S2+cDOU1V14GTwVL5ifOkm1979zdhU4euzAABzggIKI/uWO2LT984y3qp2WvX9JYFsrrsDToSP6rqzGiyM8w8sEPyTJhdsRYpP4wetqQvyyiWAN5RlxcnhAb/wkQ+pkATzzHSOa4MwRT7twWBAVZb4uiYNZO9ZL2sm6Ww65fjBPpeu38AI5LMJxWyCr0qeedc3Mt7d494rX2U2PcEFvurBeIvRzgXSXo8in1FMjynyEJgdDEl6A2bTE2azCWUxU+LHSkGEsiwCIaT+WGtJe30Vko6DHoPPZWjwlQWet7pT2Adq/pRlGYrpKmrg0jvfx433fVsQIJkXjDVjWTDmAYOI+q3TFt1tDyuwm65nrZm55lqa7Y1s1Rvv+9xrooiySScrIebDaY+iFfGiOQGKeHZYuWvPL/zIh5kcHyqaFYg42qwHSBI16yaTSTMR+oOBrtK1J0lVW9lESSWruub44IDhaMjNF9/BG195mdF4TDE9IUkzbrzjPdx/7ascvvk6aZIw3NpRXgPrQ4pPST49wVil4xVrSUdbiBGmJ0ehO7inrFJM6UIhosdHAknvcS4j63lm00nod1Qh4hotUATCENBgtW4vm1Jv7/U+svEW197zp+jvXGyef3gLLK7Eq5z97r9zkTqR5lmuWtHXHWuVKXZWLRbPuek7pwlh1w1YF1A695pIhCZboAUYloOv7f6Lv4cVDn0Qn/2D3+ILn/o4IuCMYF3gV3NWmTlDe5HYQtE6Q+0r7SKXJFpZGgK0kWcahKqsONq/D3XNjRde4uTwAUnWp8hnVGXOpWde5Orz38DxwX0mhweK8qUpvcFIGyy7hJOjB0wnR1RVQZKlDHcvBNqtKvB9zzTwWZVq8gWif+1YoT6STZXjOy/yht8OUBL5stRWJ4EcMtIZV5WWqV94/p0894H/siNAYZJ5DwsO9iYgoPl3zTud107rJ/HKuM6KYyxe0/zf6wX2NGBj8TjrysrPvRCBKBQbNJCRGMuQBspdBSTE7/roPXjP/r07/MpPfAQRTS51VkhitoCL+WStqRjz5pyzFMWM0dY4lGfXrbnUYVCdTWecHD7A2oSL129xeO8uWX+o3HRVyWjnIje/4ZtxScr+ndfIZ9NmUcj6Q4yxHD+4qzEkaxlfvEp/ayt0i0C1XllQ+cADXhaNoCizj2BtyGfwnqLMKcq86eIXiVCidvJe2YT6uxd57tv/Apff+U0Yl3SeXse37oyHMbmWx2btddbzxLGYTrT6OpY5Ek679rMIeRxPgDnXDtOpkUGW0bdlnyiuIODrip/74X9GMZvgjDSBySRRGuKYc6bnafm8I6RcliV5PmP3wh4P7u+HoK/BWUdRhqbGxlKXJZPjA8bbe+xduc7BG7e5cP0WZZFTJSlpr8fFm88xPT5k/87XyKdagm2doz/aYjY55uRwH+sS0qzP7o0XqKrPkU9OEOMoygJTJUCOSE8RtzrwvoUX7hL1jcoipyzKUGoBvtPA13tPMhxz+Z3vZXz1VlMA2Ey2xg4KD19aYOAs5pWIBHTwdLNv1TFWjVX7dAk7113XokaLGjxeW8w5XIf8tf7gas/o/AuRBAafWN4QIe1TzTgdcaX69O9+jJc//QnSxGIIppw1WkpgzRwqBzTNuZxzgQshpapKal8z3t7iYP+BTsRAbi9GSHs90l4PEWF6csx4e5edS1c4uHuH3SvX8b6mLkvIUgbbO6T9Afdf+zJ3X32Z0e7lhjHn5Gif48P7jHYu0R+MuHDrJe698jLTo0NqXyvZYppiTIkPuXz4uFjEVJ7Qwa8o2vaYUiNi6e3ssvfMS2xdu4lx6dIq3aa8N3+0H51qPrUT2ixlbitcvEkI2/27U2A1Wcm6cRYAYlVd07pjnBasPfdCpCaFmjyI4Gkzupt9ZDlztzsmR0f84o9+GGfAWbBG6W+jCddweovW23jfVlhGqDuaeEU+YzgY0ev3A/DgtSTcSCiSS0izHr6Gk+Mjxts77Fy6yuRwn9H2niJ+pRbUJVnGhVsvghgevPEKe9ee0fw5MUxOjnBpn+F4hzTrs3P1Jm98+fMKQBgXMhMKLFDk6v9IeFreV5QRaatrKl+TpD22rtxk77l30t/e0wz4VQ/LxzjJvDkEmzXK4vbFfZYn4XphasGAxQncghunCdVp1xqP2xWmTYILrPWJzr0QASGwegqM3ezbmncx5vFLP/4vmE6OSZ0lsZGQUEEEhbLVvGuqZuu6eYGL9FvGGKazKcPhkMmJwtJJoj2DJGhN/a5S5ebTKcOtMbuXrjI5OqTKMrJeTxt2oebI3o1n8XXF/duvsn3pKv3BiOlsSjGbUGQ9sl4P/JDxpcu88eUvUhZTyjyl6TQROBXKGOMJwgNCf+8S2zefZ+vydVx/2C44iw8uzg+/VBwQyqXfmiCt+hviItXdB1Zpv0UBWjzUaUK8arHtjuXMhu752u2bNNf5F6IoNDJPdbX4QJZBBV2hX//yF/nj3/sYxig07ToAQhSgCCBEjm9FBNtSayVE7DqaNZPpMYPhiMMH2lkcUGYfH190N1BbkwCj7R3KImc2OSHJsiZQbKzl0rMvUZUF+2++zu7lG2S9oYIDxQwX/LYk65NmPS2Gm53gfA9ntcmYIm0h+m8dW9efYe/Wi/R3LyJ2PtdtbmJ15tTKCdlBFhZRrFWju73uCOvqCd36JuuuYdP2eMyzAADrPl+XgtSF5DdZOfAECJHE/8lqQYFljSSixOr5bMq/+/A/AV+TJq6BspteplbBhJhsOt80edmUi6OutGMCvmZnd1ezBIzV1iYupa7qDvytfkpdVSRJQm8wZHp8yGw6oTdURM5XJVmvz6VnX+LoU7/F0YN7bF+8qomjIuSzWSjRTumNdzi8dwdT5IBQW83ULiutAdq79RIXX3xvIEwUWm6bdjT36Nu/Fz/v/LGycO0s2kYklN2Hj7odL3w8rl/UBOt/75p7fqW2Wh6L4MJZteZZ7i+OJwDiDnEiYzsTWRrnd1WSKSg/9n/86f+Pw3t3SazgnCizZ8xKiA3CIsTdtEfUYGsEG7qaryk/h04JtmE0GgXt1cLjdV2FJFCapVYVqqc3GFDXmjlR5DPwQcjSHjde+iamxwfMJieBzL4OpPE5dVkx2NojG4yZzaaUIaKKrUsAABluSURBVBZUVTX93Uu88B3/Ndff9x2kgy1ac2T+FYuHuqyUKyFuWwPS+AUBOk1LrETpFgzHuX1PSe7cqGH88udnQfg2XS/MN8uOH2/SQvAEaKIw8zqTOd6kNGXcXdOuDMw5L//RH/C7v/7LpM5oicOcsECyEBOKn0ELKkT+7O6kUu3UaSbmIcsysixTeuCq0LJso/zfzmlzsiTk3RHKKYZbW5wcHpHPpoDHGnX0s8GQS7de4o2vfAG5coNef4iv5lGu/ngPxFDmObY/5vq7v5Xtq88goQXNIiggC3MlLhjdcVZHffHfxRDD0n4xA9W3vsWiOb5u1V+frbDs1zzM9W/af95sX3/O7jgLUUlPRH5LRP5ARP5QRP5u2P53ROQVEfn98PNdne98n4h8XkQ+IyLf2dn+rSLyyfDZP5LTRHz5WvQB1LSQt2j+jxHbTPKjg31+7iMfaqDsJLRStIG1xtm228FcdoKdD956XzddvqNrEOtvRARnLalzTT5ckqQN1wHQaCOXOkzohleWZbhOYTQaYfDkk4lmI5SqbYZbe2xdvMqDN19jcnJIns8oSy3OK2YzqqrAJT164x1e/I7vZPv682sFaNXLfygBOmVxP5Mm8KebVZuOvTypl/dbJ+TrjnfavXSPe9r1PgqNMMA/9N7//e7O8thphJkTFgUZmk+IOXHR/i6Lgp/94R9gdnwQBEgb99oQXHVGKz0RGkKPrjaKAlIHYo9IuRsFqQ59hLyvSBNHmqWBeUebfM0mU4Sw2qKrVOyb1PYiKpFaoNZmzXmeU8ymAJo/J8LOpev4quTo3l2y0RjnEuXaDik7Jkm48s73kw7Ga8yNzasnnO6D6O/xF2lMu+4zOm1Vj5ewLtZyVt9qfjIva45Vx1ildU4b60xEUZRh5XcehUZ43XisNMJhXyCWendJNaIpoU2z6qriE7/x63zhEx9va4SsDZwJ0uTCGaPwdtRE0eyKgtTCmRp1b2U2ThwfNExCr98PfNWqCdMspa4VREizVIPDPlSeigmJSIEwJLzgNE3J85x8OglccgkihvHF67gk48G9N7AuCX2KlA12uHeVnZsvdt9T960tvaKzON/rd5hPPl1ptq0Thg2a6q059fPaaeWEP8P1nTYWtdsmk+lMwIKIWBH5fZSg8Ze8978ZPvobIvIJEflBEYlVWzeAr3a+HumCb3BGGuGFszerumqhVWCC3uidV77Cf/iJH8aItoJ31pA4JSFsOOqMxVhDmiYB8TKNabfEbbfA0dDtZpcFPydJ0kYgvddODVmvj0sT+oMBIlDVtXbg9pWWNHTMm0b40pS6qiimsyZp1RpIh2N2L13X3qwuofaCGMeNb/wASy03N7/DpW2LE2W9ox50UCDwO4v26G5fZW6ddWy+rs3fW3Ud67Ytmm7dfeWU851JiLz3lff+/Shr6QdEaYS/H3gReD/wGvAP4jlXHWLD9qUhIv+riPyOiPzO/v59vG8ncxfQ8d5T1QUi2ozrJz/0j6mLKdZK8H9MCKTOI3KRazpJlEE0CQ2wWl9IbyOSFjYT0KuTnoQgbZIkyoft3Nw5VAtljVAq2NC+nKqqFm8ZgP5wpB0fDo8o84JiluPLCusShqNt+sMxWX/E9fd8GzbJ1vgMq7XQw2qi+WO3h4zl0utW+u65FmNQi+fpCvZZhOWsgrRuv1W+4Opn2Pn7DML/UBC3934f7f7wl7z3t4Nw1cA/Az4QdntkGmHv/Qe999/mvf+23d1dXQCN4EPGtLoTMclRTaxf+FcfZv/Oq032gXW2QeC6MDYoLF2WReh24DQNKGgiiFC1AoMRyvbe46xVJqBeL5SBJ0pQH0ozlgQ1ZD/UVWgEvOJFtAKl5+kNBjjnmE4m5LNZY9dr3ZDQ27nA1pUzKPANY51QrZrI3c9W/R7/PqvPs+o8pwnxqrFKABe/t8lcXHf+tcK0YbxlGmFR/u04/irwqfD7Y6UR9r7N1O2OOlZJInzyN36dP/iNj6pZFkq9k4VUnggxJ4lyeSdJqiabsU3rkBbypvGVggXZmJQuUdadpKe0wNY5rW+S9jxRkLz32ID61Z0X15h+dVuOXQeTz4owGA7Jsgxfax/XoiiYnEwoqpqrgSx+9Ys93elefr5nE5zTjrdyu6z+rGuOL39v07WcDYVbdV2nCeza+4L23a05x6PQCP+wiLw/nOdLwF8PF/RYaYR9eHDdcvBwZ1RU3H7lK/zCj35IqbWDCddl7UkDcUiTpZAkjQ8kCM4sp8TMcdPVoW7Il2AI5RMJ1iWhl2o0A9Xfynq94CelTbeHNMsC/0HgZZgL6LWreF2XgDQxqDTTkoaiKKjrmkvvfB8u8B0s+zirzaWzrPanbV86hm+8pLVDTaHTj93d3qYBKU9G/DuedNW9rIPM14ELp5pvS9cUIl0bhPVRaIT/5w3feWw0whGBM8YG87RdkSYnh/zEB/9fyvyEJFEgwVkTNFBrVrW5cfMxIe+ZC/7F0QRdqwjh1srBYG1A4hKcdRixGBfLsx1pmpBlfVyqqFydF23qUKBCrsoSU9dNPlsXVgcayqq6k1FgjGGwd5nd68+ueUqr25Ws+j3+3b3fdRMx/r00IWWzFmi3PxwYECnNdP/V5u86E2zT593PFgUwnnfTfvN3sjyeiIyFOKklpNP40GLkp/7597N/56saQDUGa2NXA2nSdqKvo3U/pgEBIlQeH5ZmPsdJo4JU5oWigRDy4lwDZ2sryHnilCRRrSeBw8BITGqd/4mxpi7auChIVShB94AXy7V3f0tAZ7oaTPdQo3Y9OrcJ8j2LNupOKhEJxX31yu93j9sFFlb5TPPXsJzxcJpwr7v+TQLUPe5po/udR9JE52HUdWwEp6tTUZb89kd/ni9+6rc72kerTG0nybQrRNHRj1oK5ldkEYsxalKIaM/WRkMZ7SKXZpl283baKyl+Px4faDrWGWMgcUTNGc+jRCudVXcuQ8I3+2g5hlDj2Xv2GzCdrgvdlb6ZdGfwhU7TTus+Wz5QTLqav4Z1xzqbRmmfwXyNz3qhP8s1nxoMXrjGJaHxfmOgFZ4AIRJoqKt8XWFF+MqnP8l//KkfwRjU9zEa6zGmrVKNJluWZaRZgjEuZIK3nHLdSSvSseF9QOU6JQQ2mHJJlpAmWTPxo0Zp40ytdtHOdFq63QqrNufygRwyju7EEdEixNp7XDJg62oX7Gz32TR5Y0VvvL51k+e0VXn9RD2lTmeFFjpdC0oD4ix+/jDaY/N1b/7uquvs7LDye+deiDwgxuBLda6PDx/w0//8n0JdNmictZbEmQAsaJ5ckrrmbyMW6yRomxagiBNNJHbMoyGDrzsPMwqlMUaDtW4+aTWacxGt65o+NTSZ3E3AGK0hcqK82xHm7r5EPT9cevEbm1V/HkzwrPM5uvf1MELSvYau0K/1qc6o7c5icnU1EisqWFfFeBrwZ811du9n0zWu3Sf8PPHmnD5ORWuqasa/+8Hv5/hoH2elyUowNnZKCOUNJmQnuG6Wtp3TQvGltMT17YvrMsjEvkTWGZI0aRqLdUlNYqZ25AL3GERCzREE87HG+PYFNpniTvuoVnWpuXHeU5ZKoJiO9+iNduag4nYynW6arctSOG3bevh50QQiwO0LwIZvzct1GmGVdmy3nS1LYVGQzrJYLGrfTaanX7N9cZx7IfL4Jlj5u7/2H/jiH328KW9QKDtO6hZIMIHNJ007VLq1xzlp/l5cqY0Ide0bf0Sh5gprFR53icNY9YXi92K+nQINbTxIJ4EKZdRYIE3voO4kF2WgxIih9CVlqfwIYiwXn3v3yoK6VX5I+9l8XO2tmDUr30PnmueOs6Koznft4sXPzuS7zOfqxe91BWBRINf5UWfRjKvMuLkfVqfbxHH+hShUhd577av88r/+Ia1GDWk9di4jQc01oNUUYUUUQ0MJpUnQLSDQCpNptsfUHxFDmvWUVCTV2I9qlflCvXgdCpe16SJhPUe7MCigIEGA52HVOEEMYj3ia3o7F0kGo83P5pSVd5NGOctYNWFXT8Rl84t2y5xmOk0LrNOei/fQrfOK+2wyXRcFbd1YEqBoXtfLFcJxnHshAphNT/jXH/zHlPmELA2+SQiqGiOhEtU2eXIx302CtortGMFgjK7kXS0ktKhZpJcSqQOi1zYWU80zn6hqY+qQCx0qKpb8Be9r7f0DDegAXTAh+AFG05i8GC48954NT2S9jd/9e632WPjOZtNq3rxbverLHJKwKEhzgdkQK1/F7TDvh3UWpM5ncazim1unrTaZfIvCgmg3+aXjNnezPJ4AIfL8+x/7YV77ymdJjChfXKxUtQZrXeOXxH9dYjvspQ7BBNotwRhH0yUuPmSkaeAVTT0N1CZN6pAJRW/GWZydr0Nq4fOOlpGYeR4aFHfvaMVqWYds77quGexdIen11z+RRzDLznq8hxE6/bz9rOvszw2ZP043uLrqPKu00yaza921L7KkLlYrt9cXU8lCyxlRS8gHM3ydKjr3QnS4f5/f/pWf0QrSREsbksRhE9uBlbscCfMJp1oBa8CHBsFGn0dsGiwic4HKufw5G+uOAj9d8LmiFovxIbGm4WqTjqaJM6stWVizGtJqpdp7LjzzLlZZ4Xr87u+rLfXTIOLThHCTWbS4fZ0WW/fdqPnVXD47orZpnKaJVwnZaUhnTDfzCtmirTpXX9O5Jyo5enAfgw+82aZB3ZyNaFs78ZvsBJc0kzz2MxKROUbOLtsptEmRMdZkXTAPRbMVjLUKLDin5l08trNg5iHx7rHi8BBMmXlzo/aeKqS41FXF8MJ1XLZaC+mL159FQVllxzfn3iAQp/k7qyZ0d6FYdexF86/LfOScAxGqOnTr65hlESmN54xJwN3tzTHQRc77TtO1hevptgxNkvny+einOudCzkfdAEWLzyiuZ+uyQs69EIEndYbEaqWqtcE4k0gD3DbmagGGNjvAGH14c9s62kdEmpdhYmJqzP42GmNyNmZ6hxiTNRhnFe62irx5Pz/hYtBQIe8gMAheQqQ//Buz0auqpqph99Y7Vj+FOeEQtKFZu5qf5pBvWvFP2687VoEN+sVllA7g+rWb7GzvkiQpaZqxt3eBW7eeBYFr169z7bpWx+ztXWBrZ5fLV64CkGU9er0+29u77OzsceuZZwGN2Y3HW/R6fd773m8G4Jlbz3Lh4uUFoUl53zdpyueNm7e4ePEKly5fwfuanZ0dEM/73vd+Ll2+jPc1o9EWV6/eYDAYtgtcRFKb+3lCNZGIaHKpCyaWqKYQCCk/7WoWiRlFQndtUb/JWE0UNU1P1zZ/TWNAkZbXaOmCsziX4EK+XeS8W9RiquGsJqIuNlru+AhigtBFs86IzrmgwapKY0SDvWvYpDd3/+tMmyhMp2mPdQJymgZatX2dVto0Xn/91VDuUXPx4iXu3LmND6GEV179mvqRAlev3+CNN25z8eIlPJ7r129ov1yJzQcUSb10+Sr7+/tMJieK+nnPaDxugu7xGvN8RpHneO+5evU6r776FZ599nnAM5lOlGc9aVuLjsdbeK8VxhATgWMCcvSrVt/j+RcilN7KNsFU2/goXd44jREF9Cxxc10OYuJps81I499oDhwBhRM8dQASNOvaJgowNPGkTsZDY14J1AuTSUxr0iySPy5OSg1LCbu3XprbfhYzbJU5t+48sLkVyaafM7Uw8a12j8OFbI88zymKQhfBkPm+u7vHvXt3VROX5dxC1e+rSbu3d5G7d+8yGo2b79ShA0b0Ub708hd5/fVXuXHjVuvLNH5NoFQWKIqc2tcMBn2KouBTn/p93nzzDcbjLfb37/G1r32Zb3jne+fMSbUg9GddsOjcAwsiugpFtMw22Qku5LNpHU8aMgZivY9I7GXUxn30wejEn2f4iZpGHciukKg2Cn6QcR3hgfhUq6rS0gZjkRC9bwS29k19kEgLaMxNTCDburDWF4pjtcCczQnvmnzxu4tgQ3f74ugiaavg4ua7C8vyhQsXyWczxqMt3njjDpcvXeHgwQPwsL21w/HJMdPZhC++/AWuXL7Gy1/6IojwymuvAHD79utcunSFV1/9Ggjcuf16c677QQAHwyHb29t87ZUv05auCfv79/DUfPazf8Stm8/x6U9/ChHh8PAAay2XLl3BiOGLL3+e7e0dnrl1mc997jNAW6LSvCdYi87JWV7A2zn2xn3/lz/wnpC/1ppR/X6/gbMTl5B0OA26/7Yl3zGjekGAwnl8oMgCmiI+6yxJmuFcGjSXa5xaa5OmIhbAiw+Fga0GFJFAN9w+46oq8N43/HRFWVGUJVff8+2k/VEjcGdx+PXfeb6G1idbhoVXIWmL+8QVePG8i0IYRxfO1o5981qyaxatO+fi+eMIlR7NYrG4XyPMVBvvK8b+Vl3L/L56tK7Wjd+py4q/+3/+Pb70pa8srTDnXxMhcz5IrFZtEj/FLNQJMSdIrfkFIrGwj7YMIZxHg7GqsWyIQbkkVK4aZf6JD9U4u6zaa/DGz02qZiILUMcXAlF7xBeU9LdIesPmUOsmRHdbO/nowN7dfZrf5ky5dYIUr2cVFL7OtFyc+CKrEb9Vgr0ue6C7b4RQNt5/R4DWmZqL/uyqc68S5kZQaz8HRi2O8y9EEh15wbmkYdppIGwxOJu0nAYLfAlxBY2CpImoBBKR9sFKECjvAz1wQMDMgi/VbS/ZvMyIkC1de1jxApdCawapjV7WNbX37N18aQ49XDSpVk3KaF5F4Y73t85UW3Wc7ohcD6ug4u53Fk29buZH7cs51LO7onffw+Lx5kzbriYSaej410Se1l7X4j2vWyAWn2nct7lmNHaHb32wxXHuhQgkCIUF/BycDfPxnggw6GRwzaRqswtUa+lhW4c/IjT6kkMHb024o8s9RyOMHQqvUFXaxoRawYTwguqO6eE9tRfKqlYhdinZaKd9Qb7D890Zc5NaQGzw5RY+X7WixnNH/697zPlFZjn4uap0euU1Adqdr1o5oVfFtRavc53ZCfMaT79Xh8yCZYRwUUNuMvWWvhN6PdV13S4IrCrAb8e5FyLVOJqlHc24rq8TfRfvI4upCa2M4sRobfzYm1XTOTSIGkcXdYsaTpjXCgGSQDqgZu1BxGMbde8xokT3bXyog4yhAdYIMAwu3pxvjiA05eXdCdV94TULk82zcpVVAarwPk7k7n7t/l0B35St0B3d62v/nv8svr91geHFbXH/pfMHTdDEbUSDo+vMxu627r/RfI7nWdxHvJ5LdIc25Ye1wBzwRAhR6+N0qag0fpQ0gmMa7gE0ny4QlvjuQ/M+5LhZapl3pLtZBuovtc2V4/CLEE0QyOYz8U1Hv+Zlxxcr0mQmNBrCw/ji9fUTh1P8o87qGdQu0EXS2jQhs9JMa/RYe0hoM64XtNaqsSyIMfirx/fM38eyRlkN588JqfdNx3b9bFmAVl3POm00dx0x/au5Ds2bk/gszrCgPAFCJGRZ1gAH8ea7OWxxv7qsMCYiW4FX24cSg8AxpxRXoWius5K2sZxYFtGSRcbjQ2eShGzxKGjRDBAsteiLiPEMRQF9M1cj31wy2lUztVa7rNEE9bytv3LCeN8QWC5qoQjX17XScOk1uI7J2e5X1+3EhIiILVv/61b5+PuiZtLjxu1C7asl/2dOG9S+I/zqR2pWO+1iEe9xw/nj38050MXAe/VpGrM9HnfJD+p274vXFxC9NfJ07iFuETkEPvN2X8efwLgIvPl2X8SfwPh6vS+AZ733lxY3nntNBHzGe/9tb/dFPO4hIr/z9L6+Psa5T/t5Op6O8z6eCtHT8XQ84ngShOiDb/cF/AmNp/f1dTLOPbDwdDwd5308CZro6Xg6zvV4KkRPx9PxiOPcCpGI/CUR+YyIfF5Evvftvp7Thmjf2jsi8qnOtj0R+SUR+Vz4d7fz2feFe/uMiHxnZ/u3isgnw2f/SFYV9/xnHCJyS0R+RUT+WET+UET+Ztj+xN/bYxvdaO95+QEs8AXgBSAF/gB4z9t9Xadc858D/hTwqc62/xv43vD79wL/V/j9PeGeMuD5cK82fPZbwHeg4fKfB/6bt/m+rgF/Kvw+Bj4brv+Jv7fH9XNeNdEHgM9777/ovc+BHwX+ytt8TRuH9/7XgHsLm/8K8EPh9x8C/tvO9h/13s+89y8Dn0cbSl8Dtrz3H/M66/5F5ztvy/Dev+a9/93w+yHwx2jX9yf+3h7XOK9CdAP4aufvr4VtT9q44rVXLeHfy2H7uvu7EX5f3H4uhog8h3ZN/E2+zu7tUcZ5FaKVBY3/2a/iT26su79ze98iMgL+DfC/e+8PNu26Ytu5vrdHHedViL4GdDtb3QRefZuu5VHG7WDGEP69E7avu7+vhd8Xt7+tQ0QSVIA+4r3/ibD56+LeHsc4r0L028A7ROR5EUmBvwb81Nt8TW9l/BTw3eH37wZ+srP9r4lIJiLPA+8AfiuYRYci8u0BufpfOt95W0a4jg8Bf+y9/386Hz3x9/bYxtuNbGxAhb4LRYK+APztt/t6znC9PwK8BhToqvs9wAXgo8Dnwr97nf3/dri3z9BBqYBvAz4VPvsnhKySt/G+/ixqdn0C+P3w811fD/f2uH6epv08HU/HI47zas49HU/HEzOeCtHT8XQ84ngqRE/H0/GI46kQPR1PxyOOp0L0dDwdjzieCtHT8XQ84ngqRE/H0/GI4/8H5U4awMxAVMoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import scipy\n",
    "from PIL import Image\n",
    "from scipy import ndimage\n",
    "\n",
    "## START CODE HERE ## (PUT YOUR IMAGE NAME) \n",
    "my_image = \"1597460144660.jpg\"\n",
    "## END CODE HERE ##\n",
    "\n",
    "# We preprocess your image to fit your algorithm.\n",
    "fname = \"images/\" + my_image\n",
    "\n",
    "import imageio\n",
    "image = np.array(imageio.imread(fname))\n",
    "my_image = np.array(Image.fromarray(image).resize((64,64))).reshape((1, 64*64*3)).T\n",
    "my_image_prediction = predict(my_image, parameters)\n",
    "\n",
    "plt.imshow(image)\n",
    "print(\"Your algorithm predicts: y = \" + str(np.squeeze(my_image_prediction)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You indeed deserved a \"thumbs-up\" although as you can see the algorithm seems to classify it incorrectly. The reason is that the training set doesn't contain any \"thumbs-up\", so the model doesn't know how to deal with it! We call that a \"mismatched data distribution\" and it is one of the various of the next course on \"Structuring Machine Learning Projects\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<font color='blue'>\n",
    "**What you should remember**:\n",
    "- Tensorflow is a programming framework used in deep learning\n",
    "- The two main object classes in tensorflow are Tensors and Operators. \n",
    "- When you code in tensorflow you have to take the following steps:\n",
    "    - Create a graph containing Tensors (Variables, Placeholders ...) and Operations (tf.matmul, tf.add, ...)\n",
    "    - Create a session\n",
    "    - Initialize the session\n",
    "    - Run the session to execute the graph\n",
    "- You can execute the graph multiple times as you've seen in model()\n",
    "- The backpropagation and optimization is automatically done when running the session on the \"optimizer\" object."
   ]
  }
 ],
 "metadata": {
  "coursera": {
   "course_slug": "deep-neural-network",
   "graded_item_id": "BFd89",
   "launcher_item_id": "AH2rK"
  },
  "kernelspec": {
   "display_name": "python35",
   "language": "python",
   "name": "python35"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "320.326px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}