transfer_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Networks\n",
    "\n",
    "## Project: Write an Algorithm for Landmark Classification\n",
    "\n",
    "\n",
    "### Transfer learning\n",
    "\n",
    "In the previous notebook we have trained our own CNN and we got a certain performance. Let's see how hard it is to match that performance with transfer learning.\n",
    "\n",
    "---\n",
    "## <img src=\"static_images/icons/noun-advance-2109145.png\" alt=\">\" style=\"width:50px\"/> Step 0: Setting up\n",
    "\n",
    "The following cells make sure that your environment is setup correctly and check that your GPU is available and ready to go. You have to execute them every time you restart your notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install requirements\n",
    "!pip install -r requirements.txt | grep -v \"already satisfied\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU available\n",
      "Dataset already downloaded. If you need to re-download, please delete the directory landmark_images\n"
     ]
    }
   ],
   "source": [
    "from src.helpers import setup_env\n",
    "\n",
    "# If running locally, this will download dataset (make sure you have at \n",
    "# least 2 Gb of space on your hard drive)\n",
    "setup_env()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## <img src=\"static_images/icons/noun-advance-2109145.png\" alt=\">\" style=\"width:50px\"/> Step 1: Create transfer learning architecture\n",
    "\n",
    "Open the file `src/transfer.py` and complete the `get_model_transfer_learning` function. When you are done, execute this test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.7.6, pytest-7.1.1, pluggy-1.0.0 -- /home/giacomov/miniconda3/envs/udacity_starter/bin/python\n",
      "cachedir: .pytest_cache\n",
      "rootdir: /home/giacomov/udacity/cnns/final_project/udacity_cnn_solution, configfile: pytest.ini\n",
      "plugins: anyio-3.5.0, subtests-0.7.0\n",
      "collected 1 item                                                               \u001b[0m\u001b[1m\n",
      "\n",
      "src/transfer.py::test_get_model_transfer_learning \u001b[32mPASSED\u001b[0m\u001b[32m                 [100%]\u001b[0m\n",
      "\n",
      "\u001b[32m============================== \u001b[32m\u001b[1m1 passed\u001b[0m\u001b[32m in 1.64s\u001b[0m\u001b[32m ===============================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pytest -vv src/transfer.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## <img src=\"static_images/icons/noun-advance-2109145.png\" alt=\">\" style=\"width:50px\"/> Step 2: Train, validation and test\n",
    "\n",
    "Let's train our transfer learning model! Let's start defining the hyperparameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 64  # size of the minibatch for stochastic gradient descent (or Adam)\n",
    "valid_size = 0.2  # fraction of the training data to reserve for validation\n",
    "num_epochs = 50  # number of epochs for training\n",
    "num_classes = 50  # number of classes. Do not change this\n",
    "learning_rate = 0.001  # Learning rate for SGD (or Adam)\n",
    "opt = 'adam'      # optimizer. 'sgd' or 'adam'\n",
    "weight_decay = 0.0 # regularization. Increase this to combat overfitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAI4CAYAAAB3HEhGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABkeElEQVR4nO3dd5hU5d3G8e9vlypNBBQUFVSMFBVkUaQoYEfFrthiL1jAmGg0yRsTE1PU2HvvhWDDXqkKCiggduzYQBRFKVKe948dzYqUBWb3bPl+rmsvZk6be4aR473nnOdESglJkiRJ0uoryDqAJEmSJFUVFixJkiRJyhMLliRJkiTliQVLkiRJkvLEgiVJkiRJeWLBkiRJkqQ8sWBJkiRplUTEBxGxY9Y5pIrEgiXlmTsbSZKk6suCJUmSpLyKiBpZZ5CyYsGSykFE1I6ISyLi09zPJRFROzevaUQ8EhGzIuKriBgVEQW5eb+PiE8iYnZEvBURO2T7TiRJ+qWI+EtEDImIOyLiW+DIrDNJWfG3C1L5+CPQFegIJOAh4E/A/wG/BaYBzXLLdgVSRPwKOAXoklL6NCJaAYXlG1uSpFLbCzgA+DVQO+MsUmY8giWVj0OBc1NK01NKM4C/Aofn5i0AWgAbppQWpJRGpZQSsIjiHVS7iKiZUvogpfRuJuklSVqxMSmlB1NKi1NKc7MOI2XFgiWVj3WBD0s8/zA3DeACYCrwVES8FxFnAaSUpgKnAX8BpkfEPRGxLpIkVUwfZx1AqggsWFL5+BTYsMTzDXLTSCnNTin9NqW0EdAPOP3Ha61SSnellHrk1k3Av8s3tiRJpZayDiBVBBYsqWzUjIg6P/4AdwN/iohmEdEU+DNwB0BE7BERm0REAN9QfGrg4oj4VUT0yQ2GMQ+YCyzO5u1IkiSpNCxYUtl4jOJC9ONPHWA8MBl4FXgZ+Htu2TbAM8B3wBjgqpTSMIqvv/oX8CXwObA2cHb5vQVJkiStrCi+ll6SJEmStLo8giVJkiRJeVLqghURhRHxSkQ8spR5R0bEjIiYmPs5tsS8IyLindzPEfkKLkmSJEkVzcrcaHgQ8AbQcBnz700pnVJyQkSsBZwDFFE8ssyEiBiaUvp6VcJKkiRJUkVWqiNYEdES2B24YSW3vwvwdErpq1ypehrYdSW3IUmSJEmVQmmPYF0CnAk0WM4y+0XEdsDbwG9SSh8D6/Hzm85Ny037hYg4HjgeoF69ep0322yzUkaTJFUmEyZM+DKl1CzrHGWtadOmqVWrVlnHkCSVkWXtz1ZYsCJiD2B6SmlCRPRaxmIPA3enlOZHxAnArUCflQmYUroOuA6gqKgojR8/fmVWlyRVEhHxYdYZykOrVq1wXyZJVdey9melOUWwO9AvIj4A7gH6RMQdJRdIKc1MKc3PPb0B6Jx7/AmwfolFW+amSZIkSVKVs8KClVI6O6XUMqXUCugPPJdSOqzkMhHRosTTfhQPhgHwJLBzRDSOiMbAzrlpkiRJklTlrMwogj8TEecC41NKQ4GBEdEPWAh8BRwJkFL6KiL+BozLrXZuSumr1YssSZIkSRXTShWslNJwYHju8Z9LTD8bOHsZ69wE3LTKCSVJkiSpkij1jYYlSZIkSctnwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScqTUhesiCiMiFci4pGlzDs9Il6PiMkR8WxEbFhi3qKImJj7GZqv4JIkSZJU0dRYiWUHAW8ADZcy7xWgKKU0JyIGAOcDB+XmzU0pdVytlJIkSZJUCZTqCFZEtAR2B25Y2vyU0rCU0pzc07FAy/zEkyRJkqTKo7SnCF4CnAksLsWyxwCPl3heJyLGR8TYiNh7WStFxPG55cbPmDGjlLEkSZIkqeJYYcGKiD2A6SmlCaVY9jCgCLigxOQNU0pFwCHAJRGx8dLWTSldl1IqSikVNWvWrHTpJUmSJKkCKc0RrO5Av4j4ALgH6BMRdyy5UETsCPwR6JdSmv/j9JTSJ7k/3wOGA51WP7YkSZIkVTwrLFgppbNTSi1TSq2A/sBzKaXDSi4TEZ2AaykuV9NLTG8cEbVzj5tSXNZez2N+SVIVFxG7RsRbETE1Is5ayvzaEXFvbv6LEdGqxLyzc9PfiohdVrTNiLglIt4vMfptx9z0Q3Mj5b4aES9ExJZl+64lSZXVyowi+DMRcS4wPqU0lOJTAusD/40IgI9SSv2AtsC1EbGY4jL3r5SSBUuSVCoRUQhcCewETAPGRcTQJfYlxwBfp5Q2iYj+wL+BgyKiHcW/GGwPrAs8ExGb5tZZ3jbPSCkNWSLK+8D2KaWvI2I34Dpgm7y/YUlSpbdSBSulNJzi0/xIKf25xPQdl7H8C8Dmqx5PklTNbQ1MzZ1mTkTcA+zFz8+G2Av4S+7xEOCKKP5t317APbnT1t+PiKm57VGKbf5Mbn/2I0fLlSQtU6lvNCxJUgbWAz4u8XxabtpSl0kpLQS+AZosZ90VbfO83OmAF/94mvsSlhwtV5Kkn1iwJEn6n7OBzYAuwFrA70vOjIjeFBes3/9yVW85IkmyYEmSKrZPgPVLPG+Zm7bUZSKiBtAImLmcdZe5zZTSZ6nYfOBm/ndKIRGxBXADsFdKaebSwnrLEUmSBUuSVJGNA9pEROuIqEXxoBVDl1hmKHBE7vH+FI92m3LT++dGGWwNtAFeWt42I6JF7s8A9gam5J5vANwPHJ5Serus3qwkqfJb5VEEJUkqaymlhRFxCvAkUAjclFJ6bYmRbG8Ebs8NYvEVxYWJ3HKDKR68YiFwckppEcDStpl7yTsjohkQwETgxNz0P1N8XddVudFyF6aUisr23UuSKqMo/iVfxVJUVJTGjx+fdQxJUhmIiAnVoZy4L5Okqm1Z+zNPEZQkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLypNQFKyIKI+KViHhkKfNqR8S9ETE1Il6MiFYl5p2dm/5WROySp9ySJEmSVOGszBGsQcAby5h3DPB1SmkT4GLg3wAR0Q7oD7QHdgWuiojCVY8rSZIkSRVXqQpWRLQEdgduWMYiewG35h4PAXaIiMhNvyelND+l9D4wFdh69SJLkiRJUsVU2iNYlwBnAouXMX894GOAlNJC4BugScnpOdNy034hIo6PiPERMX7GjBmljCVJkiRJFccKC1ZE7AFMTylNKMsgKaXrUkpFKaWiZs2aleVLSZIkSVKZKM0RrO5Av4j4ALgH6BMRdyyxzCfA+gARUQNoBMwsOT2nZW6aJEmSJFU5KyxYKaWzU0otU0qtKB6w4rmU0mFLLDYUOCL3eP/cMik3vX9ulMHWQBvgpbyllyRJkqQKpMaqrhgR5wLjU0pDgRuB2yNiKvAVxUWMlNJrETEYeB1YCJycUlq0+rElSZIkqeJZqYKVUhoODM89/nOJ6fOAA5axznnAeaucUJIkSZIqiZW5D5YkSZIkaTksWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5IkSZLyxIIlSZIkSXliwZIkSZKkPLFgSZIkSVKeWLAkSZIkKU8sWJIkSZKUJxYsSZIkScoTC5YkSZIk5YkFS5JUoUXErhHxVkRMjYizljK/dkTcm5v/YkS0KjHv7Nz0tyJilxVtMyJuiYj3I2Ji7qdjbvpmETEmIuZHxO/K9h1LkiqzGlkHkCRpWSKiELgS2AmYBoyLiKEppddLLHYM8HVKaZOI6A/8GzgoItoB/YH2wLrAMxGxaW6d5W3zjJTSkCWifAUMBPbO+5uUJFUpHsGSJFVkWwNTU0rvpZR+AO4B9lpimb2AW3OPhwA7RETkpt+TUpqfUnofmJrbXmm2+TMppekppXHAgny9MUlS1WTBkiRVZOsBH5d4Pi03banLpJQWAt8ATZaz7oq2eV5ETI6IiyOi9sqEjYjjI2J8RIyfMWPGyqwqSaoiLFiSJP3P2cBmQBdgLeD3K7NySum6lFJRSqmoWbNmZZFPklTBWbAkSRXZJ8D6JZ63zE1b6jIRUQNoBMxczrrL3GZK6bNUbD5wM8WnE0qSVGoWLElSRTYOaBMRrSOiFsWDVgxdYpmhwBG5x/sDz6WUUm56/9wog62BNsBLy9tmRLTI/RkUD2gxpSzfnCSp6nEUQUlShZVSWhgRpwBPAoXATSml1yLiXGB8SmkocCNwe0RMpXi0v/65dV+LiMHA68BC4OSU0iKApW0z95J3RkQzIICJwIm55ZsD44GGwOKIOA1ol1L6tqw/A0lS5RLFv+SrWIqKitL48eOzjiFJKgMRMSGlVJR1jrLmvkySqrZl7c88RVCSJEmS8sSCJUmSJEl5YsGSJEmSpDyxYEmSJElSnliwJEmSJClPLFiSJEmSlCcWLEmSJEnKEwuWJEmSJOWJBUuSJEmS8sSCJUmSJEl5YsGSJEmSpDyxYEmSJElSnliwJEmSJClPLFiSJEmSlCdVsmB9/f0PWUeQJEmSVA1VuYI1ffY8drp4BH8Z+hrzFy7KOo4kSZKkamSFBSsi6kTESxExKSJei4i/LmWZiyNiYu7n7YiYVWLeohLzhuY5/y80qluTfluuxy0vfMC+V73AezO+K+uXlCRJkiSgdEew5gN9UkpbAh2BXSOia8kFUkq/SSl1TCl1BC4H7i8xe+6P81JK/fKUe5lq1yjkz3u244ZfF/HJrLnscflo7n95Wlm/rCRJkiStuGClYj8eBqqZ+0nLWeVg4O48ZFstO7Zbh8cH9aTDuo04ffAkfjt4Et/PX5h1LEmSJElVWKmuwYqIwoiYCEwHnk4pvbiM5TYEWgPPlZhcJyLGR8TYiNh7NfOulBaN6nLXcdswcIc23P/KNPa8YjSvf/pteUaQJEmSVI2UqmCllBblTv9rCWwdER2WsWh/YEhKqeToEhumlIqAQ4BLImLjpa0YEcfnitj4GTNmlP4drECNwgJO32lT7jx2G76bt5C9r3qe28Z8QErLOwgnSZIkSStvpUYRTCnNAoYBuy5jkf4scXpgSumT3J/vAcOBTsvY9nUppaKUUlGzZs1WJlapdNu4KY8P6km3jZvw54de48Q7JvDNnAV5fx1JkiRJ1VdpRhFsFhFr5h7XBXYC3lzKcpsBjYExJaY1jojaucdNge7A63lJvgqa1K/NTUd04U+7t+W5N6fT97JRTPjwq6ziSJIkSapiSnMEqwUwLCImA+MovgbrkYg4NyJKjgrYH7gn/fzcu7bA+IiYRPGRr3+llDIrWAAFBcGxPTdiyIndKCiAA68dy5XDprJ4sacMSpIkSVo9URGvRSoqKkrjx48v89f5dt4Czr7/VR6d/Bk9NmnKRQdtydoN6pT560pSdRYRE3LX5lZp5bUvkyRlY1n7s5W6BquqaVinJlcc3Il/7bs54z/8ir6XjmLk2/kbYEOSJElS9VKtCxZARNB/6w0YekoP1qpXi1/f9BL/fuJNFixanHU0SZIkSZVMtS9YP9p0nQY8dHIPDt56fa4e/i4HXTuGaV/PyTqWJEmSpErEglVC3VqF/HPfLbj84E6888V39L10FE9M+SzrWJIkSZIqCQvWUuy55bo8OrAnrZvW48Q7Xub/HpzCvAWLVryiJEmSpGrNgrUMGzRZg/+e2I3jt9uI28d+yN5XPs/U6d9lHUuSJElSBWbBWo5aNQr4Q9+23HxkF6bPns+el4/mv+M/piIObS9JkiQpexasUui92do8NrAnW67fiDOGTOb0wZP4bv7CrGNJkiRJqmAsWKXUvFEd7jy2K6fvtCkPTfyEPS4bxZRPvsk6liRJkqQKxIK1EgoLgoE7tOHu47oyb8Fi9r3qBW5+/n1PGZQkSZIEWLBWyTYbNeHxQT3p2aYpf334dY67bQJff/9D1rEkSZIkZcyCtYoa16vFDUcU8ec92jHi7en0vWwUL73/VdaxJEmSJGXIgrUaIoKje7Tm/gHdqVWjgP7XjeHyZ99h0WJPGZQkSZKqIwtWHmzeshGPnNqDPbdcl/88/TaH3/gi07+dl3UsSZIkSeXMgpUnDerU5JKDOnL+/lvwykez2O3SUQx/a3rWsSRJkiSVIwtWHkUEBxatz8OndqdZg9ocefM4/vnYG/ywcHHW0SRJkiSVAwtWGdhk7QY8eHJ3Duu6AdeOfI8Drh3DRzPnZB1LkiRJUhmzYJWROjUL+fvem3PVoVvx3ozv2P2yUTw6+bOsY0mSJEkqQxasMtZ38xY8NrAnG69dn5Pvepk/PPAq8xYsyjqWJEmSpDJgwSoH66+1Bv89cVtO3H5j7nrxI/a64nne+WJ21rEkSZIk5ZkFq5zULCzgrN0249ajt+bL7+az5xWjuXfcR6TkPbMkSZKkqsKCVc6237QZjw/qSecNG/P7+15l4D0TmT1vQdaxJEmSJOWBBSsDazesw21Hb8MZu/yKx179jD0uH83kabOyjiVJkiRpNVmwMlJYEJzcexPuOb4rCxYuZr+rX+CGUe95yqAkSZJUiVmwMtal1Vo8NqgnvX+1Nn9/9A2OuXU8X33/Q9axJEmSJK0CC1YFsOYatbj28M78tV97Rr/zJbtdOpKx783MOpYkSZKklWTBqiAigiO6teL+k7qxRq0aHHL9WC555m0WLfaUQUmSJKmysGBVMB3Wa8TDp/Zg747rcckz73DI9WP5/Jt5WceSJEmSVAoWrAqofu0aXHRQR/5zwJa8+sk37HbpSJ5784usY0mSJElaAQtWBbZf55Y8fGoPmjeqy9G3jOdvj7zODwsXZx1LkiRJ0jJYsCq4jZvV54GTuvHrbTfkxtHvs/81L/DhzO+zjiVJkiRpKSxYlUCdmoWcu1cHrjmsMx98+T27XzaaoZM+zTqWJEmSpCVYsCqRXTs057FBPflV8wYMvPsVfj9kMnN+WJh1LEmSJEk5FqxKpmXjNbjn+K6c3HtjBk/4mH5XPM9bn8/OOpYkSZIkLFiVUs3CAs7YZTNuO3prZs1ZQL8rRnPXix+RkvfMkiRJkrJkwarEerZpxuODerJ167X4wwOvcspdr/DN3AVZx5IkSZKqLQtWJdesQW1uPWprfr/rZjzx2ufsftkoXvno66xjSZIkSdWSBasKKCgIBvTamMEnbEtKcMA1Y7h2xLssXuwpg5IkSVJ5smBVIZ03bMxjA3uyY9t1+Ofjb3LULeP48rv5WceSJEmSqg0LVhXTaI2aXH3YVvxt7w6MeW8mfS8dxQtTv8w6liRJklQtWLCqoIjg8K4b8tDJ3WlQpwaH3vgi/3nqLRYuWpx1NEmSJKlKs2BVYW1bNOThU3uw31Ytufy5qRxy/Yt8Omtu1rEkSZKkKsuCVcWtUasGFx6wJRcftCWvffoNfS8bxdOvf5F1LEmSJKlKsmBVE/t0askjA3vSsnFdjrttPH8Z+hrzFy7KOpYkSZJUpViwqpHWTetx34BuHNW9Fbe88AH7XvUC73/5fdaxJEmSpCqjRtYBVL5q1yjknD3b023jppwxZBJ7XDaK8/bZnL07rZd1NClTCxYsYNq0acybNy/rKFVGnTp1aNmyJTVr1sw6iiRJ5caCVU3t1G4dHhvYk0H3vMJp905k9NQvOXev9qxRy6+Eqqdp06bRoEEDWrVqRURkHafSSykxc+ZMpk2bRuvWrbOOI0lSufEUwWps3TXrcvdxXRnYZxPue3kae1w+mtc//TbrWFIm5s2bR5MmTSxXeRIRNGnSxCOCkqRqx4JVzdUoLOD0nX/Fncduw3fzFrL3Vc9z+5gPSCllHU0qd5ar/PLzlCRVRxYsAdBt46Y8Nqgn227UhP976DVOuvNlvpmzIOtYUrUya9YsrrrqqpVer2/fvsyaNWu5y/z5z3/mmWeeWcVkkiSptCxY+knT+rW5+cgu/KHvZjz9+hf0vWwUEz78OutYUrWxrIK1cOHC5a732GOPseaaay53mXPPPZcdd9xxdeJJkqRSsGDpZwoKguO325ghA7pRUAAHXjuGq4ZPZfFiTxmUytpZZ53Fu+++S8eOHenSpQs9e/akX79+tGvXDoC9996bzp070759e6677rqf1mvVqhVffvklH3zwAW3btuW4446jffv27LzzzsydOxeAI488kiFDhvy0/DnnnMNWW23F5ptvzptvvgnAjBkz2GmnnWjfvj3HHnssG264IV9++WU5fwqSJFVuDhmnpeq4/po8OrAnZ9/3Kuc/8RZj3p3JRQd2pFmD2llHk8rcXx9+Le8DvrRbtyHn7Nl+ucv861//YsqUKUycOJHhw4ez++67M2XKlJ9G4bvppptYa621mDt3Ll26dGG//fajSZMmP9vGO++8w913383111/PgQceyH333cdhhx32i9dq2rQpL7/8MldddRUXXnghN9xwA3/961/p06cPZ599Nk888QQ33nhj/j4ASZKWUFlukbKytx2xYGmZGtapyRWHdKL7S03568Ovsdulo7jkoI70aNM062hStbD11lv/bIjzyy67jAceeACAjz/+mHfeeecXBat169Z07NgRgM6dO/PBBx8sddv77rvvT8vcf//9AIwePfqn7e+66640btw4n29HkqSfqQy3SFmV246ssGBFRB1gJFA7t/yQlNI5SyxzJHAB8Elu0hUppRty844A/pSb/veU0q2lSqYKISI4ZJsN6LxhY06562UOv+lFBmy/Mb/ZaVNqFnqGqaqmFR1pKi/16tX76fHw4cN55plnGDNmDGussQa9evVa6m/8atf+31HmwsLCn04RXNZyhYWFK7zGS5KksjBv3rwKXa7gf7cdmTFjRqnXKc3/Ic8H+qSUtgQ6ArtGRNelLHdvSqlj7ufHcrUWcA6wDbA1cE5E+CvRSuhXzRsw9JQeHFS0PlcNf5f+141l2tdzso4lVSkNGjRg9uzZS533zTff0LhxY9ZYYw3efPNNxo4dm/fX7969O4MHDwbgqaee4uuvHeRGklS2KnK5+tHKZlxhwUrFvss9rZn7Ke2IB7sAT6eUvkopfQ08Dey6UglVYdStVci/9tuCyw7uxFufz6bvpaN4YsrnWceSqowmTZrQvXt3OnTowBlnnPGzebvuuisLFy6kbdu2nHXWWXTturTfc62ec845h6eeeooOHTrw3//+l+bNm9OgQYO8v44kSVVZqa7BiohCYAKwCXBlSunFpSy2X0RsB7wN/Cal9DGwHvBxiWWm5aYt7TWOB44H2GCDDUr9BlT++m25Llu2bMSpd7/CiXdM4Nfbbsgf+ralTs3CrKNJld5dd9211Om1a9fm8ccfX+q8H6+zatq0KVOmTPlp+u9+97ufHt9yyy2/WB6gqKiI4cOHA9CoUSOefPJJatSowZgxYxg3btzPTjmUJKmqqV+/Pt99992KF1wJpbqIJqW0KKXUEWgJbB0RHZZY5GGgVUppC4qPUq30dVYppetSSkUppaJmzZqt7OoqZxs2qceQE7txbI/W3DbmQ/a56gXenZHfL6ek8vXRRx/RpUsXttxySwYOHMj111+fdSQAImLXiHgrIqZGxFlLmV87Iu7NzX8xIlqVmHd2bvpbEbHLirYZEbdExPsRMTH30zE3PSListzykyNiq7J915KkrKzutckrNUpBSmkWMIwlTvNLKc1MKc3PPb0B6Jx7/AmwfolFW/K/gTBUydWqUcCf9mjHTUcW8fk3c9nz8tEMmTAt61iSVlGbNm145ZVXmDRpEuPGjaNLly5ZR/rxDIorgd2AdsDBEdFuicWOAb5OKW0CXAz8O7duO6A/0J7i/dZVEVFYim2eUeKa4om5absBbXI/xwNX5/3NSpIyM3z48F/cf3JVlWYUwWbAgpTSrIioC+xEbudVYpkWKaXPck/7AW/kHj8J/KPEwBY7A2evVmJVOH02W4fHB23HoHte4Xf/ncQLU7/k3L07UL+2dwGQtNq2BqamlN4DiIh7gL2A10sssxfwl9zjIcAVUXxF8l7APblfAL4fEVNz26MU21zSXsBtKaUEjI2INZfY95WJsrgnmyRVFCd3qkut3BlQVw6byrvT83s21MZr1+fk3pv8YnrdmoWsu2bdX0x/+eWXf3b/yVVVmiNYLYBhETEZGEfxoBWPRMS5EdEvt8zAiHgtIiYBA4EjAVJKXwF/y603Djg3N01VTPNGdbjruK6ctmMbHpz4CXtePpopn3yTdSxJlV9pruX9aZmU0kLgG6DJctZd0TbPy50GeHFE/HgRWqmuKY6I4yNifESMX5khfSVJ2Vvy/pOraoWHGFJKk4FOS5n+5xKPz2YZR6ZSSjcBN61GRlUShQXBaTtuSteNmjDonlfY96oX+EPfzTiiW8W+v4EklXA28DlQC7gO+D1wbmlXTildl1uPoqKi0o64u0wV5Z5sklQW3njjDTZuVh+Aiw7smG0Yfn7/ydXhnWKVd103asLjg7ajR5um/OXh1zn+9gnMmvND1rEkVU6luZb3p2UiogbQCJi5nHWXuc2U0me525PMB27mf6cUek2xJKlULFgqE2vVq8WNRxTxp93bMvyt6fS9dBTjPvDsUCmf6tcv/q3fp59+yv7777/UZXr16sX48eOXu51LLrmEOXP+d+Pwvn37MmvWrLzlXE3jgDYR0ToialE8aMXQJZYZChyRe7w/8FzuWqmhQP/cKIOtKR6g4qXlbTMiWuT+DGBvYEqJ1/h1bjTBrsA3ZX39lSSpcrJgqcxEBMf23Ij7BnSjZo0C+l83liuee4dFi1f7rBlJJay77roMGTJklddfsmA99thjrLnmmnlItvpy11SdQvGgSW8Ag1NKry1xHfCNQJPcIBanA2fl1n0NGEzx4BVPACfnbjuy1G3mtnVnRLwKvAo0Bf6em/4Y8B4wFbgeOKkM37YkqZz8eA+sXr168cgjj+RlmxYslbktWq7JI6f2oO/mLbjwqbf59U0vMv3beVnHkiqcs846iyuvvPKn53/5y1/4+9//zg477MBWW23F5ptvzkMPPfSL9T744AM6dCi+PeHcuXPp378/bdu2ZZ999mHu3Lk/LTdgwACKiopo374955xzDgCXXXYZn376Kb1796Z3794AtGrVii+//BKAiy66iA4dOtChQwcuueSSn16vbdu2HHfccbRv356dd975Z6+Tbymlx1JKm6aUNk4pnZeb9ueU0tDc43kppQNSSpuklLb+cXTA3Lzzcuv9KqX0+PK2mZveJ6W0eUqpQ0rpsJTSd7npKaV0cm75zVNKyz8sKEmqthxHW+WiQZ2aXNa/Iz03acqfh05ht0tH8Z8Dt6TXr9bOOpr0S4+fBZ+/mt9tNt8cdvvXchc56KCDOO200zj55JMBGDx4ME8++SQDBw6kYcOGfPnll3Tt2pV+/fotc+CYq6++mjXWWIM33niDyZMns9VW/7sf7nnnncdaa63FokWL2GGHHZg8eTIDBw7koosuYtiwYTRt2vRn25owYQI333wzL774IiklttlmG7bffnsaN27MO++8w913383111/PgQceyH333cdhhx22mh+SJEmVn0ewVG4iggO7rM/Dp/Sgaf3aHHnzOP75+BssWLQ462hShdCpUyemT5/Op59+yqRJk2jcuDHNmzfnD3/4A1tssQU77rgjn3zyCV988cUytzFy5Mifis4WW2zBFlts8dO8wYMHs9VWW9GpUydee+01Xn99ebd9gtGjR7PPPvtQr1496tevz7777suoUaMAaN26NR07dgSgc+fOfPDBB6v35iVJ1VLxJbMV28pm9AiWyl2bdRrw0CndOfeR17l2xHu8+N5XXH5wJ9Zfa42so0nFVnCkqSwdcMABDBkyhM8//5yDDjqIO++8kxkzZjBhwgRq1qxJq1atmDdv5U+xff/997nwwgsZN24cjRs35sgjj1yl7fyodu3aPz0uLCws01MEJUlVU506dZg5cyZNmjSpsLf0SSkxc+ZM6tSpU+p1LFjKRJ2ahfxjn83pvnFTzrpvMn0vG8W/99uCvpu3yDqalKmDDjqI4447ji+//JIRI0YwePBg1l57bWrWrMmwYcP48MMPl7v+dtttx1133UWfPn2YMmUKkydPBuDbb7+lXr16NGrUiC+++ILHH3+cXr16AdCgQQNmz579i1MEe/bsyZFHHslZZ51FSokHHniA22+/vUzetySp+mnZsiXTpk2jot+YvU6dOrRs2bLUy1uwlKndt2jBFi0bccrdr3DSnS9z6DYb8H97tKNOzcKso0mZaN++PbNnz2a99dajRYsWHHrooey5555svvnmFBUVsdlmmy13/QEDBnDUUUfRtm1b2rZtS+fOnQHYcsst6dSpE5ttthnrr78+3bt3/2md448/nl133ZV1112XYcOG/TR9q6224sgjj2TrrYtvBXXsscfSqVMnTweUJOVFzZo1ad26ddYx8i4q4nmPRUVFaUX3bVHVsmDRYi588i2uHfkemzVvwBWHdGKTtRtkHUvVyBtvvEHbtm2zjlHlLO1zjYgJKaWijCKVG/dlklS1LWt/5iAXqhBqFhZwdt+23HJUF2bMns+elz/P4HEfV4oLHyVJkqQfWbBUofT61do8NqgnnTZYkzPvm8xp905k9rwFWceSJEmSSsWCpQpnnYZ1uP2Ybfjdzpvy8KRP2ePy0UyeNivrWJIkSdIKWbBUIRUWBKf0acO9J2zLDwsXs9/VL3Dj6Pc9ZVBlyu9Xfvl5SpKqIwuWKrQurdbisYE92X7TtfnbI69z7K3j+er7H7KOpSrox3txWAryY1XuGyJJUlXgMO2q8BrXq8X1v+7MrS98wD8ee5O+l47i0v4d2WajJllHUxVSWe7FUZms7H1DJEmqCixYqhQigiO7t6ao1VqcevcrHHz9WAbtsCmn9NmEwoKKeedvVS5V9V4ckiSpfHmKoCqVDus14uFTe7BXx/W4+Jm3OfSGsXzx7bysY0mSJEmABUuVUP3aNbjowC258IAtmfTxN+x26SiGvTk961iSJEmSBUuVU0Swf+eWPHxqD9ZuUJujbhnHeY++zg8LF2cdTZIkSdWYBUuV2iZr1+fBk7vz62035PpR73PANS/w0cw5WceSJElSNWXBUqVXp2Yh5+7VgWsO24r3v/ye3S8bxcOTPs06liRJkqqhqlewUoLn/g5fvpN1EpWzXTu04NGBPWmzTn1OvfsVzr5/MnN/WJR1LEmSJFUjVa9gfTMNxt0I1/SE8TcVFy5VG+uvtQb3nrAtJ/XamHvGfUy/K0bz1uezs44lSZKkaqLqFaw114cBL8AGXeGR38A9h8D3X2adSuWoZmEBZ+66GbcdvTVfz/mBfleM5u6XPiJZtiVJklTGql7BAmjYAg67H3b5J0x9Bq7uBu88k3UqlbOebZrx2KCebN16Lc6+/1VOvfsVvp23IOtYkiRJqsKqZsECKCiAbU+C44bBGk3gzv3g8d/DgrlZJ1M5WrtBHW49amvO3PVXPD7lc3a/bBQTP56VdSxJkiRVUVW3YP2oeYfikrXNAHjxGriuN3w+JetUKkcFBcFJvTZh8AldWbwY9r/6Ba4f+R6LF3vKoCRJkvKr6hcsgJp1YLd/wWH3wdyv4PreMOZKWOxNaauTzhuuxWMDe7JD27U577E3OPrWccz8bn7WsSRJklSFVI+C9aNNdiweAGOTneDJP8Ad+8C3n2WdSuWo0Ro1ueawzvxtr/a88O5Mdrt0FC+86yAokiRJyo/qVbAA6jWF/nfCHpfAxy/B1dvCGw9nnUrlKCI4fNtWPHBSN+rXqcGhN7zIRU+/zcJFHtGUJEnS6ql+BQsgAoqOghNGwZobwr2HwUOnwPzvsk6mctR+3UY8fEoP9tuqJZc9+w6HXP8in33jICiSJEladdWzYP2o6SZwzNPQ43R45Q64tidMm5B1KpWjerVrcOEBW3LxQVsy5dNv2O3SUTzz+hdZx5IkSVIlVb0LFkCNWrDjOXDko7BoAdy4E4y4ABYvyjqZytE+nVryyKk9WG/Nuhx723jOffh15i/0OyBJkqSVY8H6UavucOJoaL8PDPs73NwXvv4w61QqRxs1q8/9J3XjyG6tuOn599nv6hd4/8vvs44lSZKkSsSCVVLdNWH/G2Hf62H663BND5g8OOtUKke1axTyl37tue7wznz81Vz2uGwUD038JOtYkiRJqiQsWEuzxYHFR7PWaQ/3HwdDjoG5s7JOpXK0c/vmPD6oJ21bNGTQPRM547+TmPPDwqxjSZIkqYKzYC1L4w2Lr8vq8yd47YHio1kfPJ91KpWjddesyz3Hd+XUPpsw5OVp7Hn5aN747NusY0mSJKkCs2AtT0EhbHdG8UiDhTXhlt3hmb/Cwh+yTqZyUqOwgN/u/CvuOGYbvp23kL2ufJ47xn5ISinraJIkSaqALFil0bJz8T2ztjocRl9UPNLgl+9knUrlqPsmTXl8UE+6btSEPz04hZPufJlv5i7IOpYkSZIqGAtWadWuD/0uh4PugFkfwrXbwfibwSMZ1UbT+rW55cgunL3bZjz9+hf0vXQUL3/0ddaxJEmSVIFYsFZW2z1hwBhYfxt45DS451D4/susU6mcFBQEJ2y/MYNP3JYIOPCaMVwz4l0WL7ZoS5IkyYK1ahq2gMPuh13+CVOfhqu7wdRnsk6lcrTVBo15dGBPdm6/Dv96/E2OuPklZsyen3UsSZIkZcyCtaoKCmDbk+C4YVB3LbhjP3j897BgXtbJVE4a1a3JlYdsxXn7dOCl97+i72WjGP2ORzMlSZKqMwvW6mreAY4fBtucCC9eA9f3hi9eyzqVyklEcOg2G/LQKd1pVLcmh9/0Ihc8+SYLFy3OOpokSZIyYMHKh5p1Ybd/w2H3wZyZcF0vGHMlLPZ/squLzZo3ZOgp3Tmw8/pcOexd+l83lk9mzc06liRJksqZBSufNtkRBrxQ/OeTf4A79oVvP8s6lcrJGrVq8O/9t+DS/h158/PZ9L10FE++9nnWsSRJklSOLFj5Vq8p9L8L9rgYPhpbPADGGw9nnUrlaK+O6/HIqT3YYK01OOH2CZzz0BTmLViUdSxJkiSVAwtWWYiAoqPhxFGw5gZw72Ew9FSY/13WyVROWjWtx30DunFMj9bcOuZD9r3qBd6b4d+/JElSVWfBKktN28AxT0OP0+Hl2+HanvDJhKxTqZzUqlHA/+3RjhuPKOKzb+ayx+Wjuf/laVnHkiRJUhmyYJW1GrVgx3PgyEdg4Q9w484w8gJY7Clj1cUObdfhsUE96bBeI04fPInTB0/k+/kLs44lSZKkMmDBKi+tesCA56HdXvDc3+GW3eHrD7NOpXLSolFd7j6uK4N2aMODr3zCnpeP5rVPv8k6liRJkvLMglWe6q4J+90I+1xXfK+sa3rA5MFZp1I5KSwIfrPTptx5bFe+/2Eh+1z5Are+8AEppayjSZIkKU9WWLAiok5EvBQRkyLitYj461KWOT0iXo+IyRHxbERsWGLeooiYmPsZmu83UOlEwJYHwYmjYe12cP9xMOQYmDsr62QqJ9tu3ITHBvak+yZNOGfoaxx9yzje/mJ21rEkSZKUB6U5gjUf6JNS2hLoCOwaEV2XWOYVoCiltAUwBDi/xLy5KaWOuZ9++QhdJTTeEI58FHr/CV57oPho1gfPZ51K5aRJ/drceEQX/m+Pdoz74Gt2uWQkpw+eyMdfzck6miRJklbDCgtWKvbj+NI1cz9piWWGpZR+/D/DsUDLvKasqgprwPZnwDFPQWHN4uuynj0XFi3IOpnKQUFBcEyP1ow8szfH9dyIRyd/Rp//DOf/HpzC9G/nZR1PkiRJq6BU12BFRGFETASmA0+nlF5czuLHAI+XeF4nIsZHxNiI2Hs5r3F8brnxM2bMKE2sqqNlEZwwCjodBqP+AzfuBF9OzTqVysla9Wrxh75tGXFGbw4sWp+7X/qI7S4Yxj8ff4Ovv/8h63iSJElaCaUqWCmlRSmljhQfmdo6IjosbbmIOAwoAi4oMXnDlFIRcAhwSURsvIzXuC6lVJRSKmrWrNnKvIeqoXZ92OsKOPB2+PqD4ntmTbgFHACh2mjeqA7n7bM5z/52e3br0ILrRr7HducP47Jn3+E7h3WXJEmqFFZqFMGU0ixgGLDrkvMiYkfgj0C/lNL8Eut8kvvzPWA40GnV41YD7frBgBdg/a3h4UFwz6Hw/cysU6kcbdikHhcf1JEnBm3Hths34aKn32b784dx4+j3mbfA+6dJkiRVZKUZRbBZRKyZe1wX2Al4c4llOgHXUlyuppeY3jgiauceNwW6A6/nLX1V1XBdOOwB2OUfMPVpuHpbmPpM1qlUzn7VvAHX/bqIB07qRtsWDfnbI6/T+8Lh3PPSRyxctDjreJIkSVqK0hzBagEMi4jJwDiKr8F6JCLOjYgfRwW8AKgP/HeJ4djbAuMjYhLFR77+lVKyYJVGQQFsezIc9xzUXQvu2A8ePwsWOPhBddNpg8bccew23HXsNqzTsA5n3f8qO108kqGTPmXxYk8hlSRJqkiiIt7ktKioKI0fPz7rGBXHgrnw9Dnw0rXF987a7wZYp33WqZSBlBLPvDGdC598i7e+mM1mzRtwxi6/os9maxMRWceTSiUiJuSuza3S3JdJUtW2rP3ZSl2DpYzUrAt9z4dD74Pvv4TresGYq2Cxp4lVNxHBTu3W4bFBPbm0f0fmLljEMbeOZ/9rxjD2Pa/VkyRJypoFqzJpsyOcNAY23gGePBvu2Be+/SzrVMpAYUGwV8f1eOb07Tlvnw5M+3oO/a8by+E3vsjkabOyjidJklRtWbAqm3pN4eC7YY+L4aOxcHU3eOORrFMpIzULCzh0mw0ZcUZv/ti3LVM++YZ+VzzPibdP4J0vZmcdT5IkqdqxYFVGEVB0NJwwEtZcH+49FIaeCvO/yzqZMlKnZiHHbbcRI8/szWk7tmH01C/Z5ZKRnD54Ih9/NSfreJIkSdWGBasya7YpHPMM9PgNvHw7XLsdfDIh61TKUIM6NTltx00ZeWZvju25EY9O/ow+/xnO/z04henfOgKlJElSWbNgVXY1asGOf4EjH4GF8+HGnWHkBbDYG9JWZ2vVq8Uf+rZlxBm9ObBofe5+6SO2u2AY/3r8TWbN+SHreJIkSVWWBauqaNUDBoyGtv3gub/DLXvArI+yTqWMNW9Uh/P22ZxnTt+eXds359qR79Lz38O4/Nl3+G7+wqzjSZIkVTkWrKqkbmPY/ybY51r4/FW4ujtM/m/WqVQBtGpaj0v6d+LxQT3punET/vP022x//jBuHP0+8xZ4tFOSJClfLFhVTQRs2b/4aNba7eD+Y+G+Y2HurKyTqQLYrHlDrv91EQ+c1I3NWjTgb4+8Tp8Lh3PvuI9YuMj7qkmSJK0uC1ZV1bgVHPko9P4jTLkfrukBH76QdSpVEJ02aMydx3blzmO3oVnDOvz+vlfZ6eKRPDzpUxYvTlnHkyRJqrQsWFVZYQ3Y/kw45ikoqAG37A7PnguLFmSdTBVE902a8uBJ3bju8M7UKizg1LtfYffLR/Pcm1+QkkVLkiRpZVmwqoOWRXDiaOh4KIz6T/FIg19OzTqVKoiIYOf2zXlsUE8uOagj389fyNG3jGf/a8Yw9r2ZWceTJEmqVCxY1UXt+rDXFXDgbfDVe3BtT5hwC3iUQjmFBcHendbj2d9uz3n7dGDa13Pof91YDr/xRSZPm5V1PEmSpErBglXdtNsLThoDLbvAw4Pg3sPge49S6H9qFhZw6DYbMuKM3vyxb1umfPIN/a54nhNvn8A7X8zOOp4kSVKFZsGqjhquC4c/CDufB+88BVdvC1OfzTqVKpg6NQs5bruNGHlmb07bsQ2jp37JLpeM5LeDJ/HxV3OyjidJklQhWbCqq4IC6HYKHPdc8f2z7tgXnjgbFszLOpkqmAZ1anLajpsy8szeHNOjNQ9P/pQ+/xnOnx+awvRv/b5IkiSVZMGq7ppvDscPh61PgLFXwfV94IvXsk6lCmiterX44+7tGHFGLw4oWp+7XvyI7S4Yxr8ef5NZc37IOp4kSVKFYMES1KwLfc+HQ4fA9zPgut4w9mpY7I1n9UstGtXlH/tszjOnb8+u7Ztz7ch36fnvYVz+7Dt8P39h1vEkSZIyZcHS/7TZCQa8ABv3gSfOgjv3g9mfZ51KFVSrpvW4pH8nHh/Uk64bN+E/T7/NducP46bR7zNvwaKs40mSJGXCgqWfq98MDr4b9rgYPhwDV20LbzySdSpVYJs1b8j1vy7i/pO68avmDTj3kdfpc+Fw7h33EQsXeRRUkiRVLxYs/VIEFB0NJ4yENdeHew+FoQPhh++zTqYKbKsNGnPXcV2589htaNawDr+/71V2vngkD0/6lMWLvd+aJEmqHixYWrZmm8Ixz0D30+Dl2+Da7eCTl7NOpQqu+yZNefCkblx3eGdqFAan3v0Ku18+mufe/ILkja0lSVIVZ8HS8tWoBTv9FY54uHgI9xt3gpEXwmKvsdGyRQQ7t2/O44O245KDOvL9/IUcfct4DrhmDC++542tJUlS1WXBUum07gkDRkPbfvDc3+CWPWDWR1mnUgVXWBDs3Wk9nv3t9py3Twc+/noOB103ll/f9BKvTvsm63iSJEl5Z8FS6dVtDPvfBPtcC5+/Clf3gFeHZJ1KlUDNwgIO3WZDRpzRmz/03YzJ02ax5xWjGXDHBKZOn511PEmSpLyxYGnlRMCW/YuPZq29Gdx3DNx3HMzzaIRWrE7NQo7fbmNGndmbQTu0YeTbM9j54pH8dvAkPv5qTtbxVEFFxK4R8VZETI2Is5Yyv3ZE3Jub/2JEtCox7+zc9LciYpeV2OZlEfFdiecbRsSzETE5IoZHRMsyeKuSpCrAgqVV07gVHPkY9P4jTLmv+GjWhy9knUqVRIM6NfnNTpsy6vd9OKZHax6e/Cl9/jOcPz80henfzss6niqQiCgErgR2A9oBB0dEuyUWOwb4OqW0CXAx8O/cuu2A/kB7YFfgqogoXNE2I6IIaLzEa1wI3JZS2gI4F/hnXt+oJKnKsGBp1RXWgO3PhKOfhIJCuGV3ePZcWLQg62SqJNaqV4s/7t6OEWf04oCi9bnzxY/Y7oJh/PuJN5k154es46li2BqYmlJ6L6X0A3APsNcSy+wF3Jp7PATYISIiN/2elNL8lNL7wNTc9pa5zVz5ugA4c4nXaAc8l3s8bCkZJEkCLFjKh/W7wImjoOMhMOo/cOPOMPPdrFOpEmnRqC7/2Gdznj19e3Zt35xrRrxLz/OHccVz7/D9/IVZx1O21gM+LvF8Wm7aUpdJKS0EvgGaLGfd5W3zFGBoSumzJV5jErBv7vE+QIOIaLJk2Ig4PiLGR8T4GTNmlOoNSpKqFguW8qN2A9jrSjjwNvjqPbimB0y4FbzvkVZCq6b1uKR/Jx4f1JOuGzXhwqfeZrvzh3HT6PeZt8BbA6hsRcS6wAHA5UuZ/Ttg+4h4Bdge+AT4xZcypXRdSqkopVTUrFmzMs0rSaqYLFjKr3Z7wYAXoGUXeHgg3HsYfO99j7RyNmvekOt/XcT9J3Vj03UacO4jr9PnwuHcO+4jFi5anHU8la9PgPVLPG+Zm7bUZSKiBtAImLmcdZc1vROwCTA1Ij4A1oiIqQAppU9TSvumlDoBf8xNm7X6b0+SVNVYsJR/jdaDwx+Enf8O7zwFV3eDqc9mnUqV0FYbNObu47ty57Hb0KxhHX5/36vsfPFIHpn8KYsXe3S0mhgHtImI1hFRi+JBK4YuscxQ4Ijc4/2B51JKKTe9f26UwdZAG+ClZW0zpfRoSql5SqlVSqkVMCc3cAYR0TQiftxnng3cVGbvWJJUqVmwVDYKCqDbqXDcc1B3TbhjX3jibFjgCHFaed03acqDJ3XjusM7U6MwOOWuV9jj8tEMe3M6ydNQq7TcNVWnAE8CbwCDU0qvRcS5EdEvt9iNQJPc0abTgbNy674GDAZeB54ATk4pLVrWNlcQpRfwVkS8DawDnJfHtylJqkKiIv7PSVFRURo/fnzWMZQvC+bC03+Gl66DtdvDfjfAOkuOsiyVzqLFiYcnfcpFT7/NR1/NoWjDxpyxy6/YZqNfjDegCioiJqSUirLOUdbcl0lS1bas/ZlHsFT2ataFvhfAIf+F76fDdb1g7NWw2GtptPIKC4K9O63HM6dvz9/37sBHX83hoOvG8uubXuLVad7wWpIkZcuCpfKz6c4wYAxs3BueOAvu3B9mf551KlVStWoUcFjXDRl5Zm/+0HczJk+bxZ5XjGbAHROYOn121vEkSVI1ZcFS+arfDA6+B3a/CD58oXgAjDcfzTqVKrE6NQs5fruNGXVmbwbt0IaRb89g54tH8rv/TuLjr+ZkHU+SJFUzFiyVvwjocgycMBIargf3HAIPD4Ifvs86mSqxBnVq8pudNmXkmb05pkdrhk76lD7/Gc45D01h+mwHV5EkSeXDgqXsNNsUjn0Wup9WfFPia7eDT17OOpUquSb1a/PH3dsx4oxeHFC0Pne8+BHbnT+Mfz/xJt/MWZB1PEmSVMVZsJStGrVgp7/CEQ8XjzZ4404w6j+weFHWyVTJtWhUl3/ssznPnr49u7RvzjUj3qXH+c9x5bCpfD9/YdbxJElSFWXBUsXQuicMeB7a7gnPngu37gmzPso6laqAVk3rcWn/Tjw2sCfbtG7CBU++xfYXDOPm599n/kKLvCRJyi8LliqOuo1h/5th72vgs8lwdQ94dUjWqVRFtG3RkBuOKOL+k7rRZu0G/PXh1+l9wXAGj/uYhYu8ZYAkScoPC5YqlgjoeDAMGA1rbwb3HQP3HQfzvL+R8mOrDRpz9/FdufPYbWjWsA5n3jeZnS8eySOTP2Xx4op343VJklS5WLBUMTVuBUc+Br3+AFPuKz6a9eGYrFOpCum+SVMePKkb1x7emRqFwSl3vcIel49m2JvTScmiJUmSVo0FSxVXYQ3o9Xs4+kkoKIBb+sKzf4NFjgSn/IgIdmnfnMcHbcfFB23Jd/MXctQt4zjw2jG89P5XWceTJEmVkAVLFd/6XeDE0bDlITDqQrhxZ5j5btapVIUUFgT7dGrJM6dvz9/37sCHM+dw4LVjOOKml3h1mqenSpKk0rNgqXKo3QD2vhIOuBW+eg+u6Qkv3waeyqU8qlWjgMO6bsiIM3rzh76bMWnaLPa8YjQn3TmBqdNnZx1PkiRVAhYsVS7t94YBL0DLzjD0VLj3MJjjqVzKr7q1Cjl+u40ZeWZvBu3QhhFvzWDni0fyu/9O4uOv5mQdT5IkVWAWLFU+jdaDwx+Cnf8Obz8JV20L7z6XdSpVQQ3r1OQ3O23KyDN7c3T31gyd9Cl9/jOccx6awvTZ87KOJ0mSKiALliqnggLodioc9xzUaQS37wNP/AEW+D+9yr8m9Wvzpz3aMeKMXuzfeX3uePEjtj9/OOc/8SbfzHHQFUmS9D8WLFVuLbaAE0bA1sfD2Cvh+j7wxetZp1IV1aJRXf657+Y8e/r27Nx+Ha4e8S49zn+OK4dN5fv5C7OOJ0mSKgALliq/mnWh7wVwyH/h++lwXS8Ye40DYKjMtGpaj0v7d+KxgT3ZpnUTLnjyLba/YBg3P/8+8xcuyjqeJEnKkAVLVcemO8OAMbBRL3ji93Dn/jD7i6xTqQpr26IhNxxRxH0DurHJ2vX568Ov0+fCEQwe9zELFy3OOp4kScqABUtVS/1mcMi9sPt/4IPn4ept4c3Hsk6lKq7zho25+7iu3HHMNjStX4sz75vMzpeM5NHJn7F4sUdSJUmqTlZYsCKiTkS8FBGTIuK1iPjrUpapHRH3RsTUiHgxIlqVmHd2bvpbEbFLnvNLvxQBXY4tvjar4Xpwz8Hw8Gnww/dZJ1MVFhH0aNOUB0/uzrWHd6ZGQXDyXS+z5xWjGfbWdJKnrEqSVC2U5gjWfKBPSmlLoCOwa0R0XWKZY4CvU0qbABcD/waIiHZAf6A9sCtwVUQU5im7tHzNfgXHPgvdB8GEW+Da7eDTV7JOpSouItilfXMeH7QdFx+0Jd/OW8BRN4/jwGvH8NL73rNNkqSqboUFKxX7Lve0Zu5nyV/F7gXcmns8BNghIiI3/Z6U0vyU0vvAVGDrvCSXSqNGLdjpXDhiKCyYCzfsCKMugsUORKCyVVgQ7NOpJc+e3ou/7d2BD2fO4cBrx3DETS8x5ZNvso4nSZLKSKmuwYqIwoiYCEwHnk4pvbjEIusBHwOklBYC3wBNSk7PmZabtrTXOD4ixkfE+BkzZqzUm5BWqPV2MOB52GwPePavcOueMOvjFa8nraZaNQo4vOuGjDijN2fvthmTps1ij8tHc9KdE5g6fXbW8SRJUp6VqmCllBallDoCLYGtI6JDvoOklK5LKRWllIqaNWuW781LULcxHHAL7H01fDYJru4Orw7JOpWqibq1Cjlh+40ZeWZvBu7QhhFvzWDni0fyu/9OYtrXc7KOJ0mS8mSlRhFMKc0ChlF8PVVJnwDrA0REDaARMLPk9JyWuWlSNiKg4yFw4ujia7TuOwbuPx7mecqWykfDOjU5fadNGXlmb47u3pqhkz6l94XD+cvQ15g+e17W8SRJ0moqzSiCzSJizdzjusBOwJtLLDYUOCL3eH/guVQ8ZNZQoH9ulMHWQBvgpTxll1bdWq3hqMeh19nFR7Gu7gEfjsk6laqRJvVr86c92jHijF7s33l9bh/7IdufP5zzn3iTb+YsyDqeJElaRaU5gtUCGBYRk4FxFF+D9UhEnBsR/XLL3Ag0iYipwOnAWQAppdeAwcDrwBPAySklRxdQxVBYA3qdBUc/CQUFcEtfeO7vsMj/uVX5adGoLv/cd3OeOX17dm6/DlePeJee5z/HlcOmMueHhVnHkyRJKykq4r1ZioqK0vjx47OOoepk/mx4/Pcw8U5YrzPsez002TjrVKqG3vjsW/7z1Fs888Z0mtavxSm9N+HgbTagdo2qc4eLiJiQUirKOkdZc18mSVXbsvZnK3UNllRl1W4Ae18FB9wKM9+Fa3rCy7dBBfwFhKq2ti0acsMRXbhvQDc2Wbs+f3n4dfpcOILB4z9m4aLFWceTJEkrYMGSSmq/Nwx4AVp2hqGnwuDDYY43h1X567xhY+4+rit3HLMNTevX4swhk9n5kpE8OvkzFi+2+EuSVFFZsKQlNVoPDn8IdvobvPUEXN0N3h2WdSpVQxFBjzZNefDk7lx7eGcKIzj5rpfZ84rRDHtrOhXxFG9Jkqo7C5a0NAUF0H0gHPcs1G4It+8NT/4RFs7POpmqoYhgl/bNeeK07bjowC35dt4Cjrp5HAdeO4aX3vcIqyRJFYkFS1qeFlvC8cOhy3Ew5gq4vg9MfyPrVKqmCguCfbdqybOn9+Jve3fgw5lzOPDaMRx580tM+cR7uUmSVBFYsKQVqbUG7H4hHDIYvvsCrt0eXrzWATCUmVo1Cji864aMOKM3Z++2GRM/nsUel4/m5DtfZur077KOJ0lStWbBkkpr012KB8DYqBc8fibcuT/M/iLrVKrG6tYq5ITtN2bkmb0ZuEMbhr81nZ0vHsEZ/53EtK/nZB1PkqRqyYIlrYz6a8Mh98Lu/4EPRsPV28Jbj2edStVcwzo1OX2nTRl5Zm+O7t6ahyZ9Sp8LR/CXoa8xY7bXDUqSVJ4sWNLKioAux8IJI6HhunB3f3j4NPjh+6yTqZprUr82f9qjHcN/14v9Oq/H7WM/ZLvzh3HBk2/yzZwFWceTJKlasGBJq6rZr+DYZ6HbQJhwS/G1WZ++knUqiXXXrMs/992CZ07fnp3arcOVw96l5/nPceWwqcz5YWHW8SRJqtIsWNLqqFEbdv4bHDG0+AjWDTvCqItg8aKsk0m0blqPyw7uxGMDe7J167W44Mm32O784dzy/PvMX+h3VJKksmDBkvKh9XYw4HnYbA949q9waz+Y9XHWqSQA2q3bkBuO6MJ9A7qxydr1+MvDr9PnwhEMHv8xCxctzjqeJElVigVLypc11oIDboG9r4bPJsLV3eHVIVmnkn7SecPG3H1cV24/Zmua1K/FmUMms8slI3ns1c9YvNjbDkiSlA8WLCmfIqDjIXDiKGi2Kdx3DNx/Asz7NutkEgARQc82zXjo5O5cc1hnCiI46c6X6XflaIa/NZ3k/d0kSVotFiypLKy1ERz1BPQ6G179L1zTHT4am3Uq6ScRwa4dmvPEadtx0YFb8s3cBRx58zgOunYs4z74Kut4kiRVWhYsqawU1oBeZ8HRT0AUwM27wXPnwSKHy1bFUVgQ7LtVS549vRd/27sDH8z8ngOuGcORN7/ElE++yTqeJEmVjgVLKmvrbw0njoYtD4aR58NNu8DMd7NOJf1MrRoFHN51Q0ac0Zuzd9uMiR/PYo/LR3PynS8zdfp3WceTJKnSsGBJ5aF2A9j7quJBMGa+C9f0hJdvB693UQVTt1YhJ2y/MSPP7M3APpsw7K3p7HzxCM4cMolpX8/JOp4kSRWeBUsqT+33gQEvwHpbwdBTYPDhMMfrXVTxNKxTk9N3/hUjz+zNUd1b8+DET+lz4Qieeu3zrKNJklShWbCk8tZoPfj1UNjpXHjrCbiiC4y9BhbOzzqZ9AtN69fm//Zox/Df9eKgLuvTpdVaWUeSJKlCs2BJWSgogO6D4LjnYO228MTv4YoimHQvLPbGr6p41l2zLn/buwON69XKOookSRWaBUvKUost4IiH4bD7oE4jeOB4uLYnvP2U12dJkiRVQhYsKWsRsMmOcPxI2O9G+OE7uOsAuGV3+Hhc1ukkSZK0EixYUkVRUACb7w8nj4O+F8KXb8ONO8I9h8KMt7JOJ0mSpFKwYEkVTY1asPVxMHAi9P4TvDcCruoKD50M30zLOp0kSZKWw4IlVVS168P2Z8CgSbDNAJg8GC7bCp76k0O7S5IkVVAWLKmiq9cEdv0HnDoBOuwHL1wBl3aEkRfCD99nnU6SJEklWLCkymLNDWCfq4tvVLxhN3jub3BZJxh3IyxakHU6SZIkYcGSKp912sEh98BRT0Dj1vDo6XDlNjDlfu+hJUmSlDELllRZbbgtHP0EHHwPFNaCIUfB9b3h3WFZJ5MkSaq2LFhSZRYBv9oNBjwPe18Nc2bC7XvDbXvBp69knU6SJKnasWBJVUFBIXQ8pHggjF3+CZ9Nhut6wX+PhJnvZp1OkiSp2rBgSVVJjdqw7UnFQ7tvdya8/RRc0QUePg1mf551OkmSpCrPgiVVRXUaQp8/wqCJUHQ0vHJ78dDuz/wV5s7KOJwkSVLVZcGSqrL6a8PuF8Ip42Cz3WH0RXBZR3j+MlgwL+t0kiRJVY4FS6oO1toI9r8RThgJ624FT/8fXL4VvHw7LFqYdTpJkqQqw4IlVScttoTD74cjHoYGzWHoKXB1N3jjEUgp63SSJEmVngVLqo5abwfHPgsH3g5pMdx7KNy4M3zwfNbJJEmSKjULllRdRUC7fnDSWNjzMvhmGtzSF+7YHz5/Net0kiRJlZIFS6ruCmtA5yNg4Muw419h2ktwTU+47zj4+oOs00mSJFUqFixJxWrWhR6nFd9Dq/sgeGMoXF4Ej50J383IOp0kSVKlYMGS9HN1G8NOf4WBr0DHQ2DcDcVDuw/7J8yfnXU6SZKkCs2CJWnpGq4L/S6Dk1+ETXaAEf8qvlnx2Gtg4fys00mSJFVIFixJy9e0DRx4Gxz7HKzdFp74PVxRBJPuhcWLs04nSZJUoViwJJVOy87F98867H6osyY8cDxc2xPeftJ7aEmSJOVYsCSVXkTx6YLHj4D9boQfvoe7DoSb+8LHL2WdTpIkKXMWLEkrr6AANt8fTn4J+l4IM6fCjTvB3YfA9DezTidJkpQZC5akVVejFmx9XPGIg73/BO+PhKu3hQdPLr5xsSRJUjVjwZK0+mrXh+3PKL6H1jYD4NXBcNlW8OQfYc5XWaeTJEkqNxYsSflTrwns+g84dQJ02A/GXAmXbgkjLyy+XkuSJKmKs2BJyr81N4B9roYBL0CrHvDc3+CyTsU3LV60IOt0kiRJZcaCJansrNMODr4bjn4S1toIHv0tXLk1TLnPe2hJkqQqyYIlqext0BWOehwOvhdq1IEhR8P1veHd57JOJkmSlFcrLFgRsX5EDIuI1yPitYgYtJRlzoiIibmfKRGxKCLWys37ICJezc0bXxZvQlIlEAG/2hVOHA17X1M8+MXt+8Ct/eCTl7NOJ0mSlBelOYK1EPhtSqkd0BU4OSLalVwgpXRBSqljSqkjcDYwIqVUcuiw3rn5RfkKLqmSKiiEjgfDqeNhl3/CF1OKj2YNPgK+nJp1OlVAEbFrRLwVEVMj4qylzK8dEffm5r8YEa1KzDs7N/2tiNhlJbZ5WUR8V+L5BrlfNr4SEZMjom8ZvFVJUhWwwoKVUvospfRy7vFs4A1gveWscjBwd37iSaqyatSGbU+CgRNhuzPhnaeLr896+DT49rOs06mCiIhC4EpgN6AdcPCSv+QDjgG+TiltAlwM/Du3bjugP9Ae2BW4KiIKV7TNiCgCGi/xGn8CBqeUOuW2eVVe36gkqcpYqWuwcr8V7AS8uIz5a1C8E7uvxOQEPBUREyLi+OVs+/iIGB8R42fMmLEysSRVZnUaQp8/wqCJ0OUYeOWO4hEHn/kLzJ2VcThVAFsDU1NK76WUfgDuAfZaYpm9gFtzj4cAO0RE5Kbfk1Kan1J6H5ia294yt5krXxcAZy7xGglomHvcCPg0j+9RklSFlLpgRUR9iovTaSmlb5ex2J7A80ucHtgjpbQVxb8pPDkitlvaiiml61JKRSmlombNmpU2lqSqov7a0PcCOGUctN0DRl9cfA+t5y+FBXOzTqfsrAd8XOL5NH55FsVPy6SUFgLfAE2Ws+7ytnkKMDSltORh1L8Ah0XENOAx4NRVezuSpKquVAUrImpSXK7uTCndv5xF+7PE6YEppU9yf04HHqD4N4eStHRrtYb9boATRkHLInj6z3B5Z3j5Nli0MOt0qsIiYl3gAODypcw+GLglpdQS6AvcHhG/2Id6NoYkqTSjCAZwI/BGSumi5SzXCNgeeKjEtHoR0eDHx8DOwJTVDS2pGmixBRx2HxzxCDRoDkNPhau7wRsPQ0pZp1P5+QRYv8TzlrlpS10mImpQfArfzOWsu6zpnYBNgKkR8QGwRkT8OPLKMcBggJTSGKAO0HTJsJ6NIUkqzRGs7sDhQJ8SQ7H3jYgTI+LEEsvtAzyVUvq+xLR1gNERMQl4CXg0pfRE3tJLqvpa94Rjn4UDb4e0GO49DG7cCT4YnXUylY9xQJuIaB0RtSg+U2LoEssMBY7IPd4feC6llHLT++dGGWwNtKF4X7TUbaaUHk0pNU8ptUoptQLm5AbOAPgI2AEgItpSXLA8RCVJ+oUaK1ogpTQaiFIsdwtwyxLT3gO2XMVsklQsAtr1g1/1hYl3wvB/wS27wyY7wY7nQPPNs06oMpJSWhgRpwBPAoXATSml1yLiXGB8SmkoxWdZ3J472vQVxYWJ3HKDgdcpvuXIySmlRQBL2+YKovwWuD4ifkPxgBdH5kqcJEk/ExVx/1BUVJTGj/eexJKWYcFceOk6GHURzPsGNj8Aev+h+PotVXgRMaE63BfRfZkkVW3L2p+t1DDtklQh1KwL3QcVD+3e47Ti67Ku6AKPnQHfedaWJEnKjgVLUuVVtzHs+BcY+Ap0OhTG3Vg8tPuwf8C8Zd1NQpIkqexYsCRVfg1bwJ6XwskvQpsdYcS/4bKOMPZqWDg/63SSJKkasWBJqjqatoEDb4PjnoN12sMTZ8EVRTDpHli8KOt0kiSpGrBgSap61usMvx4Kh90PddaEB06Aa3rC2096Dy1JklSmLFiSqqYI2GQHOH4E7HcjLJgDdx0IN/eFj17MOp0kSaqiLFiSqraCAth8fzhlHOz+H5g5FW7aGe4+GKa/kXU6SZJUxViwJFUPhTWhy7HFQ7v3+RN8MBqu7gYPngSzPs46nSRJqiIsWJKql1r1YLszYOBE6HoSvPpfuLwzPPlHmPNV1ukkSVIlZ8GSVD3VawK7nAenvlx8CuHYq4rvoTXyAvjh+6zTSZKkSsqCJal6W3N92PsqGPACtOoBz/0dLusE426ARQuyTidJkioZC5YkAazdFg6+G45+EtbaCB79LVy5NUy5DxYvzjqdJEmqJCxYklTSBl3hqMfhkMFQoy4MORqu7wXvPpd1MkmSVAlYsCRpSRGw6S5w4ijY51qY8zXcvg/c2g8+mZB1OkmSVIFZsCRpWQoKYcv+cOp42PVf8MUUuL4PDP41fDk163SSJKkCsmBJ0orUqA1dBxQP7b797+GdZ4qvz3p4EHz7WdbpJElSBWLBkqTSqtMQev+h+GbFXY6BV+4sHnHwmb/A3FkZh5MkSRWBBUuSVlb9taHvBXDKOGi7B4y+uPgeWs9fCgvmZp1OkiRlyIIlSatqrdaw3w1wwihoWQRP/xku2wpevg0WLcw6nSRJyoAFS5JWV4st4LD74IhHoOG6MPRUuHpbeH0opJR1OkmSVI4sWJKUL617wrHPwEF3FD8ffDjcsCO8PyrbXJIkqdxYsCQpnyKg7Z4wYAz0uxy+/RRu3QPu2A8+m5x1OkmSVMYsWJJUFgprwFa/hoEvw07nwrTxcG1PuO9Y+Or9rNNJkqQyYsGSpLJUsy50HwSDJkGP38Abj8AVXeCxM+C76VmnkyRJeWbBkqTyUHdN2PEvMPAV6HQojLsRLu0Iw/4B877NOJwkScoXC5YklaeGLWDPS+HkF6HNTjDi33BZRxhzFSycn3U6SZK0mixYkpSFpm3gwFvhuOdgnQ7w5NlweRFMvBsWL8o6nSRJWkUWLEnK0nqd4YihcPgDsEZjePBEuKYHvPWE99CSJKkSsmBJUkWwcR84bjjsfxMsnAd3HwQ37wYfvZh1MkmStBIsWJJUURQUQIf94OSXYPf/wFfvwU07w90Hw/Q3sk4nSZJKwYIlSRVNYU3ocmzxiIN9/gQfjIartoUHT4JZH2edTpIkLYcFS5Iqqlr1YLsziu+hte3J8OoQuLwzPPlH+H5m1ukkSdJSWLAkqaJbYy3Y5Tw4dQJsfgCMvap4aPcRF8AP32edTpIklWDBkqTKYs31Ye8rYcAL0KonDPt78c2KX7oeFi3IOp0kScKCJUmVz9pt4eC74OinoMkm8Njv4IouxacQLl6cdTpJkqo1C5YkVVYbbANHPQaHDIaaa8B9x8B128PUZ72HliRJGbFgSVJlFgGb7gInjoJ9roW5s+COfeG2fvDJhKzTSZJU7ViwJKkqKCiELfvDqeNh13/BF6/B9X1g8K/hy3eyTidJUrVhwZKkqqRGbeg6oHho9+3PKj5d8MptYOhA+PbTrNNJklTlWbAkqSqq3QB6nw0DJxbftHjiXXBZJ3j6HJj7ddbpJEmqsixYklSV1W8Gfc+HU8ZB237w/KXFQ7uPvgQWzM06nSRJVY4FS5Kqg7Vaw37XFw+G0bILPHMOXLYVTLgVFi3MOp0kSVWGBUuSqpPmm8NhQ+DIR6HhuvDwQLiqK7w+1KHdJUnKAwuWJFVHrXrAsc/AQXcUD/U++HC4YQd4f2TWySRJqtQsWJJUXUVA2z1hwBjodwXM/hxu3RNu3xc+m5R1OkmSKiULliRVd4U1YKvD4dQJsNPfim9QfO12MOQY+Oq9rNNJklSpWLAkScVq1oXuA4vvodXjdHjzUbiiCzz6O/huetbpJEmqFCxYkqSfq7sm7HgODHwFOh0O428qHtr9ufNg3rdZp5MkqUKzYEmSlq5hC9jzEjj5JWizE4w8H94blnUqSZIqtBpZB5AkVXBNN4EDb4XPp8A67bNOI0lShWbBkiSVTvMOWSeQJKnC8xRBSZIkScqTFRasiFg/IoZFxOsR8VpEDFrKMr0i4puImJj7+XOJebtGxFsRMTUizsr3G5AkSZKkiqI0pwguBH6bUno5IhoAEyLi6ZTS60ssNyqltEfJCRFRCFwJ7ARMA8ZFxNClrCtJkiRJld4Kj2CllD5LKb2cezwbeANYr5Tb3xqYmlJ6L6X0A3APsNeqhpUkSZKkimylrsGKiFZAJ+DFpczeNiImRcTjEfHjMFPrAR+XWGYapS9nkiRJklSplHoUwYioD9wHnJZSWvJOky8DG6aUvouIvsCDQJuVCRIRxwPHA2ywwQYrs6okSZIkVQilOoIVETUpLld3ppTuX3J+SunblNJ3ucePATUjoinwCbB+iUVb5qb9QkrpupRSUUqpqFmzZiv5NiRJkiQpe6UZRTCAG4E3UkoXLWOZ5rnliIitc9udCYwD2kRE64ioBfQHhuYrvCRJkiRVJKU5RbA7cDjwakRMzE37A7ABQErpGmB/YEBELATmAv1TSglYGBGnAE8ChcBNKaXX8vsWJEmSJKliWGHBSimNBmIFy1wBXLGMeY8Bj61SOkmSJEmqRFZqFEFJkiRJ0rJZsCRJkiQpTyxYkiRJkpQnUTwWRcUSETOAD1dzM02BL/MQpzxUlqyVJSeYtayYtWxUt6wbppSq/P048rQvg8rz/agsOcGsZcWsZcOs+ZevnEvdn1XIgpUPETE+pVSUdY7SqCxZK0tOMGtZMWvZMKuWp7J85pUlJ5i1rJi1bJg1/8o6p6cISpIkSVKeWLAkSZIkKU+qcsG6LusAK6GyZK0sOcGsZcWsZcOsWp7K8plXlpxg1rJi1rJh1vwr05xV9hosSZIkSSpvVfkIliRJkiSVKwuWJEmSJOVJpStYEbFrRLwVEVMj4qylzK8dEffm5r8YEa1KzDs7N/2tiNilAmQ9PSJej4jJEfFsRGxYYt6iiJiY+xlaAbIeGREzSmQ6tsS8IyLindzPERUg68Ulcr4dEbNKzCu3zzUiboqI6RExZRnzIyIuy72PyRGxVYl55f2ZrijrobmMr0bECxGxZYl5H+SmT4yI8RUga6+I+KbE3/OfS8xb7ncng6xnlMg5Jff9XCs3r9w+14hYPyKG5f49ei0iBi1lmQrzfa0q3J9lltX92crndH+WTVb3Zyufs2Lsz1JKleYHKATeBTYCagGTgHZLLHMScE3ucX/g3tzjdrnlawOtc9spzDhrb2CN3OMBP2bNPf+ugn2uRwJXLGXdtYD3cn82zj1unGXWJZY/Fbgpo891O2ArYMoy5vcFHgcC6Aq8mMVnWsqs3X7MAOz2Y9bc8w+AphXoc+0FPLK6353yyLrEsnsCz2XxuQItgK1yjxsAby/l34AK832tCj+l/HfX/VnZZD0S92crm9X9WTZZe+H+bGVzVoj9WWU7grU1MDWl9F5K6QfgHmCvJZbZC7g193gIsENERG76PSml+Sml94Gpue1lljWlNCylNCf3dCzQsgzzLE9pPtdl2QV4OqX0VUrpa+BpYNcyygkrn/Vg4O4yzLNMKaWRwFfLWWQv4LZUbCywZkS0oPw/0xVmTSm9kMsC2X5XS/O5LsvqfM9XyUpmzfK7+llK6eXc49nAG8B6SyxWYb6vVYT7s7Lh/qwMuD8rG+7P8q+i7M8qW8FaD/i4xPNp/PJD+2mZlNJC4BugSSnXzaeVfb1jKG7TP6oTEeMjYmxE7F0G+Uoqbdb9codSh0TE+iu5br6U+vVyp6i0Bp4rMbk8P9cVWdZ7Ke/PdGUt+V1NwFMRMSEijs8o05K2jYhJEfF4RLTPTauwn2tErEHxP+L3lZicyecaxaehdQJeXGJWZf2+VlTuz8qG+7NsVNZ/H9yf5Zn7s2I1VmUl5VdEHAYUAduXmLxhSumTiNgIeC4iXk0pvZtNQgAeBu5OKc2PiBMo/q1qnwzzlEZ/YEhKaVGJaRXtc61UIqI3xTukHiUm98h9pmsDT0fEm7nfdGXlZYr/nr+LiL7Ag0CbDPOUxp7A8ymlkr8dLPfPNSLqU7xTPC2l9G1ZvpaqJvdnZcb9WZ65Pysz7s+ofEewPgHWL/G8ZW7aUpeJiBpAI2BmKdfNp1K9XkTsCPwR6JdSmv/j9JTSJ7k/3wOGU9zAM8uaUppZIt8NQOfSrptnK/N6/VniEHU5f64rsqz3Ut6faalExBYU/93vlVKa+eP0Ep/pdOAByvZUpRVKKX2bUvou9/gxoGZENKWCfq45y/uulsvnGhE1Kd4Z3ZlSun8pi1Sq72sl4P4so6zuz8pEpfr3wf1ZmXJ/BpVukIsaFF9w1pr/XdTXfollTubnFwUPzj1uz88vCn6Psr0ouDRZO1F8kWKbJaY3BmrnHjcF3qEML14sZdYWJR7vA4xN/7sg8P1c5sa5x2tlmTW33GYUX1QZWX2uuddpxbIvXt2dn19k+VIWn2kps25A8XUe3ZaYXg9oUOLxC8CuGWdt/uPfO8X/iH+U+4xL9d0pz6y5+Y0oPq+9Xlafa+7zuQ24ZDnLVKjva2X/KeW/u+7Pyiar+7NVy7u8f3cr1L8PK8jq/qwMsubmuz/78TXK+i+jDD64vhSPCPIu8MfctHMp/o0ZQB3gv7n/eF4CNiqx7h9z670F7FYBsj4DfAFMzP0MzU3vBrya+w/mVeCYCpD1n8BruUzDgM1KrHt07vOeChyVddbc878A/1pivXL9XCn+Dc5nwAKKz+M9BjgRODE3P4Arc+/jVaAow890RVlvAL4u8V0dn5u+Ue7znJT7fvyxAmQ9pcR3dSwldqJL++5kmTW3zJEUD1hQcr1y/VwpPkUmAZNL/B33rajf16rys6J/y3B/VlZZ3Z+tfE73Z9lkdX+28jkrxP7sx1YsSZIkSVpNle0aLEmSJEmqsCxYkiRJkpQnFixJkiRJyhMLliRJkiTliQVLkiRJkvLEgiVVAhHRKyIeyTqHJEmryn2ZqgsLliRJkiTliQVLyqOIOCwiXoqIiRFxbUQURsR3EXFxRLwWEc9GRLPcsh0jYmxETI6IByKicW76JhHxTERMioiXI2Lj3ObrR8SQiHgzIu6MiMjsjUqSqiz3ZdLqsWBJeRIRbYGDgO4ppY7AIuBQoB7Fd4pvD4wAzsmtchvw+5TSFhTfSfzH6XcCV6aUtgS6UXzndIBOwGlAO4rvjN69jN+SJKmacV8mrb4aWQeQqpAdgM7AuNwv5OoC04HFwL25Ze4A7o+IRsCaKaURuem3Av+NiAbAeimlBwBSSvMActt7KaU0Lfd8ItAKGF3m70qSVJ24L5NWkwVLyp8Abk0pnf2ziRH/t8RyaRW3P7/E40X4368kKf/cl0mryVMEpfx5Ftg/ItYGiIi1ImJDiv872z+3zCHA6JTSN8DXEdEzN/1wYERKaTYwLSL2zm2jdkSsUZ5vQpJUrbkvk1aTvzWQ8iSl9HpE/Al4KiIKgAXAycD3wNa5edMpPrcd4AjgmtxO5z3gqNz0w4FrI+Lc3DYOKMe3IUmqxtyXSasvUlrVI7ySSiMivksp1c86hyRJq8p9mVR6niIoSZIkSXniESxJkiRJyhOPYEmSJElSnliwJEmSJClPLFiSJEmSlCcWLEmSJEnKEwuWJEmSJOXJ/wP2y7z+PbBG7wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training:   0%|                                          | 0/32 [00:00<?, ?it/s]"
     ]
    }
   ],
   "source": [
    "from src.data import get_data_loaders\n",
    "from src.optimization import get_optimizer, get_loss\n",
    "from src.train import optimize\n",
    "from src.transfer import get_model_transfer_learning\n",
    "\n",
    "# Get a model using get_model_transfer_learning. Use one of the names reported here:\n",
    "# https://pytorch.org/vision/0.10/models.html\n",
    "# For example, if you want to load ResNet 18, use \"resnet18\"\n",
    "# NOTE: use the hyperparameters defined in the previous cell, do NOT copy/paste the\n",
    "# values\n",
    "model_transfer = # YOUR CODE HERE\n",
    "\n",
    "# train the model\n",
    "data_loaders = get_data_loaders(batch_size=batch_size)\n",
    "optimizer = get_optimizer(\n",
    "    model_transfer,\n",
    "    learning_rate=learning_rate,\n",
    "    optimizer=opt,\n",
    "    weight_decay=weight_decay,\n",
    ")\n",
    "loss = get_loss()\n",
    "\n",
    "optimize(\n",
    "    data_loaders,\n",
    "    model_transfer,\n",
    "    optimizer,\n",
    "    loss,\n",
    "    n_epochs=num_epochs,\n",
    "    save_path=\"checkpoints/model_transfer.pt\",\n",
    "    interactive_tracking=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"static_images/icons/noun-question-mark-869751.png\" alt=\"?\" style=\"width:25px\"/> __Question:__ Outline the steps you took to get to your final CNN architecture and your reasoning at each step.  Describe why you think the architecture is suitable for the current problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"static_images/icons/noun-answer-3361020.png\" alt=\">\" style=\"width:25px\"/>  __Answer:__ I decided to use ResNet18 for the base of my model, since it performs fairly well on ImageNet and is not too large of a model. Also, since ResNet18 was trained for the ImageNet task, it is a good model to use for this landmark classificaiton task, since both ImageNet and this landmark task use images of natural scenes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now play with the hyperparameters and see which performance you can get on the validation set. You should get at least 60% for a passing grade, but a good model choice and a good training strategy could get you up to 80% or so. Let's see how close you can get!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## <img src=\"static_images/icons/noun-advance-2109145.png\" alt=\">\" style=\"width:50px\"/> Step 3: Test the Model\n",
    "\n",
    "Try out your model on the test dataset of landmark images. Use the code cell below to calculate and print the test loss and accuracy.  Ensure that your test accuracy is greater than 60% and matches more or less what you got on the validation set (otherwise you're overfitting!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Testing: 100%|██████████████████████████████████| 10/10 [00:05<00:00,  1.82it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Loss: 2.868267\n",
      "\n",
      "\n",
      "Test Accuracy: 37% (470/1250)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from src.train import one_epoch_test\n",
    "from src.transfer import get_model_transfer_learning\n",
    "\n",
    "model_transfer = get_model_transfer_learning(\"resnet18\", n_classes=num_classes)\n",
    "# Load saved weights\n",
    "model_transfer.load_state_dict(torch.load('checkpoints/model_transfer.pt'))\n",
    "\n",
    "one_epoch_test(data_loaders['test'], model_transfer, loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## <img src=\"static_images/icons/noun-advance-2109145.png\" alt=\">\" style=\"width:50px\"/> Step 4: Export using torchscript\n",
    "\n",
    "Now, just like we did with our original model, we export the best fit model using torchscript so that it can be used in our application:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reusing cached mean and std for landmark_images\n"
     ]
    }
   ],
   "source": [
    "from src.predictor import Predictor\n",
    "from src.helpers import compute_mean_and_std\n",
    "\n",
    "# First let's get the class names from our data loaders\n",
    "class_names = data_loaders[\"train\"].dataset.classes\n",
    "\n",
    "# Then let's move the model_transfer to the CPU\n",
    "# (we don't need GPU for inference)\n",
    "model_transfer = model_transfer.cpu()\n",
    "# Let's make sure we use the right weights by loading the\n",
    "# best weights we have found during training\n",
    "# NOTE: remember to use map_location='cpu' so the weights\n",
    "# are loaded on the CPU (and not the GPU)\n",
    "model_transfer.load_state_dict(\n",
    "    torch.load(\"checkpoints/model_transfer.pt\", map_location=\"cpu\")\n",
    ")\n",
    "\n",
    "# Let's wrap our model using the predictor class\n",
    "mean, std = compute_mean_and_std()\n",
    "predictor = Predictor(model_transfer, class_names, mean, std).cpu()\n",
    "\n",
    "# Export using torch.jit.script\n",
    "scripted_predictor = torch.jit.script(predictor)\n",
    "scripted_predictor.save(\"checkpoints/transfer_exported.pt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from src.predictor import predictor_test\n",
    "from src.helpers import plot_confusion_matrix\n",
    "\n",
    "model_reloaded = torch.jit.load(\"checkpoints/transfer_exported.pt\")\n",
    "\n",
    "pred, truth = predictor_test(data_loaders['test'], model_reloaded)\n",
    "\n",
    "plot_confusion_matrix(pred, truth)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:udacity_starter]",
   "language": "python",
   "name": "conda-env-udacity_starter-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}