2b. Basic neural net. Practice.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic neural net. Practice\n",
    "# Keras API\n",
    "\n",
    "Source: \n",
    "- https://github.com/Hvass-Labs/TensorFlow-Tutorials\n",
    "- https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This tutorial demonstrates how to build a simple neural network using Keras with Tensorflow backend. After loading the so-called MNIST data-set with images of hand-written digits, we define and optimize a simple mathematical model. The results are then plotted and discussed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import math\n",
    "from helpers import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to import several things from Keras. Note the long import-statements. This might be a bug. Hopefully it will be possible to write shorter and more elegant lines in the future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from tf.keras.models import Sequential  # This does not work!\n",
    "from tensorflow.python.keras.models import Sequential\n",
    "from tensorflow.python.keras.layers import InputLayer, Input\n",
    "from tensorflow.python.keras.layers import Dense, Flatten"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This was developed using Python 3.6 (Anaconda) and TensorFlow version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.4.0'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.0.8-tf'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/MNIST/train-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/train-labels-idx1-ubyte.gz\n",
      "Extracting data/MNIST/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/MNIST/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "data = input_data.read_data_sets('data/MNIST/', one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set has now been loaded and consists of 70,000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Test-set:\t\t10000\n",
      "- Validation-set:\t5000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(len(data.train.labels)))\n",
    "print(\"- Test-set:\\t\\t{}\".format(len(data.test.labels)))\n",
    "print(\"- Validation-set:\\t{}\".format(len(data.validation.labels)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The class-labels are One-Hot encoded, which means that each label is a vector with 10 elements, all of which are zero except for one element. The index of this one element is the class-number, that is, the digit shown in the associated image. We also need the class-numbers as integers for the test-set, so we calculate it now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0., ...,  1.,  0.,  0.],\n",
       "       [ 0.,  0.,  1., ...,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0., ...,  0.,  0.,  0.],\n",
       "       ..., \n",
       "       [ 0.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0., ...,  0.,  0.,  0.]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.test.labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 10)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.test.labels.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data.test.cls = np.argmax(data.test.labels, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data dimensions are used in several places in the source-code below. They are defined once so we can use these variables instead of numbers throughout the source-code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# We know that MNIST images are 28 pixels in each dimension.\n",
    "img_size = 28\n",
    "\n",
    "# Images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = img_size * img_size\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "# This is used for plotting the images.\n",
    "img_shape = (img_size, img_size)\n",
    "\n",
    "# Tuple with height, width and depth used to reshape arrays.\n",
    "# This is used for reshaping in Keras.\n",
    "img_shape_full = (img_size, img_size, 1)\n",
    "\n",
    "# Number of colour channels for the images: 1 channel for gray-scale.\n",
    "num_channels = 1\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot a few images to see if data is correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4\nEGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7\n+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgi\nAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6q\neqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfe\neAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMyd\nOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABO\nOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy\n411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgw\nILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4j\nnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cC\nJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb\n4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVA\nMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy\n47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir\n2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBj\nw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDu\nvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUm\niYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6S\nH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK0\n9/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+v\neeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRe\nkpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXU\nUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8\nE4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+\newD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bs\ntNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2\nQNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32e\nxx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv\n3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrW\nfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23\nfoXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S\n3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJx\nU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDm\nlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2\nOOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c\n6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pm\nR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8V\neQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLye\nwIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQX\nN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBna\nPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfM\nrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4n\nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/\nVMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoM\nRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAAN\nGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++\nfQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwI\nwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXr\ngS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmA\nDRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohS\nSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYi\nIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVq\ne+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23\n334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCL\nFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9\nNbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2\neOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+\nM/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4\nf+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBY\nvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6\nUSzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDK\nUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQ\ndNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VH\nHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9V\nIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdf\nfHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7Z\njtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu\n3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFB\nlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23K\nDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQ\nTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp\n0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltu\nSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9Xwv\ngMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpD\nERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGI\nCBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90\nCX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq\n49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZ\nLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CR\nzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1z\nblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAm\noW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38wa\nAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlR\nGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZS\nktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr\n+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogA\nqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcY\nlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxT\nTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI\n3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0a\nAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp3\n3XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9\njooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9Fgju\nEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD8\n4he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt\n27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp\n37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneA\nBjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW\n91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBD\nZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjk\nkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBE\nDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcf\nfzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2p\nfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfE\nY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUX\nAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQW\nu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0b\nAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPH\njBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDw\nusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkD\nBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwag\nVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+e\nOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ\n4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQR\noR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5\nEgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5\nfgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJ\nOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKO\nZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4B\noLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5\nnh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0b\nN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcA\nsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjs\nM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/m\nnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz\n24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2yd\nszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY\n7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhR\nebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cya\nOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543sze\ncs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiw\nug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoEL\ngOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/\nSuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5\nlRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd1de05e630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.test.images[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.test.cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true, img_shape=img_shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequential Model\n",
    "\n",
    "The Keras API has two modes of constructing Neural Networks. The simplest is the Sequential Model which only allows for the layers to be added in sequence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "Total params: 0\n",
      "Trainable params: 0\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Start construction of the Keras Sequential model.\n",
    "model = Sequential()\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 784)               0         \n",
      "=================================================================\n",
      "Total params: 0\n",
      "Trainable params: 0\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Add an input layer which is similar to a feed_dict in TensorFlow.\n",
    "# Note that the input-shape must be a tuple containing the image-size.\n",
    "model.add(InputLayer(input_shape=(img_size_flat,)))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 784)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 128)               100480    \n",
      "=================================================================\n",
      "Total params: 100,480\n",
      "Trainable params: 100,480\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# First fully-connected / dense layer\n",
    "model.add(Dense(128))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100352"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "128*784"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 784)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 128)               100480    \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 101,770\n",
      "Trainable params: 101,770\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Last fully-connected / dense layer with softmax-activation\n",
    "# for use in classification.\n",
    "model.add(Dense(num_classes, activation='softmax'))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Compilation\n",
    "\n",
    "The Neural Network has now been defined and must be finalized by adding a loss-function, optimizer and performance metrics. This is called model \"compilation\" in Keras.\n",
    "\n",
    "We can either define the optimizer using a string, or if we want more control of its parameters then we need to instantiate an object. For example, we can set the learning-rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from tensorflow.python.keras.optimizers import Adam\n",
    "\n",
    "optimizer = Adam(lr=1e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a classification-problem such as MNIST which has 10 possible classes, we need to use the loss-function called `categorical_crossentropy`. The performance metric we are interested in is the classification accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer=optimizer,\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "Now that the model has been fully defined with loss-function and optimizer, we can train it. This function takes numpy-arrays and performs the given number of training epochs using the given batch-size. An epoch is one full use of the entire training-set. So for 10 epochs we would iterate randomly over the entire training-set 10 times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 52250 samples, validate on 2750 samples\n",
      "Epoch 1/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.4130 - acc: 0.8822 - val_loss: 0.2252 - val_acc: 0.9389\n",
      "Epoch 2/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2970 - acc: 0.9172 - val_loss: 0.2060 - val_acc: 0.9433\n",
      "Epoch 3/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2853 - acc: 0.9189 - val_loss: 0.2212 - val_acc: 0.9411\n",
      "Epoch 4/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2773 - acc: 0.9224 - val_loss: 0.2197 - val_acc: 0.9495\n",
      "Epoch 5/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2739 - acc: 0.9233 - val_loss: 0.2113 - val_acc: 0.9422\n",
      "Epoch 6/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2684 - acc: 0.9236 - val_loss: 0.2151 - val_acc: 0.9389\n",
      "Epoch 7/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2653 - acc: 0.9256 - val_loss: 0.2115 - val_acc: 0.9458\n",
      "Epoch 8/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2619 - acc: 0.9273 - val_loss: 0.2123 - val_acc: 0.9455\n",
      "Epoch 9/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2588 - acc: 0.9279 - val_loss: 0.2112 - val_acc: 0.9425\n",
      "Epoch 10/10\n",
      "52250/52250 [==============================] - 1s - loss: 0.2584 - acc: 0.9270 - val_loss: 0.2159 - val_acc: 0.9422\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(x=data.train.images,\n",
    "          y=data.train.labels, validation_split=0.05,\n",
    "          epochs=10, batch_size=128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation\n",
    "\n",
    "Now that the model has been trained we can test its performance on the test-set. This also uses numpy-arrays as input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 9152/10000 [==========================>...] - ETA: 0s"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(x=data.test.images,\n",
    "                        y=data.test.labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can print all the performance metrics for the test-set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.289508258358\n",
      "acc 0.9228\n"
     ]
    }
   ],
   "source": [
    "for name, value in zip(model.metrics_names, result):\n",
    "    print(name, value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can just print the classification accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "acc: 92.28%\n"
     ]
    }
   ],
   "source": [
    "print(\"{0}: {1:.2%}\".format(model.metrics_names[1], result[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['val_acc', 'loss', 'val_loss', 'acc'])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history_dict = history.history\n",
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_values = history_dict['loss']\n",
    "val_loss_values = history_dict['val_loss']\n",
    "acc_values = history_dict['acc']\n",
    "val_acc_values = history_dict['val_acc']\n",
    "epochs = range(1, len(loss_values) + 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7fd1dd902a20>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE9pJREFUeJzt3X+s3Xd93/HnKw6hGAINyx2COPY11O0W+iOpjlI6tHZq\nQ0lHFWdjGkFmyjokiypp2Kg6glKJNF0mmk7RRuutuG0oa02tAm3ndSppFqBbtVJ8DVnSmEU4Jk6c\nhcX8KD9mlsTJe3+cr+NzE9ufa/ue+/1en+dDOjrn+znf77nv+5V9Xvfz+Xx/pKqQJOlkzum7AEnS\n8BkWkqQmw0KS1GRYSJKaDAtJUpNhIUlqMiwkSU2GhSSpybCQJDWd23cBy+XCCy+s+fn5vsuQpFVl\nz549X66qudZ6Z01YzM/Ps7Cw0HcZkrSqJDmwlPUchpIkNRkWkqQmw0KS1GRYSJKaDAtJUtPMh8WO\nHTA/D+ecM37esaPviiRpeM6aQ2dPx44dsHUrHD48Xj5wYLwMsGVLf3VJ0tDMdM/ippuOBcVRhw+P\n2yVJx8x0WDz88Km1S9KsmumwWL/+1NolaVbNdFjceiusXbu4be3acbsk6ZiZDostW2D7dtiwAZLx\n8/btTm5L0nPN9NFQMA4Gw0GSTm6mexaSpKUxLCRJTYaFJKnJsJAkNU01LJJcmeSBJPuS3HiS9d6c\npJKMJtre0233QJI3TrNOSdLJTe1oqCRrgG3AG4CDwO4ku6pq73PWOx94J/CXE22XANcArwVeBfzX\nJN9dVU9Pq15J0olNs2dxObCvqvZX1ZPATmDzcdb7JeCXgf830bYZ2FlVT1TVF4F93edJknowzbC4\nCHhkYvlg1/asJD8IXFxV/+VUt5UkrZzeJriTnAPcDvzcGXzG1iQLSRYOHTq0fMVJkhaZZlg8Clw8\nsbyuazvqfOB7gU8leQh4HbCrm+RubQtAVW2vqlFVjebm5pa5fEnSUdMMi93ApiQbk5zHeMJ619E3\nq+rrVXVhVc1X1TzwaeCqqlro1rsmyQuTbAQ2AZ+ZYq2SpJOY2tFQVXUkyfXAncAa4I6quj/JLcBC\nVe06ybb3J/l9YC9wBLjOI6EkqT+pqr5rWBaj0agWFhb6LkOSVpUke6pq1FrPM7glSU2GhSSpybCQ\nJDUZFpKkJsNCktRkWEiSmgwLSVKTYSFJajIsJElNhoUkqcmwkCQ1GRaSpCbDQpLUZFhIkpoMC0lS\nk2EhSWoyLCRJTYaFJKnJsJAkNRkWkqQmw0KS1GRYSJKaDAtJUpNhIUlqMiwkSU2GhSSpybCQJDUZ\nFpKkJsNCktRkWEiSmgwLSVKTYSFJajIsJElNhoUkqcmwkCQ1GRaSpCbDQpLUZFhIkpoMC0lS01TD\nIsmVSR5Isi/Jjcd5/x1J7ktyT5I/T3JJ1z6f5Ntd+z1Jfn2adUqSTu7caX1wkjXANuANwEFgd5Jd\nVbV3YrUPV9Wvd+tfBdwOXNm992BVXTqt+iRJSzfNnsXlwL6q2l9VTwI7gc2TK1TVNyYWXwzUFOuR\nJJ2maYbFRcAjE8sHu7ZFklyX5EHgNuCGibc2Jvlckj9L8neP9wOSbE2ykGTh0KFDy1m7JGlC7xPc\nVbWtql4DvBv4ha75MWB9VV0GvAv4cJKXHmfb7VU1qqrR3NzcyhUtSTNmmmHxKHDxxPK6ru1EdgJX\nA1TVE1X1le71HuBB4LunVKckqWGaYbEb2JRkY5LzgGuAXZMrJNk0sfgm4Atd+1w3QU6SVwObgP1T\nrFWSdBJTOxqqqo4kuR64E1gD3FFV9ye5BVioql3A9UmuAJ4CvgZc223+I8AtSZ4CngHeUVVfnVat\nkqSTS9XZcQDSaDSqhYWFvsuQpFUlyZ6qGrXW632CW5I0fIaFJKnJsJAkNRkWkqQmw0KS1GRYSJKa\nDAtJUpNhIUlqMiwkSU2GhSSpybCQJDUZFpKkJsNCktRkWEiSmgwLSVKTYSFJajIsJElNhoUkqcmw\nkCQ1GRaSpCbDQpLUZFhIkpoMC0lSk2EhSWpaUlgkeU2SF3av/16SG5J853RLkyQNxVJ7Fh8Dnk7y\nXcB24GLgw1OrSpI0KEsNi2eq6gjwD4BfraqfB145vbIkSUOy1LB4KslbgWuBP+7aXjCdkiRJQ7PU\nsPhp4IeBW6vqi0k2Ar8zvbIkSUNy7lJWqqq9wA0ASS4Azq+qX55mYZKk4Vjq0VCfSvLSJC8HPgv8\nRpLbp1uaJGkoljoM9bKq+gbwD4H/WFU/BFwxvbIkSUOy1LA4N8krgX/MsQluSdKMWGpY3ALcCTxY\nVbuTvBr4wvTKkiQNyVInuD8CfGRieT/w5mkVJUkalqVOcK9L8odJHu8eH0uybtrFSZKGYanDUB8E\ndgGv6h7/uWuTJM2ApYbFXFV9sKqOdI/fBuamWJckaUCWGhZfSfK2JGu6x9uAr0yzMEnScCw1LP4Z\n48NmvwQ8Bvwj4J+2NkpyZZIHkuxLcuNx3n9HkvuS3JPkz5NcMvHee7rtHkjyxiXWKUmagiWFRVUd\nqKqrqmquqv5mVV1N42ioJGuAbcBPApcAb50Mg86Hq+r7qupS4Dbg9m7bS4BrgNcCVwL/vvs8SVIP\nzuROee9qvH85sK+q9lfVk8BOYPPkCt1Z4Ue9GKju9WZgZ1U9UVVfBPZ1nydJ6sGZhEUa718EPDKx\nfLBrW/whyXVJHmTcs7jhFLfdmmQhycKhQ4dOpfZB2bED5ufhnHPGzzt29F2RJC12JmFR7VWW8CFV\n26rqNcC7gV84xW23V9WoqkZzc6vz4KwdO2DrVjhwAKrGz1u3GhiShuWkYZHkm0m+cZzHNxmfb3Ey\njzK+/epR67q2E9kJXH2a265aN90Ehw8vbjt8eNwuSUNx0rCoqvOr6qXHeZxfVa1LhewGNiXZmOQ8\nxhPWuyZXSLJpYvFNHLve1C7gmiQv7G60tAn4zKn8YqvFww+fWrsk9WFJ14Y6HVV1JMn1jC9AuAa4\no6ruT3ILsFBVu4Drk1wBPAV8jfFtW+nW+31gL3AEuK6qnp5WrX1av3489HS8dkkailQty9RD70aj\nUS0sLPRdxik7OmcxORS1di1s3w5btvRXl6TZkGRPVY1a653JBLeWwZYt42DYsAGS8bNBIWlopjYM\npaXbssVwkDRs9iwkSU2GhSSpybCQJDUZFnqWlx2RdCJOcAt4/iG8Ry87Ak6+S7JnoY6XHZF0MoaF\nAC87IunkDAsBJ768iJcdkQSGhTq33jq+zMiktWvH7ZJkWAjwsiOSTs6jofQsLzsi6UTsWWhwPN9D\nGh57FhoUz/eQhsmehQbF8z2kYTIsNChDOt/D4TDpGMNCgzKU8z2ODocdOABVx4bDDAzNKsNCgzKU\n8z0cDpMWMyw0KEM532NIw2HSEHg0lAZnCOd7rF8/Hno6Xrs0i+xZSMcxlOEwcKJdw2BYSMcxlOEw\nJ9o1FKmqvmtYFqPRqBYWFvouQ1pW8/PHHw7bsAEeemilq9HZKMmeqhq11rNnIQ3YkCbaHQ6bbYaF\nNGCed6KhMCykARvKRLvnnciwkAZsKBPtQxoOUz8MC2ngtmwZT2Y/88z4uY9zUIY0HOa8ST8MC0lN\nQxgOc96kX4aFpKYhDIcNad5kFns4hoWkJel7OGwo8yZD6uGsZGgZFpJWhaHMmwylh7PSoWVYSFoV\nhjBvAsPp4ax0aBkWklaFIcybwHB6OCsdWoaFpFWj73kTGE4PZ6VDy7CQpFMwlB7OSoeWNz+SpFM0\nhBt0Hf35N900Hnpav34cFNOqa6o9iyRXJnkgyb4kNx7n/Xcl2Zvk3iR3J9kw8d7TSe7pHrumWack\nrUYrOSw3tZ5FkjXANuANwEFgd5JdVbV3YrXPAaOqOpzkZ4DbgLd07327qi6dVn2SpKWbZs/icmBf\nVe2vqieBncDmyRWq6pNVdfTgr08D66ZYjyTpNE0zLC4CHplYPti1ncjbgT+ZWP6OJAtJPp3k6mkU\nKElamkFMcCd5GzACfnSieUNVPZrk1cAnktxXVQ8+Z7utwFaA9St9kLMkzZBp9iweBS6eWF7XtS2S\n5ArgJuCqqnriaHtVPdo97wc+BVz23G2rantVjapqNDc3t7zVS5KeNc2w2A1sSrIxyXnANcCio5qS\nXAZ8gHFQPD7RfkGSF3avLwReD0xOjEuSVtDUhqGq6kiS64E7gTXAHVV1f5JbgIWq2gX8CvAS4CNJ\nAB6uqquAvw18IMkzjAPtfc85ikqStIJSVX3XsCxGo1EtLCz0XYYkrSpJ9lTVqLWel/uQJDUZFpKk\nJsNCktRkWEiSmgwLSVKTYSFJajIsJElNhoUkqcmwkCQ1GRaSpCbDQpLUZFhIkpoMC0lSk2EhSWoy\nLCRJTYaFJKnJsOjcfHPfFUjScBkWnV/8xb4rkKThMiwkSU0zHRY33wzJ+AHHXs/6kNSs//6Sni9V\n1XcNy2I0GtXCwsJpb5/AWbIrzpj7QpodSfZU1ai13kz3LDRs9nCk4TAsOu99b98V9GuIQ3IedCAN\nh8NQep6hDEMNpQ7pbOYwlFalIfZwpKFbif8fhoWep88huZtvHvcmjvYojr42LNwHOrGVGLJ1GEqD\n5TDUYu4PnciZ/NtwGEqr3qwfdCCdzEoP2RoWGiyHXZzDOZFZ//1h5YdsDQupoc8vpiHO4Qzhi3oo\nh1UPYV+sFOcspIahzBVYx7BqGFIdN998+sHlnIV0lpn1ORyH5E7MQ2elngzxi6nvn933/hjKkNwQ\n9kUfHIaSGoYy1DAUQ9gfQ6hhSHWcCYehVqGz/S8TabnM+pBcHwyLARnKER5azC+mxYawP4byh9UQ\n9sVKcRhqQM6GLq2k1cVhqFViVifLJK0u9iwGxJ6FpJU2iJ5FkiuTPJBkX5Ibj/P+u5LsTXJvkruT\nbJh479okX+ge106zTknSyU0tLJKsAbYBPwlcArw1ySXPWe1zwKiqvh/4KHBbt+3LgfcCPwRcDrw3\nyQXTqnUoZmmyTNLqMs2exeXAvqraX1VPAjuBzZMrVNUnq+pwt/hpYF33+o3AXVX11ar6GnAXcOUU\nax0E5ykkDdU0w+Ii4JGJ5YNd24m8HfiT09xWkjRF5/ZdAECStwEj4EdPcbutwFaA9evXT6EySRJM\nt2fxKHDxxPK6rm2RJFcANwFXVdUTp7JtVW2vqlFVjebm5patcEnSYtMMi93ApiQbk5wHXAPsmlwh\nyWXABxgHxeMTb90J/ESSC7qJ7Z/o2iRJPZjaMFRVHUlyPeMv+TXAHVV1f5JbgIWq2gX8CvAS4CMZ\nn5X2cFVdVVVfTfJLjAMH4Jaq+uq0apUkndxZc1JekkPAgb7rOEMXAl/uu4gBcX8s5v44xn2x2Jns\njw1V1RzHP2vC4myQZGEpZ1LOCvfHYu6PY9wXi63E/vDaUJKkJsNCktRkWAzL9r4LGBj3x2Luj2Pc\nF4tNfX84ZyFJarJnIUlqMiwGIMnFST7ZXa79/iTv7LumviVZk+RzSf6471r6luQ7k3w0yf9K8vkk\nP9x3TX1K8i+6/yd/leT3knxH3zWtpCR3JHk8yV9NtL08yV3dLR3umsZVug2LYTgC/FxVXQK8Drju\nOJdznzXvBD7fdxED8e+Aj1fV3wJ+gBneL0kuAm5gfGuD72V8wu81/Va14n6b51+F+0bg7qraBNzd\nLS8rw2IAquqxqvps9/qbjL8MZvYqu0nWAW8CfrPvWvqW5GXAjwC/BVBVT1bVX/dbVe/OBV6U5Fxg\nLfC/e65nRVXVfwOee0WLzcCHutcfAq5e7p9rWAxMknngMuAv+62kV/8W+JfAM30XMgAbgUPAB7th\nud9M8uK+i+pLVT0K/BvgYeAx4OtV9af9VjUIr6iqx7rXXwJesdw/wLAYkCQvAT4G/POq+kbf9fQh\nyU8Bj1fVnr5rGYhzgR8E/kNVXQb8X6YwxLBadGPxmxmH6KuAF3e3OFCnxoe4LvthrobFQCR5AeOg\n2FFVf9B3PT16PXBVkocY313xx5L8br8l9eogcLCqjvY0P8o4PGbVFcAXq+pQVT0F/AHwd3quaQj+\nT5JXAnTPjzfWP2WGxQBkfMnd3wI+X1W3911Pn6rqPVW1rqrmGU9cfqKqZvYvx6r6EvBIku/pmn4c\n2NtjSX17GHhdkrXd/5sfZ4Yn/CfsAq7tXl8L/Kfl/gGGxTC8HvgnjP+Kvqd7/P2+i9Jg/CywI8m9\nwKXAv+65nt50PayPAp8F7mP8HTZTZ3Mn+T3gL4DvSXIwyduB9wFvSPIFxr2v9y37z/UMbklSiz0L\nSVKTYSFJajIsJElNhoUkqcmwkCQ1GRZSQ5KnJw5pvifJsp1BnWR+8uqh0lCd23cB0irw7aq6tO8i\npD7Zs5BOU5KHktyW5L4kn0nyXV37fJJPJLk3yd1J1nftr0jyh0n+Z/c4epmKNUl+o7tHw58meVG3\n/g3dPU7uTbKzp19TAgwLaSle9JxhqLdMvPf1qvo+4NcYXy0X4FeBD1XV9wM7gPd37e8H/qyqfoDx\n9Z3u79o3Aduq6rXAXwNv7tpvBC7rPucd0/rlpKXwDG6pIcm3quolx2l/CPixqtrfXQjyS1X1N5J8\nGXhlVT3VtT9WVRcmOQSsq6onJj5jHriru2kNSd4NvKCq/lWSjwPfAv4I+KOq+taUf1XphOxZSGem\nTvD6VDwx8fppjs0lvgnYxrgXsru72Y/UC8NCOjNvmXj+i+71/+DYrT63AP+9e3038DPw7D3GX3ai\nD01yDnBxVX0SeDfwMuB5vRtppfiXitT2oiT3TCx/vKqOHj57QXc12CeAt3ZtP8v4znY/z/gudz/d\ntb8T2N5dJfRpxsHxGMe3BvjdLlACvN/bqapPzllIp6mbsxhV1Zf7rkWaNoehJElN9iwkSU32LCRJ\nTYaFJKnJsJAkNRkWkqQmw0KS1GRYSJKa/j+5lptysnYo9wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd1dd8b5278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(epochs, loss_values, 'bo')\n",
    "plt.plot(epochs, val_loss_values, 'b+')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGWdJREFUeJzt3X+UX3V95/HnK4n8iPwQyZijJCTpSi1poWSdRa21obBo\nQJeU9HQFIosu3Ry7gtQj1XDSKk1NsZbuYa3Us7FFwabSSLtu6rIgG6D2VOpmEBIMIZBmQzIBy3Aw\nVqQrJLz2j3uHfDMmc79D5s69w/f1OGfO3Pu59859zzfffF9zP5/7Q7aJiIgYzZSmC4iIiPZLWERE\nRKWERUREVEpYREREpYRFRERUSlhERESlhEVERFRKWERERKWERUREVJrWdAHjZcaMGZ47d27TZURE\nTCr333//07b7qtZ7xYTF3LlzGRgYaLqMiIhJRdLj3ayXbqiIiKiUsIiIiEoJi4iIqFRrWEhaJGmr\npG2Slh9k+RxJ6yVtknSvpFkdy/ZJerD8WldnnRERMbraBrglTQVuBM4FBoENktbZfrhjteuBW2zf\nLOls4Drg0nLZv9g+o676IiKie3UeWZwJbLO93fbzwK3A4hHrzAfuLqfvOcjy6GHXXtt0BRExrM6w\nOAnY1TE/WLZ12ggsKacvBI6VdGI5f5SkAUn/IOlXaqwzWup3f7fpCiJiWNMD3FcDCyU9ACwEdgP7\nymVzbPcDlwA3SPpXIzeWtKwMlIGhoaEJKzoiotfUGRa7gdkd87PKtpfYfsL2EtsLgBVl257y++7y\n+3bgXmDByB3YXm2733Z/X1/lBYgxCVx7LUjFF+yfTpdURLPqDIsNwCmS5kk6ArgIOOCsJkkzJA3X\ncA1wU9l+gqQjh9cB3g50DozHK9S114JdfMH+6YRFRLNqCwvbe4ErgDuBLcBa25slrZR0QbnaWcBW\nSY8CM4FVZfupwICkjRQD358ecRZVRERMoFrvDWX7duD2EW2f6Ji+DbjtINt9Czitztqi/T75yaYr\niIhhTQ9wRxxSup4i2iNhERERlRIWERFRKWERERGVEhYRMSYZS+pNCYuIGJM23IYlgTXxEhYRMem0\nIbB6TcIiIirlNiyRsIiISm24DUsCq1ny8L/+JNff3++BgYGmy4h4xZP2h0Yv1/BKIen+8g7fo8qR\nRUSMSW7D0psSFi2Sw+mYDNrwPk1gTbx0Q7VIDq0jYqKlGyoiIsZNwqJhOcMjYvLqpf+nCYuGteGU\nxBhd/i3iUNpyceBEvEczZtEiGbNop/y7xKG05b1xOHVkzGISyhkeEe3Xq13HCYsWacubrS11NKlX\nPxCiWlu6jif6PZpuqPgJbTm0bou8HnEobXlvTPpuKEmLJG2VtE3S8oMsnyNpvaRNku6VNGvE8uMk\nDUr6XJ11RkS8HL3UdVxbWEiaCtwInAfMBy6WNH/EatcDt9g+HVgJXDdi+e8B36yrxtgv3S6H1ksf\nCDE2bfn/MRHv0dq6oSS9DbjW9rvK+WsAbF/Xsc5mYJHtXZIE/MD2ceWyNwO/BdwB9Nu+YrT9pRtq\n/LTl0Doi6teGbqiTgF0d84NlW6eNwJJy+kLgWEknSpoC/BFwdY31RUREl5o+G+pqYKGkB4CFwG5g\nH/CfgdttD462saRlkgYkDQwNDR1WIW05nGyDdLu0U96j0aRGu6FGrH8M8IjtWZLWAO8AXgSOAY4A\n/sT2TwySDzvcbqh0vUTb5T0adei2G2pajTVsAE6RNI/iiOEi4JLOFSTNAJ6x/SJwDXATgO2lHeu8\nn2LM4pBBERER9aqtG8r2XuAK4E5gC7DW9mZJKyVdUK52FrBV0qPATGBVXfUcTM4AirbLezTaIhfl\nlXKIH22X92jUoQ1nQ0VExCtEwqKUM4Ci7fIejSalGyoiooelGyoiIsZNwiIiIiolLCIiolLCIiIi\nKiUsIiKiUsIiIiIqJSwiIqJSwiIiIiolLCIiolLCIiIiKiUsIiKiUsIiIiIqJSwiIqJSwiIiIiol\nLCIiolLCIiIiKiUsIiKiUsIiIiIq1RoWkhZJ2ippm6TlB1k+R9J6SZsk3StpVkf7dyQ9KGmzpA/W\nWWdERIyutrCQNBW4ETgPmA9cLGn+iNWuB26xfTqwEriubH8SeJvtM4C3AMslvaGuWiMiYnR1Hlmc\nCWyzvd3288CtwOIR68wH7i6n7xlebvt52z8u24+suc6IiKhQ54fwScCujvnBsq3TRmBJOX0hcKyk\nEwEkzZa0qfwZf2D7iZE7kLRM0oCkgaGhoXH/BSIiotD0X+xXAwslPQAsBHYD+wBs7yq7p94IXCZp\n5siNba+23W+7v6+vbyLrjojoKXWGxW5gdsf8rLLtJbafsL3E9gJgRdm2Z+Q6wHeBd9RYa0REjKLO\nsNgAnCJpnqQjgIuAdZ0rSJohabiGa4CbyvZZko4up08AfhHYWmOtERExitrCwvZe4ArgTmALsNb2\nZkkrJV1QrnYWsFXSo8BMYFXZfirwbUkbgb8Frrf9UF21RkSMxZo1MHcuTJlSfF+zpumK6ifbTdcw\nLvr7+z0wMNB0GRFRozVrYMUK2LkTTj4ZVq2CpUsnvoZly+C55/a3TZ8Oq1dPfC3jQdL9tvur1mt6\ngDsioivDH9KPPw528X3Zson/q37FigODAor5FSsmtg6Y2COcHFlExKQwd24RECPNmQM7dkxcHVOm\nFGE1kgQvvjhxdYzXEU6OLCLiFWXnzrG11+Xkk8fWXpeJPsJJWETEpNCWD+lVq4q/4DtNn160T6SJ\nDs+ERUR0pekzgNryIb10adHVM2dO0fU0Z04zg9sTHZ4Ji4io1IbB5bZ8SA/XsmNHMUaxY0czNUx0\neGaAOyIqtWVwOQ40HqcSdzvAnbCIiEptOQMoxl/OhoqIcdOWweVoTsIiIiq1ZXA5mpOwiGi5ps9C\ngnYNLkczEhYRh9CGD+k2nIU0rA1nAEVzEhYRB9GWD+k23YcoelvCIuIg2vIh3ZZbXEQkLKJ12tD9\n05YP6ZyFFG1RGRaSriyfVhdRu7Z0/7TlQzpnIUVbdHNkMRPYIGmtpEWSVHdR0bva0v3Tlg/pnIUU\nbdHVFdxlQLwT+ADQD6wF/sz2P9ZbXvdyBfcrQ5uuFG7DU9ki6jauV3C7SJTvlV97gROA2yR95rCq\njBihLd0/kFNFIzp1M2ZxlaT7gc8Afw+cZvs3gDcDv1pzfdFj2tL9ExEH6ubI4rXAEtvvsv1V2y8A\n2H4ReM9oG5ZjHFslbZO0/CDL50haL2mTpHslzSrbz5B0n6TN5bL3vozfLSah9NFHtFPlmIWktwKb\nbf+wnD8OONX2tyu2mwo8CpwLDAIbgIttP9yxzleBr9u+WdLZwAdsXyrppyl6vx6T9Abg/nKfew61\nv4xZRESM3XiOWXweeLZj/tmyrcqZwDbb220/D9wKLB6xznzg7nL6nuHlth+1/Vg5/QTwFNDXxT4j\nIqIG3YSF3HH4UXY/Tetiu5OAXR3zg2Vbp43AknL6QuBYSScesHPpTOAIoDVnXkVE9JpuwmK7pA9L\nelX5dRWwfZz2fzWwUNIDwEJgN7BveKGk1wNfpuie+okTJyUtkzQgaWBoaGicSupdbbhyOiLaqZuw\n+CDwCxQf5IPAW4BlXWy3G5jdMT+rbHuJ7SdsL7G9AFhRtu2Bl8ZG/iewwvY/HGwHtlfb7rfd39eX\nXqrD0ZYrpyOinWp7rKqkaRQD3OdQhMQG4BLbmzvWmQE8Y/tFSauAfbY/IekI4H8Bf2P7hm72lwHu\nw5NnLEf0pm4HuCvHHiQdBVwO/Cxw1HC77f842na290q6ArgTmArcZHuzpJXAgO11wFnAdZIMfBP4\nULn5vwd+CThR0vvLtvfbfrCq3nh52nLjvIhop25Onf0q8AhwCbASWApssX1V/eV1L0cWhydHFhG9\naTxPnX2j7d8BfmT7ZuDdFOMWMU7aMLCcK6cjYjTdhMUL5fc9kn4OOB54XX0l9Za2DCznyumIGE03\n3VC/DvwVcBrwJeAY4Hds/7faqxuDydoNle6fiGjSuAxwS5oC/LPt71MMQP/UONUXpQwsR8RkMGo3\nVHkh3McmqJae1KZbckdEHEo3Yxb/W9LVkmZLeu3wV+2V9YgMLEfEZNDNPZ6Gbw/+oY42ky6pcTE8\ngJwnskVEm1WGhe15E1FIL1u6NOEQEe3WzRXc/+Fg7bZvGf9yIiKijbrphvo3HdNHUdzr6TtAwiIi\nokd00w11Zee8pNdQPMgoIiJ6RDdnQ430IyDjGBERPaSbMYu/oTj7CYpwmQ+srbOoiIhol27GLK7v\nmN4LPG57sKZ6IiKihboJi53Ak7b/H4CkoyXNtb2j1soiIqI1uhmz+CrQ+fzrfWVbRET0iG7CYprt\n54dnyukj6ispIiLappuwGJJ0wfCMpMXA0/WVFBERbdPNmMUHgTWSPlfODwIHvao7IiJembq5KO8f\ngbdKOqacf7b2qiIiolUqu6Ek/b6k19h+1vazkk6Q9KlufrikRZK2StomaflBls+RtF7SJkn3SprV\nsewOSXskfX1sv1JERIy3bsYszrO9Z3imfGre+VUbSZoK3AicR3Eh38WS5o9Y7XrgFtunAyuB6zqW\n/SFwaRf1RUREzboJi6mSjhyekXQ0cOQo6w87E9hme3t5BtWtwOIR68wH7i6n7+lcbns98MMu9hMR\nETXrJizWAOslXS7p14G7gJu72O4kYFfH/GDZ1mkjsKScvhA4VtKJXfzsiIiYQJVhYfsPgE8BpwJv\nAu4E5ozT/q8GFkp6AFgI7Ka46K8rkpZJGpA0MDQ0NE4lRUTESN3edfafKG4m+GvA2cCWLrbZDczu\nmJ9Vtr3E9hO2l9heAKwo2/bQJdurbffb7u/r6+t2s4iIGKNDnjor6aeBi8uvp4G/BGT7l7v82RuA\nUyTNowiJi4BLRuxjBvCM7ReBa4CbxvwbRERE7UY7sniE4ijiPbZ/0fYfM4YuItt7gSsouq22AGtt\nb5a0suOK8LOArZIeBWYCq4a3l/R3FPegOkfSoKR3jeH3ioiIcTTaRXlLKI4G7pF0B8XZTBrLD7d9\nO3D7iLZPdEzfBtx2iG3fMZZ9RUREfQ55ZGH7a7YvAn6G4rTW3wReJ+nzkt45UQVGRETzujkb6ke2\n/8L2v6MYpH4A+HjtlUVERGuM6Rnctr9fnoF0Tl0FRURE+4wpLCIiojclLCIiolLCIiIiKiUsIiKi\nUsIiIiIqJSwiIqJSwiIiIiolLCIiolLCIiIiKiUsIiKiUsIiIiIqJSwiIqJSwiIiIiolLCIiolLC\nIiIiKiUsIiKiUsIiIiIqJSwiIqJSrWEhaZGkrZK2SVp+kOVzJK2XtEnSvZJmdSy7TNJj5ddlddYZ\nERGjqy0sJE0FbgTOA+YDF0uaP2K164FbbJ8OrASuK7d9LfBJ4C3AmcAnJZ1QV60RETG6Oo8szgS2\n2d5u+3ngVmDxiHXmA3eX0/d0LH8XcJftZ2x/H7gLWFRjrRERMYo6w+IkYFfH/GDZ1mkjsKScvhA4\nVtKJXW4bERETpOkB7quBhZIeABYCu4F93W4saZmkAUkDQ0NDddUYEdHz6gyL3cDsjvlZZdtLbD9h\ne4ntBcCKsm1PN9uW66623W+7v6+vb7zrj4iIUp1hsQE4RdI8SUcAFwHrOleQNEPScA3XADeV03cC\n75R0Qjmw/c6yLSIiGlBbWNjeC1xB8SG/BVhre7OklZIuKFc7C9gq6VFgJrCq3PYZ4PcoAmcDsLJs\ni4iIBsh20zWMi/7+fg8MDDRdRkTEpCLpftv9Ves1PcAdERGTQMIiIiIqJSwiIqJSwiIiIiolLCIi\nolLCIiIiKiUsIiKiUsIiIiIqJSwiIqJSwiIiIiolLCIiolLCIiIiKiUsIiKiUsIiIiIqJSwiIqJS\nwiIiIiolLCIiolLCIiIiKiUsIiKiUsIiIiIq1RoWkhZJ2ippm6TlB1l+sqR7JD0gaZOk88v2IyR9\nUdJDkjZKOqvOOiMiYnS1hYWkqcCNwHnAfOBiSfNHrPbbwFrbC4CLgD8p2/8TgO3TgHOBP5KUo6CI\niIbU+QF8JrDN9nbbzwO3AotHrGPguHL6eOCJcno+cDeA7aeAPUB/jbVGRMQo6gyLk4BdHfODZVun\na4H3SRoEbgeuLNs3AhdImiZpHvBmYHaNtUZExCia7tq5GPiS7VnA+cCXy+6mmyjCZQC4AfgWsG/k\nxpKWSRqQNDA0NDSBZUdE9JY6w2I3Bx4NzCrbOl0OrAWwfR9wFDDD9l7bH7F9hu3FwGuAR0fuwPZq\n2/22+/v6+mr5JSIiot6w2ACcImmepCMoBrDXjVhnJ3AOgKRTKcJiSNJ0Sa8u288F9tp+uMZaIyJi\nFNPq+sG290q6ArgTmArcZHuzpJXAgO11wEeBL0j6CMVg9/ttW9LrgDslvUhxNHJpXXVGREQ12W66\nhnHR39/vgYGBpsuIiJhUJN1vu/Js06YHuCMiYhJIWERERKWERUREVEpYREREpYRFRERUSlhERESl\nhEVERFRKWERERKWERUREVEpYREREpYRFRERUSlhERESlhEVERFRKWERERKWERUREVOr5sFizBubO\nhSlTiu9r1jRdUURE+9T2pLzJYM0aWLYMnnuumH/88WIeYOnS5uqKiGibnj6yWLFif1AMe+65oj0i\nIvbr6bDYuXNs7RERvaqnw+Lkk8fWHhHRq3o6LFatgunTD2ybPr1oj4iI/WoNC0mLJG2VtE3S8oMs\nP1nSPZIekLRJ0vll+6sk3SzpIUlbJF1TR31Ll8Lq1TBnDkjF99WrM7gdETFSbWdDSZoK3AicCwwC\nGySts/1wx2q/Day1/XlJ84HbgbnArwFH2j5N0nTgYUlfsb1jvOtcujThEBFRpc4jizOBbba3234e\nuBVYPGIdA8eV08cDT3S0v1rSNOBo4Hngn2usNSIiRlFnWJwE7OqYHyzbOl0LvE/SIMVRxZVl+23A\nj4AngZ3A9bafGbkDScskDUgaGBoaGufyIyJiWNMD3BcDX7I9Czgf+LKkKRRHJfuANwDzgI9K+qmR\nG9tebbvfdn9fX99E1h0R0VPqDIvdwOyO+VllW6fLgbUAtu8DjgJmAJcAd9h+wfZTwN8D/TXWGhER\no6gzLDYAp0iaJ+kI4CJg3Yh1dgLnAEg6lSIshsr2s8v2VwNvBR6psdaIiBiFbNf3w4tTYW8ApgI3\n2V4laSUwYHtdeQbUF4BjKAa1P2b7G5KOAb4IzAcEfNH2H1bsawh4vLZfZmLMAJ5uuogWyetxoLwe\n++W1ONDhvB5zbFf249caFjE2kgZsp7utlNfjQHk99strcaCJeD2aHuCOiIhJIGERERGVEhbtsrrp\nAlomr8eB8nrsl9fiQLW/HhmziIiISjmyiIiISgmLFpA0u7z77sOSNku6qumamiZpank34q83XUvT\nJL1G0m2SHinvwvy2pmtqkqSPlP9PvivpK5KOarqmiSTpJklPSfpuR9trJd0l6bHy+wnjvd+ERTvs\nBT5qez7FBYgfKq9B6WVXAVuaLqIl/ivFHQ1+Bvh5evh1kXQS8GGg3/bPUVzDdVGzVU24LwGLRrQt\nB9bbPgVYX86Pq4RFC9h+0vZ3yukfUnwYjLzpYs+QNAt4N/CnTdfSNEnHA78E/BmA7edt72m2qsZN\nA44u70o9nf13q+4Jtr8JjLyx6mLg5nL6ZuBXxnu/CYuWkTQXWAB8u9lKGnUD8DHgxaYLaYF5FLfA\n+WLZLfen5S1wepLt3cD1FLcEehL4ge1vNFtVK8y0/WQ5/T1g5njvIGHRIuVtTv4K+E3bPfn8Dknv\nAZ6yfX/TtbTENOBfA5+3vYDi1v3j3sUwWZR98YspQvQNFM+9eV+zVbWLi1Ncx/0014RFS0h6FUVQ\nrLH9103X06C3AxdI2kHxwKyzJf15syU1ahAYtD18pHkbRXj0qn8L/F/bQ7ZfAP4a+IWGa2qDf5L0\neoDy+1PjvYOERQtIEkWf9Bbb/6Xpeppk+xrbs2zPpRi4vNt2z/7laPt7wC5JbyqbzgEeHmWTV7qd\nwFslTS//35xDDw/4d1gHXFZOXwb8j/HeQcKiHd4OXErxV/SD5df5TRcVrXElsEbSJuAM4Pcbrqcx\n5RHWbcB3gIcoPsN66mpuSV8B7gPeJGlQ0uXAp4FzJT1GcfT16XHfb67gjoiIKjmyiIiISgmLiIio\nlLCIiIhKCYuIiKiUsIiIiEoJi4gKkvZ1nNL8oKRxu4Ja0tzOu4dGtNW0pguImAT+xfYZTRcR0aQc\nWUS8TJJ2SPqMpIck/R9Jbyzb50q6W9ImSeslnVy2z5T03yVtLL+Gb1MxVdIXymc0fEPS0eX6Hy6f\ncbJJ0q0N/ZoRQMIiohtHj+iGem/Hsh/YPg34HMXdcgH+GLjZ9unAGuCzZftngb+1/fMU93faXLaf\nAtxo+2eBPcCvlu3LgQXlz/lgXb9cRDdyBXdEBUnP2j7mIO07gLNtby9vBPk92ydKehp4ve0XyvYn\nbc+QNATMsv3jjp8xF7irfGgNkj4OvMr2pyTdATwLfA34mu1na/5VIw4pRxYRh8eHmB6LH3dM72P/\nWOK7gRspjkI2lA/7iWhEwiLi8Ly34/t95fS32P+oz6XA35XT64HfgJeeMX78oX6opCnAbNv3AB8H\njgd+4ugmYqLkL5WIakdLerBj/g7bw6fPnlDeDfbHwMVl25UUT7b7LYqn3H2gbL8KWF3eJXQfRXA8\nycFNBf68DBQBn83jVKNJGbOIeJnKMYt+2083XUtE3dINFRERlXJkERERlXJkERERlRIWERFRKWER\nERGVEhYREVEpYREREZUSFhERUen/A1G+zkWN+XAvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd1dd8b54a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(epochs, acc_values, 'bo')\n",
    "plt.plot(epochs, val_acc_values, 'b+')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction\n",
    "\n",
    "We can also predict the classification for new images. We will just use some images from the test-set but you could load your own images into numpy arrays and use those instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "images = data.test.images[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the true class-number for those images. This is only used when plotting the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_true = data.test.cls[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the predicted classes as One-Hot encoded arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred = model.predict(x=images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the predicted classes as integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4VdW19/HvQDAIKIjYKScRC0gUE+yxXAtRVFBiFGPh\nGtEoxhJvIIlRggkag43XEkV9RG+wgopIRBS70oSASrGAokEvIqJGUbGN94+95t7r9L3O7off53nO\nc3ZZZcA8e+4x55prTnN3REQkOy1KHYCISCVRpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIgmo0hQR\nSUCVpohIAqo0RUQSaJnLzp06dfKqqqo8hVIZ5s2bt9rdNy91HMWiMm7+VMbJ5FRpVlVVMXfu3FwO\nUXHM7O1Sx1BMKuPmT2WcjJrnIiIJqNIUEUlAlaaISAKqNEVEElClKSKSQE5Xz0Wa6sorrwTgiy++\nAODll18GYOLEibW2PeusswDYe++9ATj55JOLEaJInZRpiogkoExTiur4448HYMKECXW+b2a1Xrvp\nppsAmD59OgAHHHAAAF27di1EiFJCr7/+OgA77rgjANdeey0A55xzTsliqkmZpohIAso0peBCdgn1\nZ5g77bQTAIcddhgAb775Zvq9yZMnA7B06VIAxo8fD8CFF16Y/2ClpObPnw9AixapfG7bbbctZTh1\nUqYpIpKAMk0pmHA/84MPPljrvV69egGZLLJTp04AtGvXDoCvvvoqve2ee+4JwEsvvQTAhx9+WKCI\npdQWLFgAZP4OBg4cWMpw6qRMU0QkgaJnmmEc3i233ALANttsk36vdevWAJx44okAbLXVVgB07969\nmCFKnvzf//0fAO6efi1kmNOmTQNg6623rnPfMI4TYMmSJdXeO/LII/Map5TeK6+8AsB1110HwCmn\nnFLKcBqkTFNEJIGiZ5rDhg0DYPny5fVuE8blbbLJJgD07NkzL+fu0qULAMOHDwegT58+eTmu1O2o\no44CMle9ATbeeGMAOnbs2OC+9957b/pxvH9TmqfXXnsNgLVr1wLVR1yUG2WaIiIJqNIUEUmg6M3z\nW2+9FcgMH4k3vRcvXgxkBrg+/fTTAMyaNQvI3Db3zjvv1Hv8Vq1aAZkhLOFiRPw4oZmu5nlxdOvW\nLettr7jiCiBzO11cGHoUfkvzMXr0aCC19AaU92dTmaaISAJFzzQPPvjgar/jwi10wUcffQRkMs/w\n7fPiiy/We/zvfe97QOaG/3B7HsCaNWsA2G677ZoUuxTOlClTABgxYgQA69atS7+35ZZbAnD55ZcD\n0KZNmyJHJ4UQvxgcPtPhc9u2bdtShJQVZZoiIgmU9W2Um266KQAHHXRQtdfrylJruv/++4FMtgqw\nyy67ADBo0KB8hSh5Em65jGeYQRh+EqaEk+bhmWeeqfXa5puX/3LzyjRFRBIo60yzKVatWgXA0KFD\ngeq38IX+ssYGVkvxHH300UDmtspg8ODB6cejRo0qakxSHGGJk7hw40k5U6YpIpJAs8s0b7jhBiCT\ncXbo0CH9XrgyJ6UXxs/OmDEDyPRlhj6tiy66KL1tmCZMmoeZM2cCMG7cuPRru+22GwCHHnpoSWJK\nQpmmiEgCzSbTfP7554HMWL7goYceSj8O05JJ6YXJZVevXl3t9TAtoMbSNl9PPPEEUH1kSxijHaaH\nLGfKNEVEElClKSKSQLNpnj/yyCNAZu7FQw45BIC99967ZDFJbWFNoHBrbHDggQcC8Oc//7nYIUmR\nhcl64n7+85+XIJKmUaYpIpJAxWeaX3zxBQCPPvookJmw45JLLgEyU8VJ6cRXj7zsssuA2rOx9+7d\nG9DwouZs5cqVADz33HNA9cl0jjnmmJLE1BTKNEVEEqj4TDNMWhv6yA4//HAA9tlnn5LFJNVdddVV\n6cdz5syp9l64jVJ9mc3f7bffDsD7778PZD6rlUaZpohIAhWZaYYJawH+8pe/ANC+fXsALr744pLE\nJPW7+uqr630v3Paqvszm7+233672PEz9WGmUaYqIJFBRmWa4CnvuueemX/vmm28A6NevH6BxmZUm\nlGk2oxxCayJs+/XXXwPwySef1No23KJ3zTXX1HmsDTbYIP34b3/7G6BlNArt4Ycfrvb8yCOPLFEk\nuVGmKSKSgCpNEZEEKqJ5/u233wKZmVDeeuut9Hvdu3cHMheEpLKEdZuycdxxxwGw9dZbA5mhK/fc\nc09OMYTVLuNzeEr+hMHsobwqnTJNEZEEKiLTXLZsGZBZsTAuDGfR/IvlK1ykA5g0aVKTj3Pfffc1\nuk24SNSiRfV8oH///gD06dOn1j4/+clPmhyTNO7BBx8EMhdtwyztlbq6qDJNEZEEyjrTDINh+/bt\nW+31K6+8Mv24UoctrE8eeOCB9OPRo0cDtSfsCBYvXgw03E952mmnAdCtW7da7/3sZz8DoEePHk0L\nVvLm888/B2Dq1KnVXg/TwMWHfVUSZZoiIgmUdaY5duxYoPbtV/G+EDMrakySm2zXtb7rrrsKHIkU\nWuhfDivCDhgwAIDzzjuvZDHlgzJNEZEEyjLTDOO6rr/++hJHIiJNFTLNsM55c6FMU0QkgbLMNMMa\n5p9++mm118PdP5pGTERKRZmmiEgCqjRFRBIoy+Z5TWGlwieeeAKAjh07ljIcEVmPKdMUEUmgLDPN\nP/zhD9V+i4iUC2WaIiIJmLs3fWezD4C3G92weenm7puXOohiURk3fyrjZHKqNEVE1jdqnouIJKBK\nU0QkgQYrTTPbzMwWRD8rzezd2PMNCxGQmfWMnWOBmX1qZr9uZJ8hZvZBtP0SM/tljjGMN7OjG9nm\n97EYF5nZN2bWPpfzlkKJyribmT1tZouj/7sGyzfapxRlfIqZvWJmL5vZC2b2w1zOWSqlKOPovHeE\nMsty+1KU8c5mNtPM1pnZ+Vkd2N2z+gFGAr+t43UDWmR7nCQ/QCtgFdC5ke2GAGOix1sBq4FONbZp\nmeC844GjE2x/DPBYIf4PivlTrDIGtgF6R483AZYBO5RbGQP7Ah2ix0cBL5S6jCqljKNjHgDsASzI\ncvtSlPGWQB/gcuD8bI7bpOa5mXWPsoQ7gUVAFzP7OPb+IDO7NXq8pZk9YGZzzWyOme2V4FSHAkvc\nfUW2O7j7SmA50NXMRpnZ/5rZC8DtZtbSzK6O4njZzIZEMbYws7+b2atm9jjQKUGMACcAdyfcp6wV\nsozd/T13XxA9/g/wKrBttrEVq4zd/QV3D//mWUDnbGOsBIX+HLv7M8CapsRWxDJ+393nAt9kG1su\ng9t3Ak5x97lm1tBxrgVGu/ssM6sCpgC9zGxP4FR3P7OBfQeRsDIys+5AN+DNWJz7u/uXZjYUWOXu\ne5jZ94BZZvYYsBfwfaAnqSxoMXBTdLxLSWUYj9RzvnbAIcDpSeKsEAUvYzP7AdALeDHboIpdxpHT\ngKkNvF+pivE5TqxEZZyVXCrNZVEN3ZhDgB0tsyzFpma2kbvPBmbXt5OZtQaOAC7IMp4TzexAYB0w\nxN0/js75kLt/GW3TF+hhZoOi5+2B7YH9gbvd/TtghZk9HQ7q7n9s5LwDgGfc/ZMs46wkhS7jTYD7\ngXPc/bMszlOSMjazQ4CTgea41m9By7gJSvU5zlouleba2OPvSPWJBK1jjw3Yw93rXn6wfkcAs919\ndZbb3+nudXXkxuM0YKi7PxHfwMyOSRhb3CDgHznsX84KVsaWugDxADDO3SdnuVvRy9jMegNjgZ+6\n+0dNOUaZK/TnOKlSfY6zlpchR1HN/pGZbW9mLUhdGAmmA2eHJ9EfYTZq9ROa2XlmlkszYBowNDRD\nzGxHM9sIeBY4PuoT2ZZUB3ajzGxTYB/g4Rxiqgj5LGNLpQ63k7pAcG2N98qmjKNm6ETgF+6+NIeY\nKkKBPse1lFMZN0U+x2n+jtQ/ZgYQv3BzNrBv1GG7mKjvz8z2NLOb6jqQmW0M/BcwqcZbPYAPc4hx\nLPAGsMDMFgI3ksq2JwLvkOoDGQekFzUxs0vNrF89x/sZMNXdv8ghpkqSrzI+gNSX4qGWGfry0+i9\ncirjkUBHYGwUYz6boeUqn5/jCcBzQE8zW2Fm/x29VTZlbGadzWwFcC4wMoqzTUMnr6jbKM3sn8AA\nd8/6SpdUFpVx81fpZVxRlaaISKnpNkoRkQRUaYqIJKBKU0QkAVWaIiIJ5LRGUKdOnbyqqipPoVSG\nefPmrfb1aFZvlXHzpzJOJqdKs6qqirlzs7kDq/kws/VqWQCVcfOnMk5GzXMRkQRUaYqIJKBKU0Qk\nAVWaIiIJqNIUEUlAlaaISAI5DTkqlrVrU/OPDhs2DICbbsrMRNWnTx8AJkyYAEC3bt2KHJ2IrE+U\naYqIJFARmeZ7770HwC233ALABhtskH4vDMp9+OHU5Om//nWjS2hLGfjXv/4FwMCBAwFYvnx5k4/1\n2GOPpR/36NEDgC5dujQ9OCmZ8Dnu378/ANdddx0AZ511Vnqb+Oe/FJRpiogkUNaZ5gcffADA4MGD\nSxyJ5Nu0adMAWLduXc7Hmjw5sy7bbbfdBsA999yT83GleD78MLX6RTyjBDjnnHMAOO2009KvbbTR\nRsULrA7KNEVEEijLTPPaa1MLFE6alFpX7cUXX2x0n+eeew6AsHzHrrvuCsD+++9fiBClib75JrUs\nzCOPPJK3Y4YRFABXX301kBlx0bZt27ydRwrn2WefBeDdd9+t9voJJ5wAQOvWrWvtUyrKNEVEEijL\nTPP881NrxSe5SvbAAw9U+921a1cA7rvvvvQ2P/7xj/MVojTRU089BcCMGTMA+N3vfpfzMdesWZN+\nvGjRIgA+//xzQJlmOYv3Z48aNarObU4++WQAzKwoMWVDmaaISAKqNEVEEiir5nm/fv2AzMWcb7/9\nttF9OnXqBGSaYW+/nZqQ+a233gJg9913T2/73Xff5S9Yydorr7ySfjxo0CAAunfvDsCFF16Y8/Hj\nQ46kcrz88svpx+Fmh6Bly1TVdPjhhxc1pmwo0xQRSaDkmeYzzzyTfvzqq68CmU7f+i4EnXnmmenH\nffv2BaB9+/YAPPnkkwBceumltfa78cYbgdoDaKWw4mURLtCMHz8egHbt2jX5uOECUPxvqJwuGEjD\nwkXbuhx66KFFjCQZZZoiIgmULNMMEzSEPi6A1atX17ltGD507LHHAvCnP/0p/V6bNm2qbRumhhs7\ndmytYw4fPhyAL7/8EshM7tGqVaum/SOkQRMnTgSqD2QPfZnxvuamCsNU4tnlgQceCECHDh1yPr4U\nVryFEGy44YYAXHbZZcUOJ2vKNEVEEihZpvn1118D9WeXkLkF8t577wUyV8obEjLNcFX2ggsuSL8X\nbq0LGWeYfmq77bZLFLtkJ0wMHf7fIT/9yaGVctdddwGZK60AF110EaDWQzkLNzbMnDmz1nuh5di7\nd++ixpSEMk0RkQRKfvW8LqG/a9y4cUB2GWZNIYu8884706/NmTMnD9FJYz755BMAZs2aVeu9oUOH\n5nz8m2++GchMHdizZ8/0ewcddFDOx5fCamgCnkoY2aJMU0QkgZJnmnXd9TN79uycjxvuKorfBVTz\nTqNwFT6MGZT8CBMxrFixAshM75Uvy5Ytq/a8V69eeT2+FFZdmWYY7ZCPlkihKdMUEUlAlaaISAIl\na56HtcsLtbJcWNVu/vz56ddq3p55ySWXFOTc67uNN94YyAwbiU/YEW597NixY+Ljrlq1CsgMZQr2\n3XffJsUpxfX8888DmaFiceE26M6dOxc1pqZQpikikkDJMs0pU6bk9Xhh+MnixYuBhm/DCkOYNAC6\nMMJqgeGWyXA7JcARRxwBVL/poC4LFy5MPw4XfsK0fzUn5WjRQt/9lSCsOBkuyMaV8wQdNemvTUQk\ngZIPOcqXMP3YDTfcUO82VVVVANxxxx1AZiIQKYyRI0cC1TOL0MKIT9RSl8033zz9OGSW9d1ye+qp\np+YSphRJzb7o+KQqZ5xxRrHDaTJlmiIiCVR8phmWyAgTGDck3G633377FTQmSenRowdQfUXQMJqh\n5gD1msI0gHGDBw8Gat+MEPpQpTyFmxxqXjWPXynPx1SBxaJMU0QkgZJlmg0tnjZ16tRqz08//XQA\n3nvvvXqPk80yB/m+Yi/J7bbbbtV+J/GDH/ygztfj40B/+MMfNi0wKZgwFVzNq+YDBgwoRTg5U6Yp\nIpKAKk0RkQRK1jwP8+aFWdTjwgDomrdY1nXLZWjeZ7NypVS20Lyr2cxTk7y8hUHtQbi55Pzzzy9F\nODlTpikikkDJMs2BAwcCMHr06PRrDa0X1Jjw7RWGudxyyy0AbL311k0+ppSXcLFPa5tXlmnTplV7\n3qVLFyAzSUelUaYpIpJAyTLNsGpkWGkSYNKkSQCMGTMm8fH++Mc/Apm1zKX5CevVBxrUXt7CirNL\nly6t9nrr1q2Byp0wR5mmiEgCJb+NMqxtHn/ct29fILPqYJhQ+KijjgLgV7/6VXqfcCU1viKhNE9h\nddIw0cOIESNKGY40IkzZF26RXLRoEQDbb799yWLKB2WaIiIJlDzTrMthhx1W7bcIZDKW3/zmN4DW\nOC93Yex0mLYxjHr40Y9+VLKY8kGZpohIAmWZaYrUJfRtS2XZZpttALjttttKHEl+KNMUEUlAlaaI\nSAKqNEVEElClKSKSgCpNEZEEVGmKiCRgNSd0TbSz2QfA2/kLpyJ0c/fNG9+seVAZN38q42RyqjRF\nRNY3ap6LiCSgSlNEJAFVmiIiCTRYaZrZZma2IPpZaWbvxp5vWKigzKyfmb1mZkvNbFgW24+KxfaK\nmR2R4/mfN7PejWzT2swmRjHONLOuuZyzVEpVxtG5W5rZy2Y2KYtti17GsW2PNzPPdvtyU8LP8R1m\n9oGZLchy+yFhezNbYma/zPH8483s6Ea22Tn6/K4zs6yWx2xwwg53/xDoHR18JPCZu19Z46RG6oLS\nd9mcsDFm1gq4HvgvYCUw18wecvfXG9n1CncfY2a9gKfMbAuPXeUys5bu/k0+YoycAax09+5mdhLw\nV+DEPB6/KEpRxjEXAAuBNlluX+wyxsw2AYYCc/N53GIqYRnfBtwA3Jxgnzvd/Xwz2wpYaGaT3T29\n4mIByng1cA5wbLY7NKl5bmbdzWyxmd0JLAK6mNnHsfcHmdmt0eMtzewBM5trZnPMbK9GDr8XsMTd\n33b3dcB9wIBsY3P3hYABm0bfNDea2RzgMjNrZ2a3R3HMN7OjohjbmNmE6NvtfqB1FqcaANwRPb4P\n+Gm2MVaCApcxZtYNOBQYlzS2IpYxwGXRz7qkcZa7Qpexuz8DrGlKbO6+ElgOdI1aGf9rZi8At0ct\nlKujOF42syFRjC3M7O9m9qqZPQ50yuI877v7XCDrijiXPs2dgGvcvSfwbgPbXQuMdvc+wHFAKIQ9\nzeymOrbfFvh37PmK6LWsmNk+wJfuHgpra2Avdx8OjAAedfc9gIOAq8ysNfBr4CN37wGMAnaLHW9c\nPc2ydJzu/hWw1sw6ZBtnhShUGQOMAYYBice8FauMzWx3YAt3n1bzvWakkGXcZGbWHegGvBmL82B3\nP4lUK29VVMa7A2dbqnvsWOD7QE/gVGCf2PEuNbN++Ygtl/k0l0U1dGMOAXa0zFrVm5rZRu4+G5id\nw/lrGmZm/w18Chwfe31CrMnRFzjczH4fPW8NdAX2B0YDuPt8M1sUdnb3U/MYY6UpSBlH/Uz/dvcF\nZnZIgniKVsZm1gK4igrsckmo3D7HJ5rZgaQy+yHu/nF0zofcPSxH2hfoYWaDouftge1JlfHd0d/C\nCjN7OhzU3f+YrwBzqTTXxh5/R6q5FMSbPgbsEWVj2XgX6BJ73pmGvwGDK9y9rrV/43EacLS7L4tv\nEPtDSCLEudJSnelt3f3jRvapNIUq432AgWbWPzrOJmZ2h7sPbmS/YpZxB1IZy3PRvlsBj5jZEe4+\nP+nBylihyrip7nT3ui7I1Czjoe7+RHwDMzumoJFF8jLkKKrZPzKz7aNv6Hjw04Gzw5N6mrpxs4Ce\nZtbNzL5HqikwOdp3dOijaqJppDp9QyyhifYs8IvotV2BnbM41mQgfMiPAx7LIa6yl88ydvfh7t7Z\n3auAk4DHQoVZLmXs7mvcvZO7V0VxzgX6NbMKs5o8f47rZWbnmdmZTY+UacBQM2sZHW9HM9uIVBkf\nH/VtbgsckMM56pXPcZq/I/WPmUGqHzI4G9g36rBdDJwO9feFuPvXwLnA48BiYLy7vxa9vQupK+pN\ndQnQ1lJDVhYBI6PXrwc2M7MlwMVA+oPRQJ/mzcDWZraUVH/ZhTnEVSnyUsaNKKcyXh/lrYzNbALw\nHKkkaEXUtQLQA/gwhxjHAm8AC8xsIXAjqVbzROAdUvXGOGBmLJY6+zTNrLOZrSBV54yM4mxwNEfF\n3HtuqTbSVHfXEpXNlMp4/WBm/wQG5Ht4WLFUTKUpIlIOdBuliEgCqjRFRBJQpSkikkAu4zTp1KmT\nV1VV5SmUyjBv3rzV69Os3irj5k9lnExOlWZVVRVz51bsPAZNYmbr1bIAKuPmT2WcjJrnIiIJqNIU\nEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCSQ0zhNEZFC+OijjwB455136t2mW7duAFxzzTUA\n9OrVC4AddtgBgF133bUgsSnTFBFJoKwyzVWrVgFw3HHHAbDPPql1kc444wwgdedCPnzyyScAPPvs\nswAcdlhq+sZWrVrl5fgiksyUKVMAePjhhwF4+umnAXjjjTfq3WfHHXcEYPny5QCsW1d9wdDvvsv3\nitMpyjRFRBIoeaYZ+i4Adt45tWxLyAS33HJLIP8Z5o9+9CMAVq9OrUEf7rvdfvvt83Ieyd5//vMf\nAH7/+9TikYsWpRaJnD59enobtQCah2XLUmvd3XDDDQDcfPPN6fe++OILAJJMiv7aa681vlEBKNMU\nEUmgZJlmyPJC/yXAhx+m1lo6++zUonfXXXddXs85atQoAN566y0g802nDLP4xo8fD8BFF10E1L5K\nGjJQgM0226x4gUnBrFiRWqdtzJi6VmHO3k477QRkrpYXmzJNEZEESpZp/utf/wIyV8niRowYkbfz\nLFy4MP34yiuvBOCYY1LLOR9//PF5O49kJ2Qbv/nNb4BMiyO1EGXGOeekly7n+uuvB6Bjx47FCFGa\nIJQjZDLJn/zkJ0BmdMqGG24IQPv27QFo165dep/PPvsMgJ/+9KdAJovcc889Adhtt93S22600UYA\ntG3bNs//iuwo0xQRSUCVpohIAkVvnocB7Pfff3+t92677TYANt889+VZQrP80EMPrfXewIEDAdh4\n441zPo8kE7pIwkW/+txzzz3px1OnTgUyF41C0z0096R01q5dC1T/nL300ksATJo0qdq2e++9NwDz\n588Hqg8lDBcCO3fuDECLFuWbz5VvZCIiZajomeb//M//AJkhJ2GgOcDPf/7zvJ3n+eefB2DlypXp\n10499VQATjrppLydRxr39tuZNazGjRtX7b0wqUK4keHxxx+vtX+4KSFkqSeeeCIAW221Vf6Dlax8\n9dVXAPziF78AMtklwIUXXgjAIYccUue+dd2s0rVr1zxHWDjKNEVEEih6phmGloTf2267bfq9XPqo\nwm1Yl112GZC5VSs+lCX0mUpxLViwIP04DFrff//9AXjmmWcA+PLLLwG46667APjrX/+a3mfp0qVA\nptUwYMAAINPXqaFIxROGBoXPWZhgI34dYtiwYQC0adOmyNEVhzJNEZEESj5hR5gSCqBv374AdOjQ\nAYCzzjqr0f3D4Pjwe9asWdXez2c/qTRNfMqukPmHwe1B69atAfjlL38JwMSJE9PvhYkewmQOIYPR\n1fPiC1fEL7/8ciAzEfBzzz2X3iYMXm+ulGmKiCRQ9EzzvPPOA+DJJ58E4L333ku/F/q3Qkbx0EMP\nNXq8sG3N2/C22247INP3IqVz991313rtn//8JwBHH310nfuE6frqstdeewHVb8OT4pgxY0a15+H2\nxjC+cn2gTFNEJIGiZ5o//vGPAXjllVeA6ldWH330UQBGjx4NwBZbbAHA4MGD6z3eySefDMAuu+xS\n7fWwVEbIOKV0TjjhhPTj0Hp48cUXAXj11VeBzN/Dgw8+CFSfnDr0cYfXwpR+oex79uxZsNilunhf\nM2RGMFxyySXp1/r37w9Un2SjOVGmKSKSgCpNEZEELMmaHDX16dPHG+qwL4Y333wTyDTDe/fuDcBj\njz0G5Gfyjzgzm+fuffJ60DKWjzJes2ZN+nEop3BrZH0X8uITQIQbFY488kgAXn/9dSCzSulNN92U\nU3w1qYzrV/PmlLpssMEGAJx55plAZk7Mf//73wB0794dyKwJFhfWiAqTexTqAlMuZaxMU0QkgZIP\nbs/Vn//8ZyDzzRcuIuU7w5Smi9/mOGHCBACOPfZYoHbGee655wLwt7/9Lb1PGPgepvQLt1hOmzYN\nyAx+B134K7Tf/va3AFx11VX1bvPtt98CmRZC+J1EuAh84IEHAtWnCiw1ZZoiIglUZKYZshWAO+64\nA4BNNtkE0MqF5S5MFxaGroQJOsKwotByCNll3MUXXwzAkiVLgMzwpbAPZP4epDDC7ZNhFdkwTd/X\nX3+d3iasAxUyzqYIk5WHz3p85ckwGXWpKNMUEUmgIjPNMKA27ogjjgCqT2os5StknPVNVFuXsAph\nWEU0ZJpPPfVUeptwpV7TxRVGuDK+++67A5mRDHFPPPEEkMk+R44cCcCcOXMSny/0dc+bNy/xvoWi\nTFNEJIGKzzTD2sfhqp40f6E/bfLkyUD1K6thjfQRI0YUPzAB4OCDD672PNwqHTLNVq1aAZnlZwBO\nP/10AK655hog09ddjpRpiogkoEpTRCSBimqeh9vl4itMhlUMdQFo/RHWxB4+fDhQfX3tcNFh0KBB\nAOywww7FDU5qCSsyhFUqwwWiMFsVwBtvvAFkVmCoKb6WWKkp0xQRSaAiM834ZAH9+vWrts2nn34K\nZOZerKT1lCWZMDnLX/7yl/Rr4YLgH/7wBwDGjx8PZIYrSfH16NEDyAwVu/fee2ttEx82BtCyZapq\nCkMJ47cVUU8FAAAHAElEQVTVlpoyTRGRBCoq06xL+EYKGUUYshBuu9Jtdc3fKaeckn48duxYAB54\n4AEg01dWc2Z/KZ6Q5Y8ZMwbItAbjA9bff/99AKqqqoBMmYY+6nKiTFNEJIGKzzRvueUWAG699VYA\nhgwZAmQmd5DmLz4N4PTp04HMetxhgolyHiy9vggjXaZMmQLAP/7xj/R7M2fOBDKZZZgarhwp0xQR\nSaCiMs3rrrsOgD/96U/p1/bff38AzjrrLAA23XRTADbccMMiRyflIIyWCMtlhFstFy9eDGjlynIS\nVhOt+bjcKdMUEUmgojLN/fbbD4Ann3yyxJFIuQuTHO+6664ALF26FFCmKblTpikikoAqTRGRBCqq\neS6SrbBm1FtvvVXiSKS5UaYpIpKAKk0RkQRUaYqIJGBhtbcm7Wz2AfB2/sKpCN3cffPGN2seVMbN\nn8o4mZwqTRGR9Y2a5yIiCajSFBFJoMFK08w2M7MF0c9KM3s39rygM2KYWUsze9nMJmWx7ahYbK+Y\n2RE5nvt5M+ud5bbHm5lnu325KVUZm9kFZrYo+jkni+2HmNkHUVxLzOyXOZ5/vJkd3cg2Hc1scvR3\nONvMKvIezBKW8Yro87jAzGZnsX3Ryzja7mAzeyn6W2z0Hu0GB7e7+4dA7+jAI4HP3P3KGic0Un2j\n3zV2soQuABYCbbLc/gp3H2NmvYCnzGwLj3XYmllLd/8mnwGa2SbAUGBuPo9bTKUo4+gLZjDQB/gG\neMzMprh7YyPR73T3881sK2ChmU1299Wx4+a7jC8GZrt7fzPbGfh/wKF5PH5RlPhzvJ+7f5xg+6KW\nsZl1BK4D+rr7CjNrdCLPJjXPzay7mS02szuBRUAXM/s49v4gM7s1erylmT1gZnPNbI6Z7ZXF8buR\n+uMclzQ2d18IGLBp9E1zo5nNAS4zs3ZmdnsUx3wzOyo6XxszmxB9u90PtM7ydJdFP+uSxlnuClzG\nPYBZ7v6Fu38NPAsck21s7r4SWA50jVoZ/2tmLwC3Ry2Uq6M4XjazIVGMLczs72b2qpk9DnTK4lQ9\ngSejcy4CdjCzzbKNs9wV+nOciyKW8UnAfe6+IjrvqsZ2yKVPcyfgGnfvCbzbwHbXAqPdvQ9wHBAK\nYU8zu6mefcYAw4DEl/bNbB/gS3dfE720NbCXuw8HRgCPuvsewEHAVWbWGvg18JG79wBGAbvFjjfO\n6mh6m9nuwBbuPi1pjBWkUGX8CnCApZq/bYHDgS7ZBmVm3YFuwJuxOA9295OAM4BVURnvDpxtZl2B\nY4Hvk6oITwX2iR3vUjOrvqxpykvAwGibvYHO0U9zUsjPsQNPmtk8MzstSVBFLOMdgM3M7JnoC+Gk\nxmLL5d7zZe6eTbP0EGBHyyy7u6mZbeTus4Fa/RyW6oP4t7svMLNDEsQzzMz+G/gUOD72+oRYk6Mv\ncLiZ/T563hroCuwPjAZw9/lmtijs7O6n1hFjC+Aq4MQE8VWigpSxuy80s6uB6cBnwHzg2yzOc6KZ\nHUgqsx/i7h9H53zI3b+MtukL9DCzQdHz9sD2pMr47uhvYYWZPR2L54/1nO9S4FozW0CqAn0pyzgr\nSUHKOLKXu78bNbUfN7Ml7j6jkfMUu4xbAj8k1bJtC8w0s5nuvqy+AHOpNNfGHn9HqkkcxJu3Buzh\n7l9ledx9gIFm1j86ziZmdoe7D25kvyvcfUwjcRpwdM3/kNgfQrY6kPo2ey7adyvgETM7wt3nJz1Y\nGStUGePuNwM3A5jZaGBpFrvd6e7nNxKnAUPd/Yn4BmaWdfM/FuMnpPpewxflcqC5zQBSyDJ+N/q9\n0sweAvYAGqs0i1rGwArgXXf/HPg86gLYBai30szLkKOoZv/IzLaP/rjiwU8Hzg5P6mrq1jjWcHfv\n7O5VpPobHgsVppmNDv2QTTQNSF+pNbPQDH8W+EX02q7Azo3EuMbdO7l7VRTnXKBfM6swq8lnGUfb\nbBH9rgL6A/dEz88zszNzCHUaMNTMWkbH29HMNiJVxsdH/V7bAgdkEWMHM2sVPf0VMN3d1za0TyXL\nZxlb6vpBu+hxW1KZ3MLoedmUMTAJ2M/MNoji3AN4taEd8jlO83ek/jEzSNXewdnAvlGH7WLgdGi0\nL6Q+uwArc4jxEqCtpYZBLAJGRq9fT6pfYwmpK6bpyq++Ps31VD7LeFK07STgTHf/T/R6D+DDHGIc\nC7wBLDCzhcCNpFpUE4F3gMWkLjDODDs00N/1Q2Cxmb0GHExqREdzl68y3hp4wcxeAuYAD7r79Oi9\nsinj6MLxk6T62WcDf3f3JQ2dvGJuo7RUO3iqux9W6likcMzsn8CAfA8Pk/JR6WVcMZWmiEg50G2U\nIiIJqNIUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCTw/wF5WJBSe+H5cQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd1dd908048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=images,\n",
    "            img_shape=img_shape,\n",
    "            cls_true=cls_true,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples of Mis-Classified Images\n",
    "\n",
    "We can plot some examples of mis-classified images from the test-set.\n",
    "\n",
    "First we get the predicted classes for all the images in the test-set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred = model.predict(x=data.test.images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we convert the predicted class-numbers from One-Hot encoded arrays to integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred,axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot some of the mis-classified images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xnc1WP+x/HXJ62ktNlKhVQiE7+UPaOFscZEJttESGTf\nfmMZxjJ+GUkkMcYyJUaoFJOKNEgpKi1MsiSkolDI0vX743yv8z3n7l7O9z77fb+fj0eP+yzf5bq7\n7nOdz7Wbcw4REUlNjXwnQESkmKjQFBGJQIWmiEgEKjRFRCJQoSkiEoEKTRGRCFRoiohEoEJTRCQC\nFZoiIhHUTOfkpk2butatW2coKcVh3rx5a51zzfKdjlxRHld9yuNo0io0W7duzdy5c9O5RNExs0/y\nnYZcUh5XfcrjaFQ9FxGJQIWmiEgEKjRFRCJQoSkiEoEKTRGRCNLqPRcRqcjs2bMBuOaaawC44IIL\nADj22GPjx2yzzTa5T1glKdIUEYmgykWa69atA2DFihVlHtOqVSsA7r77bgD23ntvANq2bQvAb37z\nm2wmUYD169cD0KBBAwBq1ND3d1U1dOhQAP7zn/8AMHPmTAD69OkTP+bGG28Ews9iIdNfqohIBEUf\naU6aNAmA559/HoAZM2YAsGzZsjLPadeuHQAff/wxAJs2bUp6f/PmzRlOpZR08sknA2Fb1oABA4Dk\ndq5MWr16NQCNGzcGoGbNov/TLxoHHXQQAOPHjwfg559/BuCZZ56JH/Pqq68CcOuttwLQv39/oDDz\nSZGmiEgEhVeMl2L58uUAjBgxAoAHH3ww/t4PP/wAQJStiN9///0Mpk4qY7/99gNgyJAhAHTr1i2r\n9xs2bBgQRjl33nlnVu8noUsuuQSAX375BQjz4vPPP48fs3btWgAGDhwIhDVFf27z5s1zk9gUKNIU\nEYmgKCLNlStXAuE3VGW1b98eKI4euqpul112ycl9pk6dCoQ9uL79WpFm7l1xxRUAtGjRAoAvvvgi\n/p6vTd5///0A3HXXXQBs2LAh6fVCoEhTRCSCvEeavi0DwkjykEMOAeCoo44CoHbt2gA0bNgQgPr1\n68fP8d9ERx55JBBGkV27dgVg3333jR9br149oLhmH1RVuYocXnnlFSCMMH1bquRP3759t3jN9034\nz7iPNEeOHAnATjvtFD/2hhtuyHYSy6VIU0QkAhWaIiIR5K16vnHjRgB69uwZf23BggVAOAjWO/DA\nAwF45513gNjy/J6fLukblzUdr7AtWrQISB5ukk3Tpk1Lev7nP/85J/eVaHzTmR/c7v9OJk6cCMCU\nKVPix1599dUA1KlTJ5dJjFMJIyISQc4jzZ9++gmAfv36AWF0CfCnP/0JgB49epR6bmk75rVs2TLD\nKZRsmjVrFgDffPNN0uuZ7pzzHT9+MLuPZA477LCM3keyY+zYsUD4d+H/bgC+/fZbAJo1y8+GoYo0\nRUQiyFmk6YcG3X777UC4wEbit8VVV10FwNZbb52rZEkO+LyHcCiJd+KJJwJw3nnnZfSeEyZMAGD+\n/PlJ199uu+0yeh/JDh9pFiJFmiIiEeQs0vQ94nfccQcQLgTsFyaFcGCrVC2XXXZZ/HHJxVKy1Zv9\nj3/8IyvXlezwg9tvu+02YMsaiZ+8AtCoUaPcJawUijRFRCLIWaT5xhtvJD330xv9+EqpevwYu6ef\nfnqL9/xICL8gdKb4Xvkvv/wyo9eV1F188cUA3HfffUC4rNsJJ5wAhJFi4jhL38fx1ltvJV2rQ4cO\nADz00EPx1/K9MLEiTRGRCHJWZI8bNy7p+YsvvgjAzTffHH/t+OOPB5IX2ZDi48fR3XLLLcCWYzIB\nnnvuOQDq1q2b0Xt/9NFHQNhr7p1zzjkZvY+UzbdPmhkQzv7yi2/4BcP9+4l22GEHAC688EIg3PZC\nixCLiBQpFZoiIhHkrHq+Zs0aIAzJ/TS3xOq5n6zv9wnxa2J++umnALRp0waAvfbaa4vrL168GAgX\n91AHU/74nR/nzp27xXt+MHvHjh1zmqYmTZrk9H7Vme8A8nnt/w78gHX/97F+/fotzj3llFMAuP76\n67OezspSpCkiEoFF2cWxpM6dO7vSoonS+CmSJQetZtr2228PwOGHHw7Ak08+mdHrm9k851znjF60\ngEXJY798l1945e233wZgjz32iB/j97f2K3H7vz+/VGBpatWqBYSLb3h+MYfEDgXfAeQ7E3204/fY\nLq3zoSTlcXb5jsLEWqbftcF/fv3Qo2zVGNPJY0WaIiIR5KxN00+f9G0Wp512GpAcPfhdJ3/99ddK\n38e3l/gB1Yk7TxZyO0lV4BfJ8BGm59uvIZwm5/m8fuCBB8q8ro8a/SLU3pgxYwA47rjj4q+99NJL\nScf4BTpSiTAlNxo0aACEZQDAqFGjgPDz66PRQqRIU0QkgpxFmltttRUA+++/PwD//e9/tzhm+vTp\nQBh93nTTTQDMmTMn8v18W9m8efMinyuV49sNS/JbkgCMGDEi8nX90nJ+wemSkUrnzmHTlB+l4V1w\nwQWR7yep8yNeEhd3rsxCz5s3bwbCz20hU6QpIhJB3vc9T9S9e/ek574n1EeavhfVT60COPfccwG4\n++67AXjiiSeynk4pnd+PevDgwUmv+2UAIZwmV3J7iyOOOAIIayKJunTpAsD3338PQNOmTQGYOXMm\nAPfee2/8WN8L3KlTJwDatm1bmV9FKuCnRQ8ZMgRI/kyW5YMPPgBg+PDhQLgnPYTt3sXQ9qxIU0Qk\nAhWaIiIRFFT1vKRevXoB4WBp30H04IMPxo9ZtmwZADNmzCj1GoW0OkpVV3L6q+cHsgM0btwYqNw+\nUCVX7O7ZsycQDnVK5KfTajeAzPIdNqNHjwbCSQm+6QzCYUN+taNHH3006ecnn3wCJFfFfXPNRRdd\nBBR2s4oiTRGRCHI2jbIy/DfV2WefDcBTTz1V4Tl+VedjjjkGCL8RITN7a2uKXeFJHGbkB0X76LYy\nEa3yuGw+ikysPUDyUKGKOnO6desGQO/eveOv9ejRAwhXas82TaMUEcmRgm7TrFevHhBO5v/uu++A\n5AHrfi8Yv+fMmWeeCYQD46Xqa9asWamPJfN8m7TfHdIv0lIaX0P0q/P37dsXgEMOOSSbScw6RZoi\nIhEUdKTp+QHRkyZNAuCf//xn/L1Zs2YBYWTpl5YSkczzfQYvvPBCnlOSP4o0RUQiKIpIs6Qzzjij\n1MciItmmSFNEJAIVmiIiEajQFBGJQIWmiEgEKjRFRCJQoSkiEoEKTRGRCFRoiohEkNbScGa2Bvgk\nc8kpCq2cc9VmVQjlcdWnPI4mrUJTRKS6UfVcRCQCFZoiIhGUW2iaWRMzmx/8W2VmnyU8r52tRJnZ\nSjN7N7jP7BSOH2Bma4Ljl5rZ2Wnef7SZ9U7huO5mtsDMFpvZy+ncM1/ylcfBvWua2UIzG5/Csbcm\npO1dMzsmzXu/ZmadUjjuD2a2JMjjx9O5Z77kI4/NbBszmxPcY4mZ3ZjCOTnPYzO7OigzFpjZVDPb\npaLrlrvKkXPuK6BTcPGbgA3Oub+VuKkRaxvdXNHNIjrUObc+wvFjnHOXmtmOwCIzm+icW5uQzprO\nuV8ylTgzawzcC/Ryzq00s6JcyDPPeXw5sAhIdSOfO51zw8xsb+AVM9veJTTKZyGP2wNXAgc559Yr\njyP5Afitc26jmdUCZpnZC865ijYjymkeA3OBe51zP5jZYOAO4LTyTqhU9dzM2gTfHmOAxcAuZrY+\n4f1TzezvweMdzOxZM5sbfPMcUJl7pso5twr4GGgZfHM9bmavA48Gkc3QIB0LzWxAkMYaZna/mb1n\nZlOBpinc6nTgX865lcF9V2fpV8qLbOexmbUCegKPRE2bc24RYECjoFYw0szmALebWX0zezRIxztm\ndlxwv63N7OkgqngGqJvCrc4j9oFaH9xXeZxiHjvnNjvnNgZPawO1gJR7nXOVx865l51zPwRP3wRa\nVHROOm2a7YG7nXMdgM/KOW44MCTY+e0UwGdCVzN7oIxzHPCymc0zs3OiJMrM2gCtgA8T0tndOXc6\nsQ/BaudcF2B/4EIzawn0AXYFOgD9gYMSrnebmR1dyq3aAk3M7NXgD+n0KOksEtnM42HAVUT4IHlm\ndhDwo3Pu6+ClnYADnHNXAzcC/w7y+AjgLjOrC1wErHPO7QncCuybcL1HyqjGtQX2NLPXzWyWmfWK\nmtYikLU8NrPaZjYf+BKY5JybV9pxZZybqzxOdA7wYkVpS2cR4uUphNoAPYB2Fm7r2cjM6jnnZgNl\ntVce4Jz7LKhqTzWzpc65Nyq4z2lmdjiwCRgQVKcAJjjnfgyO6UXsQ3Bq8LwhsAdwGDA2qJqsNLMZ\n/qLOuevKuF9NoCOxaGkbYtWPWc655RWks5hkJY8t1l78qXNuvpn1iJCeq8zsj8B3QN+E159OqFb2\nAn5nZtcGz+sCLYnl8RAA59w7ZrbYn+yc61/G/WoCuwHdiH0Rv2pmHZxz30ZIc6HL2ufYOfcT0MnM\nGgHPmdmezrmlFdwn13kMQHDPjsDFFaQvrUJzY8LjzcRCaS8xLDagS/AfmBLn3GfBz1VmNgHoAlRU\naI5xzl1aQToNGOScm554gJmdmGraEqwEPnPOfQ98HzQB7ANUpUIzW3l8EHCSmR0fXKeBmT3mnDur\ngvPudM4NqyCdBvQu+eVlFezFXYaVwKtBG9pyM1sO7A68U5mLFaisfY4959w6M5sJHAlUVGjmOo8x\ns6OI1Xq6pfL7ZWTIUfANsM7M9jCzGkBiITQNuDAhgRX1ZtU3s/rB422IRXKLgueXmNnANJI6BRhk\nZjWD67Uzs3rATKBv0LbZnFhkUZHxwKFmtlWQzi7Ae2mkraBlMo+dc1c751o451oTaxt+yReYZjbE\nt1FV0hRgcEJafBVtJtAveO03wF4pXGs8cHhwzvbECsyP0khbQcvw53h7M2sYPN6aWKT6XvC8YPLY\nzDoDI4DjEzuOy5PJcZrXEPtl3iD2De1dCBxssY6XJcC5QWLLagvZCXjdzBYAc4DnnHPTgvf2BL5K\nI42jgGXAfDNbBIwkFm2PA1YAS4h1TMzyJ5TVphk0VL8MvEusenJ/ClWPYpepPC7PPsCqNNJ4M7CN\nxYasLAZuCl6/j1gb9FLgBhKixXLauyYDG4LfaRpwWcQRHcUoU3m8M7HmDP85nuyc+3fwXiHl8d+I\nNa89Y7GhTs9VdPOimkZpZpOBEzI85EAKhMXqVy86547Kd1okO6pCHhdVoSkikm+aRikiEoEKTRGR\nCFRoiohEoEJTRCSCdAa307RpU9e6desMJaU4zJs3b211WtVbeVz1KY+jSavQbN26NXPnpjIDq+ow\ns2q1LYDyuOpTHkej6rmISARpRZrZtnlzbH7+ww8/DMDChQsBuPfee/OWJhGp3hRpiohEoEJTRCSC\ngq6e//e//wVg4MDYwkbHH398PpMjIqJIU0QkioKONE866aSk53vtlcoSiCIi2aNIU0QkgoKKNH/+\n+WcALr/8cgA++OADAC6+OLZtx80335yfhImIBBRpiohEUFCR5uTJkwG4//77AXjooYcAOPvss/OW\nJsmOjz/+GAhHSIwePRqAjz6KbcGz2267xY/98MPYbsynnXYaAP37xzYWrFOnTk7SKpJIkaaISAQF\nFWn+61//AqBVq1YA9OnTJ+1rrl0bbjDnt/hs0qRJ2teVyvn8888BOPzwwwFYsWIFAH7bFZ9Hr7/+\n+hbnvvbaa0CYp9dff31W0ypwwQUXANC1a1cA/vjHP+YxNYVBkaaISAR5jzRfeOGF+OPnn38egNtu\nuw2ABg0aRL7e999/D8Cdd94JwPDhw7c4plu32Lbmzz77bOTrS3oaNWoEwBlnnAGEeV23bl0Azjvv\nPABatmwZP+eqq65KusYDD8R2jD3//PMBaNas2ix9mXP+/9p/Vvbbbz8A9tlnn7yladGiRUDYDn7N\nNdcA4d9WtinSFBGJQIWmiEgEea+e+yo5QMeOHYFwMHsUvpPghBNOAGD9+vVlHuubBP7zn/8AcOih\nh0a+n1ROvXr1ALjyyisB+PTTTwE49thjgbDzb9WqVfFzSlbPf/jhBwB+/PHH7CZWaNiwIQBr1qwB\n4MknnwSgTZs2AGy99dZZvf/XX38NwBNPPBF/7S9/+QsQdgj6v5VHH300q2nxFGmKiESQ90hzzJgx\n8cc+Soxi48aNQBiN+Ajz5JNPBsKOBQgj2GXLlgGKVPLJRzBlRQdffPFF/LEfjlTWc8ke39nil2W8\n4447gHCK87XXXgvAjjvuGD9n5513jnwfP/Rszpw5QFgbfPXVV4Fw0kNpShuelk2KNEVEIshbpOmH\nDfiIA2DkyJGRr9OvXz8g/Ibq27cvEH5D1qgRfi9su+22QDi8pWfPnpHvJ7kxffr0+GM/4N1HmD/9\n9BMQtm1K9hx11FFJP6dMmQLAuHHjAJg0aRIAtWvXjp/jH/uIc9OmTUBY+/PWrVsXf/z4448D8N13\n30VOY+/evSOfkw5FmiIiEeQt0hwyZAiQ3PtWv379lM717RwAEydOBKBLly5AuFNlYoTpderUCYDF\nixdXIsWSS0uWLCnzPT/Kom3btrlKTrVVs2asiPBtjDfccAMAo0aNAsIe7NL6BxKnMAPceuutFd7v\nxBNPBMLRML7XvrQ0/fWvfwXg3HPPrfC6maRIU0QkgrxFmgsWLACge/fukc9N/Mby7V1+8YbyFuOY\nP38+oA3aCtm0adOA8qe4ZmIhF6mcW265BYCjjz4agKeeegqAxx57LH6M7zPwn0Vfa/ARoueX+IOw\nHHjkkUeALds2a9WqFX/8yiuvAHDQQQel86tUmiJNEZEI8j5O04+ZjMIvSgvQokULIFxqrKSFCxfG\nH/se+x49ekS+p+SG75395ptvyjzm4IMPzlVypAwHHnhg0s9hw4Ztccwbb7wBhGMwGzdunPR+4ufw\n7rvvBsL8L8nPAoL8RZieIk0RkQhUaIqIRJD36vkRRxyR8rFLly4FkgfFNm/eHAgHrnsbNmwA4Kyz\nzoq/5tfa9KtRS+HwDf9+koLv4Et02WWXAeHwMilsvhpdVnU6cTqsHzpYUtOmTYHC+swq0hQRiSDv\nkWZiw+///u//lnus7zRK7CT485//XOqxI0aMAJI7gn7/+98DWzZIS/49/PDDQLhcX2n8sLLSJi5I\n8fE1R9gy37fbbjsgHHpWmV0cskV/fSIiEeQ90nz//fczer0JEyYA4XSvRFdccQUQDr6VwvHMM8+U\n+Z7fCyhXe8BIbpQ2rdL3TfghRoccckhO05QKRZoiIhHkLdIcOHAgAJdcckn8Nd9mlcrEfu+tt94C\n4L333gPCHla/+Edir5zfSU8Kj1+gwfNtWgA33nhjrpMjWeRHv/jl5RL5RY0vuuiinKYpCkWaIiIR\n5C3S9OOuXnrppfhrvgd19erVANx1111A2M7x1VdfbXEdv+HS2LFjk1730WohtolIyC/Q4fnxmccd\nd1z8tZ122imnaZLs8stC/vLLL1u8VwwjIwo/hSIiBUSFpohIBHkfcuQHnAPMmzcPCKvpfjiS33Pk\n7bffTvm6u+22W6aSKFngdxEdPHhwqe/7fdGl6vCf37/97W9lHlNyOnQhUqQpIhJB3iPN008/Pf54\nn332AcK9sB966CEgXGjDR4+Ji3D4fc79as7XXXcdsOXOd1JY/AIdmZ7cIIVr9913B8LPcWlr6fp9\nvAqZIk0RkQjyHmkm8pHm0KFDk35K1ZW4PFhpz6Xq8DvPJu5A69WpUweAvffeO6dpqgxFmiIiERRU\npCnVh59C5wez+wjT7xTaoUOH/CRMssbvBut3ok3k9zsvpCXgyqJIU0QkAkWakhcdO3YEoH379gB8\n/PHHQLhzYck9sqVq69evX76TkDJFmiIiEejrXPKiW7duAEyfPh2Ae+65B4C2bdvmLU2SXX58Zteu\nXQF499134++1aNEiL2mqDEWaIiIRqNAUEYlA1XPJK79W5h133JHnlEi2NWnSBIDJkycDsGbNmvh7\n7dq1y0uaKkORpohIBIo0RSSnGjdunPSz2CjSFBGJwNJZIMHM1gCfZC45RaGVc65ZvhORK8rjqk95\nHE1ahaaISHWj6rmISAQqNEVEIlChKSISQbmFppk1MbP5wb9VZvZZwvPa2UqUmTU2s2fN7D0zW2pm\nXSo4foCZrQnStdTMzk7z/qPNrHcFx5xpZu+a2UIze93MOqZzz3zJRx6bWYeEe8w3s+/M7KIKzslH\nHu9lZrPMbJOZXZrO/fJJeZzSsQea2a+pHF/uOE3n3FdAp+CiNwEbnHNJ+29abBVZc85tTiVxKboX\nmOicOynI1HopnDPGOXepme0ILDKzic65tQnprOmc+yWDaVwOHOqcW29mxwEPAAdn8Po5kY88ds4t\nSbhnLeAzYHwKp+Y6j9cCg4E+GbxmzimPy2dmNYHbgampHF+p6rmZtTGzJWY2BlgM7GJm6xPeP9XM\n/h483iGIGuea2RwzO6CCazcGujrnHgVwzv3knPsm1bQ551YBHwMtzexWM3vczF4HHjWzmmY2NEjH\nQjMbENyzhpndH0S2U4GmKdzndeec/53fBIpnmZYUZDOPS+gJLHXOrUz1hBzm8ZfOublARj+khUJ5\nHHcp8CSxL8kKpTMjqD1wpnNublBSl2U4MMQ596aZtQYmAXubWVegv3NuYInjdwPWmNnjQEfgLeBS\n59z3qSTKzNoArYAPE9J5mHPuRzMbBKx2znUxszrAm2b2EnAAsCvQAdgZWEIscsTMbgNed869UM5t\nzwFeTCV9RSZbeZzoVGBslETlKY+rqmqdx2bWEjgG6A4cmkra0ik0lwffwhXpAbSzYC8YoJGZ1XPO\nzQZml5GmzsSqRfOIVdWvAm6u4D6nmdnhwCZgQFBtBpjgnPsxOKYXsKeZnRo8bwjsARwGjA2qJivN\nbIa/qHPuuvJuamY9gDOAQypIXzHKVh4DYGZ1if3BXp5ievKSx1Vcdc/jYcDVzrnNCb9budIpNDcm\nPN4MJN6xbsJjA7o4535K8borgRU+I83sGWLhc0XGOOdKOy4xnQYMcs5NTzzAzE5MMW1JzKwTMAo4\n0jm3rjLXKHDZymPvGGB2YptVBXKex9VAdc/jzsDTQYHZFOhlZr86554v64SMDDkKSvZ1ZraHmdUA\nEhM/DbjQPwkKmvKutRL4MgjPIRY2LwnOvcTMyqsGVGQKMMhXQ8ysnZnVA2YCfYM2keZAt4ouFFRR\nxgH9nHMfpJGmopDJPE7wB0pU2wopj6ub6pjHzrmWzrnWzrnWxDqqziuvwITMjtO8htgv8waxaNG7\nEDg4aLBdApwLYGZdzeyBMq41GHjKzBYCewF+scU9ga/SSOMoYBkw38wWASOJRdvjgBXECudHgFn+\nBDO7zcyOLuVaNwGNgVEWGyJRZhWlCslYHpvZtsBv2bJHtWDy2MxamNlK4GLgJjNbaWZbp5G2YlCt\n8rgyimruuZlNBk7I9JADKRzK46qv2PO4qApNEZF80zRKEZEIVGiKiESgQlNEJIK09ghq2rSpa926\ndYaSUhzmzZu3tjqt6q08rvqUx9GkVWi2bt2auXNTmUxQdZhZtdoWQHlc9SmPo1H1XEQkAhWaIiIR\nqNAUEYlAhaaISAQqNEVEIlChKSISgQpNEZEIVGiKiESgQlNEJIK0ZgTlyi+/xJbd83t4bLXVVvlM\njmTAE088ARCfiTJs2LAyj/XLFx544IEAHHfccQCcd955ADRp0iRr6RQpSZGmiEgEBR1pTpo0CYAz\nzjgDgKZNY9sY/+lPf4ofc9ZZZwFQo4bK/0J1/fXXxx/fd999APzwww/AlrWI8syePTvp57vvvguE\nUavk3rp1sf0Ex4wZE3/tjjtiu9N89tlnpZ7Tu3dvIPzsJr5WDFTSiIhEUNCR5t577w3A2WefDcC4\nceMAOOecc+LHPPLIIwD8/e9/B6Bt27a5TKKUw9cIhg4dGn/t559/TjqmYcOGQBhpHHvssQDUrl07\nfszxxx9f6vWXL18OwNq14e6wvjYi2eVrCieeGNuw8tVXX93imN/+9rcA7LPPPgC0a9cOgOeeew6A\n008/PX7s6NGjgeKIOBVpiohEUNCRpl8Y9a677kr6+dJLL8WP6du3LwD/8z//A8B7770HQPPmzXOV\nTCnhww8/BOChhx4CYPvtt4+/169fPwD69+8PhBHlrrvumnSNxHZrb/fddwegUaNGQNjz/tFHH8WP\nUaSZGw8++CAQRpiJ+ecjzAceiO3sW6tWraRzzz//fCD8W4Dwc/zkk08CYQRbiBRpiohEoEJTRCSC\ngq6el6VXr17xx74D6JRTTgHCYSiqnueP77j5+uuvgeTGfT8cpSKDBg3a4hzfMeiHLR1yyCEAjBw5\nMn7s/vvvX9lkSwTDhw9Pev7vf/87/riizlg/PNB34gL89NNPAFxzzTUAHHzwwUBy006hUKQpIhJB\nUUaaiX7/+98D4bfbO++8A8BRRx2VtzRVV59//jkAK1asSPtaLVq0iD++4YYbgDDiHDBgQNKxZQ2i\nltzxEw4g9WF/9erViz++9dZbAejevTsAJ5xwAgCzZs3KVBIzRpGmiEgEBRVp+ojhqaeeAsLhDH6I\nSZ8+fYCwLQtg5cqVAGzcuBEo7KEKVd2GDRsA2Lx5c9LriYOYK+Pmm28G4KuvvgLCBTu8PfbYI63r\nS3T33HMPEPYlXH311fH3OnbsCECnTp1Svl6HDh2AsH164MCBQDiIPjEqzTdFmiIiEeQt0vzuu+8A\nePrpp+OvXXbZZUC4FJhf8uvXX38F4LHHHgNgxx13jJ/TsmVLALp27QpA+/bts5lsKYdvy2rcuDEQ\n1hx8FJGuwYMHAzB27FgA1q9fn5HrSnR+uquvBdx4443x9373u98BMGrUKKDsabCLFi2KP7722muB\nsG3TlwEPP/wwABdddFHG0p4uRZoiIhHkLdJ85ZVXAJgyZUr8tUsvvRQIF+TwUaQfw+WPTfzmWrVq\nFQANGjQA4JNPPgGgVatWWUu7lM+3b11yySUATJw4Mf7eVVddVenr+ql6vn3LR5qJi4D4peZq1iyo\n5voqy+fTy5kQAAAJjklEQVTnnnvuGX/t1FNPBcL+hT/84Q8A/OUvfwFgt912A5JrCpMnTwbCRT38\nAtOXX345kDzl0tdk8kWRpohIBDn/On7ttdeAcGHhxMVLfTtJSX5RB987m8hHHzNnzgTCts0JEyYk\nPZfc8dtS+Oj/H//4R/w9HyUkLu9XkRkzZgDwf//3fwB88cUXSe8nLkvm/w6OOOKIiKmWdCR+dt98\n800AbrnlFiD8jPvP5L777gskj4LxFi9eDMBf//pXIGzjLDkiI58UaYqIRKBCU0QkAvNd+5XRuXNn\n59c0TNUBBxwAwKZNm4CwOgWw7bbblnrOiy++CIRVuh122CH+nl8owE/j8p0PvoPIT6uEzAxHMrN5\nzrnOaV+oSFQmjz0/qL20PXx22WUXIBzEXB6/tqbfR8jvLeMXfJg2bVr8WL+Yy7PPPgtUbiVw5XFm\n+LJl6dKlQJhvq1evBkqfbuvz2Hci+XwcP358/Bg/xTId6eSxIk0RkQhy3hHkl2zzqzuXFV1C+E10\n7rnnAlC3bl0gjDwhjDr9MCQ/kLpnz55AcqThV3z3Q5kku3wk6AelQ5gffvpr4k6VZfHLvfkhaSed\ndFLS+4cffnj8sR8k7WsexbDnTFXlo0b/mXzrrbcAWLNmDRD+DQBcd911QPjZThzCBGG0CpmJNNOh\nSFNEJIKct2n6b5uTTz4ZgHvvvTf+nl+Iwe93fuWVVwKwzTbbAOFCHm3atKnwPh988AEQRpwAderU\nAWDevHlJ141C7V3p8VGG3z8mcX8fCPPILwcHUL9+fSAcwlQePxHC/335xSP80JVUKI9zz+//5T/z\nfq8vv3fQzjvvHD/2hRdeSPt+atMUEcmRnLdp+vapM888E0ieEunbqvyUqp122gmAl19+GUj+tqmI\nj0anTp0af22//fYDwm8vv1hIIS07VdU1a9YMSI4kM8lPhNhqq60A+Oc//wlEizQl/7beemsgXIza\n1xgKgSJNEZEI8raqgV/UIXGc5rJly4CwvctHnqm0ZZUlsf3TR5Z+iww/ZnTOnDlA2J4mxc/nu++N\nHTduHBAuZC3FpWHDhvlOQpwiTRGRCFRoiohEkLfquR9Gkrhfst8P2TfmZ9qRRx4JhMOefMeQX6HF\nr9fp0yHFy++f7ddtHT16NKDqebH48ssvAZg+fTpQ+opI+aLSQUQkgrwvb+2nRuaSn6Ll9x/xa3ve\nd999AFx88cU5T1N151df95MSEvkpdp4fuO5XBC/NsGHDAHj77bcB6N+/f0bSKbnx4YcfAvDjjz8C\nYS2xECjSFBGJIO+RZj75fUf81D6/G6Zftgy0j3qujBgxAoArrrhii/f8VF+/AIRf5MPvdpnI1x58\nG6bfK6gyU2Ylf26//fak54mfyXxTpCkiEkG1jjQ9v6eyH/zulyADRZq54pfr8xMZvv322zKP9VNx\ny+On4Q0aNAgIF8CV4rBgwQIgjDALaeKJIk0RkQgUaRIu7uCndBbSznfVhZ8ye8wxxwBh7zeEbZp+\n32y/VUppfGTiF5xu27Zt5hMrWeenTfrFespbrDzXFGmKiESgSDOBnwmkGUH549uu/IyeRH4rC6l6\n/NJvu+66KxCOy0xlwfFcU+kgIhKBCk0RkQhUPReRvPP71fvpk4VMkaaISAQqNEVEIlChKSISQVr7\nnpvZGuCTzCWnKLRyzjXLdyJyRXlc9SmPo0mr0BQRqW5UPRcRiUCFpohIBOUWmmbWxMzmB/9Wmdln\nCc+zsvuZmbUysxlmtsTMFpvZRSmcM8DM1gTpWmpmZ6eZhtFm1ruCYxqb2UQzW2hms82sQzr3zJd8\n5HFw38uD/F1sZoNTOD4fedzIzCab2YIgnWemc898UR6Xe8xJwWd4vpm9ZWYHVXhh51xK/4CbgCtL\ned2AGqleJ4X77Ax0Ch43AJYDbSs4ZwAwLHi8I7AWaFrimJoR0jAa6F3BMXcD1wWP9wKmZur/IF//\ncpjHnYAFQD2gFvAKsGsB5vGNwG3B4x2AdVHuUYj/lMdbHFOfsG9nP2BRRdetVPXczNoEkeAYYDGw\ni5mtT3j/VDP7e/B4BzN71szmmtkcMzugvGs75z53zs0PHn8LvAc0TzVtzrlVwMdASzO71cweN7PX\ngUfNrKaZDQ3SsdDMBgRprGFm95vZe2Y2FWiawq06AC8H91wMtDWzJqmms9BlM4+BPYE3nXM/OOd+\nBmYCKa/2nMM8doBfk6w+sQ/xr6mms9Apj8E5t8EFJSawDbE8L1c6bZrtgbudcx2ALTdrCQ0Hhjjn\nOgOnAD4TuprZA+XdwMx2A/YG3ko1UWbWBmgF+PlY7YHuzrnTgfOA1c65LsD+wIVm1hLoA+xKrCDs\nDxyUcL3bzOzoUm61ADgpOOZAoEXwryrJVh6/C3SzWBPHNsDvgJQ3gclhHt8DdDKzz4nl9+CED1hV\nUd3zGDPrY2bvA+OJRbvlSmfu+XLn3NwUjusBtLNgUyygkZnVc87NBmaXdZKZNQCeIfaHuiGF+5xm\nZocDm4ABzrn1wT0nOOd+DI7pBexpZqcGzxsCewCHAWOdc5uBlWY2w1/UOZe8f2zoNmC4mc0n9oFa\nQBWKQgJZyWPn3CIzGwpMAzYA75Da/12u8/hoYA7QDWgL/NvMOqb491gsqnse45wbB4wzs98CtwTX\nL1M6hebGhMebibWJeImbmRvQxTn3U6oXtljj9LPAI865iSmeNsY5d2kpryem04BBzrnpJe4XeSMg\n59w3wFnB+TWIVSU+inqdApe1PHbOPQg8CGBmQ4AtNzzfUk7zmFi0clMQXb5vZp8SKzzfrsS1ClV1\nz+M459wrZvaYmW3nnFtf1nEZGXIUlOzrzGyPoABJTPw04EL/xMw6lXcti32tPArMd84NL/HeJWY2\nMI2kTgEGmVnN4HrtzKwesfaWvkGbSHNikUW5zGw7M6sVPD0fmOac21jeOcUsk3kcHLN98LM1cDzw\nZPC8YPIYWAF0D66zE9CGqvfFGFcd8zho17XgcWdinUJlFpiQ2XGa1xD7Zd4AVia8fiFwcNBguwQ4\nN0hgWW0h3YA/AD0tHBZxZPDensBXaaRxFLAMmG9mi4CRxKLtccQ+IEuAR4BZ/oRy2kI6AkuCtpDu\nwOVppKtYZCqPAcYHx44HBgadflBYeXwTsXa5hcBUYr3O69JIWzGobnl8CrAoaGYbDvSt6OZFNY3S\nzCYDJzjnfsl3WiQ7lMdVX7HncVEVmiIi+aZplCIiEajQFBGJQIWmiEgEKjRFRCJQoSkiEoEKTRGR\nCFRoiohE8P9HGXkNvGwz8QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd1d6782550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors(images, cls_true, img_shape, cls_pred, data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save & Load Model\n",
    "\n",
    "NOTE: You need to install `h5py` for this to work.\n",
    "\n",
    "This is the file-path where we want to save the Keras model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "path_model = 'model.keras'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Saving a Keras model with the trained weights is then just a single function call, as it should be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "model.save(path_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete the model from memory so we are sure it is no longer used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "del model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to import this Keras function for loading the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from tensorflow.python.keras.models import load_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loading the model is then just a single function-call, as it should be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model1 = load_model(path_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then use the model again e.g. to make predictions. We get the first 9 images from the test-set and their true class-numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "images = data.test.images[0:9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_true = data.test.cls[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then use the restored model to predict the class-numbers for those images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_pred = model1.predict(x=images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the class-numbers as integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cls_pred = np.argmax(y_pred, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the images with their true and predicted class-numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4VdW19/HvQDAIKIjYKScRC0gUE+yxXAtRVFBiFGPh\nGtEoxhJvIIlRggkag43XEkV9RG+wgopIRBS70oSASrGAokEvIqJGUbGN94+95t7r9L3O7off53nO\nc3ZZZcA8e+4x55prTnN3REQkOy1KHYCISCVRpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIgmo0hQR\nSUCVpohIAqo0RUQSaJnLzp06dfKqqqo8hVIZ5s2bt9rdNy91HMWiMm7+VMbJ5FRpVlVVMXfu3FwO\nUXHM7O1Sx1BMKuPmT2WcjJrnIiIJqNIUEUlAlaaISAKqNEVEElClKSKSQE5Xz0Wa6sorrwTgiy++\nAODll18GYOLEibW2PeusswDYe++9ATj55JOLEaJInZRpiogkoExTiur4448HYMKECXW+b2a1Xrvp\nppsAmD59OgAHHHAAAF27di1EiFJCr7/+OgA77rgjANdeey0A55xzTsliqkmZpohIAso0peBCdgn1\nZ5g77bQTAIcddhgAb775Zvq9yZMnA7B06VIAxo8fD8CFF16Y/2ClpObPnw9AixapfG7bbbctZTh1\nUqYpIpKAMk0pmHA/84MPPljrvV69egGZLLJTp04AtGvXDoCvvvoqve2ee+4JwEsvvQTAhx9+WKCI\npdQWLFgAZP4OBg4cWMpw6qRMU0QkgaJnmmEc3i233ALANttsk36vdevWAJx44okAbLXVVgB07969\nmCFKnvzf//0fAO6efi1kmNOmTQNg6623rnPfMI4TYMmSJdXeO/LII/Map5TeK6+8AsB1110HwCmn\nnFLKcBqkTFNEJIGiZ5rDhg0DYPny5fVuE8blbbLJJgD07NkzL+fu0qULAMOHDwegT58+eTmu1O2o\no44CMle9ATbeeGMAOnbs2OC+9957b/pxvH9TmqfXXnsNgLVr1wLVR1yUG2WaIiIJqNIUEUmg6M3z\nW2+9FcgMH4k3vRcvXgxkBrg+/fTTAMyaNQvI3Db3zjvv1Hv8Vq1aAZkhLOFiRPw4oZmu5nlxdOvW\nLettr7jiCiBzO11cGHoUfkvzMXr0aCC19AaU92dTmaaISAJFzzQPPvjgar/jwi10wUcffQRkMs/w\n7fPiiy/We/zvfe97QOaG/3B7HsCaNWsA2G677ZoUuxTOlClTABgxYgQA69atS7+35ZZbAnD55ZcD\n0KZNmyJHJ4UQvxgcPtPhc9u2bdtShJQVZZoiIgmU9W2Um266KQAHHXRQtdfrylJruv/++4FMtgqw\nyy67ADBo0KB8hSh5Em65jGeYQRh+EqaEk+bhmWeeqfXa5puX/3LzyjRFRBIo60yzKVatWgXA0KFD\ngeq38IX+ssYGVkvxHH300UDmtspg8ODB6cejRo0qakxSHGGJk7hw40k5U6YpIpJAs8s0b7jhBiCT\ncXbo0CH9XrgyJ6UXxs/OmDEDyPRlhj6tiy66KL1tmCZMmoeZM2cCMG7cuPRru+22GwCHHnpoSWJK\nQpmmiEgCzSbTfP7554HMWL7goYceSj8O05JJ6YXJZVevXl3t9TAtoMbSNl9PPPEEUH1kSxijHaaH\nLGfKNEVEElClKSKSQLNpnj/yyCNAZu7FQw45BIC99967ZDFJbWFNoHBrbHDggQcC8Oc//7nYIUmR\nhcl64n7+85+XIJKmUaYpIpJAxWeaX3zxBQCPPvookJmw45JLLgEyU8VJ6cRXj7zsssuA2rOx9+7d\nG9DwouZs5cqVADz33HNA9cl0jjnmmJLE1BTKNEVEEqj4TDNMWhv6yA4//HAA9tlnn5LFJNVdddVV\n6cdz5syp9l64jVJ9mc3f7bffDsD7778PZD6rlUaZpohIAhWZaYYJawH+8pe/ANC+fXsALr744pLE\nJPW7+uqr630v3Paqvszm7+233672PEz9WGmUaYqIJFBRmWa4CnvuueemX/vmm28A6NevH6BxmZUm\nlGk2oxxCayJs+/XXXwPwySef1No23KJ3zTXX1HmsDTbYIP34b3/7G6BlNArt4Ycfrvb8yCOPLFEk\nuVGmKSKSgCpNEZEEKqJ5/u233wKZmVDeeuut9Hvdu3cHMheEpLKEdZuycdxxxwGw9dZbA5mhK/fc\nc09OMYTVLuNzeEr+hMHsobwqnTJNEZEEKiLTXLZsGZBZsTAuDGfR/IvlK1ykA5g0aVKTj3Pfffc1\nuk24SNSiRfV8oH///gD06dOn1j4/+clPmhyTNO7BBx8EMhdtwyztlbq6qDJNEZEEyjrTDINh+/bt\nW+31K6+8Mv24UoctrE8eeOCB9OPRo0cDtSfsCBYvXgw03E952mmnAdCtW7da7/3sZz8DoEePHk0L\nVvLm888/B2Dq1KnVXg/TwMWHfVUSZZoiIgmUdaY5duxYoPbtV/G+EDMrakySm2zXtb7rrrsKHIkU\nWuhfDivCDhgwAIDzzjuvZDHlgzJNEZEEyjLTDOO6rr/++hJHIiJNFTLNsM55c6FMU0QkgbLMNMMa\n5p9++mm118PdP5pGTERKRZmmiEgCqjRFRBIoy+Z5TWGlwieeeAKAjh07ljIcEVmPKdMUEUmgLDPN\nP/zhD9V+i4iUC2WaIiIJmLs3fWezD4C3G92weenm7puXOohiURk3fyrjZHKqNEVE1jdqnouIJKBK\nU0QkgQYrTTPbzMwWRD8rzezd2PMNCxGQmfWMnWOBmX1qZr9uZJ8hZvZBtP0SM/tljjGMN7OjG9nm\n97EYF5nZN2bWPpfzlkKJyribmT1tZouj/7sGyzfapxRlfIqZvWJmL5vZC2b2w1zOWSqlKOPovHeE\nMsty+1KU8c5mNtPM1pnZ+Vkd2N2z+gFGAr+t43UDWmR7nCQ/QCtgFdC5ke2GAGOix1sBq4FONbZp\nmeC844GjE2x/DPBYIf4PivlTrDIGtgF6R483AZYBO5RbGQP7Ah2ix0cBL5S6jCqljKNjHgDsASzI\ncvtSlPGWQB/gcuD8bI7bpOa5mXWPsoQ7gUVAFzP7OPb+IDO7NXq8pZk9YGZzzWyOme2V4FSHAkvc\nfUW2O7j7SmA50NXMRpnZ/5rZC8DtZtbSzK6O4njZzIZEMbYws7+b2atm9jjQKUGMACcAdyfcp6wV\nsozd/T13XxA9/g/wKrBttrEVq4zd/QV3D//mWUDnbGOsBIX+HLv7M8CapsRWxDJ+393nAt9kG1su\ng9t3Ak5x97lm1tBxrgVGu/ssM6sCpgC9zGxP4FR3P7OBfQeRsDIys+5AN+DNWJz7u/uXZjYUWOXu\ne5jZ94BZZvYYsBfwfaAnqSxoMXBTdLxLSWUYj9RzvnbAIcDpSeKsEAUvYzP7AdALeDHboIpdxpHT\ngKkNvF+pivE5TqxEZZyVXCrNZVEN3ZhDgB0tsyzFpma2kbvPBmbXt5OZtQaOAC7IMp4TzexAYB0w\nxN0/js75kLt/GW3TF+hhZoOi5+2B7YH9gbvd/TtghZk9HQ7q7n9s5LwDgGfc/ZMs46wkhS7jTYD7\ngXPc/bMszlOSMjazQ4CTgea41m9By7gJSvU5zlouleba2OPvSPWJBK1jjw3Yw93rXn6wfkcAs919\ndZbb3+nudXXkxuM0YKi7PxHfwMyOSRhb3CDgHznsX84KVsaWugDxADDO3SdnuVvRy9jMegNjgZ+6\n+0dNOUaZK/TnOKlSfY6zlpchR1HN/pGZbW9mLUhdGAmmA2eHJ9EfYTZq9ROa2XlmlkszYBowNDRD\nzGxHM9sIeBY4PuoT2ZZUB3ajzGxTYB/g4Rxiqgj5LGNLpQ63k7pAcG2N98qmjKNm6ETgF+6+NIeY\nKkKBPse1lFMZN0U+x2n+jtQ/ZgYQv3BzNrBv1GG7mKjvz8z2NLOb6jqQmW0M/BcwqcZbPYAPc4hx\nLPAGsMDMFgI3ksq2JwLvkOoDGQekFzUxs0vNrF89x/sZMNXdv8ghpkqSrzI+gNSX4qGWGfry0+i9\ncirjkUBHYGwUYz6boeUqn5/jCcBzQE8zW2Fm/x29VTZlbGadzWwFcC4wMoqzTUMnr6jbKM3sn8AA\nd8/6SpdUFpVx81fpZVxRlaaISKnpNkoRkQRUaYqIJKBKU0QkAVWaIiIJ5LRGUKdOnbyqqipPoVSG\nefPmrfb1aFZvlXHzpzJOJqdKs6qqirlzs7kDq/kws/VqWQCVcfOnMk5GzXMRkQRUaYqIJKBKU0Qk\nAVWaIiIJqNIUEUlAlaaISAI5DTkqlrVrU/OPDhs2DICbbsrMRNWnTx8AJkyYAEC3bt2KHJ2IrE+U\naYqIJFARmeZ7770HwC233ALABhtskH4vDMp9+OHU5Om//nWjS2hLGfjXv/4FwMCBAwFYvnx5k4/1\n2GOPpR/36NEDgC5dujQ9OCmZ8Dnu378/ANdddx0AZ511Vnqb+Oe/FJRpiogkUNaZ5gcffADA4MGD\nSxyJ5Nu0adMAWLduXc7Hmjw5sy7bbbfdBsA999yT83GleD78MLX6RTyjBDjnnHMAOO2009KvbbTR\nRsULrA7KNEVEEijLTPPaa1MLFE6alFpX7cUXX2x0n+eeew6AsHzHrrvuCsD+++9fiBClib75JrUs\nzCOPPJK3Y4YRFABXX301kBlx0bZt27ydRwrn2WefBeDdd9+t9voJJ5wAQOvWrWvtUyrKNEVEEijL\nTPP881NrxSe5SvbAAw9U+921a1cA7rvvvvQ2P/7xj/MVojTRU089BcCMGTMA+N3vfpfzMdesWZN+\nvGjRIgA+//xzQJlmOYv3Z48aNarObU4++WQAzKwoMWVDmaaISAKqNEVEEiir5nm/fv2AzMWcb7/9\nttF9OnXqBGSaYW+/nZqQ+a233gJg9913T2/73Xff5S9Yydorr7ySfjxo0CAAunfvDsCFF16Y8/Hj\nQ46kcrz88svpx+Fmh6Bly1TVdPjhhxc1pmwo0xQRSaDkmeYzzzyTfvzqq68CmU7f+i4EnXnmmenH\nffv2BaB9+/YAPPnkkwBceumltfa78cYbgdoDaKWw4mURLtCMHz8egHbt2jX5uOECUPxvqJwuGEjD\nwkXbuhx66KFFjCQZZZoiIgmULNMMEzSEPi6A1atX17ltGD507LHHAvCnP/0p/V6bNm2qbRumhhs7\ndmytYw4fPhyAL7/8EshM7tGqVaum/SOkQRMnTgSqD2QPfZnxvuamCsNU4tnlgQceCECHDh1yPr4U\nVryFEGy44YYAXHbZZcUOJ2vKNEVEEihZpvn1118D9WeXkLkF8t577wUyV8obEjLNcFX2ggsuSL8X\nbq0LGWeYfmq77bZLFLtkJ0wMHf7fIT/9yaGVctdddwGZK60AF110EaDWQzkLNzbMnDmz1nuh5di7\nd++ixpSEMk0RkQRKfvW8LqG/a9y4cUB2GWZNIYu8884706/NmTMnD9FJYz755BMAZs2aVeu9oUOH\n5nz8m2++GchMHdizZ8/0ewcddFDOx5fCamgCnkoY2aJMU0QkgZJnmnXd9TN79uycjxvuKorfBVTz\nTqNwFT6MGZT8CBMxrFixAshM75Uvy5Ytq/a8V69eeT2+FFZdmWYY7ZCPlkihKdMUEUlAlaaISAIl\na56HtcsLtbJcWNVu/vz56ddq3p55ySWXFOTc67uNN94YyAwbiU/YEW597NixY+Ljrlq1CsgMZQr2\n3XffJsUpxfX8888DmaFiceE26M6dOxc1pqZQpikikkDJMs0pU6bk9Xhh+MnixYuBhm/DCkOYNAC6\nMMJqgeGWyXA7JcARRxwBVL/poC4LFy5MPw4XfsK0fzUn5WjRQt/9lSCsOBkuyMaV8wQdNemvTUQk\ngZIPOcqXMP3YDTfcUO82VVVVANxxxx1AZiIQKYyRI0cC1TOL0MKIT9RSl8033zz9OGSW9d1ye+qp\np+YSphRJzb7o+KQqZ5xxRrHDaTJlmiIiCVR8phmWyAgTGDck3G633377FTQmSenRowdQfUXQMJqh\n5gD1msI0gHGDBw8Gat+MEPpQpTyFmxxqXjWPXynPx1SBxaJMU0QkgZJlmg0tnjZ16tRqz08//XQA\n3nvvvXqPk80yB/m+Yi/J7bbbbtV+J/GDH/ygztfj40B/+MMfNi0wKZgwFVzNq+YDBgwoRTg5U6Yp\nIpKAKk0RkQRK1jwP8+aFWdTjwgDomrdY1nXLZWjeZ7NypVS20Lyr2cxTk7y8hUHtQbi55Pzzzy9F\nODlTpikikkDJMs2BAwcCMHr06PRrDa0X1Jjw7RWGudxyyy0AbL311k0+ppSXcLFPa5tXlmnTplV7\n3qVLFyAzSUelUaYpIpJAyTLNsGpkWGkSYNKkSQCMGTMm8fH++Mc/Apm1zKX5CevVBxrUXt7CirNL\nly6t9nrr1q2Byp0wR5mmiEgCJb+NMqxtHn/ct29fILPqYJhQ+KijjgLgV7/6VXqfcCU1viKhNE9h\nddIw0cOIESNKGY40IkzZF26RXLRoEQDbb799yWLKB2WaIiIJlDzTrMthhx1W7bcIZDKW3/zmN4DW\nOC93Yex0mLYxjHr40Y9+VLKY8kGZpohIAmWZaYrUJfRtS2XZZpttALjttttKHEl+KNMUEUlAlaaI\nSAKqNEVEElClKSKSgCpNEZEEVGmKiCRgNSd0TbSz2QfA2/kLpyJ0c/fNG9+seVAZN38q42RyqjRF\nRNY3ap6LiCSgSlNEJAFVmiIiCTRYaZrZZma2IPpZaWbvxp5vWKigzKyfmb1mZkvNbFgW24+KxfaK\nmR2R4/mfN7PejWzT2swmRjHONLOuuZyzVEpVxtG5W5rZy2Y2KYtti17GsW2PNzPPdvtyU8LP8R1m\n9oGZLchy+yFhezNbYma/zPH8483s6Ea22Tn6/K4zs6yWx2xwwg53/xDoHR18JPCZu19Z46RG6oLS\nd9mcsDFm1gq4HvgvYCUw18wecvfXG9n1CncfY2a9gKfMbAuPXeUys5bu/k0+YoycAax09+5mdhLw\nV+DEPB6/KEpRxjEXAAuBNlluX+wyxsw2AYYCc/N53GIqYRnfBtwA3Jxgnzvd/Xwz2wpYaGaT3T29\n4mIByng1cA5wbLY7NKl5bmbdzWyxmd0JLAK6mNnHsfcHmdmt0eMtzewBM5trZnPMbK9GDr8XsMTd\n33b3dcB9wIBsY3P3hYABm0bfNDea2RzgMjNrZ2a3R3HMN7OjohjbmNmE6NvtfqB1FqcaANwRPb4P\n+Gm2MVaCApcxZtYNOBQYlzS2IpYxwGXRz7qkcZa7Qpexuz8DrGlKbO6+ElgOdI1aGf9rZi8At0ct\nlKujOF42syFRjC3M7O9m9qqZPQ50yuI877v7XCDrijiXPs2dgGvcvSfwbgPbXQuMdvc+wHFAKIQ9\nzeymOrbfFvh37PmK6LWsmNk+wJfuHgpra2Avdx8OjAAedfc9gIOAq8ysNfBr4CN37wGMAnaLHW9c\nPc2ydJzu/hWw1sw6ZBtnhShUGQOMAYYBice8FauMzWx3YAt3n1bzvWakkGXcZGbWHegGvBmL82B3\nP4lUK29VVMa7A2dbqnvsWOD7QE/gVGCf2PEuNbN++Ygtl/k0l0U1dGMOAXa0zFrVm5rZRu4+G5id\nw/lrGmZm/w18Chwfe31CrMnRFzjczH4fPW8NdAX2B0YDuPt8M1sUdnb3U/MYY6UpSBlH/Uz/dvcF\nZnZIgniKVsZm1gK4igrsckmo3D7HJ5rZgaQy+yHu/nF0zofcPSxH2hfoYWaDouftge1JlfHd0d/C\nCjN7OhzU3f+YrwBzqTTXxh5/R6q5FMSbPgbsEWVj2XgX6BJ73pmGvwGDK9y9rrV/43EacLS7L4tv\nEPtDSCLEudJSnelt3f3jRvapNIUq432AgWbWPzrOJmZ2h7sPbmS/YpZxB1IZy3PRvlsBj5jZEe4+\nP+nBylihyrip7nT3ui7I1Czjoe7+RHwDMzumoJFF8jLkKKrZPzKz7aNv6Hjw04Gzw5N6mrpxs4Ce\nZtbNzL5HqikwOdp3dOijaqJppDp9QyyhifYs8IvotV2BnbM41mQgfMiPAx7LIa6yl88ydvfh7t7Z\n3auAk4DHQoVZLmXs7mvcvZO7V0VxzgX6NbMKs5o8f47rZWbnmdmZTY+UacBQM2sZHW9HM9uIVBkf\nH/VtbgsckMM56pXPcZq/I/WPmUGqHzI4G9g36rBdDJwO9feFuPvXwLnA48BiYLy7vxa9vQupK+pN\ndQnQ1lJDVhYBI6PXrwc2M7MlwMVA+oPRQJ/mzcDWZraUVH/ZhTnEVSnyUsaNKKcyXh/lrYzNbALw\nHKkkaEXUtQLQA/gwhxjHAm8AC8xsIXAjqVbzROAdUvXGOGBmLJY6+zTNrLOZrSBV54yM4mxwNEfF\n3HtuqTbSVHfXEpXNlMp4/WBm/wQG5Ht4WLFUTKUpIlIOdBuliEgCqjRFRBJQpSkikkAu4zTp1KmT\nV1VV5SmUyjBv3rzV69Os3irj5k9lnExOlWZVVRVz51bsPAZNYmbr1bIAKuPmT2WcjJrnIiIJqNIU\nEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCSQ0zhNEZFC+OijjwB455136t2mW7duAFxzzTUA\n9OrVC4AddtgBgF133bUgsSnTFBFJoKwyzVWrVgFw3HHHAbDPPql1kc444wwgdedCPnzyyScAPPvs\nswAcdlhq+sZWrVrl5fgiksyUKVMAePjhhwF4+umnAXjjjTfq3WfHHXcEYPny5QCsW1d9wdDvvsv3\nitMpyjRFRBIoeaYZ+i4Adt45tWxLyAS33HJLIP8Z5o9+9CMAVq9OrUEf7rvdfvvt83Ieyd5//vMf\nAH7/+9TikYsWpRaJnD59enobtQCah2XLUmvd3XDDDQDcfPPN6fe++OILAJJMiv7aa681vlEBKNMU\nEUmgZJlmyPJC/yXAhx+m1lo6++zUonfXXXddXs85atQoAN566y0g802nDLP4xo8fD8BFF10E1L5K\nGjJQgM0226x4gUnBrFiRWqdtzJi6VmHO3k477QRkrpYXmzJNEZEESpZp/utf/wIyV8niRowYkbfz\nLFy4MP34yiuvBOCYY1LLOR9//PF5O49kJ2Qbv/nNb4BMiyO1EGXGOeekly7n+uuvB6Bjx47FCFGa\nIJQjZDLJn/zkJ0BmdMqGG24IQPv27QFo165dep/PPvsMgJ/+9KdAJovcc889Adhtt93S22600UYA\ntG3bNs//iuwo0xQRSUCVpohIAkVvnocB7Pfff3+t92677TYANt889+VZQrP80EMPrfXewIEDAdh4\n441zPo8kE7pIwkW/+txzzz3px1OnTgUyF41C0z0096R01q5dC1T/nL300ksATJo0qdq2e++9NwDz\n588Hqg8lDBcCO3fuDECLFuWbz5VvZCIiZajomeb//M//AJkhJ2GgOcDPf/7zvJ3n+eefB2DlypXp\n10499VQATjrppLydRxr39tuZNazGjRtX7b0wqUK4keHxxx+vtX+4KSFkqSeeeCIAW221Vf6Dlax8\n9dVXAPziF78AMtklwIUXXgjAIYccUue+dd2s0rVr1zxHWDjKNEVEEih6phmGloTf2267bfq9XPqo\nwm1Yl112GZC5VSs+lCX0mUpxLViwIP04DFrff//9AXjmmWcA+PLLLwG46667APjrX/+a3mfp0qVA\nptUwYMAAINPXqaFIxROGBoXPWZhgI34dYtiwYQC0adOmyNEVhzJNEZEESj5hR5gSCqBv374AdOjQ\nAYCzzjqr0f3D4Pjwe9asWdXez2c/qTRNfMqukPmHwe1B69atAfjlL38JwMSJE9PvhYkewmQOIYPR\n1fPiC1fEL7/8ciAzEfBzzz2X3iYMXm+ulGmKiCRQ9EzzvPPOA+DJJ58E4L333ku/F/q3Qkbx0EMP\nNXq8sG3N2/C22247INP3IqVz991313rtn//8JwBHH310nfuE6frqstdeewHVb8OT4pgxY0a15+H2\nxjC+cn2gTFNEJIGiZ5o//vGPAXjllVeA6ldWH330UQBGjx4NwBZbbAHA4MGD6z3eySefDMAuu+xS\n7fWwVEbIOKV0TjjhhPTj0Hp48cUXAXj11VeBzN/Dgw8+CFSfnDr0cYfXwpR+oex79uxZsNilunhf\nM2RGMFxyySXp1/r37w9Un2SjOVGmKSKSgCpNEZEELMmaHDX16dPHG+qwL4Y333wTyDTDe/fuDcBj\njz0G5Gfyjzgzm+fuffJ60DKWjzJes2ZN+nEop3BrZH0X8uITQIQbFY488kgAXn/9dSCzSulNN92U\nU3w1qYzrV/PmlLpssMEGAJx55plAZk7Mf//73wB0794dyKwJFhfWiAqTexTqAlMuZaxMU0QkgZIP\nbs/Vn//8ZyDzzRcuIuU7w5Smi9/mOGHCBACOPfZYoHbGee655wLwt7/9Lb1PGPgepvQLt1hOmzYN\nyAx+B134K7Tf/va3AFx11VX1bvPtt98CmRZC+J1EuAh84IEHAtWnCiw1ZZoiIglUZKYZshWAO+64\nA4BNNtkE0MqF5S5MFxaGroQJOsKwotByCNll3MUXXwzAkiVLgMzwpbAPZP4epDDC7ZNhFdkwTd/X\nX3+d3iasAxUyzqYIk5WHz3p85ckwGXWpKNMUEUmgIjPNMKA27ogjjgCqT2os5StknPVNVFuXsAph\nWEU0ZJpPPfVUeptwpV7TxRVGuDK+++67A5mRDHFPPPEEkMk+R44cCcCcOXMSny/0dc+bNy/xvoWi\nTFNEJIGKzzTD2sfhqp40f6E/bfLkyUD1K6thjfQRI0YUPzAB4OCDD672PNwqHTLNVq1aAZnlZwBO\nP/10AK655hog09ddjpRpiogkoEpTRCSBimqeh9vl4itMhlUMdQFo/RHWxB4+fDhQfX3tcNFh0KBB\nAOywww7FDU5qCSsyhFUqwwWiMFsVwBtvvAFkVmCoKb6WWKkp0xQRSaAiM834ZAH9+vWrts2nn34K\nZOZerKT1lCWZMDnLX/7yl/Rr4YLgH/7wBwDGjx8PZIYrSfH16NEDyAwVu/fee2ttEx82BtCyZapq\nCkMJ47cVUU8FAAAHAElEQVTVlpoyTRGRBCoq06xL+EYKGUUYshBuu9Jtdc3fKaeckn48duxYAB54\n4AEg01dWc2Z/KZ6Q5Y8ZMwbItAbjA9bff/99AKqqqoBMmYY+6nKiTFNEJIGKzzRvueUWAG699VYA\nhgwZAmQmd5DmLz4N4PTp04HMetxhgolyHiy9vggjXaZMmQLAP/7xj/R7M2fOBDKZZZgarhwp0xQR\nSaCiMs3rrrsOgD/96U/p1/bff38AzjrrLAA23XRTADbccMMiRyflIIyWCMtlhFstFy9eDGjlynIS\nVhOt+bjcKdMUEUmgojLN/fbbD4Ann3yyxJFIuQuTHO+6664ALF26FFCmKblTpikikoAqTRGRBCqq\neS6SrbBm1FtvvVXiSKS5UaYpIpKAKk0RkQRUaYqIJGBhtbcm7Wz2AfB2/sKpCN3cffPGN2seVMbN\nn8o4mZwqTRGR9Y2a5yIiCajSFBFJoMFK08w2M7MF0c9KM3s39rygM2KYWUsze9nMJmWx7ahYbK+Y\n2RE5nvt5M+ud5bbHm5lnu325KVUZm9kFZrYo+jkni+2HmNkHUVxLzOyXOZ5/vJkd3cg2Hc1scvR3\nONvMKvIezBKW8Yro87jAzGZnsX3Ryzja7mAzeyn6W2z0Hu0GB7e7+4dA7+jAI4HP3P3KGic0Un2j\n3zV2soQuABYCbbLc/gp3H2NmvYCnzGwLj3XYmllLd/8mnwGa2SbAUGBuPo9bTKUo4+gLZjDQB/gG\neMzMprh7YyPR73T3881sK2ChmU1299Wx4+a7jC8GZrt7fzPbGfh/wKF5PH5RlPhzvJ+7f5xg+6KW\nsZl1BK4D+rr7CjNrdCLPJjXPzay7mS02szuBRUAXM/s49v4gM7s1erylmT1gZnPNbI6Z7ZXF8buR\n+uMclzQ2d18IGLBp9E1zo5nNAS4zs3ZmdnsUx3wzOyo6XxszmxB9u90PtM7ydJdFP+uSxlnuClzG\nPYBZ7v6Fu38NPAsck21s7r4SWA50jVoZ/2tmLwC3Ry2Uq6M4XjazIVGMLczs72b2qpk9DnTK4lQ9\ngSejcy4CdjCzzbKNs9wV+nOciyKW8UnAfe6+IjrvqsZ2yKVPcyfgGnfvCbzbwHbXAqPdvQ9wHBAK\nYU8zu6mefcYAw4DEl/bNbB/gS3dfE720NbCXuw8HRgCPuvsewEHAVWbWGvg18JG79wBGAbvFjjfO\n6mh6m9nuwBbuPi1pjBWkUGX8CnCApZq/bYHDgS7ZBmVm3YFuwJuxOA9295OAM4BVURnvDpxtZl2B\nY4Hvk6oITwX2iR3vUjOrvqxpykvAwGibvYHO0U9zUsjPsQNPmtk8MzstSVBFLOMdgM3M7JnoC+Gk\nxmLL5d7zZe6eTbP0EGBHyyy7u6mZbeTus4Fa/RyW6oP4t7svMLNDEsQzzMz+G/gUOD72+oRYk6Mv\ncLiZ/T563hroCuwPjAZw9/lmtijs7O6n1hFjC+Aq4MQE8VWigpSxuy80s6uB6cBnwHzg2yzOc6KZ\nHUgqsx/i7h9H53zI3b+MtukL9DCzQdHz9sD2pMr47uhvYYWZPR2L54/1nO9S4FozW0CqAn0pyzgr\nSUHKOLKXu78bNbUfN7Ml7j6jkfMUu4xbAj8k1bJtC8w0s5nuvqy+AHOpNNfGHn9HqkkcxJu3Buzh\n7l9ledx9gIFm1j86ziZmdoe7D25kvyvcfUwjcRpwdM3/kNgfQrY6kPo2ey7adyvgETM7wt3nJz1Y\nGStUGePuNwM3A5jZaGBpFrvd6e7nNxKnAUPd/Yn4BmaWdfM/FuMnpPpewxflcqC5zQBSyDJ+N/q9\n0sweAvYAGqs0i1rGwArgXXf/HPg86gLYBai30szLkKOoZv/IzLaP/rjiwU8Hzg5P6mrq1jjWcHfv\n7O5VpPobHgsVppmNDv2QTTQNSF+pNbPQDH8W+EX02q7Azo3EuMbdO7l7VRTnXKBfM6swq8lnGUfb\nbBH9rgL6A/dEz88zszNzCHUaMNTMWkbH29HMNiJVxsdH/V7bAgdkEWMHM2sVPf0VMN3d1za0TyXL\nZxlb6vpBu+hxW1KZ3MLoedmUMTAJ2M/MNoji3AN4taEd8jlO83ek/jEzSNXewdnAvlGH7WLgdGi0\nL6Q+uwArc4jxEqCtpYZBLAJGRq9fT6pfYwmpK6bpyq++Ps31VD7LeFK07STgTHf/T/R6D+DDHGIc\nC7wBLDCzhcCNpFpUE4F3gMWkLjDODDs00N/1Q2Cxmb0GHExqREdzl68y3hp4wcxeAuYAD7r79Oi9\nsinj6MLxk6T62WcDf3f3JQ2dvGJuo7RUO3iqux9W6likcMzsn8CAfA8Pk/JR6WVcMZWmiEg50G2U\nIiIJqNIUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCTw/wF5WJBSe+H5cQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd1dd936400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=images,\n",
    "            img_shape=img_shape,\n",
    "            cls_pred=cls_pred,\n",
    "            cls_true=cls_true)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}