notebooks/Classify Image Example-ResNet50.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classify Images using Residual Network with 50 layers (ResNet-50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Turi Create\n",
    "Please follow the repository README instructions to install the Turi Create package.\n",
    "\n",
    "**Note**: Turi Create is currently only compatible with Python 2.7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import turicreate as turi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference the dataset path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url = \"data/food_images\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Label the dataset\n",
    "In the following block of code we will labels the image in the dataset of **Egg** and **Soup** images. Then we will export it as an `SFrame` data object to use it for training the image classification model.\n",
    "\n",
    "1. The first line of code loads the folder images content using the `image_analysis` property. \n",
    "\n",
    "2. The second line creates a _foodType_ key for each image in the dataset to specify whether it's an **Egg** or **Soup** based on which folder it's located in.\n",
    "\n",
    "3. The third line exports the analyzed data as an `SFrame` object in order to use it while creating our image classifier.\n",
    "\n",
    "4. The fourth line simply visualises the new labeled image into a large list.\n",
    "\n",
    "**Note**:- You do not have to run the following block of code everytime you create a classifer, unless you changed/edited the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Unsupported image format. Supported formats are JPEG and PNG\t file: /Users/ahmedbekhit/Documents/Data/Development/TuriCreate/repo/turicreate-notebook/notebooks/data/food_images/eggs/.DS_Store</pre>"
      ],
      "text/plain": [
       "Unsupported image format. Supported formats are JPEG and PNG\t file: /Users/ahmedbekhit/Documents/Data/Development/TuriCreate/repo/turicreate-notebook/notebooks/data/food_images/eggs/.DS_Store"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Unsupported image format. Supported formats are JPEG and PNG\t file: /Users/ahmedbekhit/Documents/Data/Development/TuriCreate/repo/turicreate-notebook/notebooks/data/food_images/.DS_Store</pre>"
      ],
      "text/plain": [
       "Unsupported image format. Supported formats are JPEG and PNG\t file: /Users/ahmedbekhit/Documents/Data/Development/TuriCreate/repo/turicreate-notebook/notebooks/data/food_images/.DS_Store"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Unsupported image format. Supported formats are JPEG and PNG\t file: /Users/ahmedbekhit/Documents/Data/Development/TuriCreate/repo/turicreate-notebook/notebooks/data/food_images/soup/.DS_Store</pre>"
      ],
      "text/plain": [
       "Unsupported image format. Supported formats are JPEG and PNG\t file: /Users/ahmedbekhit/Documents/Data/Development/TuriCreate/repo/turicreate-notebook/notebooks/data/food_images/soup/.DS_Store"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Materializing SFrame...</pre>"
      ],
      "text/plain": [
       "Materializing SFrame..."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Done.</pre>"
      ],
      "text/plain": [
       "Done."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = turi.image_analysis.load_images(url)\n",
    "data[\"foodType\"] = data[\"path\"].apply(lambda path: \"Eggs\" if \"eggs\" in path else \"Soup\")\n",
    "data.save(\"egg_or_soup.sframe\")\n",
    "data.explore()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the labeled SFrame\n",
    "In the following line of code we are loading the `SFrame` object that contains the images in our dataset with their labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dataBuffer = turi.SFrame(\"egg_or_soup.sframe\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create training and test data using our existing dataset\n",
    "Here, we're randomly splitting the data.\n",
    "- 90% of the data in the `SFrame` object will be used for training the image classifier.\n",
    "- 10% of the data in the `SFrame` object will be used for testing the image classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "trainingBuffers, testingBuffers = dataBuffer.random_split(0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the image classifier\n",
    "In the following line of code, we will create an image classifier and we'll feed it with the training data we have. \n",
    "\n",
    "In this example, the image classifer's architecture will be a state-of-the-art Residual Network with 50 layers, also known as **ResNet-50**.\n",
    "\n",
    "Check out the official paper here: https://arxiv.org/abs/1512.03385."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resizing images...\n",
      "Performing feature extraction on resized images...\n",
      "Completed 270/270\n",
      "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
      "          You can set ``validation_set=None`` to disable validation tracking.\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre>WARNING: The number of feature dimensions in this problem is very large in comparison with the number of examples. Unless an appropriate regularization value is set, this model may not provide accurate predictions for a validation/test set.</pre>"
      ],
      "text/plain": [
       "WARNING: The number of feature dimensions in this problem is very large in comparison with the number of examples. Unless an appropriate regularization value is set, this model may not provide accurate predictions for a validation/test set."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>WARNING: Detected extremely low variance for feature(s) '__image_features__' because all entries are nearly the same.\n",
       "Proceeding with model training using all features. If the model does not provide results of adequate quality, exclude the above mentioned feature(s) from the input dataset.</pre>"
      ],
      "text/plain": [
       "WARNING: Detected extremely low variance for feature(s) '__image_features__' because all entries are nearly the same.\n",
       "Proceeding with model training using all features. If the model does not provide results of adequate quality, exclude the above mentioned feature(s) from the input dataset."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Logistic regression:</pre>"
      ],
      "text/plain": [
       "Logistic regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 256</pre>"
      ],
      "text/plain": [
       "Number of examples          : 256"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 1</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 2048</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 2048"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 2049</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 2049"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training-accuracy | Validation-accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training-accuracy | Validation-accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 3        | 0.003906  | 1.052254     | 0.953125          | 1.000000            |</pre>"
      ],
      "text/plain": [
       "| 1         | 3        | 0.003906  | 1.052254     | 0.953125          | 1.000000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 5        | 1.000000  | 1.097437     | 0.968750          | 1.000000            |</pre>"
      ],
      "text/plain": [
       "| 2         | 5        | 1.000000  | 1.097437     | 0.968750          | 1.000000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 6        | 1.000000  | 1.135470     | 0.968750          | 1.000000            |</pre>"
      ],
      "text/plain": [
       "| 3         | 6        | 1.000000  | 1.135470     | 0.968750          | 1.000000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 7        | 1.000000  | 1.173950     | 0.777344          | 0.642857            |</pre>"
      ],
      "text/plain": [
       "| 4         | 7        | 1.000000  | 1.173950     | 0.777344          | 0.642857            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 5         | 9        | 1.000000  | 1.219042     | 0.984375          | 1.000000            |</pre>"
      ],
      "text/plain": [
       "| 5         | 9        | 1.000000  | 1.219042     | 0.984375          | 1.000000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 6         | 10       | 1.000000  | 1.260064     | 0.984375          | 1.000000            |</pre>"
      ],
      "text/plain": [
       "| 6         | 10       | 1.000000  | 1.260064     | 0.984375          | 1.000000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>TERMINATED: Iteration limit reached.</pre>"
      ],
      "text/plain": [
       "TERMINATED: Iteration limit reached."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>This model may not be optimal. To improve it, consider increasing `max_iterations`.</pre>"
      ],
      "text/plain": [
       "This model may not be optimal. To improve it, consider increasing `max_iterations`."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = turi.image_classifier.create(trainingBuffers, target=\"foodType\", model=\"resnet-50\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate the test data to determine the model accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.933333333333\n"
     ]
    }
   ],
   "source": [
    "evaluations = model.evaluate(testingBuffers)\n",
    "print evaluations[\"accuracy\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the Turi Create model to retrieve it later"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.save(\"egg_or_soup.model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Export the image classification model for Core ML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.export_coreml(\"EggSoupClassifier.mlmodel\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}