Demo_Visualisation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demo - Python visualisation libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-12-04T05:05:37.668823Z",
     "start_time": "2017-12-04T05:05:37.541871Z"
    }
   },
   "source": [
    "This notebook was used to demo some plotting libraries at BioInfoSummer 2017 and in my [PyConAU 2017 talk](https://www.youtube.com/watch?v=6d3Yk7a2qYI).\n",
    "\n",
    "Depending on what version of Jupyter you're running, you may need to launch this notebook with a higher data rate limit so that visualisation libraries are not throttled in communicating with the browser, e.g.\n",
    "\n",
    "`jupyter notebook --NotebookApp.iopub_data_rate_limit=10000000`\n",
    "\n",
    "This issue is referenced [here](https://github.com/jupyter/notebook/issues/2287).\n",
    "\n",
    "### Libraries\n",
    "\n",
    "You can find documentation for libraries used in this notebook at respective home pages:\n",
    "\n",
    "* [Matplotlib](https://matplotlib.org/)\n",
    "* [Seaborn](https://seaborn.pydata.org/)\n",
    "* [Altair](https://altair-viz.github.io/)\n",
    "* [Plotly (python API)](https://plot.ly/python/)\n",
    "* [Bokeh](http://bokeh.pydata.org/en/latest/)\n",
    "    * Note that the higher-level [bkcharts](https://github.com/bokeh/bkcharts) package mentioned in the PyConAU talk has been deprecated in favour of Holoviews\n",
    "* [Holoviews](http://holoviews.org/)\n",
    "\n",
    "I'm relatively inexperienced with some of these libraries, so there may well be better ways to set up some of the plots below! \n",
    "\n",
    "And some libraries explicitly discussed, but not demoed:\n",
    "\n",
    "* [bqplot](https://bqplot.readthedocs.io/en/stable/) - the [examples](https://github.com/bloomberg/bqplot/tree/master/examples) repository is useful\n",
    "\n",
    "### Data\n",
    "\n",
    "The dataset used in this demo was downloaded from the US [National Electronic Injury Surveillance System](https://www.cpsc.gov/Research--Statistics/NEISS-Injury-Data). For this demo, I used incidents with patient ages 5-80 years, occuring in the month of December 2016.\n",
    "\n",
    "This data was pre-parsed with the assistance of `play_neiss`: https://gist.github.com/philshem/3015de98de26df1681635244c95186aa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:30.970899Z",
     "start_time": "2019-02-19T22:55:30.329148Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:38.576716Z",
     "start_time": "2019-02-19T22:55:38.572562Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:40.053745Z",
     "start_time": "2019-02-19T22:55:39.771368Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "infile = \"neiss.csv\"\n",
    "\n",
    "use_fields = \"case_id trmt_date age narr sex diag body_part disposition location race_text prod1_text fire_involvement hospital_stratum\".split()\n",
    "categorical_fields = \"sex race_text diag body_part disposition location prod1_text fire_involvement hospital_stratum\".split()\n",
    "specified_dtypes = {field:'category' for field in categorical_fields}\n",
    "\n",
    "data = pd.read_table(infile, usecols=use_fields,\n",
    "                     parse_dates=['trmt_date'],\n",
    "                     dtype=specified_dtypes, \n",
    "                     index_col=0)\n",
    "\n",
    "# Add incident_count field to clarify summary statistics\n",
    "data['incident_count'] = 1\n",
    "\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:41.392922Z",
     "start_time": "2019-02-19T22:55:41.357040Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Create a short product description for each product, similarly for body parts and diagnosis\n",
    "products = set(data['prod1_text'].unique())\n",
    "short_products = {s:s.split('(')[0].split(',')[0].strip() for s in products}\n",
    "data['product'] = data['prod1_text'].apply(lambda p: short_products[p]).astype('category')\n",
    "\n",
    "body_parts = set(data['body_part'].unique())\n",
    "short_bodyparts = {s:s.split('(')[0].strip() for s in body_parts}\n",
    "data['body_part'] = data['body_part'].apply(lambda b: short_bodyparts[b]).astype('category')\n",
    "\n",
    "# Additionally, collapse all 'Burns' diagnoses\n",
    "diagnoses = set(data['diag'].unique())\n",
    "short_diags = {s:s.split('(')[0].strip() for s in diagnoses}\n",
    "for diag in diagnoses:\n",
    "    if diag[:5]=='Burns':\n",
    "        short_diags[diag] = \"Burns\"\n",
    "data['diagnosis'] = data['diag'].apply(lambda d: short_diags[d]).astype('category')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:42.666306Z",
     "start_time": "2019-02-19T22:55:42.389264Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# For this demo, I didn't bother with dates.\n",
    "# Instead, work with day-from-start, setting the first day to 1\n",
    "# This is equivalent to day-of-month for our dataset\n",
    "data['day'] = (data['trmt_date'] - data['trmt_date'].min()).dt.days + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:42.937804Z",
     "start_time": "2019-02-19T22:55:42.917489Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Restrict our dataset to top few products\n",
    "NUM_PRODUCTS = 12\n",
    "top_products = set(data['product'].value_counts()[:NUM_PRODUCTS].index)\n",
    "data = data[ data['product'].isin(top_products) ]\n",
    "data['product'].cat.remove_unused_categories(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:43.676700Z",
     "start_time": "2019-02-19T22:55:43.639702Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pd.set_option('display.max_colwidth', -1)\n",
    "data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matplotlib "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:49.518772Z",
     "start_time": "2019-02-19T22:55:49.513549Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Default figure size\n",
    "FIGSIZE = (7,4.5)\n",
    "plt.rcParams['figure.figsize'] = FIGSIZE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:56:23.142025Z",
     "start_time": "2019-02-19T22:56:22.721616Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "product_counts = data.groupby('product')['incident_count'].aggregate(sum).sort_values()\n",
    "\n",
    "#with plt.style.context('classic'):\n",
    "plt.barh(y=list(range(len(product_counts))), \n",
    "         width=product_counts,\n",
    "         tick_label=product_counts.index)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:56:35.416098Z",
     "start_time": "2019-02-19T22:56:35.008541Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "product_counts = data.groupby('product')['incident_count'].aggregate(sum).sort_values()\n",
    "\n",
    "with plt.style.context('classic'):\n",
    "    plt.bar(x=list(range(len(product_counts))), \n",
    "             height=product_counts,\n",
    "             tick_label=product_counts.index)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Seaborn "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:56:42.854256Z",
     "start_time": "2019-02-19T22:56:40.229741Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "#sns.reset_orig()  # if we don't want to change default matplotlib rc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:57:05.449712Z",
     "start_time": "2019-02-19T22:57:02.036483Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "with sns.axes_style('darkgrid'):\n",
    "    sns.barplot(data=data, y='product', x='incident_count', estimator=sum)#, hue=\"sex\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:58:30.084659Z",
     "start_time": "2019-02-19T22:58:26.179652Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "with sns.axes_style('darkgrid'):\n",
    "    g = sns.barplot(data=data, x='product', y='incident_count', estimator=sum, hue=\"sex\")\n",
    "    g.figure.autofmt_xdate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:58:39.827479Z",
     "start_time": "2019-02-19T22:58:39.225631Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "table = data.pivot_table(index='body_part', columns='product', \n",
    "                         values='incident_count', aggfunc=sum, fill_value=0)\n",
    "sns.heatmap(table, cmap='Greens', ax=ax)   # table-10 demonstrates a diverging colour scale - seaborn uses automatically if there are negative numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:58:47.236632Z",
     "start_time": "2019-02-19T22:58:46.721752Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "sns.boxplot(data=data, y='product', x='age')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:58:51.040808Z",
     "start_time": "2019-02-19T22:58:50.104446Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# if using kind='scatter', try joint_kws={'alpha': 0.2} to pass an argument down to the \n",
    "# matplotlib scatter call (scatter will still not be a great choice)\n",
    "sns.jointplot(data=data, x='day', y='age', kind='hex')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Altair "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:55:17.543267Z",
     "start_time": "2019-02-19T22:55:15.127417Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import altair as alt\n",
    "# This line is needed for Jupyter Notebook; don't run it for Jupyter Lab\n",
    "alt.renderers.enable('notebook')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T22:59:59.704539Z",
     "start_time": "2019-02-19T22:59:59.679706Z"
    }
   },
   "outputs": [],
   "source": [
    "sample = data.sample(5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Data Type | Shorthand Code | Description |\n",
    "|-|-|-|\n",
    "| quantitative | Q | a continuous real-valued quantity |\n",
    "| ordinal | O | a discrete ordered quantity |\n",
    "| nominal | N | a discrete unordered category |\n",
    "| temporal | T | a time or date value |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-19T23:01:35.712868Z",
     "start_time": "2019-02-19T23:01:34.006184Z"
    }
   },
   "outputs": [],
   "source": [
    "alt.Chart(sample).mark_circle(opacity=0.2).encode(x='day', y='age')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-20T01:14:26.681426Z",
     "start_time": "2019-02-20T01:14:24.328289Z"
    }
   },
   "outputs": [],
   "source": [
    "alt.Chart(sample).mark_rect().encode(y='product:N', x='count()')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-20T02:24:04.820023Z",
     "start_time": "2019-02-20T02:24:01.745701Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "alt.Chart(sample).mark_rect().encode(x=alt.X('age',bin=True),y='count()') # mark_bar or mark_rect # alt.X('age',bin=True) # alt.X('age',bin=alt.Bin(step=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-20T02:24:49.331582Z",
     "start_time": "2019-02-20T02:24:46.922105Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "alt.Chart(sample).mark_line().encode(x='day',y='mean(age)') # mark_bar or mark_rect # alt.X('age',bin=True) # alt.X('age',bin=alt.Bin(step=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-20T02:41:28.878230Z",
     "start_time": "2019-02-20T02:41:27.203234Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "alt.Chart().mark_rect()\\\n",
    "    .encode(\n",
    "        x='body_part',\n",
    "        y='product',\n",
    "        tooltip=['body_part','product','count()','mean(age)'],\n",
    "        color=alt.Color(\n",
    "            'count()',\n",
    "            scale=alt.Scale(scheme='blues'))).interactive() # color='count()')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-20T02:51:03.516884Z",
     "start_time": "2019-02-20T02:51:03.312191Z"
    }
   },
   "outputs": [],
   "source": [
    "selected = alt.selection_interval()\n",
    "\n",
    "alt.Chart(sample[~sample['narr'].isnull()].sample(200))\\\n",
    "    .mark_circle(opacity=0.8)\\\n",
    "    .encode(\n",
    "        x='day', \n",
    "        y='age', \n",
    "        tooltip='narr',\n",
    "        color=alt.condition(\n",
    "            selected,\n",
    "            alt.value('red'),\n",
    "            alt.value('blue')))\\\n",
    "    .add_selection(selected)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-02-20T03:30:20.697372Z",
     "start_time": "2019-02-20T03:30:13.751200Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "chart = alt.Chart(sample[~sample['narr'].isnull()])\n",
    "\n",
    "selected_category = alt.selection_multi(encodings=['x','y'])\n",
    "\n",
    "heatmap = chart\\\n",
    "    .mark_rect()\\\n",
    "    .encode(\n",
    "        x='product',\n",
    "        y='body_part',\n",
    "        tooltip=['body_part','product','count()','mean(age)'],\n",
    "        color=alt.condition(\n",
    "            selected_category,\n",
    "            alt.value('red'),\n",
    "            alt.Color(\n",
    "                'count()',\n",
    "                scale=alt.Scale(scheme='blues'))))\\\n",
    "    .add_selection(\n",
    "        selected_category)\n",
    "\n",
    "scatter = chart\\\n",
    "    .mark_circle(opacity=0.8)\\\n",
    "    .encode(\n",
    "        x='day', \n",
    "        y='age', \n",
    "        tooltip='narr')\\\n",
    "    .transform_filter(\n",
    "        selected_category)\n",
    "\n",
    "heatmap | scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bokeh "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:08:34.580051Z",
     "start_time": "2018-02-20T04:08:34.565076Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import bokeh\n",
    "from bokeh.plotting import figure, output_notebook, show\n",
    "from bokeh.models import ColumnDataSource, tools\n",
    "from bokeh.resources import INLINE\n",
    "\n",
    "RESOURCES = None\n",
    "# Use resources=INLINE in bokeh plots to use locally cached Javascript, and render without an internet connection\n",
    "#RESOURCES = INLINE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:28:03.808813Z",
     "start_time": "2018-02-20T04:28:03.601766Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# A basic scatter plot - will have pan and zoom\n",
    "\n",
    "subset = data.sample(200)\n",
    "\n",
    "p = figure(width=400, height=300, x_axis_label=\"day\", y_axis_label=\"patient age\")\n",
    "\n",
    "p.circle(subset['day'], subset['age'], size=10, alpha=0.7)\n",
    "\n",
    "output_notebook(resources=RESOURCES)\n",
    "show(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:29:03.985707Z",
     "start_time": "2018-02-20T04:29:03.736543Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Add a legend with click policy 'hide', and a hover tool \n",
    "\n",
    "subset = data.sample(200)\n",
    "\n",
    "# Bokeh probably has a better way to do this than two separate ColumnDataSource objects\n",
    "source_male = ColumnDataSource(subset[subset['sex']=='Male'][\"day age narr\".split()] )\n",
    "source_female = ColumnDataSource(subset[subset['sex']=='Female'][\"day age narr\".split()] )\n",
    "\n",
    "p = figure(width=400, height=300, x_axis_label=\"day\", y_axis_label=\"patient age\")\n",
    "\n",
    "p.circle(source=source_male, x='day', y='age', color='teal', alpha=0.7, size=10, legend='Male')\n",
    "p.circle(source=source_female, x='day', y='age', color='darkorange', alpha=0.7, size=10, legend='Female')\n",
    "\n",
    "p.add_tools(tools.HoverTool(tooltips=[('Narrative', '@narr')]))\n",
    "\n",
    "p.legend.click_policy='hide'\n",
    "\n",
    "output_notebook(resources=RESOURCES)\n",
    "show(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:29:36.582953Z",
     "start_time": "2018-02-20T04:29:36.318500Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Two plots with linked axes\n",
    "\n",
    "subset = data.sample(500)\n",
    "\n",
    "source1 = ColumnDataSource(subset[subset['product']=='Basketball'][\"day age narr\".split()] )\n",
    "source2 = ColumnDataSource(subset[subset['product']=='Stairs or steps'][\"day age narr\".split()] )\n",
    "\n",
    "p1 = figure(width=300, height=300, title='Basketball',\n",
    "            x_axis_label=\"day\", y_axis_label=\"patient age\")\n",
    "\n",
    "p1.circle(source=source1, x='day', y='age', size=10, alpha=0.7)\n",
    "\n",
    "p2 = figure(width=300, height=300, title='Stairs',\n",
    "            x_axis_label=\"day\", y_axis_label=\"patient age\",\n",
    "            x_range=p1.x_range, y_range=p1.y_range)\n",
    "\n",
    "p2.circle(source=source2, x='day', y='age', size=10, alpha=0.7)\n",
    "\n",
    "l = bokeh.layouts.row(p1,p2)\n",
    "\n",
    "output_notebook(resources=RESOURCES)\n",
    "show(l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:30:26.287497Z",
     "start_time": "2018-02-20T04:30:25.977503Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Two plots, linked sources for selection\n",
    "# (also linked age axis)\n",
    "\n",
    "subset = data.sample(500)\n",
    "\n",
    "source = ColumnDataSource(subset[\"day age narr\".split()] )\n",
    "source.add(subset['product'].astype('str'), 'product')\n",
    "\n",
    "product_list = list(subset['product'].unique())\n",
    "\n",
    "p1 = figure(width=300, height=300,\n",
    "            tools=\"box_zoom,box_select,lasso_select,reset\",\n",
    "            x_axis_label=\"day\", y_axis_label=\"patient age\")\n",
    "p1.circle(source=source, x='day', y='age', \n",
    "          size=10, alpha=0.4, \n",
    "          selection_color='red', selection_fill_alpha=0.4)\n",
    "\n",
    "p2 = figure(width=400, height=300, \n",
    "            tools=\"box_zoom,box_select,lasso_select,reset\",\n",
    "            y_range=product_list, x_range = p1.x_range,\n",
    "            x_axis_label=\"day\")\n",
    "p2.circle(source=source, x='day', y='product', \n",
    "          size=10, alpha=0.4, \n",
    "          selection_color='red', selection_fill_alpha=0.4)\n",
    "\n",
    "l = bokeh.layouts.row(p1,p2)\n",
    "\n",
    "output_notebook(resources=RESOURCES)\n",
    "show(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-07-30T06:32:40.601451Z",
     "start_time": "2017-07-30T06:32:40.439825Z"
    }
   },
   "source": [
    "## Holoviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:08:36.658596Z",
     "start_time": "2018-02-20T04:08:36.348482Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "hv.extension('bokeh', 'matplotlib')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T01:02:40.703042Z",
     "start_time": "2018-02-20T01:02:40.683276Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# If you want to use less points, replace data with e.g. data.sample(2000)\n",
    "\n",
    "ds = hv.Dataset(data, kdims=[\"sex\",\"product\",\"body_part\"],\n",
    "                      vdims=[\"age\",\"day\",\"incident_count\",\"narr\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T01:02:41.478049Z",
     "start_time": "2018-02-20T01:02:41.466740Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T01:02:42.655513Z",
     "start_time": "2018-02-20T01:02:42.109889Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%%output backend='bokeh' \n",
    "%%opts Scatter (size=5 alpha=0.5)\n",
    "\n",
    "scatter = ds.reindex(kdims=['product'], vdims=['age', 'day']).to(hv.Scatter, \"day\", \"age\") \n",
    "scatter   #.layout('product')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T01:02:59.209892Z",
     "start_time": "2018-02-20T01:02:47.202998Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%output backend='bokeh'\n",
    "%%opts Scatter [tools=['hover']] (size=5 alpha=0.5)\n",
    "\n",
    "# Scatter plot with hover tool that includes narrative\n",
    "scatter = ds.to(hv.Scatter, \"day\", [\"age\",\"narr\"]).grid('sex')\n",
    "\n",
    "scatter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T01:04:08.535097Z",
     "start_time": "2018-02-20T01:03:48.379556Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%%output backend='bokeh'\n",
    "%%opts Bars [xrotation=90 tools=['hover']]\n",
    "%%opts Scatter [tools=['hover']] (size=5 alpha=0.5)\n",
    "\n",
    "# the above is ok, but we'd like to know how many incidents are in each body_part for current product - add a bar plot\n",
    "# retain incident_count as our value dimension, then aggregate it over body_part and product, then specify a bar plot over body_part\n",
    "bars = ds.reindex(kdims=[\"product\",\"body_part\"],vdims=['incident_count']).aggregate(['body_part', 'product'], function=sum).to(hv.Bars, kdims=['body_part'], vdims=['incident_count'])\n",
    "\n",
    "scatter = ds.to(hv.Scatter, \"day\", [\"age\",\"narr\"]).overlay('sex')\n",
    "\n",
    "bars + scatter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-12-05T05:38:42.543367Z",
     "start_time": "2017-12-05T05:38:42.173015Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%%output backend='bokeh' \n",
    "%%opts HeatMap [colorbar=True width=600 tools=['hover'] toolbar='above' xrotation=45] (cmap='Blues')\n",
    "\n",
    "# A heatmap with a hover tool showing vdim and kdim values\n",
    "# Our first HeatMap vdim, incident_count, is used to colour the heatmap, but our second, mean age, is available to the tooltip too\n",
    "aggregated_values = ds.reindex(kdims=[\"body_part\",\"product\"], vdims=[\"incident_count\", \"age\"]).aggregate([\"body_part\",\"product\"], function={'incident_count':'sum', 'age':'mean'})\n",
    "aggregated_values.to(hv.HeatMap, kdims=[\"body_part\",\"product\"], vdims=[\"incident_count\", \"age\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotly "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:08:38.190416Z",
     "start_time": "2018-02-20T04:08:38.180669Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot\n",
    "import plotly.graph_objs as go\n",
    "import plotly\n",
    "init_notebook_mode(connected=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:20:28.707953Z",
     "start_time": "2018-02-20T04:20:28.613114Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "subset = data.sample(200)\n",
    "\n",
    "trace = go.Scatter(x=subset['day'], y=subset['age'], \n",
    "                   mode='markers',\n",
    "                   marker={'size': 10, 'opacity': 0.7})\n",
    "\n",
    "iplot([trace])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:22:49.008276Z",
     "start_time": "2018-02-20T04:22:48.891075Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "subset = data.sample(200)\n",
    "\n",
    "male = subset[ subset['sex']=='Male' ]\n",
    "female = subset[ subset['sex']=='Female']\n",
    "\n",
    "trace1 = go.Scatter(x=male['day'], y=male['age'], text=male['narr'],\n",
    "               mode='markers',\n",
    "               marker={'size': 10, 'opacity': 0.7},\n",
    "               name='Male')\n",
    "trace2 = go.Scatter(x=female['day'], y=female['age'], text=female['narr'],\n",
    "               mode='markers',\n",
    "               marker={'size': 10, 'opacity': 0.7},\n",
    "               name='Female')\n",
    "\n",
    "layout = go.Layout(xaxis={'title':'Day of month'},\n",
    "                   yaxis={'title':'Patient age'},\n",
    "                   hovermode='closest')\n",
    "\n",
    "fig = go.Figure(data=[trace1, trace2], layout=layout)\n",
    "\n",
    "iplot(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:24:21.421539Z",
     "start_time": "2018-02-20T04:24:21.296533Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "trace1 = go.Histogram(x=data[data['sex']=='Male']['age'], \n",
    "                      opacity=0.7,\n",
    "                      name='Male')\n",
    "\n",
    "trace2 = go.Histogram(x=data[data['sex']=='Female']['age'], \n",
    "                      opacity=0.7,\n",
    "                      name='Female')\n",
    "\n",
    "layout = go.Layout(barmode='overlay')\n",
    "fig = go.Figure(data=[trace1, trace2], layout=layout)\n",
    "\n",
    "iplot(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:25:03.957600Z",
     "start_time": "2018-02-20T04:25:03.734192Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "trace = go.Box(x = data['diagnosis'], y=data['age'])\n",
    "\n",
    "layout = {\n",
    "    'yaxis': {'rangemode': 'tozero',\n",
    "              'title': 'Patient age'}\n",
    "}\n",
    "\n",
    "fig = {'data': [trace], 'layout': layout}\n",
    "\n",
    "iplot(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:26:12.738643Z",
     "start_time": "2018-02-20T04:26:12.591528Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "table = data.pivot_table(index='diagnosis', columns='product', \n",
    "                         values='incident_count', aggfunc=sum, fill_value=0)\n",
    "\n",
    "trace = go.Heatmap(z=table.as_matrix(), \n",
    "                     x=table.columns, y=table.index,\n",
    "                     colorscale='Greens',\n",
    "                     reversescale=True)\n",
    "\n",
    "iplot([trace])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-02-20T04:26:26.958932Z",
     "start_time": "2018-02-20T04:26:26.910772Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "subset = data.sample(300)\n",
    "\n",
    "trace = go.Scatter3d(x=subset['location'], y=subset['product'], z=subset['age'], \n",
    "                     mode='markers',\n",
    "                     marker={'opacity': 0.7},\n",
    "                     text=subset['narr'])\n",
    "\n",
    "iplot([trace])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subset = data.sample(200)\n",
    "\n",
    "products = subset['product']\n",
    "\n",
    "trace = go.Parcoords(line=dict(color='red'), dimensions=[\n",
    "    dict(label='Day of month', values=subset['day']),\n",
    "    dict(label='Patient age', values=subset['age']),\n",
    "    dict(label='Product', values=subset['product'].cat.codes,\n",
    "         ticktext=list(subset['product'].cat.categories),\n",
    "         tickvals=list(range(len(subset['product'].cat.categories)))),\n",
    "    dict(label='Body part', values=subset['body_part'].cat.codes,\n",
    "         ticktext=list(subset['body_part'].cat.categories),\n",
    "         tickvals=list(range(len(subset['body_part'].cat.categories)))),\n",
    "    dict(label='Location', values=subset['location'].cat.codes,\n",
    "         ticktext=list(subset['location'].cat.categories),\n",
    "         tickvals=list(range(len(subset['location'].cat.categories))))\n",
    "])\n",
    "\n",
    "iplot([trace])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "66px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}