basics_plotting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic plotting examples with matplolib, seaborn, bokeh and holoviews\n",
    "Few plotting examples using DR1 data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import easyaccess as ea\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from scipy import stats\n",
    "print('easyaccess: {0}'.format(ea.__version__))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Connecting to the DES DB, by default is connecting to the dessci DB\n",
    "connection = ea.connect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running a simple query into pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The text of the query can be a block or a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"\"\"\n",
    "--\n",
    "-- Example Query --\n",
    "SELECT\n",
    "  COADD_OBJECT_ID, RA, DEC, MAG_AUTO_I, MAG_AUTO_R, MAG_AUTO_Y, TILENAME \n",
    "FROM \n",
    "  Y3_GOLD_2_2\n",
    "WHERE\n",
    "  MAG_AUTO_R between 17 and 30 and  MAG_AUTO_I between 17 and 30 and rownum < 20000\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = connection.query_to_pandas(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the tail of the table\n",
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matplotlib plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, doing ourselves the work of constructing the grid over which to plot a density, we will show a magnitude vs magnitude plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the complex number is the step, number of points to create, including the stop value\n",
    "X, Y = np.mgrid[17:27:70j, 17:27:70j]\n",
    "positions = np.vstack([X.ravel(), Y.ravel()])\n",
    "values = np.vstack([df.MAG_AUTO_R, df.MAG_AUTO_I])\n",
    "# the kernel will populate the entire grid, being evaluated in the coordinates of interest\n",
    "kernel = stats.gaussian_kde(values)\n",
    "Z = np.reshape(kernel(positions).T, X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(8,8))\n",
    "plt.imshow(np.rot90(Z), cmap=plt.cm.gist_earth_r, extent=[17, 27, 17, 27])\n",
    "plt.xlabel('MAG_AUTO_R')\n",
    "plt.ylabel('MAG_AUTO_I')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This approach has the power of a full customization, from the pint of view of the kernelization, but may be a little time consuming when doing quick statistical analysis. Lets explore another option: seaborn."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example using seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seaborn offers the possibility of doing quick plots using a series of predefined methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10,10))\n",
    "sns.set_style(\"whitegrid\")\n",
    "temp = sns.JointGrid(df.MAG_AUTO_R, df.MAG_AUTO_I,\n",
    "                     xlim=[17,27], ylim=[17,27],\n",
    "                     space=0, size=6)\n",
    "temp.plot_joint(sns.kdeplot, cmap='jet', alpha=0.8, n_levels=10)\n",
    "temp.plot_joint(plt.scatter, s= .3, color='gray')\n",
    "temp.plot_marginals(sns.distplot, color='green',kde=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some interactive plots using Bokeh and Holoviews"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`bokeh` is a dynamic library for plotting, `holoviews` is a library to create objects from the dataframe and communicate to a plotting library to show it on the notebook. We can think of `holoviews` as hyperlinks to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you get an error after importing holoviews saying `module 'pandas.core.computation' has no attribute 'expressions'`, then you should update your `dask` library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hextiles = hv.HexTiles(df, [('MAG_AUTO_R', 'R'), ('MAG_AUTO_I', 'I')], [], extents=(20,26,20,26))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following plot, we will overlay 2 objects: a hexbin histogram and a bivariate distribution based on the datapoints. The overlay operator is `*`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hextiles.options(width=500, height=500, min_count=0, tools=['hover'], colorbar=True, ) * hv.Bivariate(hextiles).options(show_legend=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More interactive plots linked together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import param\n",
    "from holoviews import streams\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using streams we will access the dynamically updated version of the plots, when some defined actions takes place."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transform data across RA =0 \n",
    "df['RA'][df['RA']>180] -= 360"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By creating a HoloViews dataset we annotate the dataset for visualization declaring which columns correspond to independent variables (called key dimensions or kdims) and dependent variables (called value dimensions or vdims)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "kdims = [('RA', 'RA(deg)'), ('DEC', 'Dec(deg)')]\n",
    "vdims = [('MAG_AUTO_R', 'R')]\n",
    "ds = hv.Dataset(df, kdims, vdims)\n",
    "ds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's plot the coordinates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Points  [height=450 width=450 tools=['hover', 'box_select']]\n",
    "points = hv.Points(ds).options({'Points': {'size': 1, 'alpha': 0.5}})\n",
    "points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's plot the histogram for MAG_R"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Histogram  [height=500 width=500] (fill_color='red', alpha=0.4)\n",
    "frequencies, edges = np.histogram(ds.data['MAG_AUTO_R'],bins=30)\n",
    "hist = hv.Histogram((frequencies, edges),vdims='# Number',kdims='MAG_R')\n",
    "hist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's make a toy linked plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following function will feed the `holoviews` histogram object with the selection of points (based in its index in the data frame)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We need a function for every time there is a stream (from a selection box)\n",
    "def update_histogram(index):\n",
    "    # this will initialize the histogram with data in case there's no selection\n",
    "    if len(index) > 0:\n",
    "        selected_flux = ds.data.iloc[index]['MAG_AUTO_R']\n",
    "    else:\n",
    "        selected_flux = ds.data['MAG_AUTO_R']\n",
    "    \n",
    "    frequencies, edges = np.histogram(selected_flux, bins=24, normed=True)\n",
    "    \n",
    "    hist = hv.Histogram((frequencies, edges), vdims='Normalized', kdims='MAG_R')\n",
    "    return hist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### As before we define the points and the histogram (hv.Histogram) which is dynamically generated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "selection = streams.Selection1D(source=points)\n",
    "print('The %s stream has contents %r' % (selection, selection.contents))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We set together the scatter plot from which we extract the positions and the histogram that is updated when a new selection is made."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Histogram  [height=450 width=450] (fill_color='red', alpha=0.4)\n",
    "dmap = hv.DynamicMap(update_histogram, streams=[selection])\n",
    "points + dmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}