mimic_alarm_violations_analysis/alarm_violations_analysis_NBPs.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "python390jvsc74a57bd0e2bfb1b1dd0bcdebdb315279aa118b1f834444d4ba3ba6d660e9f6ce7703f6a2",
   "display_name": "Python 3.9.0 64-bit"
  },
  "metadata": {
   "interpreter": {
    "hash": "e2bfb1b1dd0bcdebdb315279aa118b1f834444d4ba3ba6d660e9f6ce7703f6a2"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "source": [
    "# Analysis of `alarm_violations.csv`\n",
    "\n",
    "For each parameter ...\n",
    "* Concerning alarm violations ...\n",
    "  * Create a boxplot and stripplot based on the VALUENUM\n",
    "  * Clean the data if necessary\n",
    "  * Create histogram for cleaned VALUENUM  \n",
    "* Concerning alarm thresholds ...\n",
    "  * Create a boxplot and stripplot based on the THRESHOLD_VALUE (stratified by THRESHOLD_TYPE)\n",
    "  * Clean the data if necessary\n",
    "  * Create histogram for cleaned THRESHOLD_VALUE (stratified by THRESHOLD_TYPE)\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "## Import Data"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Overview: Import all libraries used.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "#import scipy\n",
    "#import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "#import sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "alarm_violations = pd.read_csv('./alarm_violations.csv')\n",
    "alarm_violations.head()"
   ]
  },
  {
   "source": [
    "## Parameter: Non Invasive Blood Pressure systolic\n",
    "\n",
    "* `220179` **Non Invasive Blood Pressure systolic** (NBPs), metavision, in mmHg (numeric)\n",
    "* `223751` **Non-Invasive Blood Pressure Alarm - High** (NBP Alarm - High), metavision, in mmHg (numeric)\n",
    "* `223752` **Non-Invasive Blood Pressure Alarm - Low** (NBP Alarm - Low), metavision, in mmHg (numeric)"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_violations = alarm_violations[(alarm_violations[\"ITEMID\"] == 220179)]\n",
    "display(NBPs_violations)\n",
    "NBPs_violations.VALUENUM.describe()"
   ]
  },
  {
   "source": [
    "Among the VALUENUM values is at least one negative NBPs (min = -69) and a maximum heart rate of 141146.040000 mmHg.\n",
    "These appear to be implausible outliers.\n",
    "\n",
    "Let's check whether the unit (VALUEUOM) is \"mmHg\" for all heart rates (what is expected according to `D_ITEMS.csv`)."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_violations.VALUEUOM.unique()"
   ]
  },
  {
   "source": [
    "All NBPs are given in mmHg as expected.\n",
    "\n",
    "Let's check the literature to see what NBPs can be expected based on medical knowledge.\n",
    "\n",
    "* insert\n",
    "* \"The highest pressure recorded in an individual was **370**/360.\" (Influence of breathing technique on arterial blood pressure during heavy weight lifting, https://pubmed.ncbi.nlm.nih.gov/7741618/)\n",
    "  * \"Under normal circumstances (presumably a BP patient not doing heavy exercise), blood pressure approaching 300 is very dangerous. In fact, it is very rare to have any recorded history of 300+ mm Hg of blood pressure.\" (https://www.scienceabc.com/eyeopeners/how-high-can-a-blood-pressure-go.html)\n",
    "* \"That means you can find really low BP, very close to 0/0, on a 'living' person, but it’s absolutely an emergency situation which requires intensive medical care, even CPR.\" (https://www.quora.com/What-is-the-lowest-blood-pressure-recorded-on-a-living-person)\n",
    "\n",
    "Decision for now: Consider systolic blood pressures below 0 and above 370 mmHg as implausible values to be removed before further analysis."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_violations[(NBPs_violations[\"VALUENUM\"] < 1000)].sort_values(by=\"VALUENUM\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_violations_clean = NBPs_violations[(NBPs_violations[\"VALUENUM\"] >= 0) & (NBPs_violations[\"VALUENUM\"] <= 370)]\n",
    "NBPs_violations_removed_too_low = NBPs_violations[(NBPs_violations[\"VALUENUM\"] < 0)]\n",
    "NBPs_violations_removed_too_high = NBPs_violations[(NBPs_violations[\"VALUENUM\"] > 370)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check rows that were removed because of too low VALUENUM\n",
    "display(NBPs_violations_removed_too_low.sort_values(by=['VALUENUM']))\n",
    "NBPs_violations_removed_too_low.VALUENUM.describe()\n",
    "# To me, seems to make sense to remove them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(NBPs_violations_removed_too_low[[\"VALUENUM\",\"THRESHOLD_VALUE\",\"THRESHOLD_TYPE\"]].sort_values(by=['VALUENUM']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check rows that were removed because of too high VALUENUM\n",
    "display(NBPs_violations_removed_too_high.sort_values(by=['VALUENUM']))\n",
    "NBPs_violations_removed_too_high.VALUENUM.describe()\n",
    "# To me, seems to make sense to remove them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(NBPs_violations_removed_too_high[[\"VALUENUM\",\"THRESHOLD_VALUE\",\"THRESHOLD_TYPE\"]].sort_values(by=['VALUENUM']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check cleaned HR_violations\n",
    "display(NBPs_violations_clean.sort_values(by=['VALUENUM']))\n",
    "NBPs_violations_clean.VALUENUM.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "sns.set_style(\"whitegrid\")\n",
    "\n",
    "fig, (fig_box, fig_hist) = plt.subplots(2, sharex=True, gridspec_kw={\"height_ratios\": (.15, .85)}, figsize=(10, 5))\n",
    "fig.suptitle(\"NBPs alarm violations in cleaned data set\", fontsize=18)\n",
    "sns.boxplot(data=NBPs_violations_clean, x=\"VALUENUM\", ax=fig_box)\n",
    "fig_box.set(xlabel=\"\")\n",
    "sns.histplot(data=NBPs_violations_clean, x=\"VALUENUM\", kde=True, ax=fig_hist)\n",
    "fig_hist.set_xlabel(\"VALUENUM (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "fig_hist.set_ylabel(\"Count\", fontsize=12)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "fig, axs = plt.subplots(1, 2, figsize=(20, 5))\n",
    "fig.suptitle(\"Distribution of NBPs alarm violations stratified by threshold type in cleaned data set\", fontsize=18)\n",
    "\n",
    "sns.histplot(ax=axs[0], data=NBPs_violations_clean, x=\"VALUENUM\", hue=\"THRESHOLD_TYPE\", palette=[\"darkblue\", \"darkgreen\"])\n",
    "axs[0].set_title(\"Histogram\", fontsize=12)\n",
    "axs[0].set_xlabel(\"VALUENUM (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "axs[0].set_ylabel(\"Count\", fontsize=12)\n",
    "\n",
    "sns.kdeplot(ax=axs[1], data=NBPs_violations_clean, x=\"VALUENUM\", hue=\"THRESHOLD_TYPE\", palette=[\"darkblue\", \"darkgreen\"])\n",
    "axs[1].set_title(\"Kernel density estimate (KDE)\", fontsize=12)\n",
    "axs[1].set_xlabel(\"VALUENUM (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "axs[1].set_ylabel(\"Density\", fontsize=12)\n",
    "\n",
    "plt.show(fig)"
   ]
  },
  {
   "source": [
    "### NBPs - Alarm Thresholds"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check NBPs Alarm - High threshold\n",
    "NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"HIGH\")].THRESHOLD_VALUE.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "sns.set_style(\"whitegrid\")\n",
    "\n",
    "fig, (fig_box, fig_hist) = plt.subplots(2, sharex=True, gridspec_kw={\"height_ratios\": (.15, .85)}, figsize=(7, 5))\n",
    "fig.suptitle(\"NBPs alarm thresholds of type HIGH\", fontsize=16)\n",
    "sns.boxplot(data=NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"HIGH\")], x=\"THRESHOLD_VALUE\", ax=fig_box)\n",
    "fig_box.set(xlabel=\"\")\n",
    "sns.histplot(data=NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"HIGH\")], x=\"THRESHOLD_VALUE\", kde=True, ax=fig_hist)\n",
    "fig_hist.set_xlabel(\"THRESHOLD_VALUE (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "fig_hist.set_ylabel(\"Count\", fontsize=12)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "source": [
    "The minimum value of the HIGH alarm thresholds is 0, which is surprisingly low."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check Heart rate Alarm - Low threshold\n",
    "NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"LOW\")].THRESHOLD_VALUE.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "sns.set_style(\"whitegrid\")\n",
    "\n",
    "fig = sns.stripplot(data=NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"LOW\")], x=\"THRESHOLD_VALUE\")\n",
    "fig.set_title(\"Scatterplot for NBPs alarm thresholds of type LOW\", fontsize=14)\n",
    "fig.set_xlabel(\"THRESHOLD_VALUE (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "\n",
    "plt.show(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_threshold_check_low_01 = NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"LOW\") & (NBPs_violations[\"THRESHOLD_VALUE\"] > 121) & (NBPs_violations[\"THRESHOLD_VALUE\"] <= 400)].sort_values(by=['THRESHOLD_VALUE'])\n",
    "NBPs_threshold_check_low_02 = NBPs_violations[(NBPs_violations[\"THRESHOLD_TYPE\"] == \"LOW\") & (NBPs_violations[\"THRESHOLD_VALUE\"] > 400) & (NBPs_violations[\"THRESHOLD_VALUE\"] <= 100000)].sort_values(by=['THRESHOLD_VALUE'])\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "fig, axs = plt.subplots(1, 2, figsize=(15, 5))\n",
    "fig.suptitle(\"A closer look at the suspiciously high LOW thresholds\", fontsize=16)\n",
    "\n",
    "sns.stripplot(ax=axs[0], data=NBPs_threshold_check_low_01, x=\"THRESHOLD_VALUE\")\n",
    "axs[0].set_title(\"Scatterplot for NBPs alarm thresholds of type LOW\", fontsize=14)\n",
    "axs[0].set_xlabel(\"THRESHOLD_VALUE (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "\n",
    "sns.stripplot(ax=axs[1], data=NBPs_threshold_check_low_02, x=\"THRESHOLD_VALUE\")\n",
    "axs[1].set_title(\"Scatterplot for NBPs alarm thresholds of type LOW\", fontsize=14)\n",
    "axs[1].set_xlabel(\"THRESHOLD_VALUE (Systolic blood pressure in mmHg)\", fontsize=12)\n",
    "\n",
    "plt.show(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_threshold_check_low_01.THRESHOLD_VALUE.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NBPs_threshold_check_low_02.THRESHOLD_VALUE.describe()"
   ]
  }
 ]
}