mimic_alarm_violations_analysis/alarm_violations_analysis.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "python3",
   "language": "python",
   "display_name": "Python 3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## General Insights into 'alarm_violations.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "PATH = ''\n",
    "ALARM_VIOLATIONS = pd.read_csv(PATH + 'alarm_violations.csv')\n",
    "\n",
    "ALARM_VIOLATIONS.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ALARM_VIOLATIONS['ITEMID'] = ALARM_VIOLATIONS['ITEMID'].astype(str)\n",
    "ALARM_VIOLATIONS['ICUSTAY_ID'] = ALARM_VIOLATIONS['ICUSTAY_ID'].astype(str)\n",
    "ALARM_VIOLATIONS.describe()"
   ]
  },
  {
   "source": [
    "### How many unique ICU stays exist in the MIMIC-III data set?\n",
    "There are 19,968 unique ICU stays."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays = ALARM_VIOLATIONS[\"ICUSTAY_ID\"].value_counts()\n",
    "len(unique_ICU_stays)"
   ]
  },
  {
   "source": [
    "### How often is an alarm raised? (= row count)\n",
    "There are 388,209 triggered alarms."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(ALARM_VIOLATIONS)"
   ]
  },
  {
   "source": [
    "## Stratify Alarms by ITEM ID and TYPE (HIGH/LOW)\n",
    "\n",
    "### Are alarms triggered more by falling below or exceeding thresholds?\n",
    "Approximately 11,000 more alarms were triggered by exceeding a threshold."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ALARM_VIOLATIONS_HIGH = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['THRESHOLD_TYPE'] == 'HIGH')]\n",
    "print(\"HIGH Alarms:\", len(ALARM_VIOLATIONS_HIGH))\n",
    "\n",
    "ALARM_VIOLATIONS_LOW = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['THRESHOLD_TYPE'] == 'LOW')]\n",
    "print(\"LOW Alarms:\", len(ALARM_VIOLATIONS_LOW))"
   ]
  },
  {
   "source": [
    "### How often are the respective ITEM IDs affected by an alarm being triggered?\n",
    "Most ITEM IDs were affected tens of thousands of times by alarm violations. Only the thresholds of the Minute Volume parameter were exceeded (534) or undershot (1,860) significantly less often."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "ALARM_VIOLATIONS_STRATIFIED = ALARM_VIOLATIONS\\\n",
    "    .groupby(['ITEMID', 'THRESHOLD_TYPE'])\\\n",
    "    .size()\\\n",
    "    .reset_index(name='count')\n",
    "print(ALARM_VIOLATIONS_STRATIFIED)\n",
    "ALARM_VIOLATIONS_STRATIFIED.dtypes"
   ],
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": []
  },
  {
   "source": [
    "### Bar Chart Visualization"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "ALARM_VIOLATIONS_STRATIFIED_T= ALARM_VIOLATIONS_STRATIFIED.pivot(index='ITEMID', columns='THRESHOLD_TYPE', values='count')\n",
    "ALARM_VIOLATIONS_STRATIFIED_T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# define figure\n",
    "fig, ax = plt.subplots(1, figsize=(16, 6))\n",
    "# numerical x\n",
    "x = np.arange(0, len(ALARM_VIOLATIONS_STRATIFIED_T.index))\n",
    "# plot bars\n",
    "plt.bar(x - 0.1, ALARM_VIOLATIONS_STRATIFIED_T['LOW'], width = 0.2, color = '#1D2F6F')\n",
    "plt.bar(x + 0.1, ALARM_VIOLATIONS_STRATIFIED_T['HIGH'], width = 0.2, color = '#8390FA')\n",
    "\n",
    "# x and y details\n",
    "plt.xlabel('ITEM ID',fontsize=16)\n",
    "plt.ylabel('Alarm Counts',fontsize=16)\n",
    "plt.xticks(x, ALARM_VIOLATIONS_STRATIFIED_T.index)\n",
    "\n",
    "# title and legend\n",
    "plt.title('Alarm Counts by ITEM ID and TYPE', fontsize=18)\n",
    "plt.legend(['LOW','HIGH'], loc='upper left', ncol = 2)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "source": [
    "## How many alarm violations exist per ICU stay?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays = ALARM_VIOLATIONS[\"ICUSTAY_ID\"].value_counts()\n",
    "df_unique_ICU_stays = pd.DataFrame(unique_ICU_stays)\n",
    "\n",
    "df_unique_ICU_stays = df_unique_ICU_stays.reset_index()\n",
    "df_unique_ICU_stays.columns = ['ICUSTAY_ID','AlarmCount']\n",
    "df_unique_ICU_stays['ICUSTAY_ID'] = df_unique_ICU_stays['ICUSTAY_ID'].str.rstrip('.0')\n",
    "df_unique_ICU_stays.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_alarms_per_stay = df_unique_ICU_stays['AlarmCount'].mean()\n",
    "median_alarms_per_stay = df_unique_ICU_stays['AlarmCount'].median()\n",
    "min_alarms_per_stay = df_unique_ICU_stays['AlarmCount'].min()\n",
    "max_alarms_per_stay = df_unique_ICU_stays['AlarmCount'].max()\n",
    "print('Mean Alarms per Stay:', mean_alarms_per_stay)\n",
    "print('Median Alarms per Stay:', median_alarms_per_stay)\n",
    "print('Min Alarms per Stay:', min_alarms_per_stay)\n",
    "print('Max Alarms per Stay:', max_alarms_per_stay)"
   ]
  },
  {
   "source": [
    "### Strip Plot Visualization\n",
    "There were 2,490 ICU stays with only one alarm."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "count_alarm_numbers = df_unique_ICU_stays['AlarmCount'].value_counts()\n",
    "count_alarm_numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "\n",
    "fig, ax = plt.subplots(1, figsize=(16, 6))\n",
    "\n",
    "sns.stripplot(data=df_unique_ICU_stays,x='AlarmCount')\n",
    "plt.title(\"Alarm Count - Strip Plot\",fontsize=18)\n",
    "plt.xlabel(\"Alarm Count\",fontsize=16)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "source": [
    "## Analysis of the Alarm Counts per ICU Stay by ITEM ID"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create dataframe\n",
    "unique_ICU_stays_by_ItemId = ALARM_VIOLATIONS\\\n",
    "    .groupby(['ITEMID','ICUSTAY_ID'])\\\n",
    "    .size()\\\n",
    "    .reset_index(name='AlarmCount')\n",
    "unique_ICU_stays_by_ItemId['ICUSTAY_ID']=unique_ICU_stays_by_ItemId['ICUSTAY_ID'].str.rstrip('.0')\n",
    "unique_ICU_stays_by_ItemId.sort_values(by=['AlarmCount'], inplace=True)\n",
    "unique_ICU_stays_by_ItemId\n",
    "# ICUSTAY_ID with highest alarm count for ITEM ID 220277 equals the one above in the strip plot with approx. 14,000 alarms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays_by_220045= unique_ICU_stays_by_ItemId[(unique_ICU_stays_by_ItemId[\"ITEMID\"] ==\"220045\")]\n",
    "unique_ICU_stays_by_220045.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays_by_220179= unique_ICU_stays_by_ItemId[(unique_ICU_stays_by_ItemId[\"ITEMID\"] ==\"220179\")]\n",
    "unique_ICU_stays_by_220179.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays_by_220210= unique_ICU_stays_by_ItemId[(unique_ICU_stays_by_ItemId[\"ITEMID\"] ==\"220210\")]\n",
    "unique_ICU_stays_by_220210.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays_by_220277= unique_ICU_stays_by_ItemId[(unique_ICU_stays_by_ItemId[\"ITEMID\"] ==\"220277\")]\n",
    "unique_ICU_stays_by_220277.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_ICU_stays_by_224687= unique_ICU_stays_by_ItemId[(unique_ICU_stays_by_ItemId[\"ITEMID\"] ==\"224687\")]\n",
    "unique_ICU_stays_by_224687.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "\n",
    "fig, ax = plt.subplots(1, figsize=(16, 6))\n",
    "\n",
    "sns.stripplot(x=\"ITEMID\", y=\"AlarmCount\", data=unique_ICU_stays_by_ItemId)\n",
    "plt.title(\"Alarm Count by ITEM ID - Scatter Plot\",fontsize=18)\n",
    "plt.xlabel(\"ITEM ID\",fontsize=16)\n",
    "plt.ylabel(\"Alarm Count\",fontsize=16)\n",
    "plt.gca().set_ylim(bottom=0)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Analysis of Difference Between Actual Values and Thresholds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create new column that shows the dif between actual and threshold\n",
    "ALARM_VIOLATIONS['DIF_ACTUAL_TH'] = ALARM_VIOLATIONS['VALUENUM'] - ALARM_VIOLATIONS['THRESHOLD_VALUE']\n",
    "ALARM_VIOLATIONS.head()"
   ]
  },
  {
   "source": [
    "# analyze dif by item id\n",
    "dif_analysis = ALARM_VIOLATIONS.groupby('ITEMID').describe()\n",
    "dif_analysis = dif_analysis[\"DIF_ACTUAL_TH\"].round(2)\n",
    "dif_analysis"
   ],
   "cell_type": "code",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "source": [
    "Analyze difference for each Item ID:\n",
    "\n",
    "### 220045 - Heart Rate "
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ALARM_VIOLATIONS_220045 = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['ITEMID'] == '220045')]\n",
    "ALARM_VIOLATIONS_220045"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "ALARM_VIOLATIONS_220045 = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['ITEMID'] == '220045')]\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize=(25, 5))\n",
    "fig.suptitle(\"Difference Between Actual and Threshold - 220045\", fontsize=18)\n",
    "\n",
    "sns.stripplot(data=ALARM_VIOLATIONS_220045,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[0])\n",
    "#axs[0].set_title(\"Difference Between Actual and Threshold - 220045 Scatter Plot\",fontsize=14)\n",
    "axs[0].set_ylabel(\"Difference Between Actual and Threshold\",fontsize=14)\n",
    "axs[0].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "sns.boxplot(data=ALARM_VIOLATIONS_220045,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[1])\n",
    "#axs[1].set_title(\"Difference Between Actual and Threshold - 220045 Boxplot\")\n",
    "axs[1].set_ylabel(\"Difference Between Actual and Threshold\", fontsize=14)\n",
    "axs[1].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "#sns.histplot(data=ALARM_VIOLATIONS_220045, x='DIF_ACTUAL_TH', ax=axs[2])\n",
    "#axs[2].set_title(\"HR_violations_clean histogram\")\n",
    "#axs[2].set_xlabel(\"HR_violations_clean VALUENUM\")\n",
    "\n",
    "plt.show(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ALARM_VIOLATIONS_220045.describe()\n",
    "ALARM_VIOLATIONS_220045_H = ALARM_VIOLATIONS_220045[(ALARM_VIOLATIONS_220045['THRESHOLD_TYPE'] == 'HIGH')]\n",
    "ALARM_VIOLATIONS_220045_H.describe()"
   ]
  },
  {
   "source": [
    "ALARM_VIOLATIONS_220045_L = ALARM_VIOLATIONS_220045[(ALARM_VIOLATIONS_220045['THRESHOLD_TYPE'] == 'LOW')]\n",
    "ALARM_VIOLATIONS_220045_L.describe()"
   ],
   "cell_type": "code",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 220179 - Non-Invasive Blood Pressure (Systolic)"
   ]
  },
  {
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "ALARM_VIOLATIONS_220179 = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['ITEMID'] == '220179')]\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize=(25, 5))\n",
    "fig.suptitle(\"Difference Between Actual and Threshold - 220179\", fontsize=18)\n",
    "\n",
    "sns.stripplot(data=ALARM_VIOLATIONS_220179,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[0])\n",
    "#axs[0].set_title(\"Difference Between Actual and Threshold - 220045 Scatter Plot\",fontsize=14)\n",
    "axs[0].set_ylabel(\"Difference Between Actual and Threshold\",fontsize=14)\n",
    "axs[0].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "sns.boxplot(data=ALARM_VIOLATIONS_220179,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[1])\n",
    "#axs[1].set_title(\"Difference Between Actual and Threshold - 220045 Boxplot\")\n",
    "axs[1].set_ylabel(\"Difference Between Actual and Threshold\", fontsize=14)\n",
    "axs[1].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "#sns.histplot(data=ALARM_VIOLATIONS_220179, x='DIF_ACTUAL_TH', ax=axs[2])\n",
    "#axs[2].set_title(\"HR_violations_clean histogram\")\n",
    "#axs[2].set_xlabel(\"HR_violations_clean VALUENUM\")\n",
    "\n",
    "plt.show(fig)"
   ],
   "cell_type": "code",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 220210 - Respiratory Rate"
   ]
  },
  {
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "ALARM_VIOLATIONS_220210 = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['ITEMID'] == '220210')]\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize=(25, 5))\n",
    "fig.suptitle(\"Difference Between Actual and Threshold - 220210\", fontsize=18)\n",
    "\n",
    "sns.stripplot(data=ALARM_VIOLATIONS_220210,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[0])\n",
    "#axs[0].set_title(\"Difference Between Actual and Threshold - 220045 Scatter Plot\",fontsize=14)\n",
    "axs[0].set_ylabel(\"Difference Between Actual and Threshold\",fontsize=14)\n",
    "axs[0].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "sns.boxplot(data=ALARM_VIOLATIONS_220210,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[1])\n",
    "#axs[1].set_title(\"Difference Between Actual and Threshold - 220045 Boxplot\")\n",
    "axs[1].set_ylabel(\"Difference Between Actual and Threshold\", fontsize=14)\n",
    "axs[1].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "#sns.histplot(data=ALARM_VIOLATIONS_220210, x='DIF_ACTUAL_TH', ax=axs[2])\n",
    "#axs[2].set_title(\"HR_violations_clean histogram\")\n",
    "#axs[2].set_xlabel(\"HR_violations_clean VALUENUM\")\n",
    "\n",
    "plt.show(fig)"
   ],
   "cell_type": "code",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 220277 - O2 Saturation Pulseoxymetry"
   ]
  },
  {
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "ALARM_VIOLATIONS_220277 = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['ITEMID'] == '220277')]\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize=(25, 5))\n",
    "fig.suptitle(\"Difference Between Actual and Threshold - 220277\", fontsize=18)\n",
    "\n",
    "sns.stripplot(data=ALARM_VIOLATIONS_220277,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[0])\n",
    "#axs[0].set_title(\"Difference Between Actual and Threshold - 220045 Scatter Plot\",fontsize=14)\n",
    "axs[0].set_ylabel(\"Difference Between Actual and Threshold\",fontsize=14)\n",
    "axs[0].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "sns.boxplot(data=ALARM_VIOLATIONS_220277,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[1])\n",
    "#axs[1].set_title(\"Difference Between Actual and Threshold - 220045 Boxplot\")\n",
    "axs[1].set_ylabel(\"Difference Between Actual and Threshold\", fontsize=14)\n",
    "axs[1].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "#sns.histplot(data=ALARM_VIOLATIONS_220277, x='DIF_ACTUAL_TH', ax=axs[2])\n",
    "#axs[2].set_title(\"HR_violations_clean histogram\")\n",
    "#axs[2].set_xlabel(\"HR_violations_clean VALUENUM\")\n",
    "\n",
    "plt.show(fig)"
   ],
   "cell_type": "code",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 224687 - Minute Volume"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "ALARM_VIOLATIONS_224687 = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['ITEMID'] == '224687')]\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize=(25, 5))\n",
    "fig.suptitle(\"Difference Between Actual and Threshold - 224687\", fontsize=18)\n",
    "\n",
    "sns.stripplot(data=ALARM_VIOLATIONS_224687,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[0])\n",
    "#axs[0].set_title(\"Difference Between Actual and Threshold - 220045 Scatter Plot\",fontsize=14)\n",
    "axs[0].set_ylabel(\"Difference Between Actual and Threshold\",fontsize=14)\n",
    "axs[0].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "sns.boxplot(data=ALARM_VIOLATIONS_224687,x='THRESHOLD_TYPE', y='DIF_ACTUAL_TH', ax=axs[1])\n",
    "#axs[1].set_title(\"Difference Between Actual and Threshold - 220045 Boxplot\")\n",
    "axs[1].set_ylabel(\"Difference Between Actual and Threshold\", fontsize=14)\n",
    "axs[1].set_xlabel(\"Threshold Type\",fontsize=14)\n",
    "\n",
    "#sns.histplot(data=ALARM_VIOLATIONS_224687, x='DIF_ACTUAL_TH', ax=axs[2])\n",
    "#axs[2].set_title(\"HR_violations_clean histogram\")\n",
    "#axs[2].set_xlabel(\"HR_violations_clean VALUENUM\")\n",
    "\n",
    "plt.show(fig)"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Analysis of Time Between Setting a Threshold and Raising an Alarm\n",
    "\n",
    "### Creation of Additional Columns"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "set_threshold = pd.to_datetime(ALARM_VIOLATIONS['THRESHOLD_CHARTTIME'])\n",
    "raised_alarm = pd.to_datetime(ALARM_VIOLATIONS['CHARTTIME'])\n",
    "\n",
    "ALARM_VIOLATIONS['TIME_UNTIL_ALARM'] = pd.to_timedelta(raised_alarm - set_threshold)\n",
    "\n",
    "ALARM_VIOLATIONS['SEC_UNTIL_ALARM'] = ALARM_VIOLATIONS['TIME_UNTIL_ALARM']\\\n",
    "    .dt\\\n",
    "    .total_seconds()\\\n",
    "    .astype(int)\n",
    "\n",
    "ALARM_VIOLATIONS.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "time_with_sec_info = ALARM_VIOLATIONS[ALARM_VIOLATIONS['SEC_UNTIL_ALARM'] % 60 != 0]\n",
    "time_with_sec_info.SEC_UNTIL_ALARM.describe()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Since there are no seconds information in 'alarm_violations.csv', we can only examine the time difference for a minute accuracy."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "ALARM_VIOLATIONS['MIN_UNTIL_ALARM'] = ALARM_VIOLATIONS['SEC_UNTIL_ALARM']\\\n",
    "    .divide(60)\\\n",
    "    .astype(int)\n",
    "\n",
    "del ALARM_VIOLATIONS['SEC_UNTIL_ALARM']\n",
    "\n",
    "ALARM_VIOLATIONS.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Passed Time of All Triggered Alarms"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "fig, axs = plt.subplots(\n",
    "    2,\n",
    "    1,\n",
    "    figsize = (10, 15),\n",
    "    sharex = True,\n",
    "    dpi = 72)\n",
    "fig.suptitle('Minutes Until Alarm is Triggered', fontweight='bold', color= 'black', fontsize=14, y=0.9)\n",
    "fig.subplots_adjust(hspace = 0.1)\n",
    "\n",
    "sns.stripplot(\n",
    "    ax = axs[0],\n",
    "    data = ALARM_VIOLATIONS,\n",
    "    x = 'MIN_UNTIL_ALARM',\n",
    "    palette = sns.color_palette(\"colorblind\")\n",
    "    )\n",
    "axs[0].set_xlabel(\"\")\n",
    "axs[0].grid(b=True, which='both')\n",
    "axs[0].margins(.1)\n",
    "\n",
    "sns.boxplot(\n",
    "    ax = axs[1],\n",
    "    data = ALARM_VIOLATIONS,\n",
    "    x = 'MIN_UNTIL_ALARM',\n",
    "    palette = sns.color_palette(\"colorblind\")\n",
    "    )\n",
    "axs[1].set_xlabel(\"Minutes\")\n",
    "axs[1].grid(b=True, which='both')\n",
    "axs[1].margins(.1)\n",
    "\n",
    "ALARM_VIOLATIONS.MIN_UNTIL_ALARM.describe()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "As expected, the majority of these approx. 390,000 alarms was triggered in the time period of 0 to approx. 10,000 minutes with a descending trend. The maximum time it takes for an alarm to be triggered is approximately 22 days."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "instant_alarms = ALARM_VIOLATIONS[ALARM_VIOLATIONS['MIN_UNTIL_ALARM'] < 1]\n",
    "len(instant_alarms)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Among the 390,000 triggered alarms, there are 13,963 values that were triggered within the first minute after setting the threshold. These alarms should be removed, as we assume that the majority are threshold corrections that have already been made after a few seconds."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "cleaned_alarms = ALARM_VIOLATIONS[ALARM_VIOLATIONS['MIN_UNTIL_ALARM'] >= 1]\n",
    "cleaned_alarms.MIN_UNTIL_ALARM.describe()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### First 15 Minutes After Setting a Threshold"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "alarms_within_15min = ALARM_VIOLATIONS[(ALARM_VIOLATIONS['MIN_UNTIL_ALARM'] >= 1) & (ALARM_VIOLATIONS['MIN_UNTIL_ALARM'] <= 15)]"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.histplot(\n",
    "    data=alarms_within_15min,\n",
    "    x='MIN_UNTIL_ALARM',\n",
    "    kde=True)\n",
    "\n",
    "plt.title(\n",
    "    'Alarms Triggered After 1 to After 15 Minutes',\n",
    "    fontsize=12,\n",
    "    fontweight='bold')\n",
    "plt.xlabel('Minutes', fontsize=12)\n",
    "plt.ylabel('Count', fontsize=12)\n",
    "\n",
    "alarms_within_15min.MIN_UNTIL_ALARM.describe()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "In the time period of one to two minutes after setting the threshold, 1,482 alarms are triggered. Perhaps these alarms should also be removed, as they can occur very shortly after the first minute has passed. It may be that a nurse noticed only then that she had made an incorrect input.\n",
    "\n",
    "The numbers of alarms that occur after more than two minutes after setting a threshold seem plausible."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "time_until_alarm_stratified_by_itemid = alarms_within_15min\\\n",
    "    .groupby(['ITEMID', 'MIN_UNTIL_ALARM'])\\\n",
    "    .size()\\\n",
    "    .reset_index(name='Count')\n",
    "\n",
    "time_until_alarm_stratified_by_itemid.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "sns.histplot(\n",
    "    data=alarms_within_15min,\n",
    "    x='MIN_UNTIL_ALARM',\n",
    "    hue='ITEMID',\n",
    "    multiple='stack',\n",
    "    palette=sns.color_palette('colorblind', n_colors=5))\n",
    "\n",
    "plt.title(\n",
    "    'Alarms Triggered After 1 to 15 Minutes (Stratified by ITEM ID)',\n",
    "    fontsize=12,\n",
    "    fontweight='bold')\n",
    "plt.xlabel('Minutes', fontsize=12)\n",
    "plt.ylabel('Count', fontsize=12)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "In the stratified view, you can see that most alarms were triggered because of a too low or too high systolic blood pressure - regardless of how many minutes have passed. The other parameter thresholds are exceeded or undercut with approximately the same frequency."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Open questions we have to answer with whole CHARTSEVENT table\n",
    "\n",
    "* Can you deduce something from the time of day? E.g. more alarms occur at night than during the day?\n",
    "* After how many minutes is an alarm triggered on average after setting a treshold (stratified by ITEM ID)?\n",
    "* Thesis: \"The more frequently per time unit a nurse adjusts the treshold, the more likely there is a violation.\"\n",
    "* We need an extended data set to predict violations (see alarm_violations_extended_by_normal_measurements.png)\n",
    "* We would also need to include the patient again in order to get the patient demographics"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  }
 ]
}