coxph_test/coxph_test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **CoxPH test**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function handles evaluating Cox proportional hazards model performance, test one or more classifier models against held-out dataset Using held-out test features,<br> and evaluates the peformance of the estimated model.<br> \n",
    "Can be part of a kubeflow pipeline as a test step that is run post EDA and training/validation cycles.<br>\n",
    "This function is part of the [customer-churn-prediction](https://github.com/mlrun/demos/tree/master/customer-churn-prediction) demo. <br>\n",
    "To see how the model is trained or how the data-set is generated, check out `coxph_trainer` function from the function marketplace repository"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Steps**\n",
    "1. [Setup function parameters](#Setup-function-parameters)\n",
    "2. [Importing the function](#Importing-the-function)\n",
    "3. [Running the function locally](#Running-the-function-locally)\n",
    "4. [Running the function remotely](#Running-the-function-remotely)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Setup function parameters**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_set = \"https://s3.wasabisys.com/iguazio/data/function-marketplace-data/xgb_test/test_set.csv\"\n",
    "models_path = \"https://s3.wasabisys.com/iguazio/models/function-marketplace-models/coxph_test/cx-model.pkl\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Importing the function**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2021-10-17 13:38:44,758 [info] loaded project function-marketplace from MLRun DB\n"
     ]
    }
   ],
   "source": [
    "import mlrun\n",
    "mlrun.set_environment(project='function-marketplace')\n",
    "\n",
    "fn = mlrun.import_function(\"hub://coxph_test\")\n",
    "fn.apply(mlrun.auto_mount())\n",
    "\n",
    "fn.spec.build.image=\"mlrun/ml-models\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Running the function locally**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2021-10-17 13:38:45,149 [info] starting run tasks_coxph_test uid=be4bd195e5c146a69ecdee3b6a631569 DB=http://mlrun-api:8080\n",
      "> 2021-10-17 13:38:49,428 [info] cox tester not implemented\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".dictlist {\n",
       "  background-color: #4EC64B;\n",
       "  text-align: center;\n",
       "  margin: 4px;\n",
       "  border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;}\n",
       ".artifact {\n",
       "  cursor: pointer;\n",
       "  background-color: #4EC64B;\n",
       "  text-align: left;\n",
       "  margin: 4px; border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;\n",
       "}\n",
       "div.block.hidden {\n",
       "  display: none;\n",
       "}\n",
       ".clickable {\n",
       "  cursor: pointer;\n",
       "}\n",
       ".ellipsis {\n",
       "  display: inline-block;\n",
       "  max-width: 60px;\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "}\n",
       ".master-wrapper {\n",
       "  display: flex;\n",
       "  flex-flow: row nowrap;\n",
       "  justify-content: flex-start;\n",
       "  align-items: stretch;\n",
       "}\n",
       ".master-tbl {\n",
       "  flex: 3\n",
       "}\n",
       ".master-wrapper > div {\n",
       "  margin: 4px;\n",
       "  padding: 10px;\n",
       "}\n",
       "iframe.fileview {\n",
       "  border: 0 none;\n",
       "  height: 100%;\n",
       "  width: 100%;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       ".pane-header-title {\n",
       "  width: 80%;\n",
       "  font-weight: 500;\n",
       "}\n",
       ".pane-header {\n",
       "  line-height: 1;\n",
       "  background-color: #4EC64B;\n",
       "  padding: 3px;\n",
       "}\n",
       ".pane-header .close {\n",
       "  font-size: 20px;\n",
       "  font-weight: 700;\n",
       "  float: right;\n",
       "  margin-top: -5px;\n",
       "}\n",
       ".master-wrapper .right-pane {\n",
       "  border: 1px inset silver;\n",
       "  width: 40%;\n",
       "  min-height: 300px;\n",
       "  flex: 3\n",
       "  min-width: 500px;\n",
       "}\n",
       ".master-wrapper * {\n",
       "  box-sizing: border-box;\n",
       "}\n",
       "</style><script>\n",
       "function copyToClipboard(fld) {\n",
       "    if (document.queryCommandSupported && document.queryCommandSupported('copy')) {\n",
       "        var textarea = document.createElement('textarea');\n",
       "        textarea.textContent = fld.innerHTML;\n",
       "        textarea.style.position = 'fixed';\n",
       "        document.body.appendChild(textarea);\n",
       "        textarea.select();\n",
       "\n",
       "        try {\n",
       "            return document.execCommand('copy'); // Security exception may be thrown by some browsers.\n",
       "        } catch (ex) {\n",
       "\n",
       "        } finally {\n",
       "            document.body.removeChild(textarea);\n",
       "        }\n",
       "    }\n",
       "}\n",
       "function expandPanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName');\n",
       "  console.log(el.title);\n",
       "\n",
       "  document.querySelector(panelName + \"-title\").innerHTML = el.title\n",
       "  iframe = document.querySelector(panelName + \"-body\");\n",
       "\n",
       "  const tblcss = `<style> body { font-family: Arial, Helvetica, sans-serif;}\n",
       "    #csv { margin-bottom: 15px; }\n",
       "    #csv table { border-collapse: collapse;}\n",
       "    #csv table td { padding: 4px 8px; border: 1px solid silver;} </style>`;\n",
       "\n",
       "  function csvToHtmlTable(str) {\n",
       "    return '<div id=\"csv\"><table><tr><td>' +  str.replace(/[\\n\\r]+$/g, '').replace(/[\\n\\r]+/g, '</td></tr><tr><td>')\n",
       "      .replace(/,/g, '</td><td>') + '</td></tr></table></div>';\n",
       "  }\n",
       "\n",
       "  function reqListener () {\n",
       "    if (el.title.endsWith(\".csv\")) {\n",
       "      iframe.setAttribute(\"srcdoc\", tblcss + csvToHtmlTable(this.responseText));\n",
       "    } else {\n",
       "      iframe.setAttribute(\"srcdoc\", this.responseText);\n",
       "    }\n",
       "    console.log(this.responseText);\n",
       "  }\n",
       "\n",
       "  const oReq = new XMLHttpRequest();\n",
       "  oReq.addEventListener(\"load\", reqListener);\n",
       "  oReq.open(\"GET\", el.title);\n",
       "  oReq.send();\n",
       "\n",
       "\n",
       "  //iframe.src = el.title;\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.remove(\"hidden\");\n",
       "  }\n",
       "}\n",
       "function closePanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName')\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (!resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.add(\"hidden\");\n",
       "  }\n",
       "}\n",
       "\n",
       "</script>\n",
       "<div class=\"master-wrapper\">\n",
       "  <div class=\"block master-tbl\"><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>project</th>\n",
       "      <th>uid</th>\n",
       "      <th>iter</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>labels</th>\n",
       "      <th>inputs</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "      <th>artifacts</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>function-marketplace</td>\n",
       "      <td><div title=\"be4bd195e5c146a69ecdee3b6a631569\"><a href=\"https://dashboard.default-tenant.app.dev39.lab.iguazeng.com/mlprojects/function-marketplace/jobs/monitor/be4bd195e5c146a69ecdee3b6a631569/overview\" target=\"_blank\" >...6a631569</a></div></td>\n",
       "      <td>0</td>\n",
       "      <td>Oct 17 13:38:45</td>\n",
       "      <td>completed</td>\n",
       "      <td>tasks_coxph_test</td>\n",
       "      <td><div class=\"dictlist\">v3io_user=dani</div><div class=\"dictlist\">kind=</div><div class=\"dictlist\">owner=dani</div><div class=\"dictlist\">host=jupyter-dani-6bfbd76d96-zxx6f</div></td>\n",
       "      <td><div title=\"https://s3.wasabisys.com/iguazio/data/function-marketplace-data/xgb_test/test_set.csv\">test_set</div><div title=\"https://s3.wasabisys.com/iguazio/models/function-marketplace-models/coxph_test/cx-model.pkl\">models_path</div></td>\n",
       "      <td><div class=\"dictlist\">label_column=labels</div><div class=\"dictlist\">plots_dest=plots/xgb_test</div></td>\n",
       "      <td></td>\n",
       "      <td><div class=\"artifact\" onclick=\"expandPanel(this)\" paneName=\"result6ec7c8b5\" title=\"files/v3io/projects/function-marketplace/artifacts/cox-test-summary.csv\">cox-test-summary</div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div></div>\n",
       "  <div id=\"result6ec7c8b5-pane\" class=\"right-pane block hidden\">\n",
       "    <div class=\"pane-header\">\n",
       "      <span id=\"result6ec7c8b5-title\" class=\"pane-header-title\">Title</span>\n",
       "      <span onclick=\"closePanel(this)\" paneName=\"result6ec7c8b5\" class=\"close clickable\">&times;</span>\n",
       "    </div>\n",
       "    <iframe class=\"fileview\" id=\"result6ec7c8b5-body\"></iframe>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<b> > to track results use the .show() or .logs() methods  or <a href=\"https://dashboard.default-tenant.app.dev39.lab.iguazeng.com/mlprojects/function-marketplace/jobs/monitor/be4bd195e5c146a69ecdee3b6a631569/overview\" target=\"_blank\">click here</a> to open in UI</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2021-10-17 13:38:49,497 [info] run executed, status=completed\n"
     ]
    }
   ],
   "source": [
    "coxph_run = fn.run(name='tasks_coxph_test',\n",
    "                   params = {\"label_column\"  : \"labels\",\n",
    "                             \"plots_dest\"    : \"plots/xgb_test\"},\n",
    "                   inputs = {\"test_set\"      : test_set,\n",
    "                             \"models_path\"   : models_path},\n",
    "                   local=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>covariate</th>\n",
       "      <th>coef</th>\n",
       "      <th>exp(coef)</th>\n",
       "      <th>se(coef)</th>\n",
       "      <th>coef lower 95%</th>\n",
       "      <th>coef upper 95%</th>\n",
       "      <th>exp(coef) lower 95%</th>\n",
       "      <th>exp(coef) upper 95%</th>\n",
       "      <th>z</th>\n",
       "      <th>p</th>\n",
       "      <th>-log2(p)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>gender</td>\n",
       "      <td>0.712986</td>\n",
       "      <td>2.040073e+00</td>\n",
       "      <td>0.343471</td>\n",
       "      <td>0.039795</td>\n",
       "      <td>1.386176</td>\n",
       "      <td>1.040598</td>\n",
       "      <td>3.999528</td>\n",
       "      <td>2.075826</td>\n",
       "      <td>0.037910</td>\n",
       "      <td>4.721274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>senior</td>\n",
       "      <td>-0.330137</td>\n",
       "      <td>7.188252e-01</td>\n",
       "      <td>0.444705</td>\n",
       "      <td>-1.201743</td>\n",
       "      <td>0.541468</td>\n",
       "      <td>0.300670</td>\n",
       "      <td>1.718528</td>\n",
       "      <td>-0.742374</td>\n",
       "      <td>0.457861</td>\n",
       "      <td>1.127018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>partner</td>\n",
       "      <td>-0.394449</td>\n",
       "      <td>6.740516e-01</td>\n",
       "      <td>0.432243</td>\n",
       "      <td>-1.241630</td>\n",
       "      <td>0.452732</td>\n",
       "      <td>0.288913</td>\n",
       "      <td>1.572603</td>\n",
       "      <td>-0.912562</td>\n",
       "      <td>0.361473</td>\n",
       "      <td>1.468041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>deps</td>\n",
       "      <td>0.616373</td>\n",
       "      <td>1.852199e+00</td>\n",
       "      <td>0.499075</td>\n",
       "      <td>-0.361797</td>\n",
       "      <td>1.594543</td>\n",
       "      <td>0.696424</td>\n",
       "      <td>4.926080</td>\n",
       "      <td>1.235031</td>\n",
       "      <td>0.216819</td>\n",
       "      <td>2.205436</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MultipleLines</td>\n",
       "      <td>-0.787885</td>\n",
       "      <td>4.548059e-01</td>\n",
       "      <td>1.087536</td>\n",
       "      <td>-2.919417</td>\n",
       "      <td>1.343648</td>\n",
       "      <td>0.053965</td>\n",
       "      <td>3.832999</td>\n",
       "      <td>-0.724467</td>\n",
       "      <td>0.468779</td>\n",
       "      <td>1.093020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>OnlineSecurity</td>\n",
       "      <td>-0.766683</td>\n",
       "      <td>4.645512e-01</td>\n",
       "      <td>1.299746</td>\n",
       "      <td>-3.314139</td>\n",
       "      <td>1.780772</td>\n",
       "      <td>0.036365</td>\n",
       "      <td>5.934435</td>\n",
       "      <td>-0.589872</td>\n",
       "      <td>0.555277</td>\n",
       "      <td>0.848721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>OnlineBackup</td>\n",
       "      <td>-0.466691</td>\n",
       "      <td>6.270740e-01</td>\n",
       "      <td>0.949068</td>\n",
       "      <td>-2.326829</td>\n",
       "      <td>1.393448</td>\n",
       "      <td>0.097605</td>\n",
       "      <td>4.028715</td>\n",
       "      <td>-0.491736</td>\n",
       "      <td>0.622906</td>\n",
       "      <td>0.682914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DeviceProtection</td>\n",
       "      <td>-0.412620</td>\n",
       "      <td>6.619136e-01</td>\n",
       "      <td>1.083731</td>\n",
       "      <td>-2.536694</td>\n",
       "      <td>1.711453</td>\n",
       "      <td>0.079128</td>\n",
       "      <td>5.537002</td>\n",
       "      <td>-0.380741</td>\n",
       "      <td>0.703396</td>\n",
       "      <td>0.507591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>TechSupport</td>\n",
       "      <td>0.509756</td>\n",
       "      <td>1.664885e+00</td>\n",
       "      <td>1.168080</td>\n",
       "      <td>-1.779638</td>\n",
       "      <td>2.799150</td>\n",
       "      <td>0.168699</td>\n",
       "      <td>16.430675</td>\n",
       "      <td>0.436405</td>\n",
       "      <td>0.662543</td>\n",
       "      <td>0.593915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>PaperlessBilling</td>\n",
       "      <td>0.349970</td>\n",
       "      <td>1.419025e+00</td>\n",
       "      <td>0.408827</td>\n",
       "      <td>-0.451317</td>\n",
       "      <td>1.151257</td>\n",
       "      <td>0.636789</td>\n",
       "      <td>3.162165</td>\n",
       "      <td>0.856033</td>\n",
       "      <td>0.391980</td>\n",
       "      <td>1.351150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>MonthlyCharges</td>\n",
       "      <td>-0.078399</td>\n",
       "      <td>9.245958e-01</td>\n",
       "      <td>0.194463</td>\n",
       "      <td>-0.459539</td>\n",
       "      <td>0.302742</td>\n",
       "      <td>0.631574</td>\n",
       "      <td>1.353566</td>\n",
       "      <td>-0.403154</td>\n",
       "      <td>0.686835</td>\n",
       "      <td>0.541965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Contract_1</td>\n",
       "      <td>-2.188279</td>\n",
       "      <td>1.121096e-01</td>\n",
       "      <td>0.712197</td>\n",
       "      <td>-3.584159</td>\n",
       "      <td>-0.792398</td>\n",
       "      <td>0.027760</td>\n",
       "      <td>0.452758</td>\n",
       "      <td>-3.072575</td>\n",
       "      <td>0.002122</td>\n",
       "      <td>8.880219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Contract_2</td>\n",
       "      <td>-19.940767</td>\n",
       "      <td>2.186930e-09</td>\n",
       "      <td>3478.684973</td>\n",
       "      <td>-6838.038027</td>\n",
       "      <td>6798.156493</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>inf</td>\n",
       "      <td>-0.005732</td>\n",
       "      <td>0.995426</td>\n",
       "      <td>0.006614</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Payment_1</td>\n",
       "      <td>-0.865424</td>\n",
       "      <td>4.208732e-01</td>\n",
       "      <td>0.615020</td>\n",
       "      <td>-2.070840</td>\n",
       "      <td>0.339993</td>\n",
       "      <td>0.126080</td>\n",
       "      <td>1.404937</td>\n",
       "      <td>-1.407148</td>\n",
       "      <td>0.159383</td>\n",
       "      <td>2.649426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Payment_2</td>\n",
       "      <td>0.458363</td>\n",
       "      <td>1.581483e+00</td>\n",
       "      <td>0.446978</td>\n",
       "      <td>-0.417697</td>\n",
       "      <td>1.334423</td>\n",
       "      <td>0.658562</td>\n",
       "      <td>3.797805</td>\n",
       "      <td>1.025472</td>\n",
       "      <td>0.305141</td>\n",
       "      <td>1.712453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Payment_3</td>\n",
       "      <td>0.232519</td>\n",
       "      <td>1.261774e+00</td>\n",
       "      <td>0.641176</td>\n",
       "      <td>-1.024162</td>\n",
       "      <td>1.489200</td>\n",
       "      <td>0.359097</td>\n",
       "      <td>4.433547</td>\n",
       "      <td>0.362644</td>\n",
       "      <td>0.716870</td>\n",
       "      <td>0.480216</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           covariate       coef     exp(coef)     se(coef)  coef lower 95%  \\\n",
       "0             gender   0.712986  2.040073e+00     0.343471        0.039795   \n",
       "1             senior  -0.330137  7.188252e-01     0.444705       -1.201743   \n",
       "2            partner  -0.394449  6.740516e-01     0.432243       -1.241630   \n",
       "3               deps   0.616373  1.852199e+00     0.499075       -0.361797   \n",
       "4      MultipleLines  -0.787885  4.548059e-01     1.087536       -2.919417   \n",
       "5     OnlineSecurity  -0.766683  4.645512e-01     1.299746       -3.314139   \n",
       "6       OnlineBackup  -0.466691  6.270740e-01     0.949068       -2.326829   \n",
       "7   DeviceProtection  -0.412620  6.619136e-01     1.083731       -2.536694   \n",
       "8        TechSupport   0.509756  1.664885e+00     1.168080       -1.779638   \n",
       "9   PaperlessBilling   0.349970  1.419025e+00     0.408827       -0.451317   \n",
       "10    MonthlyCharges  -0.078399  9.245958e-01     0.194463       -0.459539   \n",
       "11        Contract_1  -2.188279  1.121096e-01     0.712197       -3.584159   \n",
       "12        Contract_2 -19.940767  2.186930e-09  3478.684973    -6838.038027   \n",
       "13         Payment_1  -0.865424  4.208732e-01     0.615020       -2.070840   \n",
       "14         Payment_2   0.458363  1.581483e+00     0.446978       -0.417697   \n",
       "15         Payment_3   0.232519  1.261774e+00     0.641176       -1.024162   \n",
       "\n",
       "    coef upper 95%  exp(coef) lower 95%  exp(coef) upper 95%         z  \\\n",
       "0         1.386176             1.040598             3.999528  2.075826   \n",
       "1         0.541468             0.300670             1.718528 -0.742374   \n",
       "2         0.452732             0.288913             1.572603 -0.912562   \n",
       "3         1.594543             0.696424             4.926080  1.235031   \n",
       "4         1.343648             0.053965             3.832999 -0.724467   \n",
       "5         1.780772             0.036365             5.934435 -0.589872   \n",
       "6         1.393448             0.097605             4.028715 -0.491736   \n",
       "7         1.711453             0.079128             5.537002 -0.380741   \n",
       "8         2.799150             0.168699            16.430675  0.436405   \n",
       "9         1.151257             0.636789             3.162165  0.856033   \n",
       "10        0.302742             0.631574             1.353566 -0.403154   \n",
       "11       -0.792398             0.027760             0.452758 -3.072575   \n",
       "12     6798.156493             0.000000                  inf -0.005732   \n",
       "13        0.339993             0.126080             1.404937 -1.407148   \n",
       "14        1.334423             0.658562             3.797805  1.025472   \n",
       "15        1.489200             0.359097             4.433547  0.362644   \n",
       "\n",
       "           p  -log2(p)  \n",
       "0   0.037910  4.721274  \n",
       "1   0.457861  1.127018  \n",
       "2   0.361473  1.468041  \n",
       "3   0.216819  2.205436  \n",
       "4   0.468779  1.093020  \n",
       "5   0.555277  0.848721  \n",
       "6   0.622906  0.682914  \n",
       "7   0.703396  0.507591  \n",
       "8   0.662543  0.593915  \n",
       "9   0.391980  1.351150  \n",
       "10  0.686835  0.541965  \n",
       "11  0.002122  8.880219  \n",
       "12  0.995426  0.006614  \n",
       "13  0.159383  2.649426  \n",
       "14  0.305141  1.712453  \n",
       "15  0.716870  0.480216  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "coxph_run.artifact('cox-test-summary').show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Running the function remotely**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2021-10-17 13:38:49,644 [info] starting run tasks_coxph_test uid=c28d05f0261b4c60956eee528bf68e96 DB=http://mlrun-api:8080\n",
      "> 2021-10-17 13:38:49,776 [info] Job is running in the background, pod: tasks-coxph-test-hfj9b\n",
      "> 2021-10-17 13:38:59,015 [info] cox tester not implemented\n",
      "> 2021-10-17 13:38:59,049 [info] run executed, status=completed\n",
      "final state: completed\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".dictlist {\n",
       "  background-color: #4EC64B;\n",
       "  text-align: center;\n",
       "  margin: 4px;\n",
       "  border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;}\n",
       ".artifact {\n",
       "  cursor: pointer;\n",
       "  background-color: #4EC64B;\n",
       "  text-align: left;\n",
       "  margin: 4px; border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;\n",
       "}\n",
       "div.block.hidden {\n",
       "  display: none;\n",
       "}\n",
       ".clickable {\n",
       "  cursor: pointer;\n",
       "}\n",
       ".ellipsis {\n",
       "  display: inline-block;\n",
       "  max-width: 60px;\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "}\n",
       ".master-wrapper {\n",
       "  display: flex;\n",
       "  flex-flow: row nowrap;\n",
       "  justify-content: flex-start;\n",
       "  align-items: stretch;\n",
       "}\n",
       ".master-tbl {\n",
       "  flex: 3\n",
       "}\n",
       ".master-wrapper > div {\n",
       "  margin: 4px;\n",
       "  padding: 10px;\n",
       "}\n",
       "iframe.fileview {\n",
       "  border: 0 none;\n",
       "  height: 100%;\n",
       "  width: 100%;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       ".pane-header-title {\n",
       "  width: 80%;\n",
       "  font-weight: 500;\n",
       "}\n",
       ".pane-header {\n",
       "  line-height: 1;\n",
       "  background-color: #4EC64B;\n",
       "  padding: 3px;\n",
       "}\n",
       ".pane-header .close {\n",
       "  font-size: 20px;\n",
       "  font-weight: 700;\n",
       "  float: right;\n",
       "  margin-top: -5px;\n",
       "}\n",
       ".master-wrapper .right-pane {\n",
       "  border: 1px inset silver;\n",
       "  width: 40%;\n",
       "  min-height: 300px;\n",
       "  flex: 3\n",
       "  min-width: 500px;\n",
       "}\n",
       ".master-wrapper * {\n",
       "  box-sizing: border-box;\n",
       "}\n",
       "</style><script>\n",
       "function copyToClipboard(fld) {\n",
       "    if (document.queryCommandSupported && document.queryCommandSupported('copy')) {\n",
       "        var textarea = document.createElement('textarea');\n",
       "        textarea.textContent = fld.innerHTML;\n",
       "        textarea.style.position = 'fixed';\n",
       "        document.body.appendChild(textarea);\n",
       "        textarea.select();\n",
       "\n",
       "        try {\n",
       "            return document.execCommand('copy'); // Security exception may be thrown by some browsers.\n",
       "        } catch (ex) {\n",
       "\n",
       "        } finally {\n",
       "            document.body.removeChild(textarea);\n",
       "        }\n",
       "    }\n",
       "}\n",
       "function expandPanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName');\n",
       "  console.log(el.title);\n",
       "\n",
       "  document.querySelector(panelName + \"-title\").innerHTML = el.title\n",
       "  iframe = document.querySelector(panelName + \"-body\");\n",
       "\n",
       "  const tblcss = `<style> body { font-family: Arial, Helvetica, sans-serif;}\n",
       "    #csv { margin-bottom: 15px; }\n",
       "    #csv table { border-collapse: collapse;}\n",
       "    #csv table td { padding: 4px 8px; border: 1px solid silver;} </style>`;\n",
       "\n",
       "  function csvToHtmlTable(str) {\n",
       "    return '<div id=\"csv\"><table><tr><td>' +  str.replace(/[\\n\\r]+$/g, '').replace(/[\\n\\r]+/g, '</td></tr><tr><td>')\n",
       "      .replace(/,/g, '</td><td>') + '</td></tr></table></div>';\n",
       "  }\n",
       "\n",
       "  function reqListener () {\n",
       "    if (el.title.endsWith(\".csv\")) {\n",
       "      iframe.setAttribute(\"srcdoc\", tblcss + csvToHtmlTable(this.responseText));\n",
       "    } else {\n",
       "      iframe.setAttribute(\"srcdoc\", this.responseText);\n",
       "    }\n",
       "    console.log(this.responseText);\n",
       "  }\n",
       "\n",
       "  const oReq = new XMLHttpRequest();\n",
       "  oReq.addEventListener(\"load\", reqListener);\n",
       "  oReq.open(\"GET\", el.title);\n",
       "  oReq.send();\n",
       "\n",
       "\n",
       "  //iframe.src = el.title;\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.remove(\"hidden\");\n",
       "  }\n",
       "}\n",
       "function closePanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName')\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (!resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.add(\"hidden\");\n",
       "  }\n",
       "}\n",
       "\n",
       "</script>\n",
       "<div class=\"master-wrapper\">\n",
       "  <div class=\"block master-tbl\"><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>project</th>\n",
       "      <th>uid</th>\n",
       "      <th>iter</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>labels</th>\n",
       "      <th>inputs</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "      <th>artifacts</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>function-marketplace</td>\n",
       "      <td><div title=\"c28d05f0261b4c60956eee528bf68e96\"><a href=\"https://dashboard.default-tenant.app.dev39.lab.iguazeng.com/mlprojects/function-marketplace/jobs/monitor/c28d05f0261b4c60956eee528bf68e96/overview\" target=\"_blank\" >...8bf68e96</a></div></td>\n",
       "      <td>0</td>\n",
       "      <td>Oct 17 13:38:56</td>\n",
       "      <td>completed</td>\n",
       "      <td>tasks_coxph_test</td>\n",
       "      <td><div class=\"dictlist\">v3io_user=dani</div><div class=\"dictlist\">kind=job</div><div class=\"dictlist\">owner=dani</div><div class=\"dictlist\">host=tasks-coxph-test-hfj9b</div></td>\n",
       "      <td><div title=\"https://s3.wasabisys.com/iguazio/data/function-marketplace-data/xgb_test/test_set.csv\">test_set</div><div title=\"https://s3.wasabisys.com/iguazio/models/function-marketplace-models/coxph_test/cx-model.pkl\">models_path</div></td>\n",
       "      <td><div class=\"dictlist\">label_column=labels</div><div class=\"dictlist\">plots_dest=plots/xgb_test</div></td>\n",
       "      <td></td>\n",
       "      <td><div class=\"artifact\" onclick=\"expandPanel(this)\" paneName=\"resultaf86c597\" title=\"files/v3io/projects/function-marketplace/artifacts/cox-test-summary.csv\">cox-test-summary</div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div></div>\n",
       "  <div id=\"resultaf86c597-pane\" class=\"right-pane block hidden\">\n",
       "    <div class=\"pane-header\">\n",
       "      <span id=\"resultaf86c597-title\" class=\"pane-header-title\">Title</span>\n",
       "      <span onclick=\"closePanel(this)\" paneName=\"resultaf86c597\" class=\"close clickable\">&times;</span>\n",
       "    </div>\n",
       "    <iframe class=\"fileview\" id=\"resultaf86c597-body\"></iframe>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<b> > to track results use the .show() or .logs() methods  or <a href=\"https://dashboard.default-tenant.app.dev39.lab.iguazeng.com/mlprojects/function-marketplace/jobs/monitor/c28d05f0261b4c60956eee528bf68e96/overview\" target=\"_blank\">click here</a> to open in UI</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2021-10-17 13:39:08,990 [info] run executed, status=completed\n"
     ]
    }
   ],
   "source": [
    "fn.deploy(with_mlrun=False, # mlrun is included in our image (mlrun/ml-models) therefore no mlrun installation is needed.\n",
    "          skip_deployed=True) # because no new packages or upgrade is required, we can use the original image and not build another one.\n",
    "\n",
    "coxph_run = fn.run(name='tasks_coxph_test',\n",
    "                   params = {\"label_column\"  : \"labels\",\n",
    "                             \"plots_dest\"    : \"plots/xgb_test\"},\n",
    "                   inputs = {\"test_set\"      : test_set,\n",
    "                             \"models_path\"   : models_path},\n",
    "                   local=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Back to the top](#CoxPH-test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}