notebooks/search/08-learning-to-rank.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TKxL_NmZmOpF"
   },
   "source": [
    "# How to train and deploy Learning To Rank\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/notebooks/search/08-learning-to-rank.ipynb)\n",
    "\n",
    "In this notebook, we'll:\n",
    "\n",
    "- Connect to an Elasticsearch deployment using the official Python client.\n",
    "- Import and index a movie dataset into Elasticsearch.\n",
    "- Extract features from our dataset using Elasticsearch's Query DSL, including custom `script_score` queries.\n",
    "- Build a training dataset by combining extracted features with a human curated judgment list.\n",
    "- Train a Learning To Rank model using [XGBoost](https://xgboost.ai/).\n",
    "- Deploy the trained model to Elasticsearch using [Eland](https://eland.readthedocs.io/en/latest/).\n",
    "- Use the model as a rescorer for second stage re-ranking.\n",
    "- Evaluate the impact of the LTR model on search relevance, by comparing search results before and after applying the model.\n",
    "\n",
    "> **NOTE:**\n",
    "> - Learning To Rank is generally available for Elastic Stack versions 8.15.0 and newer and requires an Enterprise subscription or higher.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jq6mztWOmOpH"
   },
   "source": [
    "## Install required packages\n",
    "\n",
    "First we must install the packages we need for this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0nCl2nhamOpH",
    "outputId": "1e7380e7-4944-430a-db5f-180f1e299615"
   },
   "outputs": [],
   "source": [
    "!pip install -qU elasticsearch eland \"eland[scikit-learn]\" xgboost tqdm\n",
    "\n",
    "from tqdm import tqdm\n",
    "\n",
    "# Setup the progress bar so we can use progress_apply in the notebook.\n",
    "tqdm.pandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yks44hf0mOpI"
   },
   "source": [
    "## Configure your Elasticsearch deployment\n",
    "\n",
    "For this example, we will be using an [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) deployment (available with a [free trial](https://cloud.elastic.co/registration?onboarding_token=vectorsearch&utm_source=github&utm_content=elasticsearch-labs-notebook))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 71
    },
    "id": "IpnP7JUHmOpI",
    "outputId": "eb52c692-a773-4863-f930-fdedb5c6e0eb"
   },
   "outputs": [],
   "source": [
    "from getpass import getpass\n",
    "from elasticsearch import Elasticsearch\n",
    "\n",
    "# https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#finding-your-cloud-id\n",
    "ELASTIC_CLOUD_ID = getpass(\"Elastic Cloud ID: \")\n",
    "\n",
    "# https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#creating-an-api-key\n",
    "ELASTIC_API_KEY = getpass(\"Elastic Api Key: \")\n",
    "\n",
    "# Create the client instance\n",
    "es_client = Elasticsearch(\n",
    "    # For local development\n",
    "    # hosts=[\"http://localhost:9200\"]\n",
    "    cloud_id=ELASTIC_CLOUD_ID,\n",
    "    api_key=ELASTIC_API_KEY,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Enable Telemetry\n",
    "\n",
    "Knowing that you are using this notebook helps us decide where to invest our efforts to improve our products. We would like to ask you that you run the following code to let us gather anonymous usage statistics. See [telemetry.py](https://github.com/elastic/elasticsearch-labs/blob/main/telemetry/telemetry.py) for details. Thank you!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl -O -s https://raw.githubusercontent.com/elastic/elasticsearch-labs/main/telemetry/telemetry.py\n",
    "from telemetry import enable_telemetry\n",
    "\n",
    "es_client = enable_telemetry(es_client, \"08-learning-to-rank\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test the Client\n",
    "Before you continue, confirm that the client has connected with this test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client_info = es_client.info()\n",
    "\n",
    "f\"Successfully connected to cluster {client_info['cluster_name']} (version {client_info['version']['number']})\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KLAN6aq_mOpJ"
   },
   "source": [
    "## Configure the dataset\n",
    "\n",
    "We'll use a dataset derived from the [MSRD (Movie Search Ranking Dataset)](https://github.com/metarank/msrd/tree/master).\n",
    "\n",
    "The dataset is available [here](https://github.com/elastic/elasticsearch-labs/tree/main/notebooks/search/sample_data/learning-to-rank/) and contains the following files:\n",
    "\n",
    "- `movies_corpus.jsonl.gz`: Movie dataset to be indexed.\n",
    "- `movies_judgements.tsv.gz`: Judgment list of relevance judgments for a set of queries.\n",
    "- `movies_index_settings.json`: Settings to be applied to the documents and index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "gFm7i-b7mOpJ"
   },
   "outputs": [],
   "source": [
    "from urllib.parse import urljoin\n",
    "\n",
    "DATASET_BASE_URL = \"https://raw.githubusercontent.com/elastic/elasticsearch-labs/main/notebooks/search/sample_data/learning-to-rank/\"\n",
    "\n",
    "CORPUS_URL = urljoin(DATASET_BASE_URL, \"movies-corpus.jsonl.gz\")\n",
    "JUDGEMENTS_FILE_URL = urljoin(DATASET_BASE_URL, \"movies-judgments.tsv.gz\")\n",
    "INDEX_SETTINGS_URL = urljoin(DATASET_BASE_URL, \"movies-index-settings.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fhO5awX9mOpJ"
   },
   "source": [
    " ## Import the document corpus\n",
    "\n",
    "This step will import the documents of the corpus into the `movies` index .\n",
    "\n",
    "Documents contains the following fields:\n",
    "\n",
    "| Field name   | Description                                 |\n",
    "|--------------|---------------------------------------------|\n",
    "| `id`         | Id of the document                          |\n",
    "| `title`      | Movie title                                 |\n",
    "| `overview`   | A short description of the movie            |\n",
    "| `actors`     | List of actors in the movies                |\n",
    "| `director`   | Director of the movie                       |\n",
    "| `characters` | List of characters that appear in the movie |\n",
    "| `genres`     | Genres of the movie                         |\n",
    "| `year`       | Year the movie was released                 |\n",
    "| `budget`     | Budget of the movies in USD                 |\n",
    "| `votes`      | Number of votes received by the movie       |\n",
    "| `rating`     | Average rating of the movie                 |\n",
    "| `popularity` | Number use to measure the movie popularity  |\n",
    "| `tags`       | A list of tags for the movies               |\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "v5vhClAHmOpK",
    "outputId": "77ee3248-86ad-4cbf-9b3e-9dfdc9cf93f4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleting index if it already exists: movies\n",
      "Creating index: movies\n",
      "Loading the corpus from https://raw.githubusercontent.com/elastic/elasticsearch-labs/ltr-notebook/notebooks/search/sample_data/learning-to-rank/movies-corpus.jsonl.gz\n",
      "Indexing the corpus into movies ...\n",
      "Indexed 9750 documents into movies\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "import elasticsearch.helpers as es_helpers\n",
    "import pandas as pd\n",
    "from urllib.request import urlopen\n",
    "\n",
    "MOVIE_INDEX = \"movies\"\n",
    "\n",
    "# Delete index\n",
    "print(\"Deleting index if it already exists:\", MOVIE_INDEX)\n",
    "es_client.options(ignore_status=[400, 404]).indices.delete(index=MOVIE_INDEX)\n",
    "\n",
    "print(\"Creating index:\", MOVIE_INDEX)\n",
    "index_settings = json.load(urlopen(INDEX_SETTINGS_URL))\n",
    "es_client.indices.create(index=MOVIE_INDEX, **index_settings)\n",
    "\n",
    "print(f\"Loading the corpus from {CORPUS_URL}\")\n",
    "corpus_df = pd.read_json(CORPUS_URL, lines=True)\n",
    "\n",
    "print(f\"Indexing the corpus into {MOVIE_INDEX} ...\")\n",
    "bulk_result = es_helpers.bulk(\n",
    "    es_client,\n",
    "    actions=[\n",
    "        {\"_id\": movie[\"id\"], \"_index\": MOVIE_INDEX, **movie}\n",
    "        for movie in corpus_df.to_dict(\"records\")\n",
    "    ],\n",
    ")\n",
    "print(f\"Indexed {bulk_result[0]} documents into {MOVIE_INDEX}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the judgment list\n",
    "\n",
    "The judgment list contains human evaluations that we'll use to train our Learning To Rank model.\n",
    "\n",
    "Each row represents a query-document pair with an associated relevance grade and contains the following columns:\n",
    "\n",
    "| Column    | Description                                                            |\n",
    "|-----------|------------------------------------------------------------------------|\n",
    "| `query_id`| Pairs for the same query are grouped together and received a unique id. |\n",
    "| `query`   | Actual text of the query.                                             |\n",
    "| `doc_id`  | ID of the document.                                                    |\n",
    "| `grade`   | The relevance grade of the document for the query.                     |\n",
    "\n",
    "\n",
    "**Note:**\n",
    "\n",
    "In this example the relevance grade is a binary value (relevant or not relavant).\n",
    "You could also use a number that represents the degree of relevance (e.g. from `0` to `4`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "id": "XLjiKfYQqM-U",
    "outputId": "38df2283-421f-43ea-8bdf-580f1a63ac0d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>query_id</th>\n",
       "      <th>query</th>\n",
       "      <th>doc_id</th>\n",
       "      <th>grade</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>846433</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>49018</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>38234</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>567604</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>269795</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384750</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>263115</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384751</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>25913</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384752</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>567604</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384753</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>533535</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384754</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>876327</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>384755 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        query_id                query  doc_id  grade\n",
       "0       qid:5141  insidious 2 netflix  846433      0\n",
       "1       qid:5141  insidious 2 netflix   49018      1\n",
       "2       qid:5141  insidious 2 netflix   38234      0\n",
       "3       qid:5141  insidious 2 netflix  567604      0\n",
       "4       qid:5141  insidious 2 netflix  269795      0\n",
       "...          ...                  ...     ...    ...\n",
       "384750  qid:3383   2013 the wolverine  263115      0\n",
       "384751  qid:3383   2013 the wolverine   25913      0\n",
       "384752  qid:3383   2013 the wolverine  567604      0\n",
       "384753  qid:3383   2013 the wolverine  533535      0\n",
       "384754  qid:3383   2013 the wolverine  876327      0\n",
       "\n",
       "[384755 rows x 4 columns]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "judgments_df = pd.read_csv(JUDGEMENTS_FILE_URL, delimiter=\"\\t\")\n",
    "judgments_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configure feature extraction\n",
    "\n",
    "Features are the inputs to our model. They represent information about the query alone, a result document alone, or a result document in the context of a query, such as BM25 scores.\n",
    "\n",
    "Features are defined using standard templated queries and the Query DSL.\n",
    "\n",
    "To streamline the process of defining and refining feature extraction during training, we have incorporated a number of primitives directly in `eland`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "id": "LjxAj4lQqEYJ"
   },
   "outputs": [],
   "source": [
    "from eland.ml.ltr import LTRModelConfig, QueryFeatureExtractor\n",
    "\n",
    "ltr_config = LTRModelConfig(\n",
    "    feature_extractors=[\n",
    "        # For the following field we want to use the score of the match query for the field as a features:\n",
    "        QueryFeatureExtractor(\n",
    "            feature_name=\"title_bm25\", query={\"match\": {\"title\": \"{{query}}\"}}\n",
    "        ),\n",
    "        QueryFeatureExtractor(\n",
    "            feature_name=\"actors_bm25\", query={\"match\": {\"actors\": \"{{query}}\"}}\n",
    "        ),\n",
    "        # We could also use a more strict matching clause as an additional features. Here we want all the terms of our query to match.\n",
    "        QueryFeatureExtractor(\n",
    "            feature_name=\"title_all_terms_bm25\",\n",
    "            query={\n",
    "                \"match\": {\n",
    "                    \"title\": {\"query\": \"{{query}}\", \"minimum_should_match\": \"100%\"}\n",
    "                }\n",
    "            },\n",
    "        ),\n",
    "        QueryFeatureExtractor(\n",
    "            feature_name=\"actors_all_terms_bm25\",\n",
    "            query={\n",
    "                \"match\": {\n",
    "                    \"actors\": {\"query\": \"{{query}}\", \"minimum_should_match\": \"100%\"}\n",
    "                }\n",
    "            },\n",
    "        ),\n",
    "        # Also we can use a script_score query to get the document field values directly as a feature.\n",
    "        QueryFeatureExtractor(\n",
    "            feature_name=\"popularity\",\n",
    "            query={\n",
    "                \"script_score\": {\n",
    "                    \"query\": {\"exists\": {\"field\": \"popularity\"}},\n",
    "                    \"script\": {\"source\": \"return doc['popularity'].value;\"},\n",
    "                }\n",
    "            },\n",
    "        ),\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the training dataset\n",
    "\n",
    "Now that we have our basic datasets loaded, and feature extraction configured, we'll use our judgment list to come up with the final dataset for training. The dataset will consist of rows containing `<query, document>` pairs, as well as all of the features we need to train the model. To generate this dataset, we'll run each query from the judgment list and add the extracted features as columns for each of the labelled result documents.\n",
    "\n",
    "For example, if we have a query `q1` with two labelled documents `d3` and `d9`, the training dataset will end up with two rows — one for each of the pairs `<q1, d3>` and `<q1, d9>`.\n",
    "\n",
    "Note that because this executes queries on your Elasticsearch cluster, the time to run this operation will vary depending on where the cluster is hosted and where this notebook runs. For example, if you run the notebook on the same server or host as the Elasticsearch cluster, this operation tends to run very quickly on the sample dataset (< 2 mins)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 615,
     "referenced_widgets": [
      "f9cdfbc3972a4b84a557507567ca2965",
      "a6d4eb3325444f28b11ba02c3d01ed83",
      "bff504814b434aec90b2cf020b08cfa9",
      "594c5ebcb9624b63b128536a46594211",
      "e95447129da74d9ebc4c1d99165bd534",
      "7ed336be71e74521a596a7d624c1e7d1",
      "ee1a6943af1e49e6a8851f27c9811c32",
      "8c393bd522f6427f9978a89bc7dbdf3b",
      "549645ca4e7b48ef86cffdaa5507c56c",
      "0ab8ee0c2e1a42658ecbf01b0b28cf92",
      "84206a53779249fbb34c78edb17fd1e0"
     ]
    },
    "id": "xbp6_9dqqJkJ",
    "outputId": "0aadfe34-d739-4823-e5e2-310bd5fb69d3"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/16279 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 16279/16279 [01:38<00:00, 165.18it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>query_id</th>\n",
       "      <th>query</th>\n",
       "      <th>doc_id</th>\n",
       "      <th>grade</th>\n",
       "      <th>title_bm25</th>\n",
       "      <th>actors_bm25</th>\n",
       "      <th>title_all_terms_bm25</th>\n",
       "      <th>actors_all_terms_bm25</th>\n",
       "      <th>popularity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>846433</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9.555246</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13.628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>49018</td>\n",
       "      <td>1</td>\n",
       "      <td>9.857118</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>64.003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>38234</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>143.211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>567604</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>32.913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>qid:5141</td>\n",
       "      <td>insidious 2 netflix</td>\n",
       "      <td>269795</td>\n",
       "      <td>0</td>\n",
       "      <td>3.813253</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>21.058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384750</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>263115</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>68.287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384751</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>25913</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>21.026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384752</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>567604</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>32.913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384753</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>533535</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>34.773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>384754</th>\n",
       "      <td>qid:3383</td>\n",
       "      <td>2013 the wolverine</td>\n",
       "      <td>876327</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.920</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>384755 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        query_id                query  doc_id  grade  title_bm25  actors_bm25  \\\n",
       "0       qid:5141  insidious 2 netflix  846433      0         NaN     9.555246   \n",
       "1       qid:5141  insidious 2 netflix   49018      1    9.857118          NaN   \n",
       "2       qid:5141  insidious 2 netflix   38234      0         NaN          NaN   \n",
       "3       qid:5141  insidious 2 netflix  567604      0         NaN          NaN   \n",
       "4       qid:5141  insidious 2 netflix  269795      0    3.813253          NaN   \n",
       "...          ...                  ...     ...    ...         ...          ...   \n",
       "384750  qid:3383   2013 the wolverine  263115      0         NaN          NaN   \n",
       "384751  qid:3383   2013 the wolverine   25913      0         NaN          NaN   \n",
       "384752  qid:3383   2013 the wolverine  567604      0         NaN          NaN   \n",
       "384753  qid:3383   2013 the wolverine  533535      0         NaN          NaN   \n",
       "384754  qid:3383   2013 the wolverine  876327      0         NaN          NaN   \n",
       "\n",
       "        title_all_terms_bm25  actors_all_terms_bm25  popularity  \n",
       "0                        NaN                    NaN      13.628  \n",
       "1                        NaN                    NaN      64.003  \n",
       "2                        NaN                    NaN     143.211  \n",
       "3                        NaN                    NaN      32.913  \n",
       "4                        NaN                    NaN      21.058  \n",
       "...                      ...                    ...         ...  \n",
       "384750                   NaN                    NaN      68.287  \n",
       "384751                   NaN                    NaN      21.026  \n",
       "384752                   NaN                    NaN      32.913  \n",
       "384753                   NaN                    NaN      34.773  \n",
       "384754                   NaN                    NaN      25.920  \n",
       "\n",
       "[384755 rows x 9 columns]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy\n",
    "\n",
    "from eland.ml.ltr import FeatureLogger\n",
    "\n",
    "# First we create a feature logger that will be used to query Elasticsearch to retrieve the features:\n",
    "feature_logger = FeatureLogger(es_client, MOVIE_INDEX, ltr_config)\n",
    "\n",
    "\n",
    "# This method will be applied for each query group in the judgment log:\n",
    "def _extract_query_features(query_judgements_group):\n",
    "    # Retrieve document ids in the query group as strings.\n",
    "    doc_ids = query_judgements_group[\"doc_id\"].astype(\"str\").to_list()\n",
    "\n",
    "    # Resolve query params for the current query group (e.g.: {\"query\": \"batman\"}).\n",
    "    query_params = {\"query\": query_judgements_group[\"query\"].iloc[0]}\n",
    "\n",
    "    # Extract the features for the documents in the query group:\n",
    "    doc_features = feature_logger.extract_features(query_params, doc_ids)\n",
    "\n",
    "    # Adding a column to the dataframe for each feature:\n",
    "    for feature_index, feature_name in enumerate(ltr_config.feature_names):\n",
    "        query_judgements_group[feature_name] = numpy.array(\n",
    "            [doc_features[doc_id][feature_index] for doc_id in doc_ids]\n",
    "        )\n",
    "\n",
    "    return query_judgements_group\n",
    "\n",
    "\n",
    "judgments_with_features = judgments_df.groupby(\n",
    "    \"query_id\", group_keys=False\n",
    ").progress_apply(_extract_query_features)\n",
    "\n",
    "judgments_with_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create and train the model\n",
    "\n",
    "The LTR rescorer supports XGBRanker trained models.\n",
    "\n",
    "Learn more in the [XGBoost documentation](https://xgboost.readthedocs.io/en/latest/tutorials/learning_to_rank.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0]\tvalidation_0-ndcg@10:0.85757\n",
      "[1]\tvalidation_0-ndcg@10:0.86397\n",
      "[2]\tvalidation_0-ndcg@10:0.86582\n",
      "[3]\tvalidation_0-ndcg@10:0.86694\n",
      "[4]\tvalidation_0-ndcg@10:0.86738\n",
      "[5]\tvalidation_0-ndcg@10:0.86704\n",
      "[6]\tvalidation_0-ndcg@10:0.86777\n",
      "[7]\tvalidation_0-ndcg@10:0.86823\n",
      "[8]\tvalidation_0-ndcg@10:0.86925\n",
      "[9]\tvalidation_0-ndcg@10:0.86903\n",
      "[10]\tvalidation_0-ndcg@10:0.86973\n",
      "[11]\tvalidation_0-ndcg@10:0.87008\n",
      "[12]\tvalidation_0-ndcg@10:0.86990\n",
      "[13]\tvalidation_0-ndcg@10:0.87030\n",
      "[14]\tvalidation_0-ndcg@10:0.87067\n",
      "[15]\tvalidation_0-ndcg@10:0.87027\n",
      "[16]\tvalidation_0-ndcg@10:0.87144\n",
      "[17]\tvalidation_0-ndcg@10:0.87159\n",
      "[18]\tvalidation_0-ndcg@10:0.87195\n",
      "[19]\tvalidation_0-ndcg@10:0.87159\n",
      "[20]\tvalidation_0-ndcg@10:0.87171\n",
      "[21]\tvalidation_0-ndcg@10:0.87234\n",
      "[22]\tvalidation_0-ndcg@10:0.87243\n",
      "[23]\tvalidation_0-ndcg@10:0.87256\n",
      "[24]\tvalidation_0-ndcg@10:0.87294\n",
      "[25]\tvalidation_0-ndcg@10:0.87327\n",
      "[26]\tvalidation_0-ndcg@10:0.87371\n",
      "[27]\tvalidation_0-ndcg@10:0.87406\n",
      "[28]\tvalidation_0-ndcg@10:0.87410\n",
      "[29]\tvalidation_0-ndcg@10:0.87426\n",
      "[30]\tvalidation_0-ndcg@10:0.87455\n",
      "[31]\tvalidation_0-ndcg@10:0.87485\n",
      "[32]\tvalidation_0-ndcg@10:0.87482\n",
      "[33]\tvalidation_0-ndcg@10:0.87499\n",
      "[34]\tvalidation_0-ndcg@10:0.87505\n",
      "[35]\tvalidation_0-ndcg@10:0.87557\n",
      "[36]\tvalidation_0-ndcg@10:0.87594\n",
      "[37]\tvalidation_0-ndcg@10:0.87592\n",
      "[38]\tvalidation_0-ndcg@10:0.87618\n",
      "[39]\tvalidation_0-ndcg@10:0.87623\n",
      "[40]\tvalidation_0-ndcg@10:0.87648\n",
      "[41]\tvalidation_0-ndcg@10:0.87632\n",
      "[42]\tvalidation_0-ndcg@10:0.87657\n",
      "[43]\tvalidation_0-ndcg@10:0.87670\n",
      "[44]\tvalidation_0-ndcg@10:0.87724\n",
      "[45]\tvalidation_0-ndcg@10:0.87766\n",
      "[46]\tvalidation_0-ndcg@10:0.87765\n",
      "[47]\tvalidation_0-ndcg@10:0.87744\n",
      "[48]\tvalidation_0-ndcg@10:0.87800\n",
      "[49]\tvalidation_0-ndcg@10:0.87824\n",
      "[50]\tvalidation_0-ndcg@10:0.87822\n",
      "[51]\tvalidation_0-ndcg@10:0.87838\n",
      "[52]\tvalidation_0-ndcg@10:0.87867\n",
      "[53]\tvalidation_0-ndcg@10:0.87869\n",
      "[54]\tvalidation_0-ndcg@10:0.87873\n",
      "[55]\tvalidation_0-ndcg@10:0.87878\n",
      "[56]\tvalidation_0-ndcg@10:0.87899\n",
      "[57]\tvalidation_0-ndcg@10:0.87907\n",
      "[58]\tvalidation_0-ndcg@10:0.87891\n",
      "[59]\tvalidation_0-ndcg@10:0.87909\n",
      "[60]\tvalidation_0-ndcg@10:0.87914\n",
      "[61]\tvalidation_0-ndcg@10:0.87934\n",
      "[62]\tvalidation_0-ndcg@10:0.87920\n",
      "[63]\tvalidation_0-ndcg@10:0.87930\n",
      "[64]\tvalidation_0-ndcg@10:0.87915\n",
      "[65]\tvalidation_0-ndcg@10:0.87913\n",
      "[66]\tvalidation_0-ndcg@10:0.87956\n",
      "[67]\tvalidation_0-ndcg@10:0.87952\n",
      "[68]\tvalidation_0-ndcg@10:0.88009\n",
      "[69]\tvalidation_0-ndcg@10:0.88007\n",
      "[70]\tvalidation_0-ndcg@10:0.87995\n",
      "[71]\tvalidation_0-ndcg@10:0.87988\n",
      "[72]\tvalidation_0-ndcg@10:0.88003\n",
      "[73]\tvalidation_0-ndcg@10:0.88031\n",
      "[74]\tvalidation_0-ndcg@10:0.88023\n",
      "[75]\tvalidation_0-ndcg@10:0.88025\n",
      "[76]\tvalidation_0-ndcg@10:0.88039\n",
      "[77]\tvalidation_0-ndcg@10:0.88038\n",
      "[78]\tvalidation_0-ndcg@10:0.88064\n",
      "[79]\tvalidation_0-ndcg@10:0.88053\n",
      "[80]\tvalidation_0-ndcg@10:0.88062\n",
      "[81]\tvalidation_0-ndcg@10:0.88067\n",
      "[82]\tvalidation_0-ndcg@10:0.88077\n",
      "[83]\tvalidation_0-ndcg@10:0.88131\n",
      "[84]\tvalidation_0-ndcg@10:0.88132\n",
      "[85]\tvalidation_0-ndcg@10:0.88128\n",
      "[86]\tvalidation_0-ndcg@10:0.88164\n",
      "[87]\tvalidation_0-ndcg@10:0.88171\n",
      "[88]\tvalidation_0-ndcg@10:0.88180\n",
      "[89]\tvalidation_0-ndcg@10:0.88206\n",
      "[90]\tvalidation_0-ndcg@10:0.88209\n",
      "[91]\tvalidation_0-ndcg@10:0.88195\n",
      "[92]\tvalidation_0-ndcg@10:0.88197\n",
      "[93]\tvalidation_0-ndcg@10:0.88209\n",
      "[94]\tvalidation_0-ndcg@10:0.88189\n",
      "[95]\tvalidation_0-ndcg@10:0.88240\n",
      "[96]\tvalidation_0-ndcg@10:0.88259\n",
      "[97]\tvalidation_0-ndcg@10:0.88265\n",
      "[98]\tvalidation_0-ndcg@10:0.88268\n",
      "[99]\tvalidation_0-ndcg@10:0.88272\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {color: black;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>XGBRanker(base_score=None, booster=None, callbacks=None, colsample_bylevel=None,\n",
       "          colsample_bynode=None, colsample_bytree=None, device=None,\n",
       "          early_stopping_rounds=20, enable_categorical=False,\n",
       "          eval_metric=[&#x27;ndcg@10&#x27;], feature_types=None, gamma=None,\n",
       "          grow_policy=None, importance_type=None, interaction_constraints=None,\n",
       "          learning_rate=None, max_bin=None, max_cat_threshold=None,\n",
       "          max_cat_to_onehot=None, max_delta_step=None, max_depth=None,\n",
       "          max_leaves=None, min_child_weight=None, missing=nan,\n",
       "          monotone_constraints=None, multi_strategy=None, n_estimators=None,\n",
       "          n_jobs=None, num_parallel_tree=None, random_state=None, ...)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" checked><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">XGBRanker</label><div class=\"sk-toggleable__content\"><pre>XGBRanker(base_score=None, booster=None, callbacks=None, colsample_bylevel=None,\n",
       "          colsample_bynode=None, colsample_bytree=None, device=None,\n",
       "          early_stopping_rounds=20, enable_categorical=False,\n",
       "          eval_metric=[&#x27;ndcg@10&#x27;], feature_types=None, gamma=None,\n",
       "          grow_policy=None, importance_type=None, interaction_constraints=None,\n",
       "          learning_rate=None, max_bin=None, max_cat_threshold=None,\n",
       "          max_cat_to_onehot=None, max_delta_step=None, max_depth=None,\n",
       "          max_leaves=None, min_child_weight=None, missing=nan,\n",
       "          monotone_constraints=None, multi_strategy=None, n_estimators=None,\n",
       "          n_jobs=None, num_parallel_tree=None, random_state=None, ...)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "XGBRanker(base_score=None, booster=None, callbacks=None, colsample_bylevel=None,\n",
       "          colsample_bynode=None, colsample_bytree=None, device=None,\n",
       "          early_stopping_rounds=20, enable_categorical=False,\n",
       "          eval_metric=['ndcg@10'], feature_types=None, gamma=None,\n",
       "          grow_policy=None, importance_type=None, interaction_constraints=None,\n",
       "          learning_rate=None, max_bin=None, max_cat_threshold=None,\n",
       "          max_cat_to_onehot=None, max_delta_step=None, max_depth=None,\n",
       "          max_leaves=None, min_child_weight=None, missing=nan,\n",
       "          monotone_constraints=None, multi_strategy=None, n_estimators=None,\n",
       "          n_jobs=None, num_parallel_tree=None, random_state=None, ...)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from xgboost import XGBRanker\n",
    "from sklearn.model_selection import GroupShuffleSplit\n",
    "\n",
    "\n",
    "# Create the ranker model:\n",
    "ranker = XGBRanker(\n",
    "    objective=\"rank:ndcg\",\n",
    "    eval_metric=[\"ndcg@10\"],\n",
    "    early_stopping_rounds=20,\n",
    ")\n",
    "\n",
    "# Shaping training and eval data in the expected format.\n",
    "X = judgments_with_features[ltr_config.feature_names]\n",
    "y = judgments_with_features[\"grade\"]\n",
    "groups = judgments_with_features[\"query_id\"]\n",
    "\n",
    "# Split the dataset in two parts respectively used for training and evaluation of the model.\n",
    "group_preserving_splitter = GroupShuffleSplit(n_splits=1, train_size=0.7).split(\n",
    "    X, y, groups\n",
    ")\n",
    "train_idx, eval_idx = next(group_preserving_splitter)\n",
    "\n",
    "train_features, eval_features = X.loc[train_idx], X.loc[eval_idx]\n",
    "train_target, eval_target = y.loc[train_idx], y.loc[eval_idx]\n",
    "train_query_groups, eval_query_groups = groups.loc[train_idx], groups.loc[eval_idx]\n",
    "\n",
    "# Training the model\n",
    "ranker.fit(\n",
    "    X=train_features,\n",
    "    y=train_target,\n",
    "    group=train_query_groups.value_counts().sort_index().values,\n",
    "    eval_set=[(eval_features, eval_target)],\n",
    "    eval_group=[eval_query_groups.value_counts().sort_index().values],\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 490
    },
    "id": "3iSx3IuLqq7R",
    "outputId": "d81ac47f-99c6-4656-9fc1-b9699a80c458"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAHHCAYAAABX6yWOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABjoklEQVR4nO3dd1QUV8MG8GfpvUpVBFRsgIiiBuxCaLGbYEsCxvJaiLFHjVI0CkGNGjUaNZEYa4pRExHFgsaGFYyNVxTExquigIDSdr4/PMznCiggTeb5nbNH9s6dmXv3IjzcvTMrEwRBABERERGRhCnVdgOIiIiIiGobQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxERG98yIjIyGTyZCSklLbTSGidxRDMRHRO6g4BJb2mDlzZrWc88SJEwgJCUFGRka1HF/KcnNzERISgtjY2NpuCpFkqdR2A4iIqPLmzZsHW1tbhTIHB4dqOdeJEycQGhqKgIAAGBgYVMs5KuuTTz7BkCFDoK6uXttNqZTc3FyEhoYCAHr06FG7jSGSKIZiIqJ3mI+PD1xcXGq7GW8lJycH2trab3UMZWVlKCsrV1GLao5cLkd+fn5tN4OIwOUTRET12t69e9G1a1doa2tDV1cXH3zwAS5fvqxQ5+LFiwgICECTJk2goaEBc3NzfPbZZ0hPTxfrhISEYPr06QAAW1tbcalGSkoKUlJSIJPJEBkZWeL8MpkMISEhCseRyWS4cuUKhg0bBkNDQ3Tp0kXcvmnTJrRv3x6ampowMjLCkCFDcPv27Tf2s7Q1xTY2NujduzdiY2Ph4uICTU1NODo6iksUduzYAUdHR2hoaKB9+/a4cOGCwjEDAgKgo6ODmzdvwsvLC9ra2rC0tMS8efMgCIJC3ZycHEydOhVWVlZQV1dHixYtsHjx4hL1ZDIZAgMDsXnzZtjb20NdXR1r1qyBiYkJACA0NFR8bYtft/KMz8uvbVJSkjibr6+vjxEjRiA3N7fEa7Zp0yZ07NgRWlpaMDQ0RLdu3bB//36FOuX5/iGqLzhTTET0DsvMzMSjR48Uyho0aAAA+OWXX+Dv7w8vLy988803yM3NxerVq9GlSxdcuHABNjY2AICYmBjcvHkTI0aMgLm5OS5fvoy1a9fi8uXLOHXqFGQyGQYOHIj//ve/2Lp1K5YuXSqew8TEBA8fPqxwuz/66CPY2dlh4cKFYnBcsGAB5s6dCz8/P4waNQoPHz7EihUr0K1bN1y4cKFSSzaSkpIwbNgw/Oc//8HHH3+MxYsXo0+fPlizZg1mz56N8ePHAwDCwsLg5+eHxMREKCn9/3xRUVERvL298d577yEiIgLR0dEIDg5GYWEh5s2bBwAQBAF9+/bF4cOHMXLkSLRt2xb79u3D9OnTcffuXSxdulShTYcOHcKvv/6KwMBANGjQAE5OTli9ejXGjRuHAQMGYODAgQCANm3aACjf+LzMz88Ptra2CAsLw/nz57F+/XqYmprim2++EeuEhoYiJCQEbm5umDdvHtTU1BAXF4dDhw7B09MTQPm/f4jqDYGIiN45GzZsEACU+hAEQXj69KlgYGAgjB49WmG/tLQ0QV9fX6E8Nze3xPG3bt0qABCOHj0qli1atEgAICQnJyvUTU5OFgAIGzZsKHEcAEJwcLD4PDg4WAAgDB06VKFeSkqKoKysLCxYsECh/N9//xVUVFRKlJf1erzcNmtrawGAcOLECbFs3759AgBBU1NTuHXrllj+ww8/CACEw4cPi2X+/v4CAOHzzz8Xy+RyufDBBx8IampqwsOHDwVBEISdO3cKAISvv/5aoU0ffvihIJPJhKSkJIXXQ0lJSbh8+bJC3YcPH5Z4rYqVd3yKX9vPPvtMoe6AAQMEY2Nj8fn169cFJSUlYcCAAUJRUZFCXblcLghCxb5/iOoLLp8gInqHrVq1CjExMQoP4MXsYkZGBoYOHYpHjx6JD2VlZXTq1AmHDx8Wj6GpqSl+/fz5czx69AjvvfceAOD8+fPV0u6xY8cqPN+xYwfkcjn8/PwU2mtubg47OzuF9lZE69at4erqKj7v1KkTAKBXr15o3LhxifKbN2+WOEZgYKD4dfHyh/z8fBw4cAAAEBUVBWVlZUycOFFhv6lTp0IQBOzdu1ehvHv37mjdunW5+1DR8Xn1te3atSvS09ORlZUFANi5cyfkcjmCgoIUZsWL+wdU7PuHqL7g8gkiondYx44dS73Q7vr16wBehL/S6OnpiV8/fvwYoaGh2LZtGx48eKBQLzMzswpb+/9evWPG9evXIQgC7OzsSq2vqqpaqfO8HHwBQF9fHwBgZWVVavmTJ08UypWUlNCkSROFsubNmwOAuH751q1bsLS0hK6urkK9Vq1aidtf9mrf36Si4/Nqnw0NDQG86Juenh5u3LgBJSWl1wbzinz/ENUXDMVERPWQXC4H8GJdqLm5eYntKir//+Pfz88PJ06cwPTp09G2bVvo6OhALpfD29tbPM7rvLqmtVhRUVGZ+7w8+1ncXplMhr1795Z6FwkdHZ03tqM0Zd2Roqxy4ZUL46rDq31/k4qOT1X0rSLfP0T1Bb+riYjqoaZNmwIATE1N4eHhUWa9J0+e4ODBgwgNDUVQUJBYXjxT+LKywm/xTOSrH+rx6gzpm9orCAJsbW3Fmdi6QC6X4+bNmwpt+u9//wsA4oVm1tbWOHDgAJ4+faowW3zt2jVx+5uU9dpWZHzKq2nTppDL5bhy5Qratm1bZh3gzd8/RPUJ1xQTEdVDXl5e0NPTw8KFC1FQUFBie/EdI4pnFV+dRVy2bFmJfYrvJfxq+NXT00ODBg1w9OhRhfLvv/++3O0dOHAglJWVERoaWqItgiCUuP1YTVq5cqVCW1auXAlVVVW4u7sDAHx9fVFUVKRQDwCWLl0KmUwGHx+fN55DS0sLQMnXtiLjU179+/eHkpIS5s2bV2Kmufg85f3+IapPOFNMRFQP6enpYfXq1fjkk0/Qrl07DBkyBCYmJkhNTcWePXvQuXNnrFy5Enp6eujWrRsiIiJQUFCAhg0bYv/+/UhOTi5xzPbt2wMAvvrqKwwZMgSqqqro06cPtLW1MWrUKISHh2PUqFFwcXHB0aNHxRnV8mjatCm+/vprzJo1CykpKejfvz90dXWRnJyMP//8E2PGjMG0adOq7PUpLw0NDURHR8Pf3x+dOnXC3r17sWfPHsyePVu8t3CfPn3Qs2dPfPXVV0hJSYGTkxP279+PXbt2YdKkSeKs6+toamqidevW2L59O5o3bw4jIyM4ODjAwcGh3ONTXs2aNcNXX32F+fPno2vXrhg4cCDU1dVx5swZWFpaIiwsrNzfP0T1Si3d9YKIiN5C8S3Izpw589p6hw8fFry8vAR9fX1BQ0NDaNq0qRAQECCcPXtWrHPnzh1hwIABgoGBgaCvry989NFHwr1790q9Rdj8+fOFhg0bCkpKSgq3QMvNzRVGjhwp6OvrC7q6uoKfn5/w4MGDMm/JVnw7s1f98ccfQpcuXQRtbW1BW1tbaNmypTBhwgQhMTGxXK/Hq7dk++CDD0rUBSBMmDBBoaz4tnKLFi0Sy/z9/QVtbW3hxo0bgqenp6ClpSWYmZkJwcHBJW5l9vTpU2Hy5MmCpaWloKqqKtjZ2QmLFi0Sb3H2unMXO3HihNC+fXtBTU1N4XUr7/iU9dqW9toIgiD89NNPgrOzs6Curi4YGhoK3bt3F2JiYhTqlOf7h6i+kAlCDVxVQERE9I4JCAjA77//juzs7NpuChHVAK4pJiIiIiLJYygmIiIiIsljKCYiIiIiyeOaYiIiIiKSPM4UExEREZHkMRQTERERkeTxwzuIyiCXy3Hv3j3o6uqW+RGsREREVLcIgoCnT5/C0tISSkrln/9lKCYqw71792BlZVXbzSAiIqJKuH37Nho1alTu+gzFRGXQ1dUFACQnJ8PIyKiWW0MvKygowP79++Hp6QlVVdXabg69guNTd3Fs6jaOT9XIysqClZWV+Hu8vBiKicpQvGRCV1cXenp6tdwaellBQQG0tLSgp6fHXxx1EMen7uLY1G0cn6pV0aWPvNCOiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJE8mCIJQ242g+q9Hjx5o27Ytli1b9lbHCQkJwc6dOxEfH18l7XqdrKws6Ovro+nU7ShU0a7281H5qSsLiOhYhBmnlZFXJKvt5tArOD51F8embqur45MS/gHCwsKwY8cOXLt2DZqamnBzc8M333yDFi1aiPXWrl2LLVu24Pz583j69CmePHkCAwMDcXtsbCx69uxZ6jlOnz6NDh06AAD27duH4OBgXL58GRoaGujWrRuWLFkCGxubMtv4+PFjfP755/jrr7+gpKSEzMxM3L17F5aWluXuJ2eK6Z0ybdo0HDx4UHweEBCA/v37116DiIiIJODIkSOYMGECTp06hZiYGBQUFMDT0xM5OTlindzcXHh7e2P27NmlHsPNzQ33799XeIwaNQq2trZwcXEBACQnJ6Nfv37o1asX4uPjsW/fPjx69AgDBw58bfuGDx+Oy5cvIyYmBtu3bwcAfPHFFxXqo0qFahPVEkEQUFRUBB0dHejo6NR2c4iIiCQlOjpa4XlkZCRMTU1x7tw5dOvWDQAwadIkAC9mhEujpqYGc3Nz8XlBQQF27dqFzz//HDLZi5nxc+fOoaioCF9//TWUlF7M3U6bNg39+vVDQUEBVFVVSxz36tWriI6OxpkzZ+Di4oKsrCwAwB9//IF79+6Ve7aYM8X1XI8ePRAYGIjAwEDo6+ujQYMGmDt3LopXzTx58gSffvopDA0NoaWlBR8fH1y/fl3cPzIyEgYGBti5cyfs7OygoaEBLy8v3L59W6xT2mztpEmT0KNHjzLb9csvv8DFxQW6urowNzfHsGHD8ODBA3F7bGwsZDIZ9u7di/bt20NdXR3Hjh1DSEgI2rZtC+DFUoqff/4Zu3btgkwmg0wmQ2xsLHr16oXAwECF8z18+BBqamoKs8xERERUOZmZmQAAIyOjSh9j9+7dSE9Px4gRI8Sy9u3bQ0lJCRs2bEBRUREyMzPxyy+/wMPDo9RADAAnT56EgYGBONtcTElJCXFxceVuD0OxBPz8889QUVHB6dOnsXz5cnz77bdYv349gBeB9uzZs9i9ezdOnjwJQRDg6+uLgoICcf/c3FwsWLAAGzduxPHjx5GRkYEhQ4a8VZsKCgowf/58JCQkYOfOnUhJSUFAQECJejNnzkR4eDiuXr2KNm3aKGybNm0a/Pz84O3tLb4N4+bmhlGjRmHLli3Iy8sT627atAkNGzZEr1693qrdREREUieXyzFp0iR07twZDg4OlT7Ojz/+CC8vLzRq1Egss7W1xf79+zF79myoq6vDwMAAd+7cwa+//lrmcdLS0mBqalqi3NDQEGlpaeVuD5dPSICVlRWWLl0KmUyGFi1a4N9//8XSpUvRo0cP7N69G8ePH4ebmxsAYPPmzbCyssLOnTvx0UcfAXgRYFeuXIlOnToBeBGyW7VqhdOnT6Njx46VatNnn30mft2kSRN899136NChA7KzsxWWR8ybNw/vv/9+qcfQ0dGBpqYm8vLyFN6OGThwIAIDA7Fr1y74+fkBeDHjHRAQIL49U5q8vDyFIF389ou6kgBlZV6PWpeoKwkK/1LdwvGpuzg2dVtdHZ+XJ8oAIDAwEJcuXcLhw4dLbAOAwsJCcb/StgPAnTt3sG/fPmzZskWhTlpaGkaNGoWPP/4YgwcPRnZ2NkJDQzFo0CDs3bu31N/jRUVFEARBPE5Z53wThmIJeO+99xS+iVxdXbFkyRJcuXIFKioqYtgFAGNjY7Ro0QJXr14Vy1RUVMQrQgGgZcuWMDAwwNWrVysdis+dO4eQkBAkJCTgyZMnkMvlAIDU1FS0bt1arPfqWyHloaGhgU8++QQ//fQT/Pz8cP78eVy6dAm7d+9+7X5hYWEIDQ0tUT7HWQ4traIKt4Oq33wXeW03gV6D41N3cWzqtro2PlFRUeLXa9euRVxcHBYuXIiLFy/i4sWLJer/+++/AID9+/eXeR3Q9u3boaurCxUVFYXjb968GQDQrVs33L9/HwDw6aefYtSoUVi2bJnC3S6KPXjwAPfu3ROPk5ubC+DFEtGXJ83ehKGY3pqSkhJevbPf6/5Ky8nJgZeXF7y8vLB582aYmJggNTUVXl5eyM/PV6irrV25W6GNGjUKbdu2xZ07d7Bhwwb06tUL1tbWr91n1qxZmDJlivg8KysLVlZW+PqCEgpVlSvVDqoe6koC5rvIMfesEvLkdee2RfQCx6fu4tjUbXV1fC6FeEEQBEyaNAnx8fE4evQo7Ozsyqxf/Lvb09NT4ZZsxQRBwOTJk/HZZ5+hb9++CttiY2ORkpICX19fsaw4HL/33ntwdXUtcTxbW1usXLkS5ubmaNeunfhOr1wuV5j4exOGYgl4dZH5qVOnYGdnh9atW6OwsBBxcXHi8on09HQkJiYqzNYWFhbi7Nmz4qxwYmIiMjIy0KpVKwCAiYkJLl26pHCO+Pj4MhfEX7t2Denp6QgPD4eVlRUA4OzZs5Xqm5qaGoqKSs7iOjo6wsXFBevWrcOWLVuwcuXKNx5LXV0d6urqJcrz5DIU1qH7RdL/y5PL6tS9PEkRx6fu4tjUbXVtfFRVVTF+/Hhs2bIFu3btgpGREdLT0wEA+vr60NTUBPBi6UNaWhpSUlIAvPh9r6uri8aNGytckHfw4EEkJydjzJgxJbJCnz59sHz5coSFhWHo0KF4+vQpZs+eDWtra3To0AGqqqo4ffo0Pv30Uxw8eBANGzZEmzZt4O3tjXHjxmHNmjXIyMgAAAwaNIj3KSZFqampmDJlChITE7F161asWLECX3zxBezs7NCvXz+MHj0ax44dQ0JCAj7++GM0bNgQ/fr1E/dXVVXF559/jri4OJw7dw4BAQF47733xJDcq1cvnD17Fhs3bsT169cRHBxcIiS/rHHjxlBTU8OKFStw8+ZN7N69G/Pnz69U32xsbHDx4kUkJibi0aNHCjPUo0aNQnh4OARBwIABAyp1fCIiIgJWr16NzMxM9OjRAxYWFuKj+J7AALBmzRo4Oztj9OjRAF4sgXB2di6xfPHHH3+Em5sbWrZsWeI8vXr1wpYtW7Bz5044OzvD29sb6urqiI6OFsN3bm4uEhMTFX7nb968GS1btoS7u7t4TdTy5csr1EeGYgn49NNP8ezZM3Ts2BETJkzAF198gTFjxgAANmzYgPbt26N3795wdXWFIAiIiopS+MtNS0sLX375JYYNG4bOnTtDR0dH4T+Bl5cX5s6dixkzZqBDhw54+vQpPv300zLbY2JigsjISPz2229o3bo1wsPDsXjx4kr1bfTo0WjRogVcXFxgYmKC48ePi9uGDh0KFRUVDB06FBoaGpU6PhEREb1Y8lDa4+U7R4WEhLyxDgBs2bJF4ff1q4YMGYLz588jOzsbDx48wK5duxQCdI8ePSAIgsIn3BkZGWHLli14+vSpeNvYin6uAT/muZ57249XjoyMxKRJk8S3It4lKSkpaNq0Kc6cOYN27dpVeH9+zHPdVVc/CpVe4PjUXRybuq2ujk9K+Ae13YQKKf79nZmZCT09vXLvxzXFVO8UFBQgPT0dc+bMwXvvvVepQPyyuFnuMDY2rqLWUVUoKChAVFQULoV4lbl2nWoPx6fu4tjUbRyf2sXlE1TvHD9+HBYWFjhz5gzWrFlT280hIiKidwBniuu5sj5/vLwCAgJK/aS5uqx4rRERERFReXGmmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkT6W2G0BU13UKO4hCFe3abga9RF1ZQERHwCFkH/KKZLXdHHoFx6fuqs2xSQn/oEbPR1RRnCmWgNjYWMhkMmRkZLy2no2NDZYtW1Yl5wwJCUHbtm2r5FhERFQ/HD16FH369IGlpSVkMhl27txZos7Vq1fRt29f6OvrQ1tbGx06dEBqamqJeoIgwMfHp8RxEhISMHToUFhZWUFTUxOtWrXC8uXL39i2x48fY/jw4dDT04OBgQFGjhyJ7Ozst+kuvWMYiuuhHj16YNKkSeJzNzc33L9/H/r6+gCAyMhIGBgY1E7j3lJKSgpGjhwJW1tbaGpqomnTpggODkZ+fr5CHZlMVuJx6tSpWmw5ERHl5OTAyckJq1atKnX7jRs30KVLF7Rs2RKxsbG4ePEi5s6dCw0NjRJ1ly1bBpms5Gz3uXPnYGpqik2bNuHy5cv46quvMGvWLKxcufK1bRs+fDguX76MmJgY/P333zh69CjGjBlTuY7SO4nLJyRATU0N5ubmtd2MKnHt2jXI5XL88MMPaNasGS5duoTRo0cjJycHixcvVqh74MAB2Nvbi8+NjY1rurlERPQSHx8f+Pj4lLn9q6++gq+vLyIiIsSypk2blqgXHx+PJUuW4OzZs7CwsFDY9tlnnyk8b9KkCU6ePIkdO3YgMDCw1PNevXoV0dHROHPmDFxcXAAAK1asgK+vLxYvXgxLS8ty95HeXZwprmcCAgJw5MgRLF++XJwhjYyMFJdPxMbGYsSIEcjMzBS3h4SElHqsjIwMjBo1CiYmJtDT00OvXr2QkJBQofb88MMPsLKygpaWFvz8/JCZmanQ1v79+2PhwoUwMzODgYEB5s2bh8LCQkyfPh1GRkZo1KgRNmzYIO7j7e2NDRs2wNPTE02aNEHfvn0xbdo07Nixo8S5jY2NYW5uLj5UVVUr1HYiIqo5crkce/bsQfPmzeHl5QVTU1N06tSpxBKL3NxcDBs2DKtWrSr3hE9mZiaMjIzK3H7y5EkYGBiIgRgAPDw8oKSkhLi4uEr1h949nCmuZ5YvX47//ve/cHBwwLx58wAAly9fFre7ublh2bJlCAoKQmJiIgBAR0en1GN99NFH0NTUxN69e6Gvr48ffvgB7u7u+O9///vaHy7FkpKS8Ouvv+Kvv/5CVlYWRo4cifHjx2Pz5s1inUOHDqFRo0Y4evQojh8/jpEjR+LEiRPo1q0b4uLisH37dvznP//B+++/j0aNGpV6nrJ+2PXt2xfPnz9H8+bNMWPGDPTt2/e17c3Ly0NeXp74PCsrCwCgriRAWVl4Y3+p5qgrCQr/Ut3C8am7anNsCgoKSpQVFhaK5WlpacjOzkZ4eDhCQ0Px9ddfY//+/Rg4cCBiYmLQrVs3AMAXX3yB9957D76+vuK+Lx/nVSdPnsT27duxa9euMuvcvXsXJiYmJbYbGRnh7t27Ze5X1YrPU1Pnq68q+/oxFNcz+vr6UFNTg5aWlvgX9LVr18Ttampq0NfXh0wme+1f2MeOHcPp06fx4MEDqKurAwAWL16MnTt34vfffy/XOqvnz59j48aNaNiwIYAXb0V98MEHWLJkiXhuIyMjfPfdd1BSUkKLFi0QERGB3NxczJ49GwAwa9YshIeH49ixYxgyZEiJcyQlJWHFihUKSyd0dHSwZMkSdO7cGUpKSvjjjz/Qv39/7Ny587XBOCwsDKGhoSXK5zjLoaVV9Mb+Us2b7yKv7SbQa3B86q7aGJuoqKgSZefOnRPfxXv8+DEAoH379rCzs8O9e/fg4OAAFxcXhIaGYurUqTh9+jT27NmDb7/9VuF4Lx/nZbdu3cLcuXPh5+eHgoKCUtsAAImJicjJySmxPT8/H5cuXSpzv+oSExNTo+erb3Jzcyu1H0MxlSohIQHZ2dkl1uE+e/YMN27cKNcxGjduLAZiAHB1dYVcLkdiYqIYiu3t7aGk9P+reMzMzODg4CA+V1ZWhrGxMR48eFDi+Hfv3oW3tzc++ugjjB49Wixv0KABpkyZIj7v0KED7t27h0WLFr02FM+aNUthv6ysLFhZWeHrC0ooVFUuV5+pZqgrCZjvIsfcs0rIk/OWX3UNx6fuqs2xuRTiVaKsffv28PX1BfAigI4ZMwbu7u5iGQD8888/OHHiBHx9fXHw4EGkpaXh448/VjhOREQEunTpggMHDohlV65cwZgxYzBu3DjMnz//tW178OAB9uzZo3DewsJCZGdnl2hPdSooKEBMTAzef/99Lvl7C8Xv9FYUQzGVKjs7GxYWFoiNjS2xrSrvXPHqf3qZTFZqmVyuOKtx79499OzZE25ubli7du0bz9OpU6c3/uWtrq4uzoq/LE8uQyHvtVon5cllvA9uHcbxqbtqY2xKC3kqKipiuaqqKjp06ICkpCSFujdu3ICNjQ1UVVUxe/bsEu9UOjo6YunSpejTp4+43+XLl+Hp6Ql/f3+Eh4e/sW1dunRBRkYGLl68iPbt2wMADh8+DLlcjs6dO9d4QFVVVWUofguVfe0YiushNTU1FBWV/Xb/m7YDQLt27ZCWlgYVFRXY2NhUqh2pqam4d++eeNXuqVOnxGUSb+Pu3bvo2bMn2rdvjw0bNijMNJclPj6+xBXKRERUs7Kzs5GUlCQ+T05ORnx8PIyMjNC4cWNMnz4dgwcPRrdu3dCzZ09ER0fjr7/+Eidoii+cflXjxo1ha2sLALh06RJ69eoFLy8vTJkyBWlpaQBevPNoYmICADh9+jQ+/fRTHDx4EA0bNkSrVq3g7e2N0aNHY82aNSgoKEBgYCCGDBnCO09ICENxPWRjY4O4uDikpKRAR0enxCyrjY0NsrOzcfDgQTg5OUFLSwtaWloKdTw8PODq6or+/fsjIiICzZs3x71797Bnzx4MGDBA4QrdsmhoaMDf3x+LFy9GVlYWJk6cCD8/v7e6Pdzdu3fRo0cPWFtbY/HixXj48KG4rfi4P//8M9TU1ODs7AwA2LFjB3766SesX7++0uclIqK3d/bsWfTs2VN8Xrxkzd/fH5GRkRgwYADWrFmDsLAwTJw4ES1atMAff/yBLl26lPscv//+Ox4+fIhNmzZh06ZNYrm1tTVSUlIAvFhzmpiYqHBB1ubNmxEYGAh3d3coKSlh0KBB+O67796yx/QuYSiuh6ZNmwZ/f3+0bt0az549U7ilGfDiDhRjx47F4MGDkZ6ejuDg4BK3ZZPJZIiKisJXX32FESNG4OHDhzA3N0e3bt1gZmZWrnY0a9YMAwcOhK+vLx4/fozevXvj+++/f6u+xcTEICkpCUlJSSXuRiEI/3819fz583Hr1i2oqKigZcuW2L59Oz788MO3OjcREb2dHj16KPysLs1nn31W4l7Dr/Pq8UJCQsq81ejr2mFkZIQtW7aU+7xU/8iEN313EklUVlYW9PX18ejRI37wRx1TfBW5r68v193VQRyfuotjU7dxfKpG8e/vzMxM6OnplXs/fngHEREREUkeQzFVir29PXR0dEp9vPzhHERERETvAq4ppkqJiooq8xNjyrvmmIiIiKiuYCimSrG2tq7tJhARERFVGS6fICIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyWMoJiIiIiLJYygmIiIiIsljKCYiIiIiyauyUJyRkVFVhyIiIiIiqlGVCsXffPMNtm/fLj738/ODsbExGjZsiISEhCprHBERERFRTahUKF6zZg2srKwAADExMYiJicHevXvh4+OD6dOnV2kDiYiIiIiqm0pldkpLSxND8d9//w0/Pz94enrCxsYGnTp1qtIGEhERERFVt0rNFBsaGuL27dsAgOjoaHh4eAAABEFAUVFR1bWOiIiIiKgGVGqmeODAgRg2bBjs7OyQnp4OHx8fAMCFCxfQrFmzKm0gEREREVF1q1QoXrp0KWxsbHD79m1ERERAR0cHAHD//n2MHz++ShtIRERERFTdKhWKVVVVMW3atBLlkydPfusGERERERHVtErfp/iXX35Bly5dYGlpiVu3bgEAli1bhl27dlVZ44iIiIiIakKlQvHq1asxZcoU+Pj4ICMjQ7y4zsDAAMuWLavK9hERERERVbtKheIVK1Zg3bp1+Oqrr6CsrCyWu7i44N9//62yxhERERER1YRKheLk5GQ4OzuXKFdXV0dOTs5bN4qIiIiIqCZVKhTb2toiPj6+RHl0dDRatWr1tm0iIiIiIqpRlbr7xJQpUzBhwgQ8f/4cgiDg9OnT2Lp1K8LCwrB+/fqqbiMRERERUbWqVCgeNWoUNDU1MWfOHOTm5mLYsGGwtLTE8uXLMWTIkKpuIxERERFRtapwKC4sLMSWLVvg5eWF4cOHIzc3F9nZ2TA1Na2O9hERERERVbsKrylWUVHB2LFj8fz5cwCAlpYWAzERERERvdMqdaFdx44dceHChapuCxERERFRrajUmuLx48dj6tSpuHPnDtq3bw9tbW2F7W3atKmSxhERERER1YRKheLii+kmTpwolslkMgiCAJlMJn7CHRERERHRu6BSoTg5Obmq20FEREREVGsqFYqtra2ruh1EdVansIMoVNF+c0WqMerKAiI6Ag4h+5BXJKvQvinhHyAkJAShoaEK5S1atMC1a9fE5ydPnsRXX32FuLg4KCsro23btti3bx80NTUBADY2Nrh165bCMcLCwjBz5swyz/38+XNMnToV27ZtQ15eHry8vPD999/DzMysQn0gIqKqV6lQvHHjxtdu//TTTyvVGKpfAgICkJGRgZ07d9Z2U4hKsLe3x4EDB8TnKir//+Pw5MmT8Pb2xqxZs7BixQqoqKggISEBSkqK1ybPmzcPo0ePFp/r6uq+9pyTJ0/Gnj178Ntvv0FfXx+BgYEYOHAgjh8/XkW9IiKiyqpUKP7iiy8UnhcUFCA3NxdqamrQ0tJiKK5lISEh2LlzZ6kfxV0fJCQkIDw8HMeOHcOjR49gY2ODsWPHKnxfxsbGomfPniX2vX//PszNzWuyuVRHqaiolPm9MHnyZEycOFFh1rdFixYl6unq6pb7+ykzMxM//vgjtmzZgl69egEANmzYgFatWuHUqVN47733KtELIiKqKpW6JduTJ08UHtnZ2UhMTESXLl2wdevWqm4j1ZL8/PzabkKpzp07B1NTU2zatAmXL1/GV199hVmzZmHlypUl6iYmJuL+/fvig/fUpmLXr1+HpaUlmjRpguHDhyM1NRUA8ODBA8TFxcHU1BRubm4wMzND9+7dcezYsRLHCA8Ph7GxMZydnbFo0SIUFhaWeb5z586hoKAAHh4eYlnLli3RuHFjnDx5suo7SEREFVKpUFwaOzs7hIeHl5hFpsqJjo5Gly5dYGBgAGNjY/Tu3Rs3btwQt9+5cwdDhw6FkZERtLW14eLigri4OERGRiI0NBQJCQmQyWSQyWSIjIwEAKSmpqJfv37Q0dGBnp4e/Pz88L///U88ZkhICNq2bYv169fD1tYWGhoaAIDff/8djo6O0NTUhLGxMTw8PJCTk1PuvoSGhsLExAR6enoYO3asQtju0aMHPv/8c0yaNAmGhoYwMzPDunXrkJOTgxEjRkBXVxfNmjXD3r17xX0+++wzLF++HN27d0eTJk3w8ccfY8SIEdixY0eJc5uamsLc3Fx8vPr2N0lTp06dEBkZiejoaKxevRrJycno2rUrnj59ips3bwJ48f9h9OjRiI6ORrt27eDu7o7r16+Lx5g4cSK2bduGw4cP4z//+Q8WLlyIGTNmlHnOtLQ0qKmpwcDAQKHczMwMaWlp1dJPIiIqv0otnyjzYCoquHfvXlUeUrJycnIwZcoUtGnTBtnZ2QgKCsKAAQMQHx+P3NxcdO/eHQ0bNsTu3bthbm6O8+fPQy6XY/Dgwbh06RKio6PF9ZL6+vqQy+ViID5y5AgKCwsxYcIEDB48GLGxseJ5k5KS8Mcff2DHjh1QVlbG/fv3MXToUERERGDAgAF4+vQp/vnnHwiCUK5+HDx4EBoaGoiNjUVKSgpGjBgBY2NjLFiwQKzz888/Y8aMGTh9+jS2b9+OcePG4c8//8SAAQMwe/ZsLF26FJ988glSU1OhpaVV6nkyMzNhZGRUorxt27bIy8uDg4MDQkJC0Llz5zLbmpeXh7y8PPF5VlYWAEBdSYCycvn6SzVDXUlQ+LciXp2tbdWqFdq1a4dmzZph69ataNmyJQBg1KhR+PjjjwEAEREROHDgANatWyd+737++ecKx1BWVsb48eMxb948qKurlzhv8SxyQUGBQrkgCCgqKipR/i4r7kt96lN9wbGp2zg+VaOyr1+lQvHu3bsVnguCgPv372PlypWvDR1UfoMGDVJ4/tNPP8HExARXrlzBiRMn8PDhQ5w5c0YMgs2aNRPr6ujolFgvGRMTg3///RfJycmwsrIC8OKCSXt7e5w5cwYdOnQA8GLJxMaNG2FiYgIAOH/+PAoLCzFw4EDxriOOjo7l7oeamhp++uknaGlpwd7eHvPmzcP06dMxf/58cdbWyckJc+bMAQDMmjUL4eHhaNCggXgBU1BQEFavXo2LFy+Wuu7yxIkT2L59O/bs2SOWWVhYYM2aNXBxcUFeXh7Wr1+PHj16IC4uDu3atSu1rWFhYSXuSAAAc5zl0NLivbfrovku8grvExUVVWq5qakp9u/fLz7Pz89XqKuvr4+4uLgy93/+/DkKCwuxceNGNGzYsMT2W7duIT8/H7/++it0dHQUyp88eVLmcd9lMTExtd0EKgPHpm7j+Lyd3NzcSu1XqVDcv39/hecymQwmJibo1asXlixZUqmGkKLr168jKCgIcXFxePToEeTyF7/8U1NTER8fD2dn51JnRsty9epVWFlZiYEYAFq3bg0DAwNcvXpVDMXW1tZiIAZeBFZ3d3c4OjrCy8sLnp6e+PDDD2FoaFiu8zo5OSnM7rq6uiI7Oxu3b98WQ/bLn4CorKwMY2NjheBdfLuqBw8elDj+pUuX0K9fPwQHB8PT01Msb9GihcKFUW5ubrhx4waWLl2KX375pdS2zpo1C1OmTBGfZ2VlwcrKCl9fUEKhqnK5+ks1Q11JwHwXOeaeVUKevGK3ZLsU4lWiLDs7G+np6ejcuTMCAgIQGhoKTU1N+Pr6inWCg4Ph5eWlUPayLVu2QElJqcz/H507d8b8+fOhoqIiHiMxMREPHz7EiBEj0KlTpwr1oy4rKChATEwM3n//faiqqtZ2c+glHJu6jeNTNYrf6a2oSoXi4oBG1adPnz6wtrbGunXrYGlpCblcDgcHB+Tn54v3Sa0Or35kt7KyMmJiYnDixAns378fK1asEO/damtrWyXnfPU/vkwmUyiTyV6Enle/765cuQJ3d3eMGTNGnGl+nY4dO5Z6sVQxdXX1Ut/2zpPLUFjBe+FSzciTyyp8n2JVVVVMmzZN/D927949BAcHQ1lZGR9//DHU1NQwffp0BAcHo127dmjbti1+/vlnJCYm4o8//oCqqipOnjyJuLg49OzZE7q6ujh58iSmT5+Ojz/+WLyY8+7du3B3d8fGjRvRsWNHNGjQACNHjsSMGTNgamoKPT09fP7553B1dUWXLl2q4+WpdaqqqvzFXkdxbOo2js/bqexrV6mrjubNm1fq1PSzZ88wb968SjWE/l96ejoSExMxZ84cuLu7o1WrVnjy5Im4vU2bNoiPj8fjx49L3V9NTa3ER223atUKt2/fxu3bt8WyK1euICMjA61bt35te2QyGTp37ozQ0FBcuHABampq+PPPP8vVl4SEBDx79kx8furUKejo6CjMWFfG5cuX0bNnT/j7+yusT36d+Ph4WFhYvNV5qX4ovlC1RYsW8PPzg7GxMU6dOiW+SzJp0iTMmjULkydPhpOTEw4ePIiYmBg0bdoUwIs/oLZt24bu3bvD3t4eCxYswOTJk7F27VrxHAUFBUhMTFT4Wbl06VL07t0bgwYNQrdu3WBubl7qBaJERFTzKjVTHBoairFjx5a46Ck3NxehoaEICgqqksZJlaGhIYyNjbF27VpYWFggNTVV4X6pQ4cOxcKFC9G/f3+EhYXBwsICFy5cgKWlJVxdXWFjY4Pk5GTEx8ejUaNG0NXVhYeHBxwdHTF8+HAsW7YMhYWFGD9+PLp37w4XF5cy2xIXF4eDBw/C09MTpqamiIuLw8OHD9GqVaty9SU/Px8jR47EnDlzkJKSguDgYAQGBr7VXSAuXbqEXr16wcvLC1OmTBGv3FdWVhZDzbJly2Brawt7e3s8f/4c69evx6FDhxTWjJJ0bdu27Y11Zs6cWean07Vr1w6nTp167f42NjYlLkjV0NDAqlWrsGrVqvI3loiIakSlkokgCOJb2i9LSEio0DpXKp2SkhK2bduGc+fOwcHBAZMnT8aiRYvE7Wpqati/fz9MTU3h6+sLR0dHhIeHQ1n5xbrXQYMGwdvbGz179oSJiQm2bt0KmUyGXbt2wdDQEN26dYOHhweaNGmC7du3v7Ytenp6OHr0KHx9fdG8eXPMmTMHS5YsgY+PT7n64u7uDjs7O3Tr1g2DBw9G3759ERISUunXBnhxi7iHDx9i06ZNsLCwEB/F66KBF2F86tSpcHR0RPfu3ZGQkIADBw7A3d39rc5NRERE9ZNMKO+9tfBiBlMmkyEzMxN6enoKwbioqAjZ2dkYO3YsZ0GoXsjKyoK+vj4ePXoEY2Pj2m4OvaSgoABRUVHw9fXlurs6iONTd3Fs6jaOT9Uo/v1dnFfLq0LLJ5YtWwZBEPDZZ58hNDQU+vr64jY1NTXY2NjA1dW1IockIiIiIqp1FQrF/v7+AABbW1u4ubnxrxiJe/leq6/au3cvunbtWoOtISIiIqq8Sl1o1717d/Hr58+fK3xsL4AKTVXTuys+Pr7MbaV9eAERERFRXVWpUJybm4sZM2bg119/RXp6eontr94OjOqnlz9Fj4iIiOhdVqm7T0yfPh2HDh3C6tWroa6ujvXr1yM0NBSWlpbYuHFjVbeRiIiIiKhaVWqm+K+//sLGjRvRo0cPjBgxAl27dkWzZs1gbW2NzZs3Y/jw4VXdTiIiIiKialOpmeLHjx+jSZMmAF6sHy7+ZLUuXbrg6NGjVdc6IiIiIqIaUKlQ3KRJEyQnJwMAWrZsiV9//RXAixlkAwODKmscEREREVFNqFQoHjFiBBISEgC8+CjUVatWQUNDA5MnT8b06dOrtIFERERERNWtUmuKJ0+eLH7t4eGBa9eu4dy5c2jWrBnatGlTZY0jIiIiIqoJlQrFL3v+/Dmsra1hbW1dFe0hIiIiIqpxlVo+UVRUhPnz56Nhw4bQ0dHBzZs3AQBz587Fjz/+WKUNJCIiIiKqbpUKxQsWLEBkZCQiIiKgpqYmljs4OGD9+vVV1jgiIiIioppQqVC8ceNGrF27FsOHD4eysrJY7uTkhGvXrlVZ44iIiIiIakKlQvHdu3dL/YhfuVyOgoKCt24UEREREVFNqlQobt26Nf75558S5b///jucnZ3fulFERERERDWpUnefCAoKgr+/P+7evQu5XI4dO3YgMTERGzduxN9//13VbSQiIiIiqlYVmim+efMmBEFAv3798Ndff+HAgQPQ1tZGUFAQrl69ir/++gvvv/9+dbWViIiIiKhaVGim2M7ODvfv34epqSm6du0KIyMj/PvvvzAzM6uu9hERERERVbsKzRQLgqDwfO/evcjJyanSBhERERER1bRKXWhX7NWQTERERET0LqpQKJbJZJDJZCXKiIiIiIjeZRVaUywIAgICAqCurg4AeP78OcaOHQttbW2Fejt27Ki6FhIRERERVbMKhWJ/f3+F5x9//HGVNoaIiIiIqDZUKBRv2LChutpBRERERFRr3upCOyIiIiKi+oChmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkT6W2G0BU13UKO4hCFe3absYbpYR/UNtNICIiemfVq5ni2NhYyGQyZGRkvLaejY0Nli1bViNtKu18MpkMO3furLHz1xap9LOuCQsLQ4cOHaCrqwtTU1P0798fiYmJCnXWrl2LHj16QE9Pr8z/MwsWLICbmxu0tLRgYGBQrnMLgoCgoCBYWFhAU1MTHh4euH79ehX0ioiIqHq906G4R48emDRpkvjczc0N9+/fh76+PgAgMjKy3L/M66KUlBTIZDLEx8fXdlPqlHXr1qFr164wNDSEoaEhPDw8cPr0aYU6AQEBkMlkCg9vb+9aanHNOnLkCCZMmIBTp04hJiYGBQUF8PT0RE5OjlgnNzcX3t7emD17dpnHyc/Px0cffYRx48aV+9wRERH47rvvsGbNGsTFxUFbWxteXl54/vz5W/WJiIioutWr5RNqamowNzev7WbUSQUFBVBVVa3tZlSJ2NhYDB06FG5ubtDQ0MA333wDT09PXL58GQ0bNhTreXt7Y8OGDeJzdXX12mhujYuOjlZ4HhkZCVNTU5w7dw7dunUDAPGPydjY2DKPExoaKu5fHoIgYNmyZZgzZw769esHANi4cSPMzMywc+dODBkypGIdISIiqkHv7ExxQEAAjhw5guXLl4szgZGRkeJbwbGxsRgxYgQyMzPF7SEhIaUeKyMjA6NGjYKJiQn09PTQq1cvJCQklKsdN27cQL9+/WBmZgYdHR106NABBw4cqJI+2traAgCcnZ0hk8nQo0cPcdv69evRqlUraGhooGXLlvj+++/FbcUzzNu3b0f37t2hoaGBzZs3IyAgAP3798fChQthZmYGAwMDzJs3D4WFhZg+fTqMjIzQqFEjhSCZn5+PwMBAWFhYQENDA9bW1ggLCyt3H+7fvw8fHx9oamqiSZMm+P3330u089dff0XXrl2hqamJDh064L///S/OnDkDFxcX6OjowMfHBw8fPhT327x5M8aPH4+2bduiZcuWWL9+PeRyOQ4ePKhwbnV1dZibm4sPQ0PDcre7PsnMzAQAGBkZVet5kpOTkZaWBg8PD7FMX18fnTp1wsmTJ6v13ERERG/rnQ3Fy5cvh6urK0aPHo379+/j/v37sLKyEre7ublh2bJl0NPTE7dPmzat1GN99NFHePDgAfbu3Ytz586hXbt2cHd3x+PHj9/YjuzsbPj6+uLgwYO4cOECvL290adPH6Smpr51H4uXBBw4cAD379/Hjh07ALwIhUFBQViwYAGuXr2KhQsXYu7cufj5558V9p85cya++OILXL16FV5eXgCAQ4cO4d69ezh69Ci+/fZbBAcHo3fv3jA0NERcXBzGjh2L//znP7hz5w4A4LvvvsPu3bvx66+/IjExEZs3b4aNjU25+zB37lwMGjQICQkJGD58OIYMGYKrV68q1AkODsacOXNw/vx5qKioYNiwYZgxYwaWL1+Of/75B0lJSQgKCirzHLm5uSgoKCgR+mJjY2FqaooWLVpg3LhxSE9PL3e76wu5XI5Jkyahc+fOcHBwqNZzpaWlAQDMzMwUys3MzMRtREREddU7u3xCX18fampq0NLSEpdMXLt2TdyupqYGfX19yGSy1y6pOHbsGE6fPo0HDx6Ib68vXrwYO3fuxO+//44xY8a8th1OTk5wcnISn8+fPx9//vkndu/ejcDAwLfpIkxMTAAAxsbGCn0IDg7GkiVLMHDgQAAvZpSvXLmCH374Af7+/mK9SZMmiXWKGRkZ4bvvvoOSkhJatGiBiIgI5ObmimtLZ82ahfDwcBw7dgxDhgxBamoq7Ozs0KVLF8hkMlhbW1eoDx999BFGjRoF4MVrExMTgxUrVijMbE+bNk0M7V988QWGDh2KgwcPonPnzgCAkSNHvvYt/C+//BKWlpYKM5Te3t4YOHAgbG1tcePGDcyePRs+Pj44efIklJWVSz1OXl4e8vLyxOdZWVkAAHUlAcrKQoX6XRsKCgpKlAUGBuLSpUs4fPhwqdsLCwvFfUvbDgBFRUVlHr88x5LL5ZDJZG/cvyKKj1WVx6Sqw/Gpuzg2dRvHp2pU9vV7Z0NxVUlISEB2djaMjY0Vyp89e4YbN268cf/s7GyEhIRgz549uH//PgoLC/Hs2bMqmSkuTU5ODm7cuIGRI0di9OjRYnlhYaF4gWExFxeXEvvb29tDSen/3yAwMzNTmEFUVlaGsbExHjx4AODFMpX3338fLVq0gLe3N3r37g1PT89yt9fV1bXE81cvHGzTpo1CewDA0dFRoay4Pa8KDw/Htm3bEBsbCw0NDbH85fWrjo6OaNOmDZo2bYrY2Fi4u7uXeqywsDBxHe3L5jjLoaVVVEYP646oqCiF52vXrkVcXBwWLlyIixcv4uLFiyX2+ffffwEA+/fvh46OTqnHTUhIQEFBQYnjv6p4NviPP/5AkyZNxPJr167B1tb2jftXRkxMTJUfk6oOx6fu4tjUbRyft5Obm1up/SQfirOzs2FhYVHqBUfluXPFtGnTEBMTg8WLF6NZs2bQ1NTEhx9+iPz8/KpvLF60F3hxB4ZOnTopbHt1BlRbu+S9dV+92E4mk5VaJpfLAQDt2rVDcnIy9u7diwMHDsDPzw8eHh4Ka4Pf1svnl8lkpZYVt+dlixcvRnh4OA4cOKAQrEvTpEkTNGjQAElJSWWG4lmzZmHKlCni86ysLFhZWeHrC0ooVC19drkuuRTyYrZdEARMmjQJ8fHxOHr0KOzs7Mrcp/h7xNPTs8zv90ePHkFVVRW+vr6vPb8gCAgJCUFBQYFYNysrC0lJSZg5c+Yb96+IgoICxMTE4P333683F5DWJxyfuotjU7dxfKpG8Tu9FfVOh2I1NTXxrd3KbAdehL60tDSoqKhUaK1ssePHjyMgIAADBgwA8CK0pqSkVPg4pVFTUwMAhT6YmZnB0tISN2/exPDhw6vkPG+ip6eHwYMHY/Dgwfjwww/h7e2Nx48fl+vCrVOnTuHTTz9VeO7s7PzWbYqIiMCCBQuwb9++UmfEX3Xnzh2kp6fDwsKizDrq6uql3qEiTy5DYZHsrdpbE4p/gI4fPx5btmzBrl27YGRkJK6l1tfXh6amJoAXs7ppaWni9+q1a9egq6uLxo0bi+OampqKx48f4+7duygqKsLly5cBAM2aNRNnlVu2bImwsDDx+3/SpEkICwtDy5YtYWtri7lz58LS0hIffvhhtfyAV1VV5S+OOozjU3dxbOo2js/bqexr906HYhsbG8TFxSElJQU6OjolZhNtbGyQnZ2NgwcPwsnJCVpaWtDS0lKo4+HhAVdXV/Tv3x8RERFo3rw57t27hz179mDAgAFvDFx2dnbYsWMH+vTpA5lMhrlz55Y6q1kZpqam0NTURHR0NBo1agQNDQ3o6+sjNDQUEydOhL6+Pry9vZGXl4ezZ8/iyZMnCjOdVeHbb7+FhYUFnJ2doaSkhN9++w3m5ublvv/zb7/9BhcXF3Tp0gWbN2/G6dOn8eOPP75Vm7755hsEBQVhy5YtsLGxEd+219HRgY6ODrKzsxEaGopBgwbB3NwcN27cwIwZM9CsWTNx7XJ9tnr1agBQuFsJAGzYsAEBAQEAgDVr1igsFSm+VdvLdYKCghQu3iz+Y+bw4cPisRMTE8W7WwDAjBkzkJOTgzFjxiAjIwNdunRBdHS0wtIWIiKiuuidvfsE8GLpgrKyMlq3bg0TE5MS63jd3NwwduxYDB48GCYmJoiIiChxDJlMhqioKHTr1g0jRoxA8+bNMWTIENy6davEVfSl+fbbb2FoaAg3Nzf06dMHXl5eaNeuXZX0T0VFBd999x1++OEHWFpaivd+HTVqFNavX48NGzbA0dER3bt3R2RkpHgLt6qkq6uLiIgIuLi4oEOHDkhJSUFUVJTCuuTXCQ0NxbZt29CmTRts3LgRW7duRevWrd+qTatXr0Z+fj4+/PBDWFhYiI/FixcDeLGM5OLFi+jbty+aN2+OkSNHon379vjnn38kca9iQRBKfRSHXQAICQl5Y53IyMhS67wctl/dRyaTYd68eUhLS8Pz589x4MABNG/evPo7TURE9JZkgiDU/cvqiWpBVlYW9PX10XTqdhSqlFyfXdekhH9Q202oMcUX/vn6+vItxjqI41N3cWzqNo5P1Sj+/Z2ZmQk9Pb1y7/dOL58gqglxs9xL3J2EiIiI6pd3evlETbC3txfXqr762Lx581sde+HChWUe28fHp4p6UD02b95cZtvt7e1ru3lEREREFcKZ4jeIiooq8ybQ5Vlz/Dpjx46Fn59fqduK7xJQV/Xt27fELeGK8S0fIiIietcwFL9BRT/BrSKMjIzKdVuzukhXVxe6urq13QwiIiKiKsHlE0REREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkqtd0AorquU9hBFKpol7t+SvgHAIC7d+/iyy+/xN69e5Gbm4tmzZphw4YNcHFxAQCEhIRg27ZtuH37NtTU1NC+fXssWLAAnTp1eu3xV61ahUWLFiEtLQ1OTk5YsWIFOnbsWPkOEhEREWeKa4JMJsPOnTsBACkpKZDJZIiPj6/VNlU3qfSzLE+ePEHnzp2hqqqKvXv34sqVK1iyZAkMDQ3FOs2bN8fKlSvx77//4tixY7CxsYGnpycePnxY5nG3b9+OKVOmIDg4GOfPn4eTkxO8vLzw4MGDmugWERFRvfXOhuKQkBC0bdu2tptRrWJjYyGTyZCRkVHbTalTwsLC0KFDB+jq6sLU1BT9+/dHYmKiQp0ePXpAJpMpPMaOHVtjbfzmm29gZWWFDRs2oGPHjrC1tYWnpyeaNm0q1hk2bBg8PDzQpEkT2Nvb49tvv0VWVhYuXrxY5nG//fZbjB49GiNGjEDr1q2xZs0aaGlp4aeffqqJbhEREdVb72worir5+fm13YRqJwgCCgsLa7sZVebIkSOYMGECTp06hZiYGBQUFMDT0xM5OTkK9UaPHo379++Lj4iIiBpr4+7du+Hi4oKPPvoIpqamcHZ2xrp168qsn5+fj7Vr10JfXx9OTk5l1jl37hw8PDzEMiUlJXh4eODkyZNV3gciIiIpqdVQHB0djS5dusDAwADGxsbo3bs3bty4IW6/c+cOhg4dCiMjI2hra8PFxQVxcXGIjIxEaGgoEhISxFnAyMhIAEBqair69esHHR0d6Onpwc/PD//73//EYxbPMK9fvx62trbQ0NAAAPz+++9wdHSEpqYmjI2N4eHhUSJklebMmTN4//330aBBA+jr66N79+44f/78W782KSkp6NmzJwDA0NAQMpkMAQEBAAC5XI6wsDDY2tpCU1MTTk5O+P3338V9i2eY9+7di/bt20NdXR3Hjh1Djx498Pnnn2PSpEkwNDSEmZkZ1q1bh5ycHIwYMQK6urpo1qwZ9u7dKx7ryZMnGD58OExMTKCpqQk7Ozts2LCh3P24du0a3NzcoKGhAQcHBxw5cqREO/ft2wdnZ2doamqiV69eePDgAfbu3YtWrVpBT08Pw4YNQ25urrhfdHQ0AgICYG9vDycnJ0RGRiI1NRXnzp1TOLeWlhbMzc3Fh56eXoXG4G3cvHkTq1evhp2dHfbt24dx48Zh4sSJ+PnnnxXq/f3339DR0YGGhgaWLl2KmJgYNGjQoNRjPnr0CEVFRTAzM1MoNzMzQ1paWrX1hYiISApq9UK7nJwcTJkyBW3atEF2djaCgoIwYMAAxMfHIzc3F927d0fDhg2xe/dumJub4/z585DL5Rg8eDAuXbqE6OhoHDhwAACgr68PuVwuBuIjR46gsLAQEyZMwODBgxEbGyueNykpCX/88Qd27NgBZWVl3L9/H0OHDkVERAQGDBiAp0+f4p9//oEgCG/sw9OnT+Hv748VK1ZAEAQsWbIEvr6+uH79OnR1dSv92lhZWeGPP/7AoEGDkJiYCD09PWhqagJ4sXxg06ZNWLNmDezs7HD06FF8/PHHMDExQffu3cVjzJw5E4sXL0aTJk3Etaw///wzZsyYgdOnT2P79u0YN24c/vzzTwwYMACzZ8/G0qVL8cknnyA1NRVaWlqYO3curly5gr1796JBgwZISkrCs2fPyt2P6dOnY9myZWjdujW+/fZb9OnTB8nJyTA2NhbrhISEYOXKldDS0oKfnx/8/Pygrq6OLVu2IDs7GwMGDMCKFSvw5ZdflnqOzMxMAICRkZFC+ebNm7Fp0yaYm5ujT58+mDt3LrS0tMpsa15eHvLy8sTnWVlZAAB1JQHKym/+XihWUFAAuVyO9u3bIzQ0FADg4OCAixcvYvXq1Rg2bJhYt0uXLjhz5gzS09Px448/ws/PD8eOHYOpqWmpxwWAwsJC8WsAKCoqgiAICmX1XXFfpdTndwnHp+7i2NRtHJ+qUdnXr1ZD8aBBgxSe//TTTzAxMcGVK1dw4sQJPHz4EGfOnBHDTrNmzcS6Ojo6UFFRgbm5uVgWExODf//9F8nJybCysgIAbNy4Efb29jhz5gw6dOgA4MXb0Bs3boSJiQkA4Pz58ygsLMTAgQNhbW0NAHB0dCxXH3r16qXwfO3atTAwMMCRI0fQu3fvirwcCpSVlcV+m5qawsDAAMCL4LZw4UIcOHAArq6uAIAmTZrg2LFj+OGHHxRC8bx58/D+++8rHNfJyQlz5swBAMyaNQvh4eFo0KABRo8eDQAICgrC6tWrcfHiRbz33ntITU2Fs7OzeMcEGxubCvUjMDBQHOfVq1cjOjoaP/74I2bMmCHW+frrr9G5c2cAwMiRIzFr1izcuHEDTZo0AQB8+OGHOHz4cKmhWC6XY9KkSejcuTMcHBzE8mHDhsHa2hqWlpa4ePEivvzySyQmJmLHjh1ltjUsLEwMsS+b4yyHllZRufscFRUFAwMD6OjoICoqSiwvLCzE9evXFcpe1r9/f+zbtw8zZ87Ehx9+WGJ7QUEBlJSUEBUVhcePH4vlFy5cgEwmK/O49VlMTExtN4Feg+NTd3Fs6jaOz9t5+d3liqjVUHz9+nUEBQUhLi4Ojx49glwuB/BiCUR8fDycnZ1LzP69ztWrV2FlZSUGYgBo3bo1DAwMcPXqVTEUW1tbi4EYeBEU3d3d4ejoCC8vL3h6euLDDz9UuFNAWf73v/9hzpw5iI2NxYMHD1BUVITc3FykpqaWu90VkZSUhNzc3BJhNz8/H87OzgplxUH2ZW3atBG/VlZWhrGxscIfAMVvzRffzWDcuHEYNGgQzp8/D09PT/Tv3x9ubm7lbm9xcAcAFRUVuLi44OrVq2W2yczMDFpaWmIgLi47ffp0qcefMGECLl26hGPHjimUjxkzRvza0dERFhYWcHd3x40bNxQudnvZrFmzMGXKFPF5VlYWrKys8PUFJRSqKpejty9cCvFCr169cOfOHfj6+orlhw4dQvPmzRXKXqWpqQkbG5sy67Rv3x5ZWVnidrlcjgkTJmDcuHGvPW59U1BQgJiYGLz//vtQVVWt7ebQKzg+dRfHpm7j+FSN4nd6K6pWQ3GfPn1gbW2NdevWwdLSEnK5HA4ODsjPzxeXClQHbW3Fe84qKysjJiYGJ06cwP79+7FixQp89dVXiIuLg62t7WuP5e/vj/T0dCxfvhzW1tZQV1eHq6trtV3Al52dDQDYs2cPGjZsqLBNXV1d4fmr/QRQ4j+ZTCZTKJPJZAAg/oHi4+ODW7duISoqCjExMXB3d8eECROwePHit+9MKW16tT3FZcXteVlgYCD+/vtvHD16FI0aNXrtOYrv/ZuUlFRmKFZXVy/xGgJAnlyGwiLZG/tRTFVVFVOnToWbmxsWLVoEPz8/nD59GuvXr8fatWuhqqqKnJwcLFiwAH379oWFhQUePXqEVatW4e7duxgyZIj4Gri7u2PAgAEIDAwEAEydOhX+/v7o2LEjOnbsiGXLliEnJwejRo2S5A9QVVVVSfb7XcHxqbs4NnUbx+ftVPa1q7UL7dLT05GYmIg5c+bA3d0drVq1wpMnT8Ttbdq0QXx8vMLbxC9TU1NDUZHiW9qtWrXC7du3cfv2bbHsypUryMjIQOvWrV/bHplMhs6dOyM0NBQXLlyAmpoa/vzzzzf24/jx45g4cSJ8fX1hb28PdXV1PHr06I37lYeamhoAKPSzdevWUFdXR2pqKpo1a6bweHmGvCqZmJjA398fmzZtwrJly7B27dpy73vq1Cnx68LCQpw7dw6tWrV6q/YIgoDAwED8+eefOHTo0Bv/cAEg3i/ZwsLirc5dXh06dMCff/6JrVu3wsHBAfPnz8eyZcswfPhwAC/+ELt27RoGDRqE5s2bo0+fPkhPT8c///wDe3t78Tg3btxQ+H4aPHgwFi9ejKCgILRt2xbx8fGIjo4ucfEdERERVUytzRQbGhrC2NgYa9euhYWFBVJTUzFz5kxx+9ChQ7Fw4UL0798fYWFhsLCwwIULF2BpaQlXV1fY2NggOTkZ8fHxaNSoEXR1deHh4QFHR0cMHz4cy5YtQ2FhIcaPH4/u3buXupSgWFxcHA4ePAhPT0+YmpoiLi4ODx8+LFd4s7Ozwy+//AIXFxdkZWVh+vTpVTbLbW1tDZlMhr///hu+vr7Q1NSErq4upk2bhsmTJ0Mul6NLly7IzMzE8ePHoaenB39//yo5d7GgoCC0b98e9vb2yMvLw99//12hULtq1SrY2dmhVatWWLp0KZ48eYLPPvvsrdo0YcIEbNmyBbt27YKurq545wV9fX1oamrixo0b2LJlC3x9fWFsbIyLFy9i8uTJ6Natm8JSjerWu3fvMteVa2hovHZ9c7GUlJQSZYGBgeLMMREREVWNWpspVlJSwrZt23Du3Dk4ODhg8uTJWLRokbhdTU0N+/fvh6mpKXx9feHo6Ijw8HAoK79Y2zlo0CB4e3ujZ8+eMDExwdatWyGTybBr1y4YGhqiW7du4gcjbN++/bVt0dPTw9GjR+Hr64vmzZtjzpw5WLJkCXx8fN7Yjx9//BFPnjxBu3bt8Mknn2DixIml3jmgMho2bIjQ0FDMnDkTZmZmYhCaP38+5s6di7CwMLRq1Qre3t7Ys2dPuWZMK0pNTQ2zZs1CmzZt0K1bNygrK2Pbtm3l3j88PBzh4eFwcnLCsWPHsHv37jJvOVZeq1evRmZmJnr06AELCwvxUTzOampqOHDgADw9PdGyZUtMnToVgwYNwl9//fVW5yUiIqL6SyaU575jRBKUlZUFfX19PHr0SOEWclT7CgoKEBUVBV9fX667q4M4PnUXx6Zu4/hUjeLf35mZmRX6jALJf6IdERERERFD8Rvo6OiU+fjnn3/e6thjx44t89hjx46toh5Uj4ULF5bZ9vIsOyEiIiKqS2r1lmzvguK7FpTm1VuiVdS8efMwbdq0UrfV5EcSV8bYsWPh5+dX6rbqvJ0eERERUXVgKH6Dlz9Fr6qZmppW2UV5Nc3IyKhCH6xCREREVJdx+QQRERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJHkMxEREREUkeQzERERERSR5DMRERERFJnkptN4CorhIEAQDw9OlTqKqq1nJr6GUFBQXIzc1FVlYWx6YO4vjUXRybuo3jUzWysrIA/P/v8fJiKCYqQ3p6OgDA1ta2lltCREREFfX06VPo6+uXuz5DMVEZjIyMAACpqakV+k9F1S8rKwtWVla4ffs29PT0ars59AqOT93FsanbOD5VQxAEPH36FJaWlhXaj6GYqAxKSi+W3Ovr6/OHUx2lp6fHsanDOD51F8embuP4vL3KTGbxQjsiIiIikjyGYiIiIiKSPIZiojKoq6sjODgY6urqtd0UegXHpm7j+NRdHJu6jeNTu2RCRe9XQURERERUz3CmmIiIiIgkj6GYiIiIiCSPoZiIiIiIJI+hmIiIiIgkj6GYqBSrVq2CjY0NNDQ00KlTJ5w+fbq2m1TvhYSEQCaTKTxatmwpbn/+/DkmTJgAY2Nj6OjoYNCgQfjf//6ncIzU1FR88MEH0NLSgqmpKaZPn47CwsKa7kq9cPToUfTp0weWlpaQyWTYuXOnwnZBEBAUFAQLCwtoamrCw8MD169fV6jz+PFjDB8+HHp6ejAwMMDIkSORnZ2tUOfixYvo2rUrNDQ0YGVlhYiIiOru2jvvTWMTEBBQ4v+St7e3Qh2OTfUICwtDhw4doKurC1NTU/Tv3x+JiYkKdarqZ1lsbCzatWsHdXV1NGvWDJGRkdXdvXqPoZjoFdu3b8eUKVMQHByM8+fPw8nJCV5eXnjw4EFtN63es7e3x/3798XHsWPHxG2TJ0/GX3/9hd9++w1HjhzBvXv3MHDgQHF7UVERPvjgA+Tn5+PEiRP4+eefERkZiaCgoNroyjsvJycHTk5OWLVqVanbIyIi8N1332HNmjWIi4uDtrY2vLy88Pz5c7HO8OHDcfnyZcTExODvv//G0aNHMWbMGHF7VlYWPD09YW1tjXPnzmHRokUICQnB2rVrq71/77I3jQ0AeHt7K/xf2rp1q8J2jk31OHLkCCZMmIBTp04hJiYGBQUF8PT0RE5OjlinKn6WJScn44MPPkDPnj0RHx+PSZMmYdSoUdi3b1+N9rfeEYhIQceOHYUJEyaIz4uKigRLS0shLCysFltV/wUHBwtOTk6lbsvIyBBUVVWF3377TSy7evWqAEA4efKkIAiCEBUVJSgpKQlpaWlindWrVwt6enpCXl5etba9vgMg/Pnnn+JzuVwumJubC4sWLRLLMjIyBHV1dWHr1q2CIAjClStXBADCmTNnxDp79+4VZDKZcPfuXUEQBOH7778XDA0NFcbnyy+/FFq0aFHNPao/Xh0bQRAEf39/oV+/fmXuw7GpOQ8ePBAACEeOHBEEoep+ls2YMUOwt7dXONfgwYMFLy+v6u5SvcaZYqKX5Ofn49y5c/Dw8BDLlJSU4OHhgZMnT9Ziy6Th+vXrsLS0RJMmTTB8+HCkpqYCAM6dO4eCggKFcWnZsiUaN24sjsvJkyfh6OgIMzMzsY6XlxeysrJw+fLlmu1IPZecnIy0tDSF8dDX10enTp0UxsPAwAAuLi5iHQ8PDygpKSEuLk6s061bN6ipqYl1vLy8kJiYiCdPntRQb+qn2NhYmJqaokWLFhg3bhzS09PFbRybmpOZmQkAMDIyAlB1P8tOnjypcIziOvw99XYYiole8ujRIxQVFSn8MAIAMzMzpKWl1VKrpKFTp06IjIxEdHQ0Vq9ejeTkZHTt2hVPnz5FWloa1NTUYGBgoLDPy+OSlpZW6rgVb6OqU/x6vu7/SVpaGkxNTRW2q6iowMjIiGNWzby9vbFx40YcPHgQ33zzDY4cOQIfHx8UFRUB4NjUFLlcjkmTJqFz585wcHAAgCr7WVZWnaysLDx79qw6uiMJKrXdACIiAPDx8RG/btOmDTp16gRra2v8+uuv0NTUrMWWEb1bhgwZIn7t6OiINm3aoGnTpoiNjYW7u3sttkxaJkyYgEuXLilcG0F1G2eKiV7SoEEDKCsrl7gS+H//+x/Mzc1rqVXSZGBggObNmyMpKQnm5ubIz89HRkaGQp2Xx8Xc3LzUcSveRlWn+PV83f8Tc3PzEhenFhYW4vHjxxyzGtakSRM0aNAASUlJADg2NSEwMBB///03Dh8+jEaNGonlVfWzrKw6enp6nER4CwzFRC9RU1ND+/btcfDgQbFMLpfj4MGDcHV1rcWWSU92djZu3LgBCwsLtG/fHqqqqgrjkpiYiNTUVHFcXF1d8e+//yr8so+JiYGenh5at25d4+2vz2xtbWFubq4wHllZWYiLi1MYj4yMDJw7d06sc+jQIcjlcnTq1Emsc/ToURQUFIh1YmJi0KJFCxgaGtZQb+q/O3fuID09HRYWFgA4NtVJEAQEBgbizz//xKFDh2Bra6uwvap+lrm6uioco7gOf0+9pdq+0o+ortm2bZugrq4uREZGCleuXBHGjBkjGBgYKFwJTFVv6tSpQmxsrJCcnCwcP35c8PDwEBo0aCA8ePBAEARBGDt2rNC4cWPh0KFDwtmzZwVXV1fB1dVV3L+wsFBwcHAQPD09hfj4eCE6OlowMTERZs2aVVtdeqc9ffpUuHDhgnDhwgUBgPDtt98KFy5cEG7duiUIgiCEh4cLBgYGwq5du4SLFy8K/fr1E2xtbYVnz56Jx/D29hacnZ2FuLg44dixY4KdnZ0wdOhQcXtGRoZgZmYmfPLJJ8KlS5eEbdu2CVpaWsIPP/xQ4/19l7xubJ4+fSpMmzZNOHnypJCcnCwcOHBAaNeunWBnZyc8f/5cPAbHpnqMGzdO0NfXF2JjY4X79++Lj9zcXLFOVfwsu3nzpqClpSVMnz5duHr1qrBq1SpBWVlZiI6OrtH+1jcMxUSlWLFihdC4cWNBTU1N6Nixo3Dq1KnablK9N3jwYMHCwkJQU1MTGjZsKAwePFhISkoStz979kwYP368YGhoKGhpaQkDBgwQ7t+/r3CMlJQUwcfHR9DU1BQaNGggTJ06VSgoKKjprtQLhw8fFgCUePj7+wuC8OK2bHPnzhXMzMwEdXV1wd3dXUhMTFQ4Rnp6ujB06FBBR0dH0NPTE0aMGCE8ffpUoU5CQoLQpUsXQV1dXWjYsKEQHh5eU118Z71ubHJzcwVPT0/BxMREUFVVFaytrYXRo0eX+KOeY1M9ShsXAMKGDRvEOlX1s+zw4cNC27ZtBTU1NaFJkyYK56DKkQmCINT07DQRERERUV3CNcVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERERCR5DMVEREREJHkMxUREREQkeQzFRERUJwUEBEAmk5V4JCUl1XbTiKgeUqntBhAREZXF29sbGzZsUCgzMTGppdYoKigogKqqam03g4iqCGeKiYiozlJXV4e5ubnCQ1lZudS6t27dQp8+fWBoaAhtbW3Y29sjKipK3H758mX07t0benp60NXVRdeuXXHjxg0AgFwux7x589CoUSOoq6ujbdu2iI6OFvdNSUmBTCbD9u3b0b17d2hoaGDz5s0AgPXr16NVq1bQ0NBAy5Yt8f3331fjK0JE1YUzxUREVC9MmDAB+fn5OHr0KLS1tXHlyhXo6OgAAO7evYtu3bqhR48eOHToEPT09HD8+HEUFhYCAJYvX44lS5bghx9+gLOzM3766Sf07dsXly9fhp2dnXiOmTNnYsmSJXB2dhaDcVBQEFauXAlnZ2dcuHABo0ePhra2Nvz9/WvldSCiypEJgiDUdiOIiIheFRAQgE2bNkFDQ0Ms8/HxwW+//VZq/TZt2mDQoEEIDg4usW327NnYtm0bEhMTS13y0LBhQ0yYMAGzZ88Wyzp27IgOHTpg1apVSElJga2tLZYtW4YvvvhCrNOsWTPMnz8fQ4cOFcu+/vprREVF4cSJE5XqNxHVDs4UExFRndWzZ0+sXr1afK6trV1m3YkTJ2LcuHHYv38/PDw8MGjQILRp0wYAEB8fj65du5YaiLOysnDv3j107txZobxz585ISEhQKHNxcRG/zsnJwY0bNzBy5EiMHj1aLC8sLIS+vn7FOkpEtY6hmIiI6ixtbW00a9asXHVHjRoFLy8v7NmzB/v370dYWBiWLFmCzz//HJqamlXWnmLZ2dkAgHXr1qFTp04K9cpa90xEdRcvtCMionrDysoKY8eOxY4dOzB16lSsW7cOwIulFf/88w8KCgpK7KOnpwdLS0scP35cofz48eNo3bp1mecyMzODpaUlbt68iWbNmik8bG1tq7ZjRFTtOFNMRET1wqRJk+Dj44PmzZvjyZMnOHz4MFq1agUACAwMxIoVKzBkyBDMmjUL+vr6OHXqFDp27IgWLVpg+vTpCA4ORtOmTdG2bVts2LAB8fHx4h0myhIaGoqJEydCX18f3t7eyMvLw9mzZ/HkyRNMmTKlJrpNRFWEoZiIiOqFoqIiTJgwAXfu3IGenh68vb2xdOlSAICxsTEOHTqE6dOno3v37lBWVkbbtm3FdcQTJ05EZmYmpk6digcPHqB169bYvXu3wp0nSjNq1ChoaWlh0aJFmD59OrS1teHo6IhJkyZVd3eJqIrx7hNEREREJHlcU0xEREREksdQTERERESSx1BMRERERJLHUExEREREksdQTERERESSx1BMRERERJLHUExEREREksdQTERERESSx1BMRERERJLHUExEREREksdQTERERESSx1BMRERERJL3f8h/TlY4DgcJAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from xgboost import plot_importance\n",
    "\n",
    "plot_importance(ranker, importance_type=\"weight\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import the model into Elasticsearch\n",
    "\n",
    "Once the model is trained we can use Eland to load it into Elasticsearch.\n",
    "\n",
    "Please note that the `MLModel.import_ltr_model` method contains the `LTRModelConfig` object which defines how features should be extracted for the model being imported."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zAMwvqYlq9py",
    "outputId": "c0f60ce3-fb07-47a5-9e37-fccbd1f30bcc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<eland.ml.ml_model.MLModel at 0x2ae5734c0>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from eland.ml import MLModel\n",
    "\n",
    "LEARNING_TO_RANK_MODEL_ID = \"ltr-model-xgboost\"\n",
    "\n",
    "MLModel.import_ltr_model(\n",
    "    es_client=es_client,\n",
    "    model=ranker,\n",
    "    model_id=LEARNING_TO_RANK_MODEL_ID,\n",
    "    ltr_model_config=ltr_config,\n",
    "    es_if_exists=\"replace\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the rescorer\n",
    "\n",
    "Once the model is uploaded to Elasticsearch, you will be able to use it as a rescorer in the _search API, as shown in this example:\n",
    "\n",
    "```\n",
    "GET /movies/_search\n",
    "{\n",
    "   \"query\" : {\n",
    "      \"multi_match\" : {\n",
    "         \"query\": \"star wars\",\n",
    "         \"fields\": [\"title\", \"overview\", \"actors\", \"director\", \"tags\", \"characters\"]\n",
    "      }\n",
    "   },\n",
    "   \"rescore\" : {\n",
    "      \"window_size\" : 50,\n",
    "      \"learning_to_rank\" : {\n",
    "         \"model_id\": \"ltr-model-xgboost\",\n",
    "         \"params\": { \n",
    "            \"query\": \"star wars\"\n",
    "         }\n",
    "      }\n",
    "   }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Xgr5MWWIrEk9",
    "outputId": "e296cf37-afd1-43fb-e839-6c65cb65c072"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Star Wars', 10.971989, '11'),\n",
       " ('Star Wars: The Clone Wars', 9.923633, '12180'),\n",
       " ('Andor: A Disney+ Day Special Look', 8.9880295, '1022100'),\n",
       " (\"Family Guy Presents: It's a Trap!\", 8.845748, '278427'),\n",
       " ('Star Wars: The Rise of Skywalker', 8.053349, '181812'),\n",
       " ('Star Wars: The Force Awakens', 8.053349, '140607'),\n",
       " ('Star Wars: The Last Jedi', 8.053349, '181808'),\n",
       " ('Solo: A Star Wars Story', 8.053349, '348350'),\n",
       " ('The Star Wars Holiday Special', 8.053349, '74849'),\n",
       " ('Phineas and Ferb: Star Wars', 8.053349, '392216')]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = \"star wars\"\n",
    "\n",
    "# First let's display the result when not using the rescorer:\n",
    "search_fields = [\"title\", \"overview\", \"actors\", \"director\", \"tags\", \"characters\"]\n",
    "bm25_query = {\"multi_match\": {\"query\": query, \"fields\": search_fields}}\n",
    "\n",
    "bm25_search_response = es_client.search(index=MOVIE_INDEX, query=bm25_query)\n",
    "\n",
    "[\n",
    "    (movie[\"_source\"][\"title\"], movie[\"_score\"], movie[\"_id\"])\n",
    "    for movie in bm25_search_response[\"hits\"][\"hits\"]\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Star Wars', 4.1874104, '11'),\n",
       " ('Star Wars: The Clone Wars', 2.3627238, '12180'),\n",
       " ('Star Wars: The Rise of Skywalker', 1.7667875, '181812'),\n",
       " ('Star Wars: The Force Awakens', 1.3336482, '140607'),\n",
       " ('Star Wars: The Last Jedi', 1.3336482, '181808'),\n",
       " ('Rogue One: A Star Wars Story', 1.1134433, '330459'),\n",
       " ('LEGO Star Wars Summer Vacation', 1.082971, '980804'),\n",
       " (\"Doraemon: Nobita's Little Star Wars 2021\", 0.9138395, '782054'),\n",
       " ('LEGO Star Wars Terrifying Tales', 0.89640737, '857702'),\n",
       " ('Solo: A Star Wars Story', 0.65811557, '348350')]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now let's display result when using the rescorer:\n",
    "\n",
    "ltr_rescorer = {\n",
    "    \"learning_to_rank\": {\n",
    "        \"model_id\": LEARNING_TO_RANK_MODEL_ID,\n",
    "        \"params\": {\"query\": query},\n",
    "    },\n",
    "    \"window_size\": 100,\n",
    "}\n",
    "\n",
    "rescored_search_response = es_client.search(\n",
    "    index=MOVIE_INDEX, query=bm25_query, rescore=ltr_rescorer\n",
    ")\n",
    "\n",
    "[\n",
    "    (movie[\"_source\"][\"title\"], movie[\"_score\"], movie[\"_id\"])\n",
    "    for movie in rescored_search_response[\"hits\"][\"hits\"]\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As also shown in the feature importance graph above, we can see in this results list that the `title_bm25` and `popularity` features are weighted highly in our trained model. Now all results include the query terms in the title, showing the importance of the `title_bm25` feature. Similarly, more popular movies now rank higher, for example `Rogue One: A Star Wars Story` is now in sixth position."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.12.3 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  },
  "vscode": {
   "interpreter": {
    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0ab8ee0c2e1a42658ecbf01b0b28cf92": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "549645ca4e7b48ef86cffdaa5507c56c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "594c5ebcb9624b63b128536a46594211": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0ab8ee0c2e1a42658ecbf01b0b28cf92",
      "placeholder": "​",
      "style": "IPY_MODEL_84206a53779249fbb34c78edb17fd1e0",
      "value": " 4233/4233 [05:50&lt;00:00, 11.77it/s]"
     }
    },
    "7ed336be71e74521a596a7d624c1e7d1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "84206a53779249fbb34c78edb17fd1e0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8c393bd522f6427f9978a89bc7dbdf3b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a6d4eb3325444f28b11ba02c3d01ed83": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7ed336be71e74521a596a7d624c1e7d1",
      "placeholder": "​",
      "style": "IPY_MODEL_ee1a6943af1e49e6a8851f27c9811c32",
      "value": "100%"
     }
    },
    "bff504814b434aec90b2cf020b08cfa9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8c393bd522f6427f9978a89bc7dbdf3b",
      "max": 4233,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_549645ca4e7b48ef86cffdaa5507c56c",
      "value": 4233
     }
    },
    "e95447129da74d9ebc4c1d99165bd534": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ee1a6943af1e49e6a8851f27c9811c32": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f9cdfbc3972a4b84a557507567ca2965": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a6d4eb3325444f28b11ba02c3d01ed83",
       "IPY_MODEL_bff504814b434aec90b2cf020b08cfa9",
       "IPY_MODEL_594c5ebcb9624b63b128536a46594211"
      ],
      "layout": "IPY_MODEL_e95447129da74d9ebc4c1d99165bd534"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}