BIDS_Net.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/fjadidi2001/Image_Inpaint/blob/main/BIDS_Net.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Bidirectional Interaction Dual-Stream Network (BIDS-Net)"
      ],
      "metadata": {
        "id": "kJsyOcVtV11y"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "> The article proposes a novel approach for image inpainting called the Bidirectional Interaction Dual-Stream Network (BIDS-Net), integrating CNN and Transformer models to enhance inpainting quality by leveraging their complementary strengths."
      ],
      "metadata": {
        "id": "Ql2YbLBmWBQM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Methodology Overview\n",
        "1. Dual-Stream Structure:\n",
        "\n",
        "- CNN Stream: Captures rich local patterns and refines details.\n",
        "- Transformer Stream: Models long-range contextual correlations for global information.\n",
        "- Both streams are based on a U-shaped encoder-decoder structure to facilitate efficient multi-scale context reasoning.\n",
        "\n",
        "2. Bidirectional Feature Interaction (BFI):\n",
        "\n",
        "Implements **bidirectional feature alignment and fusion** between the CNN and Transformer streams.\n",
        "Employs **Selective Feature Fusion (SFF)** for adaptive feature integration by learning channel weights.\n",
        "\n",
        "3. Fast Global Self-Attention:\n",
        "\n",
        "Utilizes a **kernelizable fast-attention mechanism** for the Transformer, reducing computational complexity to linear.\n",
        "\n",
        "4. Loss Functions:\n",
        "\n",
        "Combines pixel-wise reconstruction, adversarial, perceptual, and style losses to ensure inpainting quality and perceptual consistency."
      ],
      "metadata": {
        "id": "Ub2RaBV8WZ-L"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Channel allocation: Optimal performance when CNN and Transformer streams have equal importance.\n",
        "- Fusion methods: Bidirectional fusion outperforms unidirectional and unified-path approaches.\n",
        "- Specific fusion techniques: SFF surpasses element-wise addition and concatenation.\n",
        "- Number of random features: Optimal trade-off achieved with 72 orthogonal random features."
      ],
      "metadata": {
        "id": "RnduiCJ-ZoCn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Mask Creation Process**\n",
        "\n",
        "#### 1. **Purpose of Masking in Image Inpainting**\n",
        "   - Masks simulate corrupted regions by marking areas of an image for restoration.\n",
        "   - Masks represent regions with **value 1** (corrupted) and **value 0** (uncorrupted), facilitating selective processing during training.\n",
        "\n",
        "#### 2. **Mask Datasets**\n",
        "   - **Mask Set I**: Contains irregular shapes with various hole-to-image area ratios (10%–60%) to simulate real-world image corruption scenarios.\n",
        "   - **Mask Set II**: Focuses on **large-scale corruptions**, derived from a large mask sampling strategy, targeting challenges in **large-hole inpainting**.\n",
        "\n",
        "#### 3. **Techniques for Mask Creation**\n",
        "   - **Random Irregular Masks**:\n",
        "     - Generated using freehand-like curves and random polygons.\n",
        "     - Often involve **random rotations** and **flipping** for augmentation.\n",
        "   - **Large-Hole Masks**:\n",
        "     - Created by sampling large continuous regions, ensuring high diversity in shape and size.\n",
        "   - **Tools and Libraries**:\n",
        "     - Python libraries like **OpenCV** and **NumPy** for procedural generation of irregular shapes.\n",
        "     - **External mask datasets** for additional diversity, e.g., Mask datasets from previous works such as [29].\n",
        "\n",
        "---\n",
        "\n",
        "### **Model Architecture: BIDS-Net**\n",
        "\n",
        "#### 1. **Overall Structure**\n",
        "   - A **dual-stream network** combining **CNN** and **Transformer** models in a parallel design.\n",
        "   - Built on a **U-shaped encoder-decoder structure** for multi-scale feature extraction.\n",
        "\n",
        "#### 2. **Key Components**\n",
        "   - **CNN Stream**:\n",
        "     - Focus: Capturing **local patterns** for texture refinement.\n",
        "     - Built with **pre-activation residual blocks** for efficient and robust feature learning.\n",
        "   - **Transformer Stream**:\n",
        "     - Focus: Modeling **long-range contextual correlations**.\n",
        "     - Uses **fast global self-attention** for scalability and reduced computational overhead.\n",
        "   - **Bidirectional Feature Interaction (BFI)**:\n",
        "     - Bridges the CNN and Transformer streams with **feature alignment** and **adaptive fusion**.\n",
        "\n",
        "#### 3. **Detailed Implementation Steps**\n",
        "   - **Input Projection**:\n",
        "     - Corrupted images and masks are projected into separate feature spaces for the CNN and Transformer streams.\n",
        "     - Transformer features are downsampled to balance computational cost and performance.\n",
        "   - **Encoding Stage**:\n",
        "     - Each stream extracts features using **convolutional blocks (CNN)** and **Transformer blocks**.\n",
        "     - Features are fused bidirectionally via the **BFI module**.\n",
        "   - **Bottleneck Stage**:\n",
        "     - Features from both streams interact for enhanced context reasoning at the lowest spatial resolution.\n",
        "   - **Decoding Stage**:\n",
        "     - Outputs from both streams are upsampled and concatenated for final refinement.\n",
        "   - **Output Projection**:\n",
        "     - Combined features are transformed back to the image space for inpainting results.\n",
        "\n",
        "---\n",
        "\n",
        "### **Relevant Techniques and Algorithms**\n",
        "\n",
        "#### 1. **Fast Global Self-Attention**\n",
        "   - Reduces standard attention's quadratic complexity to linear using:\n",
        "     - **Kernelizable Attention**: Positive orthogonal random features replace softmax attention.\n",
        "     - Ensures **scalability** and efficiency for high-resolution images.\n",
        "\n",
        "#### 2. **Selective Feature Fusion (SFF)**\n",
        "   - Adapts weights for each channel during fusion, ensuring:\n",
        "     - CNN benefits from Transformer’s global context.\n",
        "     - Transformer incorporates CNN’s local details.\n",
        "   - Based on the **Selective Kernel Convolution** technique.\n",
        "\n",
        "#### 3. **Loss Functions**\n",
        "   - **Pixel-wise Reconstruction Loss**: Ensures pixel-level consistency.\n",
        "   - **Adversarial Loss**: Improves texture realism by incorporating a discriminator network.\n",
        "   - **Perceptual Loss**: Derived from a pre-trained VGG-19, enhancing perceptual similarity.\n",
        "   - **Style Loss**: Preserves stylistic details using Gram matrices.\n",
        "\n",
        "---\n",
        "\n",
        "### **Tools and Libraries**\n",
        "   - **Frameworks**: PyTorch (1.10.1), TensorFlow for alternate implementations.\n",
        "   - **Visualization**: Matplotlib or OpenCV for displaying masks and inpainted results.\n",
        "   - **GPU Hardware**: Tested on NVIDIA GeForce RTX 3090 for performance.\n",
        "\n",
        "---\n",
        "\n",
        "### **Considerations**\n",
        "   - **Mask Diversity**: Critical for generalization across various corruption scenarios.\n",
        "   - **Computational Efficiency**: Striking a balance between accuracy and runtime, particularly with Transformer integration.\n",
        "   - **Evaluation Metrics**:\n",
        "     - Quantitative: PSNR, SSIM, FID, LPIPS.\n",
        "     - Qualitative: Visual coherence and texture consistency.\n",
        "\n"
      ],
      "metadata": {
        "id": "8WqwVU4dfuIh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Dataset"
      ],
      "metadata": {
        "id": "Ax71wBu70bTv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install datasets -q"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sFeJjh1M0dJO",
        "outputId": "23baa99c-53f8-4b43-9c43-4dc4e5c46dff"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/480.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m480.6/480.6 kB\u001b[0m \u001b[31m32.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/116.3 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m12.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/179.3 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m179.3/179.3 kB\u001b[0m \u001b[31m18.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/143.5 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.5/143.5 kB\u001b[0m \u001b[31m14.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/194.8 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.8/194.8 kB\u001b[0m \u001b[31m20.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "gcsfs 2024.10.0 requires fsspec==2024.10.0, but you have fsspec 2024.9.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "import glob\n",
        "import random\n",
        "from PIL import Image\n",
        "from datasets import Dataset, DatasetDict\n",
        "import torchvision.transforms as transforms\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "from torch.utils.data import Dataset, DataLoader, random_split\n",
        "from torchvision import transforms\n",
        "import cv2\n",
        "from tqdm import tqdm\n",
        "import wandb"
      ],
      "metadata": {
        "id": "5BNpU_sl0joJ"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_name=\"caltech256-BIDS\"\n",
        "\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')\n",
        "\n",
        "import os\n",
        "\n",
        "CHECKPOINTS_DIR = '/content/drive/MyDrive/ckpts'\n",
        "\n",
        "def save_checkpoint(model, optimizer, epoch):\n",
        "    os.makedirs(CHECKPOINTS_DIR, exist_ok=True)\n",
        "    checkpoint_path = f'{CHECKPOINTS_DIR}/{model_name}.pth'\n",
        "    torch.save({\n",
        "        'epoch': epoch,\n",
        "        'model_state_dict': model.state_dict(),\n",
        "        'optimizer_state_dict': optimizer.state_dict(),\n",
        "    }, checkpoint_path)\n",
        "    print(f\"ckpt saved for {model_name} at epoch {epoch}.\")\n",
        "\n",
        "def load_checkpoint(model, optimizer):\n",
        "    ckpt_path = f'{CHECKPOINTS_DIR}/{model_name}.pth'\n",
        "    if not os.path.exists(ckpt_path):\n",
        "        print(f\"no ckpt found for {model_name} starting from epoch 0.\")\n",
        "        return 0\n",
        "\n",
        "    checkpoint = torch.load(ckpt_path)\n",
        "    model.load_state_dict(checkpoint['model_state_dict'])\n",
        "    optimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n",
        "    start_epoch = checkpoint['epoch'] + 1\n",
        "    print(f\"ckpt loaded for {model_name} from {ckpt_path}. resuming from epoch {start_epoch}.\")\n",
        "\n",
        "    return start_epoch"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IN73lAG-0ux8",
        "outputId": "16da645d-5346-491d-aee3-5f0a92a93735"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import kagglehub\n",
        "# Download latest version\n",
        "path = kagglehub.dataset_download(\"jessicali9530/caltech256\")\n",
        "print(\"Path to dataset files:\", path)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hdg8NxId0utK",
        "outputId": "d3412496-48d0-4260-adb2-3650ad3d837c"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading from https://www.kaggle.com/api/v1/datasets/download/jessicali9530/caltech256?dataset_version_number=2...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 2.12G/2.12G [01:51<00:00, 20.4MB/s]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting files...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Path to dataset files: /root/.cache/kagglehub/datasets/jessicali9530/caltech256/versions/2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Complete Image Inpainting Implementation\n",
        "\n"
      ],
      "metadata": {
        "id": "6PmoDqKU5X76"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install wandb"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fLrToOiI6tTN",
        "outputId": "352f0d0e-c79f-44db-e09f-307986250a77"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: wandb in /usr/local/lib/python3.11/dist-packages (0.19.4)\n",
            "Requirement already satisfied: click!=8.0.0,>=7.1 in /usr/local/lib/python3.11/dist-packages (from wandb) (8.1.8)\n",
            "Requirement already satisfied: docker-pycreds>=0.4.0 in /usr/local/lib/python3.11/dist-packages (from wandb) (0.4.0)\n",
            "Requirement already satisfied: gitpython!=3.1.29,>=1.0.0 in /usr/local/lib/python3.11/dist-packages (from wandb) (3.1.44)\n",
            "Requirement already satisfied: platformdirs in /usr/local/lib/python3.11/dist-packages (from wandb) (4.3.6)\n",
            "Requirement already satisfied: protobuf!=4.21.0,!=5.28.0,<6,>=3.19.0 in /usr/local/lib/python3.11/dist-packages (from wandb) (4.25.5)\n",
            "Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.11/dist-packages (from wandb) (5.9.5)\n",
            "Requirement already satisfied: pydantic<3,>=2.6 in /usr/local/lib/python3.11/dist-packages (from wandb) (2.10.5)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from wandb) (6.0.2)\n",
            "Requirement already satisfied: requests<3,>=2.0.0 in /usr/local/lib/python3.11/dist-packages (from wandb) (2.32.3)\n",
            "Requirement already satisfied: sentry-sdk>=2.0.0 in /usr/local/lib/python3.11/dist-packages (from wandb) (2.20.0)\n",
            "Requirement already satisfied: setproctitle in /usr/local/lib/python3.11/dist-packages (from wandb) (1.3.4)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.11/dist-packages (from wandb) (75.1.0)\n",
            "Requirement already satisfied: typing-extensions<5,>=4.4 in /usr/local/lib/python3.11/dist-packages (from wandb) (4.12.2)\n",
            "Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.11/dist-packages (from docker-pycreds>=0.4.0->wandb) (1.17.0)\n",
            "Requirement already satisfied: gitdb<5,>=4.0.1 in /usr/local/lib/python3.11/dist-packages (from gitpython!=3.1.29,>=1.0.0->wandb) (4.0.12)\n",
            "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.11/dist-packages (from pydantic<3,>=2.6->wandb) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.27.2 in /usr/local/lib/python3.11/dist-packages (from pydantic<3,>=2.6->wandb) (2.27.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.0.0->wandb) (3.4.1)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.0.0->wandb) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.0.0->wandb) (2.3.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests<3,>=2.0.0->wandb) (2024.12.14)\n",
            "Requirement already satisfied: smmap<6,>=3.0.1 in /usr/local/lib/python3.11/dist-packages (from gitdb<5,>=4.0.1->gitpython!=3.1.29,>=1.0.0->wandb) (5.0.2)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 1. Constants and Configurations\n"
      ],
      "metadata": {
        "id": "UwXKh2fvB6OO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "config = {\n",
        "    'BATCH_SIZE': 2,\n",
        "    'EPOCHS': 10,\n",
        "    'LEARNING_RATE': 0.001,\n",
        "    'IMAGE_SIZE': 128, # 'image_size' is defined here\n",
        "    'TRAIN_SPLIT': 0.8,\n",
        "    'num_heads': 4,  # Fewer attention heads to reduce memory\n",
        "\n",
        "    'checkpoint_dir': 'checkpoints',\n",
        "    'results_dir': 'results',\n",
        "    'accumulation_steps': 4,  # New parameter for gradient accumulation\n",
        "    'SAVE_INTERVAL': 5,\n",
        "    'num_workers': 2 # Add the num_workers key to the config\n",
        "}"
      ],
      "metadata": {
        "id": "ymLw4YDBB4PR"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 2. Dataset and Mask Generation"
      ],
      "metadata": {
        "id": "9vDfz6YkCIac"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class Caltech256Dataset(Dataset):\n",
        "    def __init__(self, root_dir, transform=None):\n",
        "        self.root_dir = root_dir\n",
        "        self.transform = transform\n",
        "        # self.image_size = image_size\n",
        "        self.image_paths = glob.glob(os.path.join(root_dir, \"**/*.jpg\"), recursive=True)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.image_paths)\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        img_path = self.image_paths[idx]\n",
        "        image = Image.open(img_path).convert('RGB')\n",
        "\n",
        "        if self.transform:\n",
        "            image = self.transform(image)\n",
        "\n",
        "        # Correct height and width extraction\n",
        "        height, width = image.shape[1], image.shape[2]  # Corrected line\n",
        "        mask = generate_random_mask(height, width)\n",
        "        mask = torch.from_numpy(mask).float()\n",
        "\n",
        "        return image, mask\n",
        "\n",
        "def generate_random_mask(height, width):\n",
        "    if random.random() > 0.5:\n",
        "        return generate_irregular_mask(height, width)\n",
        "    return generate_large_hole_mask(height, width)\n",
        "\n",
        "def generate_irregular_mask(height, width, max_vertices=12, max_brush_width=50):\n",
        "    mask = np.zeros((height, width), dtype=np.uint8)\n",
        "    num_vertices = random.randint(3, max_vertices)\n",
        "    vertices = np.array([[\n",
        "        random.randint(0, width),\n",
        "        random.randint(0, height)\n",
        "    ] for _ in range(num_vertices)], dtype=np.int32)\n",
        "    cv2.fillPoly(mask, [vertices], 1)\n",
        "\n",
        "    for _ in range(random.randint(1, 5)):\n",
        "        start_point = (random.randint(0, width), random.randint(0, height))\n",
        "        end_point = (random.randint(0, width), random.randint(0, height))\n",
        "        thickness = random.randint(10, max_brush_width)\n",
        "        cv2.line(mask, start_point, end_point, 1, thickness)\n",
        "\n",
        "    return mask\n",
        "\n",
        "def generate_large_hole_mask(height, width, min_size=0.3, max_size=0.6):\n",
        "    mask = np.zeros((height, width), dtype=np.uint8)\n",
        "    hole_size = random.uniform(min_size, max_size)\n",
        "    hole_height = int(height * hole_size)\n",
        "    hole_width = int(width * hole_size)\n",
        "\n",
        "    x = random.randint(0, width - hole_width)\n",
        "    y = random.randint(0, height - hole_height)\n",
        "    cv2.rectangle(mask, (x, y), (x + hole_width, y + hole_height), 1, -1)\n",
        "\n",
        "    return mask\n"
      ],
      "metadata": {
        "id": "3RPPYyEkCEW-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 3. Model Architecture\n",
        "\n",
        "\n",
        "class PreActResidualBlock(nn.Module):\n",
        "    def __init__(self, in_channels, out_channels, downsample=False):\n",
        "        super(PreActResidualBlock, self).__init__()\n",
        "        stride = 2 if downsample else 1\n",
        "        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1, stride=stride)\n",
        "        self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1)\n",
        "        self.norm1 = nn.BatchNorm2d(in_channels)\n",
        "        self.norm2 = nn.BatchNorm2d(out_channels)\n",
        "        self.relu = nn.ReLU(inplace=True)\n",
        "\n",
        "        # Skip connection with downsampling if needed\n",
        "        self.skip = nn.Sequential(\n",
        "            nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride),\n",
        "            nn.BatchNorm2d(out_channels)\n",
        "        ) if in_channels != out_channels or downsample else nn.Identity()\n",
        "\n",
        "    def forward(self, x):\n",
        "        identity = self.skip(x)\n",
        "\n",
        "        out = self.norm1(x)\n",
        "        out = self.relu(out)\n",
        "        out = self.conv1(out)\n",
        "\n",
        "        out = self.norm2(out)\n",
        "        out = self.relu(out)\n",
        "        out = self.conv2(out)\n",
        "\n",
        "        return out + identity\n",
        "\n",
        "class BIDS_Net(nn.Module):\n",
        "    def __init__(self, in_channels=3, out_channels=3, num_heads=4):  # Fewer heads\n",
        "        super(BIDS_Net, self).__init__()\n",
        "\n",
        "        # CNN Stream with downsampling\n",
        "        self.cnn_stream = nn.ModuleList([\n",
        "            PreActResidualBlock(in_channels, 64),\n",
        "            PreActResidualBlock(64, 128, downsample=True),  # Downsample to 64x64\n",
        "            PreActResidualBlock(128, 256, downsample=True)  # Downsample to 32x32\n",
        "        ])\n",
        "\n",
        "        # Transformer Stream with downsampling\n",
        "        self.trans_proj = nn.Conv2d(in_channels, 64, kernel_size=3, stride=2, padding=1)  # Initial downsampling\n",
        "        self.trans_stream = nn.ModuleList([\n",
        "            FastGlobalSelfAttention(64),\n",
        "            FastGlobalSelfAttention(128),\n",
        "            FastGlobalSelfAttention(256)\n",
        "        ])\n",
        "\n",
        "        # Adjust channel adjustments to handle downsampled features\n",
        "        self.trans_ch_adj = nn.ModuleList([\n",
        "            nn.Conv2d(64, 128, kernel_size=3, stride=2, padding=1),  # Downsample\n",
        "            nn.Conv2d(128, 256, kernel_size=3, stride=2, padding=1)  # Downsample\n",
        "        ])\n",
        "        # BFI Modules\n",
        "        self.bfi1 = BFI(64)\n",
        "        self.bfi2 = BFI(128)\n",
        "        self.bfi3 = BFI(256)\n",
        "\n",
        "        # Output Layer\n",
        "        self.output = nn.Conv2d(256, out_channels, kernel_size=1)\n",
        "\n",
        "\n",
        "    def forward(self, x, mask):\n",
        "        # CNN Stream\n",
        "        cnn_feat1 = self.cnn_stream[0](x)\n",
        "        cnn_feat2 = self.cnn_stream[1](cnn_feat1)\n",
        "        cnn_feat3 = self.cnn_stream[2](cnn_feat2)\n",
        "\n",
        "        # Transformer Stream\n",
        "        trans_feat = self.trans_proj(x)\n",
        "        trans_feat1 = self.trans_stream[0](trans_feat)\n",
        "\n",
        "        trans_feat = self.trans_ch_adj[0](trans_feat1)\n",
        "        trans_feat2 = self.trans_stream[1](trans_feat)\n",
        "\n",
        "        trans_feat = self.trans_ch_adj[1](trans_feat2)\n",
        "        trans_feat3 = self.trans_stream[2](trans_feat)\n",
        "\n",
        "        # BFI Fusion\n",
        "        fused1 = self.bfi1(cnn_feat1, trans_feat1)\n",
        "        fused2 = self.bfi2(cnn_feat2, trans_feat2)\n",
        "        fused3 = self.bfi3(cnn_feat3, trans_feat3)\n",
        "\n",
        "        # Output\n",
        "        out = self.output(fused3)\n",
        "        return out * mask + x * (1 - mask) # Applying the mask to combine inpainted and original parts\n",
        "\n",
        "class FastGlobalSelfAttention(nn.Module):\n",
        "    def __init__(self, dim, num_heads):\n",
        "        super(FastGlobalSelfAttention, self).__init__()\n",
        "        self.num_heads = num_heads\n",
        "        self.scale = (dim // num_heads) ** -0.5\n",
        "        self.to_qkv = nn.Linear(dim, dim * 3)\n",
        "        self.to_out = nn.Linear(dim, dim)\n",
        "\n",
        "    def forward(self, x):\n",
        "        B, C, H, W = x.shape\n",
        "        x = x.flatten(2).transpose(1, 2)  # (B, H*W, C)\n",
        "\n",
        "        qkv = self.to_qkv(x).chunk(3, dim=-1)\n",
        "        q, k, v = map(lambda t: t.view(B, -1, self.num_heads, C // self.num_heads).transpose(1, 2), qkv)\n",
        "\n",
        "        attn = (q @ k.transpose(-2, -1)) * self.scale\n",
        "        attn = attn.softmax(dim=-1)\n",
        "\n",
        "        out = (attn @ v).transpose(1, 2).reshape(B, H * W, C)\n",
        "        out = self.to_out(out)\n",
        "        out = out.transpose(1, 2).view(B, C, H, W)\n",
        "\n",
        "        return out\n",
        "\n",
        "class BFI(nn.Module):\n",
        "    def __init__(self, channels):\n",
        "        super(BFI, self).__init__()\n",
        "        self.cnn_norm = nn.BatchNorm2d(channels)\n",
        "        self.trans_norm = nn.BatchNorm2d(channels)\n",
        "        self.fusion = nn.Conv2d(channels * 2, channels, kernel_size=1)\n",
        "\n",
        "    def forward(self, cnn_feat, trans_feat):\n",
        "        if cnn_feat.size() != trans_feat.size():\n",
        "            raise ValueError(\"Mismatched feature dimensions in BFI.\")\n",
        "        cnn_feat = self.cnn_norm(cnn_feat)\n",
        "        trans_feat = self.trans_norm(trans_feat)\n",
        "        fused = torch.cat([cnn_feat, trans_feat], dim=1)\n",
        "        return self.fusion(fused)\n",
        "\n",
        "\n",
        "    def forward(self, x, mask):\n",
        "        # CNN Stream\n",
        "        cnn_feat1 = self.cnn_stream[0](x)\n",
        "        cnn_feat2 = self.cnn_stream[1](cnn_feat1)\n",
        "        cnn_feat3 = self.cnn_stream[2](cnn_feat2)\n",
        "\n",
        "        # Transformer Stream\n",
        "        trans_feat = self.trans_proj(x)\n",
        "        trans_feat1 = self.trans_stream[0](trans_feat)\n",
        "\n",
        "        trans_feat = self.trans_ch_adj[0](trans_feat1)\n",
        "        trans_feat2 = self.trans_stream[1](trans_feat)\n",
        "\n",
        "        trans_feat = self.trans_ch_adj[1](trans_feat2)\n",
        "        trans_feat3 = self.trans_stream[2](trans_feat)\n",
        "\n",
        "        # BFI Fusion\n",
        "        fused1 = self.bfi1(cnn_feat1, trans_feat1)\n",
        "        fused2 = self.bfi2(cnn_feat2, trans_feat2)\n",
        "        fused3 = self.bfi3(cnn_feat3, trans_feat3)\n",
        "\n",
        "        # Output\n",
        "        out = self.output(fused3)\n",
        "        return out * mask + x * (1 - mask)"
      ],
      "metadata": {
        "id": "7avkeJ_cCk24"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 4. Training and Evaluation Functions\n",
        "def train_one_epoch(model, dataloader, optimizer, criterion, device):\n",
        "    model.train()\n",
        "    total_loss = 0\n",
        "    pbar = tqdm(dataloader, desc='Training')\n",
        "\n",
        "    for images, masks in pbar:\n",
        "        images = images.to(device)\n",
        "        masks = masks.to(device)\n",
        "\n",
        "        masked_images = images * (1 - masks.unsqueeze(1))\n",
        "\n",
        "        optimizer.zero_grad()\n",
        "        outputs = model(masked_images, masks.unsqueeze(1))\n",
        "        loss = criterion(outputs, images)\n",
        "\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "\n",
        "        total_loss += loss.item()\n",
        "        pbar.set_postfix({'loss': loss.item()})\n",
        "\n",
        "    return total_loss / len(dataloader)\n",
        "\n",
        "def validate(model, dataloader, criterion, device):\n",
        "    model.eval()\n",
        "    total_loss = 0\n",
        "\n",
        "    with torch.no_grad():\n",
        "        for images, masks in dataloader:\n",
        "            images = images.to(device)\n",
        "            masks = masks.to(device)\n",
        "\n",
        "            masked_images = images * (1 - masks.unsqueeze(1))\n",
        "            outputs = model(masked_images, masks.unsqueeze(1))\n",
        "            loss = criterion(outputs, images)\n",
        "\n",
        "            total_loss += loss.item()\n",
        "\n",
        "    return total_loss / len(dataloader)"
      ],
      "metadata": {
        "id": "JT7b3onAC4CL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 5. Checkpoint Functions\n",
        "def save_checkpoint(model, optimizer, epoch, best_loss, path):\n",
        "    torch.save({\n",
        "        'epoch': epoch,\n",
        "        'model_state_dict': model.state_dict(),\n",
        "        'optimizer_state_dict': optimizer.state_dict(),\n",
        "        'best_loss': best_loss\n",
        "    }, path)\n",
        "\n",
        "def load_checkpoint(model, optimizer, path):\n",
        "    if not os.path.exists(path):\n",
        "        return 0, float('inf')\n",
        "\n",
        "    checkpoint = torch.load(path)\n",
        "    model.load_state_dict(checkpoint['model_state_dict'])\n",
        "    optimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n",
        "    return checkpoint['epoch'] + 1, checkpoint['best_loss']"
      ],
      "metadata": {
        "id": "aBcxQrthDFnu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 6. Visualization Functions\n",
        "def visualize_results(model, dataloader, device, save_path=None):\n",
        "    model.eval()\n",
        "    denorm = transforms.Normalize(\n",
        "        mean=[-0.485 / 0.229, -0.456 / 0.224, -0.406 / 0.225],\n",
        "        std=[1 / 0.229, 1 / 0.224, 1 / 0.225]\n",
        "    )\n",
        "\n",
        "\n",
        "    with torch.no_grad():\n",
        "        images, masks = next(iter(dataloader))\n",
        "        images = images[:5].to(device)  # Take first 5 samples\n",
        "        masks = masks[:5].to(device)\n",
        "\n",
        "        masked_images = images * (1 - masks.unsqueeze(1))\n",
        "        outputs = model(masked_images, masks.unsqueeze(1))\n",
        "\n",
        "        # Denormalize images\n",
        "        images = denorm(images).cpu()\n",
        "        masked_images = denorm(masked_images).cpu()\n",
        "        outputs = denorm(outputs).cpu()\n",
        "\n",
        "        # Create visualization\n",
        "        fig, axes = plt.subplots(5, 3, figsize=(12, 20))\n",
        "\n",
        "        for i in range(5):\n",
        "            axes[i, 0].imshow(images[i].permute(1, 2, 0).clip(0, 1))\n",
        "            axes[i, 0].set_title('Original')\n",
        "            axes[i, 1].imshow(masked_images[i].permute(1, 2, 0).clip(0, 1))\n",
        "            axes[i, 1].set_title('Masked')\n",
        "            axes[i, 2].imshow(outputs[i].permute(1, 2, 0).clip(0, 1))\n",
        "            axes[i, 2].set_title('Inpainted')\n",
        "\n",
        "            for ax in axes[i]:\n",
        "                ax.axis('off')\n",
        "\n",
        "        plt.tight_layout()\n",
        "\n",
        "        if save_path:\n",
        "            plt.savefig(save_path)\n",
        "            plt.close()\n",
        "        else:\n",
        "            plt.show()\n"
      ],
      "metadata": {
        "id": "YX2M-KKTDS_A"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "path = kagglehub.dataset_download(\"jessicali9530/caltech256\")\n",
        "print(\"Path to dataset files:\", path)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1UfTZZzjFMJo",
        "outputId": "d31e0d4e-9282-4890-c852-4fe93cd62d5b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Path to dataset files: /root/.cache/kagglehub/datasets/jessicali9530/caltech256/versions/2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(path)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Gp1X40NLFSno",
        "outputId": "76509bab-5c50-4f3f-f7e2-dcb71a394a2c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/root/.cache/kagglehub/datasets/jessicali9530/caltech256/versions/2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def main():\n",
        "    # Memory optimization settings\n",
        "    torch.cuda.empty_cache()  # Clear cache before starting\n",
        "    os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'\n",
        "    # Define device\n",
        "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "    # Initialize wandb with lower log frequency\n",
        "    wandb.init(project=\"image-inpainting\",\n",
        "               name=\"BIDS-Net-training\",\n",
        "               config=config)\n",
        "\n",
        "    # Data transforms with smaller image size\n",
        "    transform = transforms.Compose([\n",
        "        transforms.Resize((config['IMAGE_SIZE'], config['IMAGE_SIZE'])),\n",
        "        transforms.ToTensor(),\n",
        "        transforms.Normalize(mean=[0.485, 0.456, 0.406],\n",
        "                           std=[0.229, 0.224, 0.225])\n",
        "    ])\n",
        "    # Define the dataset\n",
        "    dataset = Caltech256Dataset(root_dir=path, transform=transform) # Assuming 'path' contains the dataset path\n",
        "\n",
        "\n",
        "\n",
        "    train_size = int(0.8 * len(dataset))\n",
        "    val_size = len(dataset) - train_size\n",
        "    train_dataset, val_dataset = random_split(dataset, [train_size, val_size])\n",
        "\n",
        "    train_loader = DataLoader(\n",
        "        train_dataset,\n",
        "        batch_size=config['BATCH_SIZE'],\n",
        "        shuffle=True,\n",
        "        num_workers=config['num_workers'],\n",
        "        pin_memory=False,  # Disabled pin_memory to reduce memory usage\n",
        "        persistent_workers=True  # Keep workers alive between batches\n",
        "    )\n",
        "\n",
        "    val_loader = DataLoader(\n",
        "        val_dataset,\n",
        "        batch_size=config['BATCH_SIZE'],\n",
        "        shuffle=False,\n",
        "        num_workers=config['num_workers'],\n",
        "        pin_memory=False,\n",
        "        persistent_workers=True\n",
        "    )\n",
        "\n",
        "    # Initialize model with memory optimizations\n",
        "    model = BIDS_Net().to(device)\n",
        "    if torch.cuda.device_count() > 1:\n",
        "        model = nn.DataParallel(model)\n",
        "\n",
        "    optimizer = optim.Adam(model.parameters(), lr=config['LEARNING_RATE'])\n",
        "    criterion = nn.L1Loss()\n",
        "    scaler = torch.cuda.amp.GradScaler()  # For mixed precision training\n",
        "\n",
        "\n",
        "    # Modified training loop with memory optimizations\n",
        "    for epoch in range(config['EPOCHS']):\n",
        "        model.train()\n",
        "        running_loss = 0.0\n",
        "\n",
        "        for i, (images, masks) in enumerate(train_loader):\n",
        "            images = images.to(device)\n",
        "            masks = masks.to(device).unsqueeze(1)\n",
        "\n",
        "            with torch.cuda.amp.autocast():\n",
        "                masked_images = images * (1 - masks)\n",
        "                outputs = model(masked_images, masks)\n",
        "                loss = criterion(outputs, images) / config['accumulation_steps']\n",
        "\n",
        "            scaler.scale(loss).backward()\n",
        "\n",
        "            if (i + 1) % config['accumulation_steps'] == 0:\n",
        "                scaler.step(optimizer)\n",
        "                scaler.update()\n",
        "                optimizer.zero_grad()\n",
        "\n",
        "            running_loss += loss.item() * config['accumulation_steps']\n",
        "\n",
        "            # Clear cache periodically\n",
        "            if i % 10 == 0:\n",
        "                torch.cuda.empty_cache()\n",
        "\n",
        "            # Log less frequently\n",
        "            if i % 100 == 0:\n",
        "                print(f'Epoch [{epoch+1}/{config[\"epochs\"]}], '\n",
        "                      f'Step [{i+1}/{len(train_loader)}], '\n",
        "                      f'Loss: {running_loss/(i+1):.4f}')\n",
        "\n",
        "        # Validation with memory optimization\n",
        "        model.eval()\n",
        "        val_loss = 0.0\n",
        "        with torch.no_grad():\n",
        "            for images, masks in val_loader:\n",
        "                images = images.to(device)\n",
        "                masks = masks.to(device)\n",
        "\n",
        "                with torch.cuda.amp.autocast():\n",
        "                    masked_images = images * (1 - masks.unsqueeze(1))\n",
        "                    outputs = model(masked_images, masks.unsqueeze(1))\n",
        "                    loss = criterion(outputs, images)\n",
        "\n",
        "                val_loss += loss.item()\n",
        "\n",
        "                # Clear cache periodically\n",
        "                torch.cuda.empty_cache()\n",
        "\n",
        "        val_loss /= len(val_loader)\n",
        "\n",
        "        # Save checkpoint with reduced frequency\n",
        "        if (epoch + 1) % 5 == 0:\n",
        "            torch.save({\n",
        "                'epoch': epoch,\n",
        "                'model_state_dict': model.state_dict(),\n",
        "                'optimizer_state_dict': optimizer.state_dict(),\n",
        "                'loss': running_loss / len(train_loader),\n",
        "            }, f'{config[\"checkpoint_dir\"]}/checkpoint_epoch_{epoch+1}.pth')\n",
        "\n",
        "        # Visualize results less frequently\n",
        "        if (epoch + 1) % 10 == 0:  # Changed from 5 to 10\n",
        "            visualize_results(model, val_loader, device, epoch, config['results_dir'])\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        },
        "id": "JAeq1Awg5bwo",
        "outputId": "3892f785-8d29-43dc-958d-15392f54d1fc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-67-525c14c81f5a>:54: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.\n",
            "  scaler = torch.cuda.amp.GradScaler()  # For mixed precision training\n",
            "<ipython-input-67-525c14c81f5a>:66: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.\n",
            "  with torch.cuda.amp.autocast():\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "'BFI' object has no attribute 'cnn_stream'",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-67-525c14c81f5a>\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m    122\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    123\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0m__name__\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"__main__\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 124\u001b[0;31m     \u001b[0mmain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m<ipython-input-67-525c14c81f5a>\u001b[0m in \u001b[0;36mmain\u001b[0;34m()\u001b[0m\n\u001b[1;32m     66\u001b[0m             \u001b[0;32mwith\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcuda\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mamp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mautocast\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     67\u001b[0m                 \u001b[0mmasked_images\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mimages\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mmasks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 68\u001b[0;31m                 \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmasked_images\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmasks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     69\u001b[0m                 \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcriterion\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mimages\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'accumulation_steps'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     70\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1734\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiled_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[misc]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1735\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1736\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1737\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1738\u001b[0m     \u001b[0;31m# torchrec tests the code consistency with the following code\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1745\u001b[0m                 \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_pre_hooks\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_hooks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1746\u001b[0m                 or _global_forward_hooks or _global_forward_pre_hooks):\n\u001b[0;32m-> 1747\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mforward_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1748\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1749\u001b[0m         \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-66-7516bf54a27d>\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, x, mask)\u001b[0m\n\u001b[1;32m     81\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     82\u001b[0m         \u001b[0;31m# BFI Fusion\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 83\u001b[0;31m         \u001b[0mfused1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbfi1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcnn_feat1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrans_feat1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     84\u001b[0m         \u001b[0mfused2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbfi2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcnn_feat2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrans_feat2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     85\u001b[0m         \u001b[0mfused3\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbfi3\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcnn_feat3\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrans_feat3\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1734\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiled_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[misc]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1735\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1736\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1737\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1738\u001b[0m     \u001b[0;31m# torchrec tests the code consistency with the following code\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1745\u001b[0m                 \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_pre_hooks\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_hooks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1746\u001b[0m                 or _global_forward_hooks or _global_forward_pre_hooks):\n\u001b[0;32m-> 1747\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mforward_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1748\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1749\u001b[0m         \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-66-7516bf54a27d>\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, x, mask)\u001b[0m\n\u001b[1;32m    131\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmask\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    132\u001b[0m         \u001b[0;31m# CNN Stream\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 133\u001b[0;31m         \u001b[0mcnn_feat1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcnn_stream\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    134\u001b[0m         \u001b[0mcnn_feat2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcnn_stream\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcnn_feat1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    135\u001b[0m         \u001b[0mcnn_feat3\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcnn_stream\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcnn_feat2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m__getattr__\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m   1929\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmodules\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1930\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mmodules\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1931\u001b[0;31m         raise AttributeError(\n\u001b[0m\u001b[1;32m   1932\u001b[0m             \u001b[0;34mf\"'{type(self).__name__}' object has no attribute '{name}'\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1933\u001b[0m         )\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'BFI' object has no attribute 'cnn_stream'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA54AAAMfCAYAAAC+YwiaAAAABHNCSVQICAgIfAhkiAAAABl0RVh0U29mdHdhcmUAZ25vbWUtc2NyZWVuc2hvdO8Dvz4AAAAvdEVYdENyZWF0aW9uIFRpbWUAU3VuIDEyIEphbiAyMDI1IDAxOjQ2OjAzIFBNICswMzMwMpxSJgAAIABJREFUeJzs3WdgVGXC9vF/em8EQgogKfQOAVEQBIUAIkUREEFQFEVFKa7YKIIoK1VRUUT02VVsCIiiFFfBQhEwEIqBFGp6SA8pk8y8H1zmXaQGMpwkXr9POHPKdSYzMdfc9znHzmKxWBARERERERGxEXujA4iIiIiIiEjNpuIpIiIiIiIiNqXiKSIiIiIiIjal4ikiIiIiIiI25Wh0ABERqRni9xaQkVRqdIway8XVnva3+RodQ0RE5KqoeIqISKWxc3TAN8DZ6Bg1jqnEjKnQZHQMERGRq6biKSIilapOAzejI9Q4yXGFRkcQERG5JjrHU0RERERERGxKxVNERERERERsSsVTREREREREbErFU0RERERERGxKxVNERERERERsSsVTREREREREbErFU0RERERERGxKxVNERERERERsSsVTREREREREbErFU0RERERERGxKxVNERERERERsSsVTRESqpDFjBmEylV7zdjIz03nrrddstn0RERG5PEejA4iIiFzKkiVzcXJywsPDk/37o3nyyedwdXVjxozJtG9/I7m52fj5+fPAA48zZswgPvxwLQCvvvo8w4aNYe3aT4mNPcCOHT/RuXO387a/detmtm7dRFhYY/bv/53IyJsoLS1l377dPPPMLDw8PJk3bwZubu5kZqYzbtxEwsObMG/eDCwWC7Vq1Wbbti2sWLGapKQTLFu2GF9fP4qKzjBhwnN4eXlf3xdMRESkCtKIp4iIVGn29vaEhjZi1KhH6N69Fzt3/oyDgwO5udmMGfMYkydPZ/fu7eTmZl9w/R49+tCmTeQFS+fZ7Xt6ejFy5MN06dKD7Ows7rvvITp06ExMzB5yc3Po23cQ//jHSwwcOIwNG77iyJFDFBWd4ZlnZnHnnfeQkZEGwIoVbzJ06GgmTZpGp05dWbfuc5u9LiIiItWJRjxFRKTKCwgIBMDV1Y3c3BzrYw4ODgD4+fmTnZ111duvVas2AE5OztSq5W/9d0lJCU5OTmzbtpXdu7eTmZmOp6cXGRlpBAYGA1C3bhDe3j4AJCefYt26z/nuuzWUlpYQGBhy1ZlERERqEhVPERGpllJTkzGbzdjb25OZmUatWrVxcnLCZCrFycmZtLQU67Jms/mq97NmzSc0btycvn0HsX79lxw+fBA/P39On84A/jyHNC8vF4CQkPoMHXo/4eFNyMnJws7O7toOUkREpIZQ8RQRkWrJ19eP5ctfJzU1mZtu6o63tw9RUQN47bXp1K/fECcnZ8xmM8HB9YmO/o3Nm7+hV6/+Fd5P+/Y38tFHyzhxIpEGDUKJj4/FxcWFsrIyFi6cRUBAELVrBwAwZsxjLF/+Bp6eXuTkZPHwwxPx8fGr7EMXERGpduwsFovF6BAiIlL9xe8tIDOtnEaRPjbfV3b2aebMeY7585fZfF8XUlZWRnT0b3TseDMmUylPPDGKd9/9zGb7S44rxFRoov1tvjbbh4iIiC1pxFNERKSCHB0d2bZtCxs3foXJZGLkyHFGRxIREanSNOIpIiKV4nqOeP7daMRTRESqO91ORURERERERGxKxVNERERERERsSsVTREREREREbErFU0RERERERGxKxVNERERERERsSsVTREREREREbEr38RQRkWuSnZ1NYmIiB3eepnH4jUbHqbEOHNhPibsj4eHhBAQEGB1HRESkQlQ8RUSkwo4ePUpiYiIJCQkUFRURFhZGgwaNjI5VowWHhJCXF8eqVauwWCyEhYURHh5OWFgYjo7637mIiFRtdhaLxWJ0CBERqdry8vJISEggMTGRxMREAgICrMUnODgYgPi9BWSmldMo0sfgtDVPclwhpkIT7W/zBSA9PZ2jR4+SkJDAsWPHCA0NtZbQ2rVrG5xWRETkfCqeIiJyQSdOnCAxMZH9+/djMpmsxSYsLAwPD4/zlj9bPEMan/+cXJuctJJziuf/Ki0ttX4hkJiYiIODAw0bNqRRo0aEhYVhb6/LOYiIiPFUPEVEBIAzZ84QHx9PXFwcx48fx9fXl7CwMFxdXencufNl14/fW0D8vsLrkPTvKaCeywWL51/t37+f/Px8EhMTOXXqFGFhYdSpU4e2bdvi5+d3HZKKiIicT8VTRORvLCMjg4SEBOLj40lOTiY8PBwHBwd69uyJt7e30fHkGhUXF7Nnzx6ysrJITEzEw8OD8PBwwsPDadCggdHxRETkb0TFU0Tkb+bEiRPEx8dz+PBhysvLiYiIIDw8nIiICOzs7IyOJzZ06tQpEhISSEhIoKCggIYNG1rP1XVzczM6noiI1GAqniIiNVxJSQnHjh0jLi6O+Ph4vL29iYiIwNvbm7Zt2xodTwxy6tQp0tPTrReNCgkJITw8HB8fH5o3b250PBERqWFUPEVEaqiEhAQOHjzIsWPHqF27No0aNSIiIkLn+cl5LBaLdSQ0Pj4eJycnGjVqROPGjQkJCTE6noiI1AAqniIiNUhqaiqHDh3i4MGDeHt706xZM1q0aHHBq9CKXExSUhJHjhwhLi4Ok8lEo0aN8Pf3p0OHDkZHExGRakrFU0SkmisoKODgwYN8++231K1bl+bNm9O8eXMCAgKMjiY1QHp6OnFxccTExFBSUkLjxo1p1KgRjRo1MjqaiIhUIyqeIiLVVFxcHPv37+fIkSO0bNmSevXq6ZxNsans7Gzi4uI4cuQIKSkp1hLauHFjHB0djY4nIiJVmIqniEg1cubMGfbu3cu+fftwd3enVatWtGrVCicnJ6Ojyd9MQUEBR44cYdu2bZw5c4YWLVrQvHlzQkNDjY4mIiJVkIqniEg1kJqayu+//050dDRt27albdu2uuiLVBn5+fnWc4uLi4s13VtERM6j4ikiUoUlJiaye/duUlNTad++Pe3bt8fd3d3oWCIXlZKSwqFDh9i6dSvh4eE0b96cZs2a4enpaXQ0ERExkIqniEgVdPjwYXbt2kVRURGRkZG0a9fO6EgiFZaYmMj+/fs5ePAgLVu2pGXLloSFhRkdS0REDKDiKSJShcTFxbFjxw5MJhOdO3emefPmRkcSuWYmk4m9e/eyd+9eHBwcaNOmDa1atcLZ2dnoaCIicp2oeIqIVAGpqan8/PPPZGdn06VLF1q0aGF0JBGbOHbsGHv27OHEiRN06NCBtm3b4u3tbXQsERGxMRVPEREDmc1mtmzZwp49e+jevTudOnUyOpLIdZGVlcXOnTuJiYmhY8eOREZGqoCKiNRgKp4iIgaJj4/n+++/p169evTs2VMXDZK/pTNnzvDLL7+wd+9ebrrpJm688UZNwRURqYFUPEVEDPDjjz+yf/9+evfuTdOmTY2OI2K4vLw8fvjhB1JTU7nllls03VxEpIZR8RQRuY4KCgr4+uuvcXZ2pl+/fri5uRkdSaRKOXr0KJs2baJevXr06NFDMwFERGoIFU8RkeskOTmZtWvX0rJlS7p162Z0HJEqbePGjSQlJdGvXz8CAwONjiMiItdIxVNE5Do4evQoq1atIioqitatWxsdR6Ra2L17Nzt37mTIkCHUrVvX6DgiInIN7I0OICJS0x09epRPP/2UAQMGqHSKVEBkZCQdO3bk+++/NzqKiIhcIxVPEREbSktL48svv2Tw4ME0adLE6Dgi1U6nTp0oLy9n//79RkcREZFroOIpImIjpaWlfPXVV/Ts2VNXrhW5Bp06deKPP/4wOoaIiFwDFU8RERvZsGEDoaGhtG/f3ugoItVa06ZNOXr0KOnp6UZHERGRq6TiKSJiA/v37yczM5NevXoZHUWkRmjUqBGnTp0yOoaIiFwlR6MDiIjUNBaLha1bt9K/f3+jo4hUe1u3bqW4uJhTp04RHx9PRkYG9vb2+lJHRKSa0e1UREQq2Y4dO0hNTWXQoEFGRxGp9sxmMz179qSgoAAAJycnNm7ciLe3t8HJRESkIjTVVkSkku3Zs4dOnToZHUOkRrC3t6djx47W/+7YsaNKp4hINaTiKSJSieLi4vDy8iI4ONjoKCI1xuDBg/Hy8sLJyUkzCUREqimd4ykiUoni4uJ065RqxlxudAK5nDat29KuXQeKiopo17a9fmbVgL2D0QlEpKpR8RQRqUTHjx8nMjLS6BhSATkZpWxZlWF0DLmMrmGTsZjN/LiyECg0Oo5cQpc7/al7g6vRMUSkilHxFBGpJCUlJeTn5xMQEGB0FKkgr1rOtI+qY3QMkWov5odMoyOISBWlczxFRCpJRkYGdeqovIiIiIj8lYqniEglOX36NP7+/kbHEBEREalyVDxFRCpJVlYWtWrVMjqGiIiISJWj4ikiUklycnLw8/MzOoaIiIhIlaPiKSJSSXJzc/Hx8TE6hoiIiEiVo+IpIlJJ8vLy8Pb2NjqGiIiISJWj4ikiUkkKCgrw9PQ0OoaIiIhIlaPiKSJSCc6cOYOrqyv29vq1KiIiIvJX+gtJRKQSFBcX4+bmZnQMERERkSpJxVNEpBIUFxfj6upqdAwRERGRKknFU0SkEpSUlODi4mJ0DBEREZEqScVTRKQSmEwmnJycjI4hf3NjxgyyyXZ3797Od9+tve7rXm+bN3/DpEkPkp6eanQUEZEax9HoACIiNUFZWRkODg5Gx5AqIDn5JMuXv0FpaSkWixkXF1cee+wf1K4dcE3bTUyM4733FuPu7kFhYQGTJ08nICDwitc/cuQQ06dPomnTllgsFgoLCxg+/AEiI2+yLmOxWLCzsztv3f9d5krNnDmFmTMXXNW6F7J58zf89NP35OfnkpGRRlhYYzw8PHn22ZcBMJvN13xxr717d/Hoo1Mu+7pe674u9joDZGVl8sYbr+Ls7IzZbKa4uIiJE18E4LPPPuTxx5+ptBwiIteTiqeISCUwm80qnoLZXM5LLz3NpEnTaNq0JQAxMXvYs2cHUVED+OCDt0hJSaK8vIyWLdsxePC9LFkyFycnJzw8PNm/P5onn3yO+fNn8sILc6lTpy67dv3K9u0/0ahRM8aNm0RoaATr1n3O1q2buOee+5k3bwYWi4WAgEDM5vJL5mvatCUzZy4A4PTpDKZOHc9rr73D7NnPEBxcn4EDh5Gensrmzd/g5fXnPWknT57Ojz9uICXlFEOHjmbBglm4uLiQl5fLyJEPExHRlF9++YHNm7/BZColMvJmvL19OHAgmlWr/o2fnz8pKae4++6RLFo0G1dXN7Kzs7jnnlG0bt2Bxx8fSffuvcjLyyUx8QgvvbTogrMHevXqT69e/YmJ2cN3361l6tTZALz++isA+Pj4MmzYGObNm4GbmzuZmemMGzcRf/86TJs2kU6dupCRkYaHhyePPDKZpUvnU1hYQH5+Hu3adaJZs1b88cd+iouLeOCBx/nXv945L+v48SPo0KEzoaERHDoUg7OzM66ubvzxx346d+5GdvZpjh1LYObMBSQnn2TZssX4+vpRVHSGCROeY8+e7WzZsgk3N3cmT56Gk5Pzece5ZcsmWrRowz333A9AdPRvnD6dwfr1XxIbe4AdO35i585frMc8YsTYC/5Mli1bzOnT6RQU5HPrrVH06tWfJUvmXjLzxcqwiEhl0NdkIiKVIDk5mdOnTxsdQwyWmZmBn5+/tXQCtG7dgaioASQmxpGQcJjnn3+FadNeY8OGteTn52Fvb09oaCNGjXqE7t17sXPnz9x++x1s3boJgB9+2EBU1AD69h1EaGgEZWVl7Nmzgw4dOnPkyCEKCwt45plZ3H33SPLycq84a25uNl5e3jg4OJKamsyUKTNo3Lg5y5e/wYsvzuXpp2fi5OTMrl2/WtfZsOErGjYMZ/Lk6Tz22D94773XAVix4k2mT5/Hyy+/gaOjIz169KFWrdoMGTLKuu7GjeuIiGjK5MnTmTx5GsuWLQbAZCqlW7dePPTQk3h5eXPsWHyFXnNHR0fCwhoxZsxj5Obm0LfvIP7xj5cYOHAYGzZ8hZ2dPbm52YwcOY7Jk6ezfftPWCwWdu3axtixE5g5cwGtWrWjSZMWNGnSgmHDxrB79/YLZi0vL6NXr/7cdls/7O3tCQ9vwujR4/Hw8CQoKIQHHnicsjITp09nsGLFmwwdOppJk6bRqVNX1q37HAcHR5ycnJg6dfYFSyfAzTd3Z9eubbz77kK2bt1MaGgETZq0oEePPrRpE0nnzt3OOeYL/UzKy8upV68Bzz33Cs899wqrVv0b4LKZRURsSSOeIiKVIDAwkOLiYqNjiMHKy8sxm80XfC41NYng4PrW/65Tpy5ZWZkA1qmdrq5u5Obm0Lv3AKZPn8idd95DUtIJmjRpAUBaWgrLli1iyJCRhIU15tdff6Ru3SAAvLy88fT0umS+2NgDzJw5BYvFgrOzM1OmzAAgMDAYe3t7CgrycXV1tZai4OB6pKWl4OHhCUBKyikSEo6QlHQCAAcHBwoK8nFxcbGO+A8YMJTy8vNHXlNTk2jZsh0Afn7+5OZmn/NanD3+kpKSSx7DhYSENADAycmJbdu2snv3djIz062vR+3aAdbRPAcHB+zs7Jg8eRpLl84nJyebAQOGEh7e5Iqynt0XgL9/HQCcnV3w8/P/bwZnSktLSE4+xbp1n/Pdd2soLS0hMDDkvPUvJDAwhNdee4e0tBRiYw/w2mvT6d6993nTf89u50I/E/jzvfLWW6/h4OBwzmt6qcwiIrak4ikiIlJJ6tYNIj09lZiYPbRu3QH487zBvXt30b17b779drV12czMdGvh+isvL2/8/euwevVKunW7HYCTJ4/x7ruLmDJlurUw+PvXIS0tBYCcnCzy8/Mume9/p9qelZubYy1lnp5eFBcXYzKV4uTkTFLSSbp27WktXkFB9ahTJ5C77hqByWQiPT0FT08vioqKMJlKcXBwZOXK5dx771gsFss5+wkKqkdy8knrsZ8tQJXhbP41az6hcePm9O07iPXrv+Tw4YMXXL6o6AwODo48//yrlJaW8Mgjw+natecVZb3S6aghIfUZOvR+wsObkJOThZ2dHTExv192va+//oJWrdrTsGE4desG4ePjy/r1X9Kv313nfKlxNseFfia//76T9PRUpk6dzcmTx9i+fesVZRYRsSUVTxERkUpib2/Pyy+/zjvvLOBf/3oXR0dHvL19eeyxp/H1rUVERFNeffUFyspMDBw4HHd3j4tuq1ev/rz66gt88MEaAJYtW0xhYb71nMa2bTsycOAwnJ1dmDPnWfz8/C9aZCvioYee5OWXn8XLyxtXVzc6dOjMDz98B0BU1AAWL57DwoWzyMrKJCpqICEhDXjooSd56aWnMZvNREbejIODA66ubrz77kIiIppa1120aDYLF84iNzeHRx+dcs1Z/6p9+xv56KNlnDiRSIMGocTHx3L8eMJ5yzk7u/DVV5/x9ddfUF5eRu/ed57zfGVkHTPmMZYvfwNPTy9ycrJ4+OGJV7Rey5btWL78DVxcXLC3t6ekpISxYyfg6upGdPRvbN78zXlZ//ozadGiDStXLufNN/9JUFAIQUH1+PHHDRU+BhGRymRn+etXkiIiUmExMTEcO3aMAQMGGB1FKigrtZQ9P+TSPqryRuBqmu+/X09qajIjRz5sdBSp4mJ+yKR5J0/q3uBqdBQRqWI04ikiIlKDbNy4jtjYA+c85ujoeM5tOCoiKekEq1ev5OGHn6qMeJd18OA+vv9+/XmPDxs2hsDA4OuS4XpYseLN86ZGN2wYzsCBwwxKJCJiWxrxFBGpBBrxrL404ilSeTTiKSIXo9upiIiIiIiIiE1pqq2ISCVISEggLS3N6BgiIiIiVZKKp4hIJQgPD7feP09EREREzqWptiIiIiIiImJTKp4iIiIiIiJiUyqeIiIiIiIiYlMqniIiIiIiImJTKp4iIiIiIiJiUyqeIiIiIiIiYlO6nYqISCWwWCyYzWajY8g1OBqTb3QEERGRGkvFU0SkEtjZ2WFvr0kk1VX9CFejI8hlxMXHYTKV0bxZM6OjyCV4ergYHUFEqigVTxER+VurFehMrUBno2PIZWSVFVNcXEzTTl5GRxERkaugr+dFRESkyjt06BCHDx82OoaIiFwljXiKiIhIlde8eXOKi4uNjiEiIldJxVNERESqvISEBEwmk9ExRETkKql4ioiISJUXHh6uEU8RkWpMxVNEpBIkJCSQmppqdAwRERGRKknFU0SkEoSHh+Pg4GB0DJEaKzs7WyOeIiLVmIqniIiIVHl+fn4qniIi1ZhupyIiIiIiIiI2peIpIiIiIiIiNqWptiIilcBkMlFWVmZ0DJEaS+d4iohUbyqeIiKVwMnJCUdH/UoVsRWd4ykiUr1pqq2IiIiIiIjYlL6eFxERkSrvxIkTlJSUGB1DRESukoqniIiIVHkNGjTQVFsRkWpMU21FRERERETEplQ8RURERERExKZUPEVERERERMSmVDxFRERERETEplQ8RURERERExKZUPEVERERERMSmVDxFRCrByZMnSUtLMzqGSI1lNpuxWCxGxxARkauk+3iKiFSC+vXrU15ebnQMkRrL3t4eOzs7o2OIiMhV0oiniIiIiIiI2JSdRfNWRESu2sGDB5kxYwbFxcWUlpbi6+tLcHAwixcvNjqaSI3w008/sWTJEvLz87Gzs8PT05MHH3yQvn37Gh1NREQqQCOeIiLXoEWLFnh4eJCamkpWVhanT59m2LBhRscSqTG6deuGxWIhMzOTjIwMSktLVTpFRKohFU8RkWvUpUsX6787dOhAhw4dDEwjUvPcdNNN1n/feOONBiYREZGrpeIpInKNevbsSfPmzfH19WX48OE4OzsbHUmkRunVqxcNGzYkKCiI/v37Gx1HRESugq5qKyJX7MQfZ4yOUCU5E8xNre/kqPcxfJ0j9DpdRINm7kZHqDKyUkspyC4zOka14esUQYdmUZQUl+DrpM9YRbh6OBDQwMXoGCIiuriQiFy5w7vzyUjVH8tSceUl5XS/u7bRMaqMrNRSfv8xF1cvff8rtmMqMtPiRk8VTxGpEvR/PBGpEGdXB8LbeRsdQ6qR7NQSkmLzjY5RJTW7yc/oCFKDxfxw2ugIIiJWOsdTREREREREbErFU0RERERERGxKxVNERERERERsSsVTREREREREbErFU0RERERERGxKxVNERERERERsSsVTREREREREbErFU0RERERERGxKxVNERERERERsSsVTREREREREbErFU0SkAnJzs1myZO55j48ZM6jC2xo79u5LPr916yY++mjZFW8vO/s0Tz89rsI5RK7Upd6zFXm/vv76K8TE7Knw/s+uZzKZePLJ0axa9e8Kb+NqfPrpB8THx16XfYmI1FSORgcQkb+nxMQjPP/8BJo2bYnFYqGwsIDhwx8gMvIm6zJmsxl7+4t/P/bJJyvIyEjjkUcm4eLietHlZs6cwsyZCzhy5BDTp0+iadOWAJSVleHt7cOkSS/i5OR8Rbl9fPyYMOHZKzzKKzNt2kSKi4vw9PSiqOgMoaERPPLI5Erdx18lJZ3gnXcW4uDggLOzM1OnvoyDgwMA//nPt6xY8SYff/ytTTNI5bn77h40b97G+t+hoRE8+OATV7z+okWzefDBCfj4+LJ793ZWr/4YBwdHCgryCAlpwPjxT+Ph4VnpucvLy1m6dD5ZWZk4OTmRlXWaUaPG0bp1h0uud/p0Bt7ePgwZMgqAwsIC3n9/CUFBIdxzz/0AZGam89lnH/L448/wn/98i7OzC7fcctsVZ/vf9YcPf+DqD/IvPvpoGYmJceTl5Z7zO89kKuWJJ0YxaNC99O1b8S+yRESqOhVPETFM06YtmTlzAfDnH5JTp45n+fJVjB8/gg4dOhMaGkHTpi1Ztmwxvr5+FBWdYcKE5/Dy8qa0tISsrEyiogawefM39O8/BIBffvmBzZu/wWQqJTLyZry9fThwIJpVq/5N69YdztmnxWJh8uSxnDx5nHr1GrBgwSxcXFzIy8tl5MiHCQ6uz/z5M/H09CIzM50RI8YSGBjC3LkvMn/+MubNm4HFYiEgIBCzuRyA+fNn0qfPIFq2bMv69V9SUlJC376DmDdvBm5u7mRmpjNu3ETCw5uc81qMHz+FsLDGmEyljBkziHHjJlmfKyo6w6JFs3F1dSM7O4t77hlF69YdzjvWHj2irOt88skKzGYz99330AVf+6VL5/Poo1OoV+8GDh2KobS0BDc3d1JTk9i58xf8/Pwr7wctNufq6sbs2YvPe3zZssWcPp1OQUE+t94aRa9e/Vm6dD6FhQXk5+fRrl0nmjVrxc6dv1BeXs4999zPihVL+Oc/38HLyxuALVs2ceTIIdq162Td7vHjibz//hJ8ff3Izc2xfhnzxx/7WbJkLsePJzJ48L106dKDLVs28sMPG3BxccHXtxaPP/6MdTvHjsWTmprEyy+/AUBKyini4v4cWdy582fWr1+Nt7cPnp5ePProFOt6a9as5PjxRNav/5I77rib775bw6BBw1m9+mPKyspwdHTkX/96h9jYA2zZspHVq1fi7e1Dw4bhbN26mZMnj1JWVsZNN3Xn9tvvYMmSuTg5OeHh4cn+/dE8+eRzfP75/xEbe4AdO35i+/at3HZbPxo1anbeZ7FBg1CmTZtIp05dyMhIw8PD86JfHMXGHuDkyeNMnz6P/Pw8UlJOWZ9bvvwNIiKaXu1bQESkylPxFJEqITc32/qHbnl5Gb169eeGG8KYPfsZhg4dTYsWbfj++/WsW/c59933EJs2fc0tt9xGkyYt+PLLj7njjruxs7NjxYo3ee+9L7Czs+Obb1bRo0cfPv/8/xgyZBRHjhwiNvYAM2dOwWQqJSMjjT59BhEW1oh16z6nYcNw7r33QdLTU1mw4CXGjp2AyWTiiSemUlpaSkFBnjXvkSOHKCwsYObMBeTn57F27aeXOLYc+vYdRMeOXdi2bQsbNnx1zh/fAO+/vwQPD0+OHUtg0KDh2NnZWZ/buHEdERFNGTp0NNnZp5k2bSJvvvnv846R+oNjAAAgAElEQVT1rG+/Xc2ZM4WMHTvhopnS01PZtm0Lx44lUKdOXZo2bfnf0acFTJr0Ii+++FSFfn5S9ZSXl1OvXgPGjZtIQUE+U6Y8xO2338GuXdtYsOA9fH1rkZh4hPDwJgQH1+Phhyfyww/fceutUdbPIsCtt/Y+b9v/939LGTFiLE2btmTr1k2sXr2SJk2a4+rqxoQJz5Kbm8PUqY/SpUsPTCYTzz03Bzc3dx57bAR5ebnW7TRoEIa7uwfz5s2gVav2tGjRhm7dbgfgnXcWsnTpSlxd3Vi4cBYHDkRb1xs4cBjJyae44467KSsrIzExjiFDRtG5czd+/vl7evToQ48efXBxceXWW6M4diyB8PAmlJeXc+BANHPnvo3ZbOaJJ0bSrdvt2NvbExraiKioAaxf/yU7d/5sXb9z525s374VuPBncc6cJeTmZjNy5Djs7OwYM2bQRYvnoUMxeHv78Pbb88jOPs3QoaMB2LHjJ7y9fWnQIPTaf/AiIlWUiqeIGOZsCbRYLDg7OzNlygzrcyEhDQBITj7FunWf8913aygtLSEwMASz2cyGDV8RElKf9eu/JCXlFL/99gstWrTFxcXFOmV0wIChlJeXn7PPsyOef54jdr/1j9yUlFMkJBwhKekEAA4ODjRu3JyuXXsya9Yz2Nvb88ADj+Pl5QRARkYadesGAeDl5Y2np9dFj9PJyYlt27aye/d2MjPTL7js2LETCAtrjNlsZs6cZ9m//3frc6mpSbRs2Q4APz9/cnOzKSjIP+9Ys7NPc+xYPP/+97ILjn79r/T0VHr3vhNf31q8/vorbNu2hfj4WPr3H4Kvb61LritVT3FxES+88P+/aGjbtiN33XUfaWkpvPXWazg4OFBSUoKdnR2TJ09j6dL55ORkM2DA0HNG38vLy6yfmbKyMmbMmITFYsHd3YMXX/yndbnU1CTrZzQwMISff/4PTZo0p169GwDw8fGloCDfuvy77y7E1dWN3NwcSktLrI87OTnx/POvkp+fx+HDB1m9eiXl5WU88shk8vNzefPNP/eZk5NNTk72BY9969ZNJCef5NVXn8dsNpOdnUWPHn0uuGxKyilOn85g/vyZALi7e5KdnQVAQEAggDXnhVzoswhQu3aA9cuis5/JCzGZSrG3t2f8+KfJzExn+vRJzJ69mI0b1zFt2mt8992ai64rIlLdqXiKiGH+d9rrX539Iy4kpD5Dh95PeHgTcnKysLOzY+fOn+nSpQf33vsg8Of5Xa+++gI33ngLRUVFmEylODg4snLlcu69dywWi+W87Ts5OTFs2AO8//4Spk6dTVBQPerUCeSuu0ZgMplIT08hLS2Fli3bEhU1gIMH97Fy5XLrSIa/fx3S0lIAyMnJIj//z9FQZ2cXSkqKAUhLS8HXtxZr1nxC48bN6dt3EOvXf8nhwwcv+prY29vj5eVNVtZp62NBQfVITj4J/Hnemb9/nf+eD3rusfbrdxf16t3ACy/MZdasf/Dqq29dtBCHhze2vi4eHp6YTKXs27ebzMx0tm7dRFpaMh9++DZjxjx20axSdbi6ujFnzpJzHtu1axvp6alMnTqbkyePsX37VoqKzuDg4Mjzz79KaWkJjzwynK5de2JnZ4fFYqZz527MmDGZqKgB1KpVmzlzlpCRkcbLL089Z9tBQfVISjpB06YtSU4+SXBwfQBSUpKAP0f5vbx8KCkp5qOPlvHBB2spLy/j119/xGw2W7ezb99ucnNz6NbtdiIjb6JVq3aMHXs3Hh6e1KpVm6eeegEnJydSUk7h71+HPXt2nHfs3323lnnz3rWep/3++0uso6Nn92VnZ4fZbCY4uD7BwfV5+umZwJ9Ths9+gXQh/5v17HH/9bNYERERTa2jp25u7pSXl7Ft2xYAFi6cxalTx7Gzs6N589YEBgZf8tx1EZHqRsVTRKq0MWMeY/nyN/D09CInJ4uHH57Il19+bP3DEf4sTi4uLiQmHuGhh57kpZeexmw2Exl5Mw4ODri6uvHuuwvPGwXp3r0Xa9as5NChGKKiBrB48RwWLpz133NHB9K4cXOWLl2An18t8vJy6d///1/Rs0mTFjg7uzBnzrP4+flTp05d4M9piStXvs/Bg3vJzEzHx8eP9u1v5KOPlnHiRCINGoQSHx9LYuKRc7IsXboADw9PysrKqF27Dl269ODXX38AICpqAIsWzWbhwlnk5uZYz3X767Ha2dnh6OhEnTp1uf/+R1iw4CWmT593zrTds8aPf5oFC2bh6uqKg4Mjo0ePP+f1eeKJUSqd1Vx4eGNWrlzOm2/+k6CgEIKC6rFt2xZ27drG119/QXl5Gb173wlA48bNmTdvBs8+O4fHH3+GuXNfxNHREVdXN5ycnJg48cVztj169Hjef38JPj6+FBYW8OSTz/H77zspLy9j6dL5HDuWwOjRj+Li4krDhhEsWjQbb29fbrnlNr744l//k7EJb789778X/3GmqOiM9f394INP8Morz+Hi4kp5edk5MyLO2rt3F3XrBp1zcbCePfvyr3+9w6OPTiE6+jc2b/6G8PAmfPLJCp54YiqNGjVl7twXMZlKCQlpcNGLMAUH17euf9bFPotXqkOHzuza9SsvvzyVgoJ8HnroKTp2vJk777wHgPXrv8Te3oEbbgir0HZFRKoDO8uFhgJERC7g8O58cnMshLfzvvzCIv+VnVpCUmw+3e+ubXSUKiMrtZTff8ylXe+KjZiJVETMD6dpcaMnAQ1cjI4iIqIRTxGRmio3N5sPP1x63uNduvQ457Y1IlK5Nm5cR2zsgXMec3R0PO+iYiIifycqniIiNZSPjx9PPfW80TFE/naiogYQFTXA6BgiIlXKxe/MLiIiIiIiIlIJVDxFRERERETEplQ8RURERERExKZUPEVERERERMSmVDxFRERERETEplQ8RURERERExKZUPEVERERERMSmVDxFRERERETEplQ8RURERERExKZUPEVERERERMSmHI0OICLVy+lTRZw+VWR0jCqhpKQEBwcHHB0r/1dpUVERbm5ulb5do7h7ORgdocoxlZj57es0o2MAFkpKSrFYLLi6uhod5oqUl5dTXl6Os7Oz0VFEROQK2VksFovRIUSkesjPLjM6QpWyYcN3hIaG0qRJ00rf9o8//khOdja9o3rj4eFZ6ds3gpefvus8qyp8lpKSkjh06BCHY2Np2qwZzZo1IyQkxOhYVyQzI4NNmzczYsQIo6NUC/rsiUhVoN9EInLF9MfLuTJzTnFreGebvC4D7urFL7/8wup1K7n55ptp27Ztpe9DjGPUZ6m4uJgdO3YQHx+PxWKhTZs29L3z0Wo3uu7pG8jpVadw9bTg5ORkdBwREbkC+itSROQqxMfHk5mZSZ06dWy2j65du5Kfn8/GjRtxc3OjSZMmNtuX1GwnTpwgJiaGmJgY/Pz86Nu3Lw0bNjQ61lWzs7PDw8ODtLQ06tWrZ3QcERG5AiqeIiJXITc3l6ZNK3+K7V/17duXJk2asHbtWg4ePMhdd91l831KzVBaWsrnn39OaWkpZWVltG7dmokTJ+Lu7m50tEoRERFBcnKyiqeISDWh4ikichVyc3Np0KDBddlXWFgY/fv3Z+vWrfzwww/07NnzuuxXqqdTp04RExPDvn37qF+/PrfeeithYWFGx6p0rq6unD592ugYIiJyhVQ8RUSuQk5ODs2aNbtu+2vcuDE33HADa9asYd26dQwYMOC67VuqvvLycvbt20dMTAzFxcW0bt2aCRMm4OlZMy5MdSERERFs3brV6BgiInKFVDxFRK5Cbm4uQUFB13WfLi4uDB8+nG+++YaVK1cSFRWFv7//dc0gVUtKSop1dDMiIoKuXbsSERFhdKzrok6dOqSnpxsdQ0RErpCKp4hIBeXl5ZGXl2fY/s9Ou125ciXDhg0jICDAsCxy/VksFmvZzMnJoV27djz66KN4e3sbHe26cnd3x8PDgxMnTly3ae8iInL1VDxFRCqoqKiIoqIiQzN0794dT09PPvroIwYNGlQjz+GTc6Wnp1un09avX58bb7zxb3+lYz8/P4qLi42OISIiV0DFU0SkggoLC/Hx8TE6Bh06dMDLy4svvviCvn370rp1a6MjiQ0cOHCAmJgYMjMzadOmDQ888AC1atUyOlaVEBAQQEZGBo0bNzY6ioiIXIaKp4hIBRUVFVWJ4gl/XnRo1KhRrFmzht9++42HHnrI6EhSCbKysvjiiy8oKioiMDCQtm3b0rx5c6NjVTne3t4cOnSILl26GB1FREQuQ8VTRKSCqsqI51nBwcEMGjSIjRs38uOPP9KjRw+jI8lViouLY+/evZw8eZJmzZoRGRlJnTp1jI5VZQUGBnLgwAGjY4iIyBVQ8RQRqaCqNOJ5VkhICPfddx+rVq1i48aNREVFGR1JrlBxcTF79+5l3759uLi40LZtW4YMGYKdnZ3R0ao8Ly8v8vPzjY4hIiJXQMVTRKSCCgsLqV+/vtExzuPi4sKIESNYtWqV7vVZDSQlJbFv3z727t1Lq1at6Nu3r67OWkEqniIi1Ye90QFERKqbqjjieZadnR333HMPOTk5fPLJJ1gsFqMjyV/ExMTwwQcfsHbtWnx9fXnqqae48847VTqvgr29PcXFxSQmJhodRURELkMjniIiFXTmzBl8fX2NjnFJ999/Pxs3bmTlypUMGTIEFxcXoyP9rWVnZ1tHN4OCgrjpppto2rSp0bFqhHr16mFvr+/RRUSqOhVPEZEKOnPmDN7e3kbHuKyoqCh+/PFHPv74Y4YMGVItMtc0CQkJ7N27l6NHj9KmTRvuu+8+XSyokrm7u+teniIi1YCKp4hIBTVs2JDCwkI8PDyMjnJZPXr0wMXFhZUrV9KzZ0/d7/A6KCwsJCYmhv379+Po6Ejbtm0ZPHiwRuVsxN3dnZKSEqNjiIjIZah4iohU0O+//87tt99udIwrdvPNN5OUlMT333+Pv78//v7+RkeqkdLT09m7dy/R0dF4e3vTr18/brjhBqNj1Xiurq4qniIi1YCKp4hIBZhMJhwcHHBwcDA6SoXcc8897Nq1i88++4zhw4dTq1YtoyPVGHFxcURHR5OcnEy7du0YP368pjVfR25ubpSWlhodQ0RELkPFU0SkAoqLi3F1dTU6xlXp2LEjFouFTz/9lKioKMLDw42OVG2ZTCb27dtHdHQ0jo6OtGvXjqFDhxod62/Jzc2NM2fOGB1DREQuQ8VTRKQCSkpKqvUVYjt16kRsbCybNm1i2LBhGvmsoKysLKKjozlw4AD169fn9ttvJzQ01OhYf2suLi7k5eUZHUNERC5DxVNEpAKq84jnWffffz+//fYbn332mcrnFUpMTLRenbZdu3aMGjVKr1sV4eLiQllZmdExRETkMlQ8RUQqIDU1lYKCAqNjXLNOnToB8PnnnxMVFaVRu4uIjo4mOjoas9lMu3btGDRokK5OW8U4OjpisViMjiEiIpeh4ikiUgG1atWqMVeF7dSpE3/88QebNm1i+PDh+Pj4GB2pSsjLy+P3338nJiaGoKAgunfvrvNhq6DS0lLKy8sxmUyUlJRQVFREUVGRRqJFRKooO4u+JhQRuWLx8fHs2bOHYcOGGR2l0mzbto3du3dz7733MnjwYKZNm0ZUVJTRsa67kydPEh0dzR9//EG7du1o2LCh7ntahSUnJzNgwIBzHluxYgWtW7c2KJGIiFyKRjxFRCrAYrFgZ2dndIxKdfPNN1NSUsKdd95JWVkZv/76a40unn/88Qcvv/wyH3/8MQAHDx7k999/Jzc3l44dO9KnTx+cnZ0NTimXExwcTGRkJLt37wagadOmKp0iIlWYiqeISAXU1EkiL7zwgvUCLXFxcRw9erRGnveZlJTEW2+9xeHDh7ntttu466678PPzIzIykmbNmhkdTyqoT58+1uJ5yy23GJxGREQuRcVTRKSCatqIJ0C3bt2Ij4/n2LFjJCQkkJCQUOOKZ3Z2Nu+++y47duwAIDc3l8GDBxMcHGxwMrlaXbp0ITIyksLCQm677Taj44iIyCXoHE8RkQqIjY3l4MGD3H333ZW2zZSjxcTvM/5KuYUFBWRlZZOWnoazs3ONm7YY+0csycnJAJSbyzGXm3Fzc6Nb925XtP4tg2rbMt4F/bw287rvs7o5fuwYZ4qKNGJ9BTr29sPV3cHoGCLyN6URTxGRCjh58iSZmZVfBsrNdoQ08qz07VaMF+EEAc0NzmEbgeEdr3rdI79lV2KSK5dxspRWt9aMqyjbSmB4K6MjVAuxO7KMjiAif3MqniIiFRASEkJ+fr5Ntl0r2MUm25VrU3Km3ND9+wW5UANnd4uIyN+M7oItIlJBNfEcTxERERFbUvEUEamAmng7FRERERFbU/EUEakAs9mMvb1+dYqIiIhUhP56EhGpAF0IXERERKTiVDxFRCpIU21FREREKkbFU0SkAnSOp4iIiEjFqXiKiFRARkYGOTk5RscQERERqVZ0H08RkQqoXbs2JSUlRscQERERqVY04ikiIiIiIiI2peIpIiIiIiIiNqXiKSIiNpWcfJIXXphw0edff/0VYmL2XMdE8lfjxg2lvLzc5vvJzj7N00+Pu+btjBkzCIBPP/2A+PjYa96eiIjYns7xFBGpYe6+uwetWrXHzs6OvLxc+vUbzG239WPVqn+zadPX1K0bbF32wQefYO3aT+ja9TY6drz5gtubNm0irq6uvPDCXOtjixfPITU1iblz37b58VxIUtIJ3nlnIQ4ODjg7OzN16ss4ODgAcOrUcR5/fCRLl64kOLi+IfmqoqKiM6xY8SZJSSdwcHCkpKSYUaPG0apVe6OjnWfZskVYLBYeeWTyJZcbPvyBCm03MzOdzz77kMcff+Za4p1n0aLZPPjgBHx8fC/4fHp6Kh9++DZJSSd4/fUPAUhMjOO99xbj7u5BYWEBkydPp3btABYtmo3FYsFkMhEYGMzgwffi61urUvOKiBhBxVNEpIZxdXVj5swFABw6FMNnn33Ibbf1A6BPn0HcddeICm8zPz+P9PRUAgICyc/PIyMj1frc8eOJvP/+Enx9/cjNzWHChGdxcnLin/+cRt26wXh4eAKQmprM4sUvW8vq2LF38/77X1q3k5R0gmXLFuPr60dR0RkmTHgOLy/vC+ZZunQ+jz46hXr1buDQoRhKS0twc3PHZDLx3nuv06JFmwofY0339tvzCA1tZC1dWVmZrF690volxeef/x/JySdJS0thzpwllJWZmDdvBm5u7mRmpjNu3ET8/eswbdpEOnXqQkZGGh4enjzyyGSWLJmLk5MTHh6e7N8fzZNPPkdAQCALFszCxcWFvLxcRo58GH//OpfNeeTIIerVu4HCwgKOH0/khhvCAJg3bwYWi4WAgEDM5j9HZxctms1tt/Vj06av6dNnEC1btmX9+i8pKSmhT5+BzJ8/E09PLzIz0xkxYiybNn1NbOwBduz4iV27tuHs7Iyrqxt//LGfzp27kZ19mmPHEpg5cwHJySfPez/+/vtOtm7dRFhYY2JjD9CrV38CA4PZufMXysvLmTx5Ovb2508mO348gXvvfZD582daHzt8+CDjxk0iNDSCdes+Z+vWTfTs2Zfk5JMsWLAcs9nMAw8MIipqgIqniNQImmorIlLDlJQU8+qrzzN79jPMmzeDwYPvveZt9urVn2++WQXApk3ruPXWKOtz//d/SxkxYiyTJ0+nZ88+rF69ks2bvyEy8maeeup5WrZse0X7WLHiTYYOHc2kSdPo1Kkr69Z9ftFl09NT2bZtC6+9Np2dO3/GxcUVgA8/fJu77hpx0cL6d7Zr1zYGDRpu/e9atWrz0ENPAn/en7Zr155MmTIDP79aHDsWT25uDn37DuIf/3iJgQOHsWHDV9jZ2ZObm83IkeOYPHk627f/BIC9vT2hoY0YNeoRunfvxc6dP7Nhw1c0bBjO5MnTeeyxf/Dee69fUc7167+kR48+9O59p/U9d+TIIQoLC3jmmVncffdI8vJyL7udU6eOYzKZeOKJqTz//KvUrh1Ajx59aNMmks6du2Fvb094eBNGjx6Ph4cnQUEhPPDA45SVmTh9OuOC70d7e3vc3NwZOfJhHnlkEv/5z7c0adKC4OB6PPzwxAuWToCOHbvg7X3uaGjfvoMIDY2grKyMPXt20KFDZ/z969CyZTumTh3PpEkPMmLEQ+etJyJSXWnEU0SkAsrKyigrKzM6xiW5uLjy3HOvAH9Or5w06UEWLVoBwLffrmbPnu3WZadMmXFF22zf/ka+/XYNJSXF/Pbbr7zwwlz+859vAUhNTSIkpAEAgYEh/PzzfygvL6N16w4ABAXVu6J9JCefYt26z/nuuzWUlpYQGBhy0WXT01Pp3ftOfH1r8frrr7Bt2xZcXd1wcnKiTZtIvv129RXt8++krMyExWK56PNnpyW7urpRUlJCrVpObNu2ld27t5OZmY6npxcAtWsHYGdnB2Cd3gwQEBBoXT83N4fMzHQSEo6QlHTivGUvJi0thf37o1m8+GUA4uMPk5ubTUZGGnXrBgHg5eVtzXIpjRs3p2vXnsya9Qz29vY88MDj5y1zdgTW2dkFPz9/AJycnCktLbno+7FOnboAuLm5U1p6bbdWSktLYdmyRQwZMpKwsMYkJsaRkHCYuXPfpry8nEmTHqRFi7ZXdLwiIlWdiqeISAU4Ojri6Fh9fnW6uLhiMpkoKSkGoF+/u65qqi1A9+69WLp0Pu3b33jOyE5QUD2Skk7QtGlLkpNPEhxcH09PL9LSUgA4efIYAK6urtYcRUVnyM/PO2f7ISH1GTr0fsLDm5CTk2UtNxcSHt7YWqI8PDwpLy/jp582U1ZWxvz5M4mNPcD77y/h+edfvaLC83fQsWMXPvvsQ0aMGAv8eb7je+8tZurUly+4/Jo1n9C4cXP69h3E+vVfcvjwwQrtLyioHnXqBHLXXSMwmUykp6dcdp21az9hypQZ1qnSu3Zt45tvvqRDh87W91NOTtZ57x1nZxfreystLQVf31qkpaXQsmVboqIGcPDgPlauXE6/fndhNpuvKP+F3o8xMb9fcFk7Ozsslivb7lknTx7j3XcXMWXKdGvpLSjIw9PTGzs7OxwdHbGzs7NOKxYRqe6qz19PIiJyRYqLi5g5c4r13wMGDK2Uc8R69erPmDGDWLbss3MeHz16PO+/vwQfH18KCwt48snnMJvNvPLK8yQmHsHHxw+LxYKvby38/evw9tvz8Pb2wdfX75ztjBnzGMuXv4Gnpxc5OVk8/PBEfHzOXeas8eOfZsGCWbi6uuLg4Mjo0ePp3r239flXX32e0aPHq3T+jwkTnmXZskVMnPgAHh6e/z2H8PGLTg9t3/5GPvpoGSdOJNKgQSjx8bEcP55wxfuLihrA4sVzWLhwFllZmURFDbzktOvCwgKio39j3LhJ1sc6dOjMihVLuOeeUTg7uzBnzrP4+flbRx3PuvXW3qxc+T4HD+4lMzPd+r5ZunQBfn61yMvLpX//uwkOrk909G9s3vzNZfNf6P14MY0bN2fevBk8++ycC07z/uCDt0hIOExS0glmzpzCwIHDWL16JYWF+bz++p+zE9q27ciAAUPZsmUTr702nfLyMjp16nrRCxaJiFQ3dpZLzbsREZFz7N27l6SkJO64445K22bK0WIORxfSqrt/pW1TKk/JmXKiN2Vwx9jA677v1UuS6XZvMJcY/P1bW7hwFr169a+SV+ataratTqHXfQG4uuvLGBExhkY8RUQE+PPiPn+dwtiwYTgDBw4zKFHVzCTXbuPGdcTGHjjnMUdHxwrd5mTHjp+IjT3A2LFPVna8q5Kbm82HHy497/EuXXoQGXmTAYlERKoWFU8REQH+vKdnVVMVM8m1i4oaQFTUgGvaRufO3ejcuVslJbp2Pj5+PPXU80bHEBGpsnQ7FREREREREbEpFU8RkQooKCjgzJkzRscQERERqVZUPEVEKsDT0xN3d3ejY4iIiIhUKyqeIiIiIiIiYlMqniIiIiIiImJTKp4iIiIiIiJiUyqeIiIiIiLy/9i77/CoyoSNw7/0SQ+E9JAQQgmQ0EEUBUGaiooNhY9dFVRUBBQsYEVFwVUERFeKiKiLiisiyiJFFwQVFAhSQ4CQQhLSe09mvj9YZs0SICjxZMJzX1cuyJQzz5mZM5ln3vecEWlQKp4iIiIiIiLSoFQ8RUREREREpEE5Gh1ARMSWJCcnk5mZ2SDLLsqtapDlim0rztPzQkREbJ+Kp4jIBQgLC8PBweGiL7e6vIajv+Rd9OXKxeHiaswEIS9fRz0vzqOgoICamhqaN29udJRGzdX94r9uiYhcCBVPERGDBUWYCIowGR1DGqFBo/2NjtDo7dhxnKKiIgYNijI6ioiInIOKp4iIiNisvLw8SkpKjI4hIiLnoeIpIiIiNqtZs2Y4OurtjIhIY6dXahEREbFZqampGvEUEbEBKp4iIiJis0JCQigqKjI6hoiInIe+x1NE5AJkZGSQk5NjdAwRERERm6IRTxGRCxAQEEB1dbXRMURERERsikY8RURExGaVlZVRVlZmdAwRETkPjXiKiIiIzXJ1ddUsBBERG6ARTxEREREREWlQKp4iIiJis+Li4jhy5IjRMURE5Dw01VZE5AKkp6eTnZ1tdAwR+Y+oqCh9nYqIiA1Q8RQRuQBBQUGYzWajY4iIiIjYFE21FREREZtVU1OjD4NERGyAiqeIiIjYLAcHB+zt9XZGRKSx0yu1iIiIiIiINCgVTxEREREREWlQKp4iIiIiIiLSoFQ8RUREREREpEGpeIqIiIiIiEiDUvEUERERERGRBuVodAARERGR3+vEiROUlJQYHUNERM5DxVNERERsVmhoKEVFRUbHEBGR89BUWxEREREREWlQGvEUEamHrVu3kpOTw7FjxygsLKSqqgqAESNGGJxM5NK0adMmiouLicJleMgAACAASURBVI+Pp7y8nOLiYuzs7LjpppuMjiYiInWws1gsFqNDiIjYguuvv56MjAzr748//jh33HGHgYlELm2DBg0iPz8fAEdHRzZs2ICXl5fBqUREpC6aaisiUk99+vSx/r9Xr15cffXVxoUREXr37m39/2WXXabSKSLSiKl4iojU0/DhwwkMDARg8ODBBAQEGJxI5NJ266230rx5c5ycnLjuuuuMjiMiIuegfTxFBICqCs26P5/ojl3p1eNyUlNP0LvXFbrP6sHJxc7oCDZBz6XfJ6pdND26XUZBQQHduvTU/fg7aTsVkT+D9vEUEQDys6r4flW20TFsQnV1DY6ODkbHaPRi+noREe1udAybsG7ZSaoq9ef49zKbzdjbaxLX71FdZeGWh4ONjiEilwCNeIqIlcndgR7X+hsdQ5qAwzvyjY5gc7oP88fVQx9oyJ/HYrbw/afpRscQkUuEPh4UERERERGRBqXiKSIiIiIiIg1KxVNEREREREQalIqniIiIiIiINCgVTxEREREREWlQKp4iIiIiIiLSoFQ8RUREREREpEGpeIqIiIiIiEiDUvEUERERERGRBqXiKSIiIiIiIg1KxVNEREREREQalIqniMhvzJz55Bmnvfrqsxw6tO+CljN//ivs3bvrrOfn5eXw2GP3X9Ayx4279YIuL9KQtmzZwEcfLT7r+fV9vv6ebeF/b3/evJnMmvXUBS/j99i58yfWrVv9p9yWiEhT4mh0ABFpmp599hHKy8twd/eguroKLy8fHn30WZycnAAwm83Y25/9s6+SkmKWLl1AUFAIt9/+17Ne7ttv/4WzswtXXXUNt946gJiY7gBYLBaqqiqZNOkpAgOD6537mWderfdl6+Of//yQjRu/Jji4JdXV1QBMn/7yRb2N/1VRUc7HH7/Hl19+ysqVG3Fycqa0tIQ5c17AYrFQWFjA6NHj6N79MlatWsGhQ3txdXWjvLyMxx6bgbOzS4Pmk4b1z39+yIYNXxEQ8N/n/dixDxMR0aZe1//tNlVaWsJ7773FyZOp1NTUUFFRzq23jqFv3wENkv3rr//J9u3f4+bmQWFhPn369GPEiDvPe719+2JZuvRz6+8ff/weWVkZjB//KC4uJgBmzJjKjBlzyM7O5NNP32fChCcuKNvcuS8xduxEeva8/MJW6hz279/D0qVvEh3djXHjJgKQkHCEJUvm4ebmTklJMVOmPIe9vT0LFszGx6cZBQX5DBx4Lf36DbpoOURE/gwqniLSYB58cCqtW7cDTo0A/vDDv/n1150AeHv7MHr0OObMeREXFxcKCwsYM+Y+2rSJAmDdui8YMeJOVq36B9XV1Tg6OpKcfJylSxfg4OCAm5s79933CKtWrcDLy5tWrSIxmVyZMWOO9faXLl3A9u3fM2LEnXz00RJSUo5TXV3N5Zf3Z9Cg69m48Wt27NiKo6MTjo6OPPbYDO6+ewTvv7+aNWtW8vPPPxAaGkZWVgYAGzZ8RXZ2JqNHjyMlJZGlSxcwY8YcFi+eR05OJsXFRVx99VAGDx5e634YOvQmbrllNADTp08gLS0FX18/6/nbtn3Hxo1f4+npBcCUKc9x4kRSrXV99NFnrZf/9dedrF79CU89Ncta5H/rxIkk+vUbzI8/braelpBwhP79h9Cv3yASE4/x7rtv0r37ZXzzzWreeutDnJ1dmDFjKunpqYSHt/6dj7g0FsOGjbA+507bvHk93333DS4uLvj4NGfChCc4duwwH3ywCE9PL/Lzc3nwwcdqbVOffLKMqKhoHn741EyA0tISVqxYyuWX97cut6yslLlzX8JkciUvL5fbb/8LnTv3wN7enmXL3iYnJ4vCwgKeffZVUlNTePfd+Xh7N6OgII+nnppVK+PHH7/Hhx9+jb29PVVVlaxduwqLxUJaWgqLF8/Dx6cZZWWlTJw43Xqdb7/9F/n5uSxa9Abjx0+hsrKC3Nxshg69kY0bv2b48NvYtGkt+/fH8s9/fkhy8nHi4vazffv3tGzZ6ozl7t69gy1bNtC6dTvi4vYzePBwAgOD2bFjGzU1NXTt2ouTJ1MZM+Z+li17m/T0VGpqqomO7sbNN49iwoQx9O8/mMLCAhIS4nnhhbl1bqcAWVknuemmOzh2LN562uHDB7j//keJiGjDmjUr2bJlA/7+gUREtOHuux8iISGeZcv+ruIpIjZHxVNEGtypUbZ8PD29cHR0JCwsghtuuJ01a1bSqlUko0aNJTPzJHPmvMCrr75DdXU1CQlHuO22v9CnTz+2bt3EgAHDWL78Hf761/FERrZny5YNWCxmevW6gsjI9rRs2Yry8jJmzJgKQGLiMfr0uYqhQ28kMfEY+/fHMnv23zGbzTz88Bj69RvE999vYuTIvxIT052EhHhqamqseT///COWLVuNvb09EyaMOeu61dTUEBoaxv33P0JxcRFTp957RvHctGkthw/vJysrg2bNfImIaEthYb71/HfffZMlS1bi5OTM/Pmv8MsvP7Bhw1e11rWoqACAI0cOsWrVCp5+uu7SCRAZ2f6M06Kju1r/v23bd1x+eT/g1EjY5Mn34O8fiLe3j0pnE1ZVVcX06S/j6urGQw+NprCwgNjYn4mKiuaOO+4mM/Mk7u4etbapPXt+4bHHZliX4ebmzr33Tqq13PXr19CmTRQjR95FXl4Ozz77CG+99SFms5mrrx5KREQb3n77b+zYsQ1vbx/uvXcyrVpFsnDhHHbv3kGnTl2syxo+/Daef34K3br1JioqmhtvHImdnR3vvfcWI0feRadOXdi0aS1r1qwkNDQMgGuuuY4VK5YyfvwU4NQHRFdddQ3t23fi88//wfXX38qAAcNYuXI5t932F2Jjf8bFxUSfPv146aUnzlhuWFgErq5ujBlzHykpiSxZMp8XX5xLcHAo9933CDt3/gic+jDn2LHDzJz5JgDjx9/BoEHXU1VVSb9+gwkMDGbWrKdITDxK27Yd6nxMBgwYxtat39Y67dprRwBQXV3Nrl3bueuuB2jZMoJ161bz8svTOHEimalTn/sDzwQREWOoeIpIg3nnnTm4u3tQU1ND587d6dGjD9u3f09IyKk3jOnpJzh2LJ7U1GQAHBwcgFP7bqWlpTBr1lOYzWby8nIZMGAY6ekn8PcPAqB//yFn3N5vRzxfffVZunTpiaurG+npJ8jJyeL112cA4ObmQV5eLhMmPMHKlctZvnwhXbr0tI7OFhcX4ebmbp0KHBwces71zMhI5+23/4aDgwMVFRVnnD9o0PXW0acvvviYFSuWMnz4rdbbMplMODk5W28rIyP9rOv6t789x7XX3vy7psNWVlawePE8wsIiuP76WzGbzbz33lvMm7cMNzd35s17mQMHfq1VBMQ2/etfq9i16yfr71OnPg/AokVvYDK5UlCQT2VlBTfccDufffYB06Y9REBAEPff/2it5VRVVWKxWLCzs2P9+jV8//1GCgsLGD/+v5c7eTKV6OhuADRr5ktBQZ71vNDQcAD8/QPJzc2mefMWrFr1D9zc3Dl0aN8ZhWzUqLFUVVVy5Egcv/66kyVL5jFr1tukpZ1gzZqVrFv3BZWVFQQGhtS53mazmW+++ZKQkJasXfs56ekn+PnnbfTseUWdlz/bcv38AgBwdXWjsvLMbfr0egcHt7T+7ucXQG5udq3rm0yudb4mnE9GRjqLF8/lttvG0Lp1O7788lO6du3FnXfeQ1ZWBq+8Mp25c9+74OWKiBhJxVNEGsxvp9r+lp2dHQBBQaH4+QVyyy2jqaqqIjMzHYB161bz2muLrGVs6dIF7N8fS1BQKKmpyURFRbNu3Wq6deuNnZ0dZrP5jNu4++6HePbZyfTseTnBwS0JDm5pHblJSkogICCI2NifmTTp1JS96dMnkJSUAIC7uwfFxUWYzWbs7OxISUkEwNnZhYqKcuDUG0OA3bt3kJl5kieffImUlER++mnLOe8Tb28fTpxIsv7u4eFJeXk5VVWVODk5k5qawpVXDqxzXQHefHM5zz8/hejorrRr1/H8D8J/VFZWMHPmNEaNGkuHDjEA/9lnrwKTydWa5fTIqti26667pdZU24qKcj76aDHLlq2mpqaaH374N2azmaSkBG655f8YM+Z+Pv74PTZvXl9rm+rd+0o+++wD7rjjboYOvZGhQ29k7tyXKCsrsy47KCiUtLQUALKzM2tNI8/ISCM0NJzMzJN069abhQvnMG3aTIKDWzJr1lNYLBbrZbOzM9m48WtGjRpLx46d6dixMzt3/kRGRjohIS0ZOfKvREa2Jz8/Fzs7O/bs+eWM9d6xYyt9+w5g1KixwKl9xWfNeppevfrWuq3T61fXcvfu3V3nfWpnZ4fF8t/XmqCgUP71r1W18p8unH9ESkoiixbNZerU52jWzBeAoqIC6/3q4eFJYaG2UxGxPSqeImKYoUNvZN68l3njjRf/s0/WTWRlZRAQEGQtnQADB17LBx8s5O67H2LJknk4O7vg6urGkCE3EBnZno8/fg9//8Bayw4ICKJ37yv5/PN/cOed99C2bRSzZz9DVVUlISFhjB37MIcPH2Dt2s9xcTHRrJmvdSTW3t6em28ezVNPPUxgYDA+Ps2xWCx07dqTtWs/57333sLBwQGLxUJkZDtWrHiXt956laCgEIKCQvn3v7+plWX9+i+tR7itqalh8uTaR9+8995JzJw5DU9PL0wmV3r06IOfX8AZ6wqnRmCeeOJFXnjhMV555S3rfqG/9e9/f8OWLRvJzDzJzJnT6NKlJ2ZzDSdOJPHpp+8D4OPTnEceeZqRI//KzJlP4u7uQWVlJT169PljD6o0Si4uJlq1asPcuS/h5eXDVVddw2effUDv3lfy4YeLrQeteeihxzl6NM66TU2Y8ARLly5g6tT78PDwBCAmphvduvXmhx++A7CW0TfeeJGCgnweeODUdHcHBwe+/34TGRlpFBTk06tXX5KSEnjnnTmEhobRrl1H1q9fQ8eOnQHw9fWjqqqSZ56ZhMnkSmVlJV279iI8vDV33/0Q7777Jh4enuTn53LffY/UuZ6ff/6PWlOD3d09cHFxITHxKCaTK4sWvcGIEaOIjf2ZjRu/rvdyAdq168hrrz3PFVdcDUBERBvatIli1qynqa6u4qab7sTNzb3ej4nZbObFFx8nJyeLkpJiUlISmTz5KRYvnkdJSRHz578CQNeuvRg+/DbmzXuZuLj95Ofncd99k+t9OyIijYWd5bcfAYrIJSs/q4qf1+fR41p/o6NIE3B4Rz5BYU5ERNf/jfilbN2yk8QM9MPVw8HoKHIJsZgtfP9pOrc8XP8jf4uI/F4a8RQRsVHr168hLm5/rdMcHR0v+GsiRKThFBTk8f7775xxet++Ay7qV7OIiDR2Kp4iIjbq9D53ItJ4eXs3O2N6vYjIpejs394uIiIiIiIichGoeIqIiIiIiEiDUvEUERERERGRBqXiKSIiIiIiIg1KxVNEREREREQalI5qKyIiDeK7774jINGOFi1a4OvrS4sWLWjWrJnRsURERMQAKp4iItIgYmJicPLJIzs7m8TERLKzsykpKaFFixa1yujpf+3s7IyOLCIiIg1ExVNERBqEn58fEdGtap1WWVlJdnY22dnZ5OTksG/fPnJycsjNzbUW0P8tpU5OTsasgIiIiFw0Kp4iIvKncXZ2Jjg4mODg4Fqnm81mayHNzs7myJEj/PTTT+Tk5ODl5YW3tzeBgYG1SqnJZDJoLURERORCqXiKSC3Hdhc2yHItFgsWiwV7ex3TTM5kb2+Pv78//v7+Z5yXm5vLoUOHsLOzIykpid27d5OdnY2Li4t1hDQ7O5u+ffvSokUL3N3dDViDPy4tvuR3X9disQBourKNMJvN2NnZ6fESkUuKneX0XysRuaTlZ1WRdaLioi6zpLiE9JPppKenU1hYSFBgIFEdOuDm5nZRb0caJ0cnOyKiG64EFhYWWkdI9+/fj6OjI9nZ2djZ1T6g0ekfLy+vBsvyRx2JLa73Za2jw1mn1j0vL48WLXzx8/enXbt2DZhSLpa9e/eSmpqKh4cHoSGhhISG4OzsbFiett08DLttEbl0qHiKyEWVk5PD0aNHOXz4MNnZ2bRv35727dvTpk0bo6PJJaK4uJicnJxaU3ezs7OpqqqiRYsWODo60qZNG2sh9fHxMTryeZ04cYKkpCSSkpJITEwkNDSU8PBw649GzmxTfHw8Bw4c4ODBg3Ts2JFOnTrpwwMRabJUPEXkD8vNzSU+Pp7Dhw9TXFxMREQE7du3JzIy0uhoIlZlZWVkZ2eTnp5OQUGBtZCWlZXVGhk9/dO8eXPDsmZkZFhLZmpqKl5eXoSHhxMWFkarVq0MHR2Ti6+iosJaQDMzM4mJiaFTp05n7AstImLLVDxF5Hc5evQoWVlZHD58mPz8fOvIZuvWrY2OJnJBKioqrEfZ/e0IaWVlJR4eHvj5+eHn59eghTQ/P5/ExERr2bSzs6Nt27aEh4fTqlUrTU+/hOTk5FhLqKOjIx07dqRly5a0bNnS6GgiIn+IiqeI1FtpaSkHDx7k4MGDFBYW0qZNG9q1a6eyKU1SaWkp+fn51iKalZVFVlYWpaWltGjRolYh9fPzw9vbu97LrqioIDExkePHj3P8+HEqKyutJbNVq1Y2Mf1XGl5SUhIHDhzgyJEjhISE0LlzZ03FFRGbpeIpIudkNps5dOgQBw8e5NixY3Ts2JEOHTrQtm1bo6OJGOL0COnpIpqVlWUdIfXx8SE4ONg6Ourn54enpycA+/btIzc3l6SkJNLS0qwl093dnZiYGIPXShozs9nM3r172bt3L8XFxcTExNClS5dGfcAsEZH/peIpInVKSEiwjm6Gh4fToUMHOnbsiKOjvoVJpC65ubkUFxdbS2laWhrHjh0jPz+f8vJyWrRoQXBwMD169KBLly54eOhIonLh0tLSrCU0MjKSmJgYjYKKiE1Q8RQRq9TUVA4dOsShQ4fw9PS0jm6eHrERkXPLyMjg2LFj/PrrrxQWFtK6dWuCg4Px8vKivLzcWkqzs7NxcHDA2dmZNm3a4O/vj5+fH/7+/vpwR+qlpqbGWkArKiro0KEDnTt3vqAp3yIifyYVT5FLXGlpKfv372f//v2Ul5cTHR1NVFQU/v7+RkcTafRqampISEggISGBY8eOAdCmTRtcXFzo16/fOb/mpLCwkPj4eKqrq8nMzLT++Pr64ufnR0BAgLWMap9POZc9e/Zw8uRJ9uzZQ8eOHenatSthYWFGxxIRqUXFU+QSlZycbP20PDo6mpiYGCIiIoyOJWITjh49yqFDh4iLi8PX15cOHToQGRl5UT6w+W0JzcrKIjMzk4qKCvz9/TGZTLRp08ZaSF1cXC7C2khTUV1dze7du9m1axfe3t706NGD9u3bGx1LRARQ8RS55Ozbt4/Y2FhKS0vp2rUr3bp105tXkXrYunUrJSUlHDp0CF9fX6Kiov60qeilpaVkZWWxY8cOPDw8rMX09Ne9/HaqbosWLRo8jzR+Bw4c4Oeff8be3p7evXvToUMHoyOJyCVOxVPkErFz50527tyJl5eXPgUXqae0tDTi4uI4dOgQrq6uREVFERUV1SDf5fl75Obm1hodzcjIoKioyFpC/f39qayspGfPnphMJqPjigHi4uL48ccfcXFx4fLLL9fXX4mIYVQ8RZq4Xbt2sWPHDvz8/Ljsssu034/IeVRVVVn3e87Pz7dORbeVkcTKykprCc3KyiI+Pp7y8nLc3d0JDAys9ePm5mZ0XPmT7N27l61btxIWFka/fv10ECIR+dOpeIo0UYcOHeKHH37A09OTvn37EhoaanQkkUYtNzeXPXv2sGfPHlq2bEnXrl2b1PfVZmdnc/LkyVo/bm5uKqOXmM2bN7Nz504GDhxI9+7djY4jIpcQFU+RJiYrK4utW7eSk5ND//799f1uIueRmZnJrl27iI2NpVevXvTq1euSOYpsTk6OtYRmZGSwb98+QkJCCAwMJCAggICAAAIDA/Hy8jI6qlxE6enprF+/nmbNmjF48GB92CAifwoVT5EmZNeuXWzatImrrrqKK664wug4Io1aYWEhO3bsYOfOnfTt25crrrhC36HJqfslLS2NtLQ0Tp48SXp6OiaTqdaoaEBAAB4eHkZHlT9o06ZNHD9+nOuvv57g4GCj44hIE6fiKdIEmM1m1q1bR0ZGBsOHD9d3cIqcx+7du9myZQudOnViwIABODk5GR2pUSsoKCA1NZX09HTS0tJIT0/H29uboKCgWoXU2dnZ6KhygWJjY9m8eTO33nqrjgEgIg1KxVPExpWVlfHll1/i4eHB8OHDjY4j0qgVFxezceNGCgsLGTZsGAEBAUZHslk5OTmkpqbWGh0NCgoiODjYWkj1IZhtOHDgAN9++y2jR4+2mYNoiYjtUfEUsXErV6607qcjImeXlZXFmjVriIiIYODAgUbHaZLS0tI4ceIEqamppKamUl1dbS2ip//V/oSN0/bt2zl+/DijRo0yOoqINFEqniI2bPPmzeTm5nLLLbcYHUWkUSsrK2PFihVER0dz2WWXGR3nklFaWsqJEyesP6mpqQQEBBASEkJwcDDBwcH4+voaHVP+Y/ny5fTu3ZsOHToYHUVEmiAVTxEbVVJSwvz585k8eTLu7u5GxxFp1L799lsqKyu59tprjY5yyUtPTyclJYWUlBROnDiBo6MjoaGhhISEEBISQlBQkNERL1mxsbGkpKRw4403Gh1FRJogHb5PxEYdOnSIDh06qHSKnEdRURG//vorkyZNMjqKAEFBQQQFBdG7d2/g1IGLkpKSSE5O5ueff6aqqoqWLVsSGhpKaGiojrb6JwoPD+fXX381OoaINFEqniI2Kjk5mY4dOxodQ6TRS0hIIDg4WF+V0kh5e3vTuXNnOnfuDJyanpuYmEhiYiK//PILVVVVhIWFER4eTlhYGH5+fgYnbrq8vb0pLS01OoaINFH6KyxigyoqKkhNTdW+nSL1kJSURKdOnYyOIfXk5uZGx44drR+s/XZEdMeOHVRXV1tLaHh4OM2bNzc4cdPh4OBAVVUVVVVV+oohEbnoVDxFbFBeXh4VFRVGxxCxCWlpaQwdOtToGPI7/e+IaG5uLsnJySQnJ7Nt2zbs7e1rjYj6+PgYnNi22dnZqXSKSIPQwYVEbMzw4cOxt7ensLAQf39/7O3t+eSTT4yOJdLojBgxAmdnZzIyMvDz86OiooKvvvrK6FhykeXk5FhHRJOTk3F0dCQ8PJyqqioGDRqEh4eH0RFtQv/+/XF3d6empob169cbHUdEmiCNeIrYmOjoaDZt2gRAcXExEyZMMDiRSON0zTXXsHz5cuDUUaBvuukmgxNJQ/D19cXX15fu3bsDkJmZaZ2Wu2jRIpo1a0arVq1o1aoVERER2NnZGZy4cWrTpo31wEI9e/Zk1KhRTJ061eBUItKUqHiK2JibbrqJX375hYKCAmJiYhg4cKDRkUQapeuuu47t27dz+PBhAgMDNd32EuHv74+/vz89e/YEICUlhePHj7Nt2zZWrlxJREQEbm5uXHbZZfj7+xuctvGIjIy0Fs8hQ4aodIrIRecwY8aMGUaHEJH6CwgIID4+noSEBEaNGkW/fv2MjiTSKDVv3pzMzExiY2O55pprGD16tNGRxADe3t6Eh4fTpUsXevbsiclkYt++fRw8eJDt27eTlZVFZWUl7u7uODs7A6c+4IuLiyMyMhJvb2+D1+DPUVFRwaZNm+jatSuTJ0+mWbNmRkcSkSZGI57S6CTsKzE6QqM35IoxFKSZ6NRqoO6vemgd0zS/61SP/fl1ajWQrm1P0LfLzbq/6qGpbiunmUwmoqKiiIqKAk7tH5qYmEhcXBzffPMNzZs3JyIigvT0dFJTUyktLeXBBx+CkqY/MupaE050xDXcft0oKAloMttLU39Oi9gSHVxIGp0v30kjIMLN6BiNnsVi0b5K9WCPmd7DmubXLRzeWURedo3RMRo9iwW0qZyfiwt0G3BpHxE2OTmZxMREpkyZYj2td+/L6Bf5OK06Xdr3jS1y97Aj5spLY8RaxBZoxFMapYiuXjg62RsdQ2xcxvEyirPLjI7RoOwd7Yns5mV0DLFxqfEl1JRXGR3DcGFhYbzwwgsAODs7ExgYSElJCU5OTrTtpeJpS5IPFgP6YE6kMVHxFBEREfmPpUuX8uGHHxIZGYmvry/BwSFsWFZgdCwREZun4ikiIiLyG3/5y1+s/z+1Q5KKp4jIH6W5jCIiIiIiItKgVDxFRERERESkQal4ioiIiIiISINS8RQREREREZEGpeIpIiIiIiIiDUrFU0RERERERBqUiqeIiIiIiIg0KBVPERERERERaVAqniKN0CefLOPo0bhap61bt5pPP33/gpazZcsGPvpo8TkvM27crRe0zPnzX2Hv3l0XdB2xbVu3fsv77//9T7mti/H8Or2tFBTksWDB7IuUTKTpuvvuEWc9rz5/R0RE6sPR6AAiDemf//yQjRu/JigoFIvFQmVlBZMnP01gYDAAFosFOzu7cy5j8eK5WCwWxo+fctbLZGdn8umn7zNhwhM8++wjlJeX4e7ugZ2dHcXFRYwceRe9el1R79x33nlPvS9bH/HxB3nuuUeJiooGoKiogAkTnqB163YX9XZ+q7S0hDlzXsBisVBYWMDo0ePo3v0ybrttINHR3QAICAjiwQcfo6yslAULZlNRUU5xcRFTpz6Pv39gg2WTuu3evYOVK5fj6OhEZWUFrVu3Zdy4iUbHOsOePb+wYcNX3HDD7XToEHPWy3l7N2PixGkXtOy5c19i7NiJeHv7/NGYVr99fajLH9lWHn/8BVq08L9oWaXhnf4b4eHhSVlZKRERbRg/fgrx8Qd59tlHaNeuo/WyAwcOIyamOy+99ATz579f5/Li4w8yefLdvP32P2jdui0AJ0+mMW7cLbz6oCrZiwAAIABJREFU6kKio7v+GatVi8ViYcOGr1ix4l0efnia9e/f0qULOH78KNXVVfTo0Yfbb/8rq1at4NChvbi6ulFeXsaUKc/h7OyCvb3GRkSaGhVPafKGDr2JW24ZDcDmzetZuXI5nTt3Z/PmDbi6ujFlyrN8+ulyUlKOU11dzeWX92fQoOuBU3/QQ0PDKSkpJikpgfDw1pSXlzF37kzAQlFRIVOnPs8HHywkLm4/27d/D8CDD061lrpffvmRdeu+oFevK9ixYytr167Cy8sbDw9PHnhgKseOHeaDDxbh6elFfn4ujz02g2XL3uaaa67D0dGJZcveJiKiDYWFBUREtOHkyTTmzZvJ7NmnRqDGjbuVpUs/Z/Pm9Xz33Te4uLjg49P8jDe5UVHRzJgxB4CPP36P2NhfahXPpKQEli5dgI9PMwoK8pk4cRoeHp5nrOtpOTlZzJ79DE888SJ+fgFn3O8JCUfo338I/foNIjHxGO+++yZRUdGEhUVYc5z21VefERPTjWuvvZkTJ5KwWCx/4BGX3yM9/QQLF87htdcW4e3dDIA1a1Zy9OhhAI4fP8qiRW9w6NA+br55NP37D67zObdgwWycnJxwd/dg375YJk2ajoeHJ88++wi9e/clKysDd3cPxo+fUuf2UB9btmxk4sRpLFky31o8Dx7ce8a2kp2dyezZz/DYYzPq3GY2bvyaHTu24ujohKOjIzfccDs7dmyjpqaGsWMf5rnnHqVPn34cP36Eli1b4ezswsGDe7nppjvo1esKPvpoyRmvGxMmjKF//8EUFhaQkBDPCy/MrfX60KdPvzPWR9vKpef034iqqkruvnsE99//KAAdOsSc8ZhnZ2eed3nR0d1Ys2YljzzyNABr135OVNSpbaOsrJS5c1/CZHIlLy+X22//C5079+C1157HYrHg7x+I2VwDwOuvz2DYsBFER3dl7drPqaiowNe3hfV26nrO1+X0h6+9e19pPa2gIA+TyZWZM+djNpu5554R3H77X/nmm9W89daHODu7MGPGVJKTj9OyZStcXd0u4B4VEVug4imXlLy8XDw9vXBwcMTJyYknn3yJxMRj7N8fy+zZf8dsNvPww2Po128Qzs4urF37OQ88MJXKygo++mgJEyY8wfr1a2jfvhO33DKagwf3kpOTxYABw3BxMdGnTz/Wrl3FO+/Mwd3dg8zMdAICgrn33kkALFz4Bu+8swKTyZU33niR/ftjiYvbT1RUNHfccTeZmSdxcPjvZvnxx+9Z36AsXjz3nOtWVVXF9Okv4+rqxkMPjaawsKDW+UeOHGLWrKcoLS0lK+skr7zyVq3zly9/h9GjxxEVFc2WLRtYtWoFAQFBZ6wrQGFhAR9/PIMpU56ts3QCtT5l37btOy6/vB9FRYXk5+fx2mvPU1xcxIABw7j66iEcOPArnTp1Yf78V6ipqW6Uo2xN3Z49OxkwYJi1dALceONI4NRUW5PJxPjxU0hJSWTJkvn07z+4zuecvb09ERFtGTr0Rtau/ZwdO7YyaNBwCgryGDPmfuzs7Lj77hGMHz+lzu3hfJKTj9O8eQtcXd1wc3MnM/Mk/v6BF7StnPb995sYOfKvxMR0JyEhnvDwSIKDQ7nvvkcAKCoqZMyY+0hMPMbrr8/grbc+5NChfaxf/yV+fgF1vm5UVVXSr99gAgODmTXrKRITj9Z6fajLH9tWJtVrXaVxWbp0Ae7uHiQmHmPEiDvPO/PmfEJDw8nISKOoqBCTycSRI4do2zYKgPXr19CmTRQjR95FXl4Ozz77CJMmTaekpJgZM+ZQVFTI6tWfnPc2zvW38n+5urpx5ZUD+fXXndbTvL2b8X//dy8A+/fH0r79qRk4Y8c+zOTJ9+DvH4i3t0+tEV8RaVpUPKXJW7/+S/bu3YXFYsHX14/77pvMzp0/ERISBpwa6cnJyeL112cA4ObmQV5eLgD79sUyb95MAI4ePUxBQR7p6SeIiekOQMeOnQGIjf251m2efgO8bt1qkpMTCAkJo6SkmKKiAt5661UA8vPzyM/P44Ybbuezzz5g2rSHCAgIsn7yDZCdnYGf36kpp0FBoZSWlpxzXRctegOTyZWCgnwqKytqnde2bQemT38FgISEeGbNeprXXltkPf/kyVTrfRIYGMLWrd9iNtecsa7r1q1m2bK3CQ9vTVBQ6DnzVFZWsHjxPMLCIrj++luprKxg2rSZtGvXkcrKCiZNuovOnbtTXV1FSEgYI0fexebN6/nHP97loYceP+ey5eKqrq6ipqbmrOeffqxdXd1qPbfqes6dniZ9+nSAFi38rW+uHRwczro9nM8XX3xMTk4Ws2Y9RUlJMV9++Sn33Tf5grcVgAkTnmDlyuUsX76QLl16njH1/PRIj7OzM82a+QLg4uJCRUXFOV83Tn8YYzK5UlFRezs8m9+7rXz22QfWD7bEdowbN5HWrdthNpt5+eVp7Nu3GxcXE4cO7ePpp//7wdvNN4+mVavIei1zyJAbWb/+S3x8mtO//xCSko4Bp17bT0/ZbtbMl4KCPLKyMggICALA09MLDw/P8y7/bM/508uprzVrVpKYeIzHH5+B2WzmvffeYt68Zbi5uTNv3svs3PkTUVHR9cokIrZFxVOavN9Ota1LcHBLgoNb8thjM4BTU04DAoJYtOgNpk59nk6dugCnpsx+/fXnBAWFkpqaDMDevbusb9bNZnMdt30jEyaMITn5OGFhETRv3oLJk5/GycmJ9PQT+Pr6kZh4jFtu+T/GjLmfjz9+j82b11uv7+vrR1bWSTw9vUhOPk6LFv6YTCYqKsqBU1OoiooKqago56OPFrNs2Wpqaqr54Yd/15nnNC8vH/Lzc2uddnq9oqKiSUtLITi4Jb6+fnWu64MPPsaePb+wevUnjBhxZ523UVlZwcyZ0xg1aqx1OmReXi4nT6bRrl1HnJyccXR0wmKx0KZNlHXZ7u6eVFVVnTW7NIzu3S/j6acnMWzYTdZ9Bj/5ZNlZ3/Re6HPuf7m7e9S5Pezatf2s18nPzyUnJ4sXX/zviObTT0+krKy0zm3ltLq2GTj1RnrSpOkATJ8+gaSkBOzs7LBYzNjZnXv/srO9bpzNue6bP7Kt1NRUnzOnNG729vZ4enqRm5tDUFDI755qC3DllQN5+umJuLiYePrpWSxb9jZw6rU9LS3FuixfXz98ff3IyEgHTm1Xp7cJZ2cX67aSkZGOj09z6/Iv9Dlfl0WL3qBly1bW7a6qqoqKigpMJlcAPDw8KSjIo7paz2uRpkjFUy554eGtads2itmzn6GqqpKQkDDuuONuYmN/rjX62KNHH957bwGvvrqQBQtmMWvWUxQVFTJlynPU1NQQG/szGzd+XWvZ9vb2jB07gYUL5/DKK28xduzDvPLKdFxcTNTUVDN16vMUFOTz4YeLrftWPvTQ49Yj2t5xx90sWDCb8PBIKirK8fX1w8enOb6+fvz976/h5eWNj08zXFxMtGrVhrlzX8LLy4errrqGzz77oNYUvri4/cyYcWofuvLyMh5++MlaWe+660GWLl2At7cPJSXFTJp0KuecOS/UWlc4NQJ0zz0TmD59AlFR0daDFv3WmjUrOXEiyXok3lP7AD7Otm3fsn79l1gsFgYOvBZfXz9uvXUM8+e/zA8/fEdxcRETJ07/4w+sXJCQkDAeeugxXnllOk5OztTU1NCpUxd69erLjz9uPuPyZ3vOXYi6todzWbNmJX37Dqh1Wu/eV/LNN1/Wua2cVtc2A3D48AHWrv0cFxcTzZr5EhISRrt2HXnttefPeiCg0+p63Rg79uE6Lxsc3NL6+jB48PA61+v3biuTJj11zpzSOJ3eHaO6upoWLfzo23cACQnxf2iZjo6OxMR0p7S0BBcXk/X0oUNvZO7cl3jjjRcpKMjngQem0r59J5ydXXj55Wk0a+ZrHaW/+uohrFixlAMH9pCdnVlr6v2FPOcTEo7wwQcLOX78CEePxrFt27dcf/2tbNz4NdHR3fj55x8AeOSRpxk58q/MnPkk7u4eVFZWctddD+Do6PSH7gsRaZzsLDoygTQyX76TRp+bA3F00hHt5I/JOF5GcXYZvYc1P/+FbdDhnUUU5FuI7OZldJRGKTPzJK+/PoO//W2h0VEavdT4EmrKq+g24OIdzbepsFhg1YJUrv6/EKOjyAVIPliMo10NMVd6Gx1FRP5DI54i8occOPArmzatPeP0O+642/q1NSIXoqAgj/fff+eM0/v2HUDPnpfXezlz575E1669Lma0P2T9+jXExe2vdZqjo+N5R1dFoHE+fxpjJhFpvDTiKY2ORjzlYtGIp0j9aMTz7DTiaZs04inS+OidvYiIiIiIiDQoFU8RERERERFpUCqeIiIiIiIi0qBUPEVERERERKRBqXiKiIiIiIhIg1LxFBERERERkQal4ikiIiIiIiINSsVTREREREREGpSKp4iIiIiIiDQoFU8RERERERFpUI5GBxCpy861mUZHaNRqasxUV1fj4uJsdJRGzy+kad9HWUmlZCWVGh2jUauoqMTZ2Qk7OzujozRqwa1NRkdo1LavPml0BLlAYe1djY4gIr9hZ7FYLEaHEPmtsuIaoyM0enl5eaxevZp77rnH6Cg2wdXDwegIDULbSv38/e9/Z9y4e/VBTT001W3lj7BYoLxE25qt0nNapPHQiKc0OvojcX5llfbgUKn76hKnx79+LPYVuHrY4+Ki+0sunJ2dtjURkYtB+3iK2KDCwkIqKiqMjiFiE+zs7LS9iIiIGEzFU8QGeXl54eLiYnQMEZtgsVi0vYiIiBhMxVNERJo8Hc5ARETEWCqeIjYoPz9fUwdFLoCOaCsiImIsFU8RG+Tj46OpgyIiIiJiM1Q8RUSkSXN3d9cMAREREYOpeIqISJNWUlKiGQIiIiIGU/EUERERERGRBqXiKSIiTZqjo6Om2oqIiBhMxVNERJq06upqTbUVERExmIqniIiIiIiINCgVTxEREREREWlQKp4iItKkmUwm7eMpIiJiMBVPERFp0srLy7WPp4iIiMFUPEVERERERKRBqXiK2KD8/Hzs7OyMjiEiIiIiUi8qniI2yMfHB4vFYnQMEREREZF6UfEUERERERGRBqXiKSIiIiIiIg1KxVNERJo0Nzc3fZ2KiIiIwVQ8RUSkSSstLdXXqYiIiBhMxVPEBuXn51NZWWl0DBERERGRenE0OoCIXDgfHx+cnZ2NjiHSqOXm5gJQUVFBXl4ezs7ONG/e3OBUIiIilyYVTxERaZKmTJnC/v37Afjiiy9o3bo1K1euNDiViIjIpUlTbUVEpEkaOXJkrd+HDBliUBIRERFR8RQRkSapV69e9O7dG4COHTsyYMAAgxOJiIhculQ8RUSkSfLz82PgwIEADBo0iMjISIMTiYiIXLq0j6eISCOXHFdKwr4So2PYJM/Ky/m/Ia/ga2nP5s+yjI5jk4IiTLTv6Wl0DBERsXEqniI2KD8/H4vFYnQM+RM5mhwJauNudAyb1Kb71UZHsFk5qeVGRxARkSZCxVPEBvn4+GBnZ2d0DPmT+fjrK3Tkz6XiKSIiF4v28RQREREREZEGpeIpIiIiIiIiDUrFU0RERERERBqUiqeIiIiIiIg0KBVPERERERERaVAqniIiIiIiItKgVDxFbFB+fj4VFRVGxxARERERqRcVTxEb5OPjg4uLi9ExRERERETqRcVTREREREREGpSKp4iIiIiIiDQoFU8REWkQ48bdetbztmzZwEcfLa7XcubPf4W9e3f97ttPTDzGQw+NZteu7Re8jN9j5swn/5TbERERsSWORgcQEZE/1623DqBjxy7W3yMi2jB27MP1vv6MGVOZMWMOADt3/sSqVf/AwcGR4uJCQkLCePDBx3B397jouXNzs3nzzVk4OztjNpspLy/jkUeeoUUL/3Ne7/DhAwwYMIwePfoAUFJSzNKlCwgKCuH22/8KwLff/gtnZxeuuuoa5s59ibFjJ+Lt7VPvbNnZmXz66ftMmPAEzzzz6u9fyd9ISDjCkiXzcHNzp6SkmClTnsPfP5DbbhtIdHQ3AAICgnjwwcdITU1m4cI3cHBwwNnZmSefnImDg8NFySEiInIxqHiKiFxiTCZXXnppXq3TyspKee2153F1dSM7O5P773+EyMj2vPPO65SUFFNUVEi3br3x8PBk//5Y/vnPD7nssqt4770FvPrqQjw9vQDYvHkD8fEH6datt3XZ27Z9x8aNX1svM2XKcwAcOrSPBQtmk5SUwM03j6Jv3wEsXjyPnJxMiouLuPrqoQwePNy6nM2bN9CpUxdrWYyN/ZmcnCxatPDno4+WkJJynOrqai6/vD+DBl0PQHFxERs3fk11dRVt23aga9derFv3BSNG3MmqVf+gurqa8vIyVq1agZeXN2azmR07tlFTU8OUKc+xYsXSM5Y7YcIY+vcfTGFhAQkJ8bzwwlw++GAhcXH72b79exYufIP3319NUlICS5cuwMenGQUF+UycOI0DB35ly5YNtG7djri4/QwePJz+/QfX+TgdPnyA++9/lIiINqxZs5ItWzZw/fW3EhYWYS3+p73zzus88MBUQkPDOXhwL5WVFZhMJuzsNLFJREQaBxVPERGhoCCfa68dQa9effnxx818882XPPTQ4/zyy4/MmbMEH5/mJCTE06pVG1auXM5tt/2FL774mKuvHmotlABXXz3kjGW/++6bLFmyEicnZ+bPf4VffvkBOFWAJ06cRkFBPk8++QB9+vQjNDSM++9/hOLiIqZOvbdW8bziiv688cZL5OZmExUVQ5cuPfDxaU5i4jH2749l9uy/YzabefjhMfTrNwgADw9PrrnmOkpKiujatRfV1dUkJBzhttv+Qp8+/di6dRMDBgyjV68riIxsz1VXXcOXX37Cffc9QnLy8TqXW1VVSb9+gwkMDGbWrKdITDzKgAHDcHEx0adPPxYufAOA5cvfYfTocURFRbNlywZWrVpBhw4xuLq6MWbMfaSkJLJkyfyzFs9rrx0BQHV1Nbt2beeuux6gqKiQ/Pw8XnvteYqLixgwYBhXXz2EzMyT/PjjZhITj+HnF0BUVLRKp4iINCoqniIil5jy8jKefnqi9feuXXsxcOC1/PjjFnbu/Ins7Ew8PDyxs7NjypRneeed18nPz+PGG0fSqlUb6/VqaqqpqakBTpWj559/FIvFgpubu3W6aXFxESaTCScnZwCCg0PJyEjH29uH0NBwALy9fSguLgIgIyOdt9/+Gw4ODmd8V21gYAh/+9tCMjLSiYvbz9/+9hz9+w/By8ubnJwsXn99BgBubh7k5eXWue5btmwgLS2FWbOewmw2k5eXy4ABw+q8bHr6ibMu188vADhVns/2nbonT6YSEhJmzb5167d06BBjva6rqxuVlef+Pt6MjHQWL57LbbeNoXXrdlRWVjBt2kzatetIZWUFkybdRefO3cnMPMmQITfg49Oc+fNf4ccfN9O9+2W4ubmfc/kiIiJ/FhVPERtkZ2dndASxYSaTKy+/vKDWae+++ybt2nXk2mtHsHbt5xw+fICyslIcHBx56qlZVFZWMH78nVxxxdVYLBYA+vTpx/PPT2Ho0Btp3rwFL7+8gKysjFoH1/Hw8KS8vJyqqkqcnJxJTU3hyisHUlJSRHp6KnBqtNXT05vdu3eQmXmSJ598iZSURH76aUutjF999RkxMd1p1SqSgIAgvL19WLv2c8aMuZ/g4JY89tgMAJKSEggICKpz3detW81rry2yFuGlSxewf38sdnZ2mM1m4NT2ZbGYCQ5uWe/lAtbrnxYUFEpqajJRUdGkpaUQHNzyXA/LGVJSElm0aC5Tpz5Hs2a+AOTl5XLyZBrt2nXEyckZR0cnLBYLkZHtrI+Lu7sHNTXVF3RbIiIiDU3FU8QG2dvbn/EmV+SP6N79Mj76aDHJyQmEhUVw9GgcqanJfPnlp3z11WfU1FQzZMgN2NvbYzK5smjRG4wfP4UJE55g9uxncHR0xGRyxcnJiUceeabWsu+9dxIzZ07D09MLk8mVHj36sHnzempqqnnnnddJTDzGXXc9QGRkO1aseJe33nqVoKAQgoJC+fe/v7EuJzq6G++++yYuLi7Y29tTUVHBuHETCQ9vTdu2Ucye/QxVVZWEhITVebCkPXt+ISAgyFo6AQYOvJYPPljINddcx8cfv4e/fyDt2nXktdeeZ9q0l+u1XIDg4JbExv7Mxo1fW0+7664HWbp0Ad7ePpSUFDNp0nT27t1d78dk8eJ5lJQUMX/+K8Cpkenrr7+Fbdu+Zf36L7FYLAwceC2+vn48+OBjzJnzIiaTCQcHR+6660EcHfUnXkREGg87y+mPSEXEZhQVFbFs2TImTZpkdBT5EyTHlZJ8pIIOVzQzOopcYo7FFuLtY0f7np5GRxERERunj0NFbJCDgwPV1ZpKJ9IUrF+/hri4/bVOc3R0ZMKEJwxKJCIicvGpeIrYICcnJxVPkSZi6NAbGTr0RqNjiIiINCgda13EBp0unpopLyIiIiK2QMVTxEY5OTlRVVVldAwRERERkfNS8RSxUS4uLpSXlxsdQ0RERETkvFQ8RWyUyWRS8RQRERERm6DiKWKjXF1dVTxFRERExCaoeIrYKDc3N0pLS42OISIiIiJyXiqeIjbKzc2NkpISo2OIiIiIiJyXiqeIjfLw8FDxFBERERGboOIpYqM8PDwoKioyOoaIiIiIyHk5Gh1ARH4fLy8v4uLijI4hf6L8zEqjI4iIiIj8LiqeIjaqWbNm5OXlGR1D/iTV5dWk7C8wOoZNOnHiBMHBIdjb2xkdxSZ5+5iMjiAiIk2AiqeIjWrevDnFxcVUVFTg4uJidBxpQGFRboRFuRkdw2a9/vpybh8xUduJiIiIgbSPp4gNCwkJISUlxegYIo1aixYtjI4gIiJyyVPxFLFhrVq14vjx40bHEGnU0tPTdQRoERERg6l4itiwtm3bEh8fb3QMkUbLbDbj6emJl5eX0VFEREQuaSqeIjbM398fNzc3jh49anQUkUbp6NGjBAUF4eioQxqIiIgYScVTxMZ17dqV3bt3Gx1DpFGKj48nNDTU6BgiIiKXPBVPERvXrVs3srKySE5ONjqKSKMTHx9PeHi40TFEREQueSqeIk3AFVdcwdatW42OIdKoHDx4kICAAAIDA42OIiIicslT8RRpArp16wbAzp07DU4i0jjU1NSwefNmevXqZXQUERERQcVTpMkYNGgQGzZs4OTJk0ZHETHcZ599RlRUFO3atTM6ioiIiKDiKdJkBAQEMGzYML7++muqqqqMjiNiiPLycj766CN8fHwYOHCg0XFERETkP+wsFovF6BAicvF8++235OTkMHLkSKOjiPypjh07xtq1a+natSv9+vUzOo6IiIj8hoqnSBP09ddfU1lZyS233GJ0FJEGZzab2bhxI0ePHmXQoEG0b9/e6EgiIiLyP1Q8RZqor776irL/b+/O46KqF/+Pvwdm2EUWkUUlccENNRU1s6tRrtlVK8tb3+7NtDIzW9TMFsvKyjJTr62WZfeWlb8W89ti2i0tcymXbpkLCpIKIiADKLLN8vujX/PTHFw5HBhfz3+CM3M+533Gx4N48/mcc8rKdM0118jf39/sOIAhNm/erFWrViklJUV/+ctfFBwcbHYkAADgBcUT8GErVqxQdna2hg0bpqioKLPjADVm69atWrt2rSIiInTRRRcpMTHR7EgAAOAkKJ6Aj9uwYYPWrFmjK664Qu3atTM7DnBONm/erI0bNyosLEzdu3dX69atzY4EAABOA8UTOA9kZmZq+fLlatWqlfr16yc/P25ojfqjuLhYmzdv1pYtW5SYmKiuXbuqRYsWZscCAABngOIJnCeqqqq0cuVKZWVl6dJLL1X79u3NjgSc1JYtW5Senq7s7Gx16tRJnTp1UuPGjc2OBQAAzgLFEzjPZGRkaPXq1QoNDdXFF1+sZs2amR0J8Ni/f79+/fVXbd26VQ6HQ4MHD1aHDh24QRYAAPUcxRM4T23atEnr1q1TQkKCUlNTuTkLTGO327Vp0yZlZWXJ6XQqJSVFHTp0UEREhNnRAABADaF4Aue5H3/8URs3btTRo0c1ePBgluCiVhw8eFDp6enatWuXSkpKFBwcrEGDBumCCy4wOxoAADAAxROAJGnbtm366aefZLfb1aVLF1144YUKCQkxOxZ8yN69e7V27VoVFRXJ5XIpOTlZrVu3pmwCAHAeoHgCOE5OTo62bNmin376SV26dFGXLl0UHx9vdizUU7t371Z6erp2796t0NBQtW7dWk2aNFHLli3NjgYAAGoRxROAV0ePHtWWLVu0ZcsWRUVFqUuXLjwHFKdUVFSkzMxM7dmzR3l5eQoLC/PMbEZFRZkdDwAAmITiCeCUtm7dqg0bNqi0tFTt2rVT+/bt1aRJE7NjoY7IzMzU+vXrVVpaqtLSUrVo0UJJSUmKjY3l8ScAAEASxRPAGcjPz9f27du1Y8cOSVLbtm0VGxurNm3amJwMtWn//v3Kzc31zGzGx8erqqpKgwYN4g8SAADAK4ongLOSnZ3teQRGeHi42rVrp3bt2ik8PFypqakaMWKE7rzzToWFhZkdFeeopKREBw4c0J49e7Rnzx5JUrNmzTwzm8HBwSYnBAAAdR3FE8A5y8zM1Pbt27V9+3Y1adJEs2fPlsVi0ciRIzV+/HiKSR22bt06zZ8/X4sXLz5u+/79+z1Fs6ioSI0bN1ZSUpKaN2+u2NhYk9ICAID6iuIJ4KyVHXGesO2GG25Qfn6+JMnf31/Dhw/X7bffXtvRcBo2/bROCxYs0LZt27Ry5Urt2bNHWVlZysrKUsOGDT1Fs3nz5vLz8zM7LgAAqMcongDOWnF+ldYsO3TctpycHFksfvL395fFYpHkVkxMjDkBUa2KoHS998kLyszMlCTFxsZq7Nixat68uZKSktSgQQP0xFX8AAAgAElEQVSTEwIAAF9iNTsAgPrNFuivboOPLZZxpmXB6dm+1q6X3/xQWfuzPNssFouGDh1qXigAAODTKJ4AcB4ae9tYVVr/qt27d2v//v3KyMgwOxIAAPBhFE8AOA81T2quxDbt1a9fP7OjAACA8wB3iwAAAAAAGIriCQAAAAAwFMUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDUTwBAAAAAIaieAIAAAAADEXxBAAAAAAYiuIJANXIydmnhx6aUO3r8+Y9pZ9/3lSLiQAAAOonq9kBAOAP06bdo/LyMoWFNVBZ2VElJbXS2LET9cEH/9aKFf+r2NgEz3tHj75TmZnpysrK0Jgx3svhBx/8Wx9//K4WLfpENptNkvTVV59p7twn9emna2vlnP4sM3OXXnttrkJCQlVaekQTJz6ivLwDmjVrupKSWkmSeva8RIMHX6Xs7L165ZXn5e/vr4CAAN1//wz5+/ubkhsAAOBcUDwB1Cnjxk1SixbJqqqq1KhRw3XbbfdKkgYNGq6rr77huPdmZqafcryEhGZas+Y/SksbJElatepLNWvWXJL022+ZWrhwviIiIlVcXKQJE6bKZrPpmWemKTY2QaGhYZ5xRo0arkWLlkqSnn76QY0cOcrzWmVlhWbPflyBgYEqKSnWjTfeqlat2nrNs3Pnr7rttnuVlNRKy5Yt0erVK9SsWXOlpQ3UqFF3HPfel19+TrffPklNm16gbdt+VmVlhYKDQ055zgAAAHUNxRNAnbJw4XyFhoYpKytDw4f/TRaL5ZzGS0sbqC+/XKa0tEHatWu7mjVrrqIiuyTprbde1g03jFHbtilavXqFPvposaKiopWaerGuvvoGrV//rfbs2XXKYyxf/omaN2+p668frby8XM2e/ZieeeZlr+8dPHi4JMnhcGjTpvW66abblZGRrl9+2aKZMx9WVVWlbrhhjFq2bKO8vFytXbtKWVkZiomJVdu2Kef0WQAAAJiF4gmgThkzZoJatEiWy+XSk09O1S+/bJYkff75R9q0aZ3nfZMmPXpa44WGhikhoZnS07fp008/0MiRo/TLL1skSbm52WrSJFGSFBfXRN999x85nQ516tRNkhQf3/S0jnHgwH5lZKQrO3uvJJ1yOezBgwe0YMEcjRhxo1q0SFZkZLTat++kJk0SlZubrenTJ+mVV95TXl6uBgz4qyIiojRv3lNau3aVLrnkstPKBAAAUJdQPAHUSX5+fmrQIFyFhYckSVdccfUJS21P17BhI/Xee2+qoqLcUzSl34tldvZetW2bopycfUpIaKawsAY6ePCAJGnfvizPe202m6qqKmWzBXheP3acmJg4XX31DaqqqlJe3vGvH2vfviy9+uocTZr0iCIjoyVJ2dl7ZbX+fg1qSEiYXC6XJKlly2S53W5Jvxdop9NxVucPAABgNoongDrl5ZdnKzQ0TA6HQ40axah37zQtXfruOY15wQUtVFhYoKuuuv647TfdNE4LF85Xw4YRKi09orvuekAul0tPPfWgMjPT1bBhpKf4DRw4VM8++4iaNWsumy3AUw7/eG3u3Cf1/POPq7CwQAMHDjuu4B5rwYK5Ki09rHnznpIkXXhhd/Xo0Vtz5z6p4OAQVVSUa9y4yZKkceMma/bsxxUUFCR/f6tuumncOX0OAAAAZrG4//itCgDOUHF+lTZ8WaRug2PMjoIzsH2tXYnJgUpsw42KAABA7WDGE0C998YbL+jw4ZLjtjVv3lLDho00KVHdzAQAAGAWZjwBnDVmPOsnZjwBAEBt8zM7AAAAAADAt1E8AQAAAACGongCAAAAAAxF8QQAAAAAGIriCQAAAAAwFMUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDWc0OAKD+2/VjkdkRapzb7ZYkWSwWk5MAAADUfxRPAOekVecQsyMYIiMjQyXFxerStavZUQAAAOo9iieAs9YwxqaGMTazYxjCXlklZ1CpWnQMNTsKAABAvUfxBAAv0tPTVVTke0uIAQAAzEDxBAAvkpOTlZ+fb3YMAAAAn0DxBAAv9u3bJ7vdbnYMAAAAn0DxBAAvmjVrpqCgILNjAAAA+ASe4wkAAAAAMBQzngDgRU5OjgoLC82OAQAA4BMongDgRUJCgmw233xUDAAAQG1jqS0AAAAAwFAUTwAAAACAoSieAOBFaWmpysrKzI4BAADgEyieAOBFaGiogoODzY4BAADgEyieAAAAAABDcVdbAPAiPT1dRUVFZscAAADwCRRPAPAiOTlZ+fn5ZscAAADwCSy1BQAAAAAYiuIJAAAAADAUxRMAAAAAYCiKJwAAAADAUBRPAAAAAIChKJ4A4EVlZaUqKyvNjgEAAOATKJ4A4EVAQIACAgLMjgEAAOATKJ4AAAAAAENRPAEAAAAAhrK43W632SEAoK745ZdfNHXqVFVWVsrpdCokJETjxo3TkCFDzI4GAABQbzHjCQDH6Nixoxo1aiS73a6SkhJZLBZKJwAAwDmieALAn/Tu3dvzdffu3U1MAgAA4BsongDwJ5dddpnat2+vhIQEXXvttWbHAQAAqPesZgcAUHcdyqlU4cHz8VmWcerW5koVFRXJWt5Mu7YcMTtQrbNaLUrqGGp2DAAA4CO4uRCAah3KqdSWb4sVHh1odhTUIqfDpbimNoonAACoMcx4Ajilll3DzY6AWpT+Q5HZEQAAgI/hGk8AAAAAgKEongAAAAAAQ1E8AQAAAACGongCAAAAAAxF8QQAAAAAGIriCQAAAAAwFMUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDUTwBnBdGjRpudgQAAIDzltXsAADOHzk5+/T66/9UZWWl3G6XAgODdMcd96lRo8bnNG5m5i699tpchYSEqrT0iCZOfESNG8ed9v5Hj5Zq9uzH5Ha7VVJSrBtuGKOuXXtqxIjLlJLSRZIUGxuvceMmq6zsqObPn6mKinIdOXJYkyY9Wu2xpk+fpOnTZ5/RuVRUlOvdd9/QJ5+8ryVLVspmC6g230cfLdb27T8rODhE5eVlmjjxEQUEBMrPj78pAgCAuoXiCaBWuFxOPfbYZN177zS1bZsiSfr5503atGm9Bg4cqjfffFEHDmTL6XQoJaWLrrrqes2fP1M2m02hoWH65ZctuuuuB/Tcc9P10EMzFRMTqx9//F7r1n2r1q3b6bbb7lVSUistW7ZEq1ev0LXX/kOzZj0qt9utxo3j5HI5q82WmblLffsOUJ8+/ZSVlaHXX/+n2rZNUWJi0gnF8X//9/+oY8cuGjz4Ku3f/5vcbrfXMb/66jNt3bpFH3zwbw0Zco3mzHlCQUHBstsLde21f1enTt287rd//2/q06e/1q5dddJ8Xbv21PLlS/XCC/9WQECgpk+fpL179ygxMUlBQcFn9G8DAABgNIongFqRn5+nyMhoT+mUpE6duqlTp27KzNyljIydmjHjn5KksWNHql+/IfLz81NSUmsNHDhUn332oTZs+E79+g3R6tUrNGLE3/X118s1fPjf1KZNB0mSw+HQpk3rddNNtys9fZtKS49o+vTZOny4REuXvldttpSUCz1fr1nztXr16qPDh0tUVGTXrFmP6siRw0pLG6RLLx2gX3/9rzp06Kx5856S0+nQmDETvI6ZljZIS5a8pREj/q6lS99Tq1Ztdd11N8luP6Rp0+7RCy/82+t+LVu2Oa18kjR69J26++6b1bhxnBo2jFBycvtqzxEAAMBMrMcCUCucTqdcLpfX13Jzs5WQ0MzzfUxMrAoLCyTJs4w1KChYFRUV6tt3gL7//htVVJQrO3uvp3QePHhATz/9oEaMuFEtWiQrP/+gYmPjJUkNGoQrLKzBSfNVVlbohReeUXh4Qw0Zco0iI6M0deoM3XffY3rooaf13ntvqLCwQA5HlZo0SdTddz+orl176p13Xj/luR97fpGR0Soutp9yn1Plc7lceuONFzR79mt67LHn5efnr40b16m09PAZjw0AAGA0iieAWhEXl6C8vFz9/PMmz7affvpRixa9pPj4psrJ2efZXlCQp5iYWK/jNGgQrujoGH300WL16dNPkrRvX5bmz5+pO++8Xx07dpUkRUfH6ODBA5KkoqJCHT5cUm22ysoKzZgxVZdffoWGDr1OkmS3Fyo3N0eSZLMFyGq1ye12q1WrtnI6f1+2GxraQFVVVV7HtFgsnmW4x55fQUGeoqNjTvFpnTqf0+lURUWFZ1ltWFgDFRfb5XBUv6QYAADALCy1BVAr/Pz8NGPGPL3yymz961+vymq1Kjw8QnfcMVkREVFq1aqtnn76ITkcVRo27G8KCQmtdqz+/a/U008/pDff/FiStGDBXJWWHta8eU9Jki68sLuGDRupgIBAPfnkVEVGRldbZCVp2bIl2r//N73//iJJUkRElMaPv09r1vxHX375idxuty67bLCio2N0zTU3at68J/X991/ryJHDmjDhgWrPNygoWK+++rxuummc5sx5Qs8//7iKi4t0++2Tqs3yzTfLtXr1SuXl5WrGjKnq3DlVLpfzhHz33POQrrvuH5ox436FhoapsrJSN910u6xW28n+GQAAAExhcVd3ZwwA571DOZXa8m2xuvQ/sxk61G/pPxQprqlNSR2rL/8AAABnghlPAOeNL79cph07th63zWq1avz4KWc9ZnGxXYsWvXzC9t6905Sa2qtWswAAANRVzHgCqBYznucnZjwBAEBN4+ZCAAAAAABDUTwBAAAAAIaieAIAAAAADEXxBAAAAAAYiuIJAAAAADAUxRMAAAAAYCiKJwAAAADAUBRPAAAAAIChKJ4AAAAAAENRPAEAAAAAhrKaHQBA3Xa02KHvPzhgdoxaV1FRKbfbraCgQLOjmCKuqc3sCAAAwIdY3G632+wQAOqmqorz98fDli1bVFBQoP79+5sdxTS2QIvZEQAAgI9gxhNAtc7n4uFvc8vP6jqvPwMAAICawjWeAOBFeXm5KisrzY4BAADgEyieAOCFzWaT1cqiEAAAgJpA8QQAL/z9/eXnx49IAACAmsBvVQAAAAAAQ1E8AQAAAACG4gImAPAiMzNTRUVFZscAAADwCRRPAPCiRYsWys/PNzsGAACAT2CpLQAAAADAUMx4AoAXeXl5stvtZscAAADwCRRPAPCicePGslgsZscAAADwCSy1BQAAAAAYiuIJAAAAADAUS20BwAuu8QQAAKg5FE8A8IJrPAEAAGoOS20BAAAAAIaieAIAAAAADEXxBAAAAAAYiuIJAAAAADAUxRMAAAAAYCjuagsAXuzcuVPFxcVmxwAAAPAJFE8A8KJNmzbKz883OwYAAIBPYKktAAAAAMBQzHgCwDEOHTqkiooKFRQUyG63KycnR2VlZWrZsqXZ0QAAAOoti9vtdpsdAgDqipKSEvXv319Op9Oz7a233lKHDh1MTAUAAFC/sdQWAI4RHh6unj17er5PSUmhdAIAAJwjiicA/MmQIUNktf5+JUKfPn1MTgMAAFD/cY0nAPxJ9+7d1aNHD9ntdoonAABADeAaTwDn5LdtR7XrpyNmx6hx+QX5Ki8rV7NmzcyOUuNiEwPV8ZKGZscAAADnEWY8AZyzkIYBSkgONTtGjWqlSLMjGMKeWyHJecr3AQAA1CSKJ4Aa0SDKZnYEnIbfiycAAEDt4uZCAAAAAABDUTwBAAAAAIaieAIAAAAADEXxBAAAAAAYiuIJAAAAADAUxRMAAAAAYCiKJwAAAADAUBRPAAAAAIChKJ4AAAAAAENRPAEAAAAAhqJ4AsBZ2rhxnb74Ymmt7wsAAFDfWM0OAADHuuaaNHXs2FWS5Ha7VVVVqbvuelBxcQle3z9t2j26+eY71KJFsmfb228vUHR0Yw0ePNyz7X/+5wq9887nJz32Tz/9qBUr/ld//eu1ateuY7XvmzPnCY0ePUGpqb3O5NTOeV8AAID6iuIJoE4JCgrW9OmzPd8vXDhf69d/q+HD/6a3335N+/btkcPhUK9efdWv35AaPfbq1Ss1YcJUvfbaPE/xXLPma61c+amqqiqVmnqxOnTorA0b1sjpdOrCC7srNzdb5eXlatGitS67bLCOHDmsKVPGavbs1zVr1qMKDg5RQUGebrvtHjkcjhP2vfHG2/Tmmy/qwIFsOZ0OpaR00VVXXa/x429U3779VVJSrMzMdD322BzZbLYaPV8AAIDaQvEEUKeUl5dp+vRJkqSsrAxddNFfNHDgUGVlZWjr1i2aOfMluVwu3XnnjerTp1+NHXfv3j2Kimqk4OAQhYSEKi8vV40bx+mNN17Qa6/9H1ksFn366Qdq06aDEhKa6tZb79HGjWslSQMG/FWvvTZPl102WN999x+lpQ1ScXGRBg8eru7de2vt2lVavvwTjR8/5YR9MzN3KSNjp2bM+KckaezYkerXb4iqqirVp09/xcUl6OmnH1RW1m61bt2uxs4XAACgNlE8AdQpx854PvPMNHXunKrg4BAdOLBfhw7l67nnpkuSQkLCZLcXeh3D7ZbcbteftrlPetyPP35Xhw7l6+mnH1Rp6RF98sn7uv760QoMDJS/v78kaejQ67zum5iYpCNHSnT4cIlWr16hKVMel9vt1tq1q7Vx4zoVFOQpLKyB131zc7OVkNDM831MTKwKCws8X//xmVRUVJw0PwAAQF1G8QRQZ40adYemTbtbqam9lJDQTAkJzTR58nRJ0m+/ZSo2Nt7rfk2bJmrTpvW64oqrJUm7dm1XVFSjao9TVFSoQ4fy9fjjczzbHnpogvz9/VVWVqaqqkr5+1u1ePHr+p//uVUWi+WEYpuWNkjLli1RYGCQoqIa6fXX/6nk5PYaPHi4PvvsQ+3c+asknbBvfHxTff75R57vCwryPIUTAADAV1A8AdRZsbHx6tHjEn344Tv6299uVuvWbTVz5sOqqqpUkyaJGj36TknSiy/OUkhIqCTpkksuU//+V2rXru26776xCggIkNPp1H33PVbtcZYtW6LevdOO29ajxyVavvwT3XLLXXrssclyuVxKTb1YFotFycntNWvWo7r44ks9709LG6Sbbhqqe++dJknq2rWn3n57gfbuzVRiYpJ2796hzMz0E/ZNSmqlVq3a6umnH5LDUaVhw/7mORcAAABfYXGfav0ZAJzEb9uOKntPpdpcFGF2FJyGvduOyGpxquMlDc2OAgAAziPMeAI4b/z663/11VefnbB95MhR1T6uBQAAAOeO4gngvNGhQ2d16NDZ7BgAAADnHT+zAwAAAAAAfBvFEwAAAABgKIonAAAAAMBQFE8AAAAAgKEongAAAAAAQ1E8AQAAAACGongCAAAAAAxF8QQAAAAAGIriCQAAAAAwlNXsAAB8Q2FOhdkRAAAAUEdRPAGcM2eVQ7kZh82OUaMKDxWqoqJC8QnxZkepcRGNbGZHAAAA5xmL2+12mx0CAOqaTZs2KT8/X4MGDTI7CgAAQL3HjCcAeHHgwAHZ7XazYwAAAPgEiicAeBEfHy+rlR+RAAAANYHfqgDAi4yMDBUVFZkdAwAAwCdQPAHAi5YtWyo/P9/sGAAAAD6B53gCAAAAAAxF8QQAAAAAGIriCQBeuFwu8bQpAACAmkHxBAAv/Pz8ZLFYzI4BAADgEyieAAAAAABDcVdbAPAiPT2dx6kAAADUEIonAHiRnJzM41QAAABqCEttAQAAAACGongCgBelpaU6evSo2TEAAAB8AsUTALwIDQ1VSEiI2TEAAAB8AsUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDUTwBAAAAAIaieAIAAAAADGU1OwAA1EUHDx5UYWGh2TEAAAB8AsUTALyIjY2Vnx+LQgAAAGoCv1UBAAAAAAzFjCcAHOPrr79WVlaWMjMzVVpaqpycHFksFt18881mRwMAAKi3LG632212CACoS4YOHaqcnBxJUkBAgFauXKnQ0FCTUwEAANRfLLUFgD+56KKLPF/36NGD0gkAAHCOKJ4A8CdXXXWVmjRpooCAAA0YMMDsOAAAAPUe13gC0GG7w+wIdUrTuNZK7XyJ8vLy1LlDTz6fP2kQyf86AADAmeEaTwBa/3mh7HlVZseoc9xulywWFoYcy9/fogF/b2x2DAAAUM/wZ2sAkqTmncLV+IJgs2OgDistdmjH94VmxwAAAPUQf8oHAAAAABiK4gkAAAAAMBTFEwAAAABgKIonAAAAAMBQFE8AAAAAgKEongAAAAAAQ1E8AQAAAACGongCAAAAAAxF8QQAAAAAGIriCQAAAAAwFMUTAAAAAGAoiicAn/HFF0v1/vuLzI4BAACAP7GaHQCA79m8eYOWLHlLVqtNlZUVatGitcaMmSCbLeCcxl29eoW++GKpAgODFBwcosmTp8tqPf0fY3l5uVq06CVlZ+/VvHmLJEmZmbv02mtzFRISqtLSI5o48RE1atRYc+Y8IbfbraqqKsXFJejmm8d7HbOgIE/vv79I48dPOaNz2bcvS2+88YIcDoeqqioVGBikMWMmKDExqdp9pk+fpOnTZ5/RcebMeUKjR09QeHhDWSyWM9oXAACgplA8AdSo7Oy9euWV2Zo161U1bBgpSVq2bIl2796pkJBQLVw4XxERkSouLtKECVNltVo1bdo96tGjt/LzDyo0NExdu16k77//Wvfc87Ak6d57R+vhh5/Rrl079MQTc2WzBeiJJ6YoMzNdDodDb775opKSWqmkpFhJSa2qzfbbbxm6/vrReu656Z5tO3f+qttuu1dJSa20bNkSrV69QpddNlg5Ofs0e/brcrlcuvnm4Ro16g6vxe1f/3pFO3Zs1fr138rhcGjlyk/VoEG4JGnixEfk53fiwpKqqko98cQUTZ06Qy1aJEuS0tO36dFHJ2rBgiWaN+9JDRo0XCkpF+qzzz5URUWFwsMbauvWLfrgg39r//7fZLMFKDAwSFlZuzVu3GT5+1s1d+4MzZz5kiRpzJhrNGXK49qwYY2cTqduv32SwsIanN0/KgAAwDliqS2AGvXjj2uVljbIUzolaejQ69SuXUe99dbLuuGGMZo48RFddtkgffTRYlksfioutuvGG2/TxImPaN26b9Wt20Xatu1nVVVVKSdnn0JDwxQdHaNbbrlLNluADh3KV3FxkRITk/Tuu29o3LhJuuOO+xQVFX3SbN2791Z4eMRx2wYPHq6kpFZyOBzatGm9unW7SNHRMUpJ6aL77x+ne+8drRtuuKXa2cK0tEHq3DlVF13UR6+//k89/PBMTZ48XTZbgH788Xuv+2Rm7lJcXBNP6ZSk5OT2atr0Au3atb3a40RFNdKIEX+XxeKnpKRWuuWWuzRkyDVauvQ9r/u0adNBCQlNdeut9yg0NOyknw0AAICRmPEEUKOqqirldDq9vpabm60mTRIlSXFxTfTdd/+RJDVq1NhT7Pz9/eXn56cePS7Rpk3rtGvXDg0YMNQzxs8/b9LSpe/p/vufUFBQsAoKDiomJk6SFB/fVEePlp5x5oMHD2jBgjkaMeJGtWiRrMzMXcrI2KmZM1+S0+nUvfeO1sUXX+qZyfTmyJHDCgoK8iwnTkhoqoMHD3h9b1VVlVwu1wnb3W63/P39TytzfHxTSVJ0dIwKCwtOax8AAACzMOMJoEb17PkXffXVZyooyPNse++9N7V+/beKj2+q7Oy9kqScnH1KSGhW7Tj9+1+pb7/9Sps3r1evXn0kSd9++5W+/voLPfjg04qJiZX0e/HKz8+VJO3du+eM8+7bl6X582fqzjvvV8eOXSVJR46UKCwsXBaLRVarVRaLRRUV5dWO4XK5FBbWQOXl5aqqqpQkZWdXf35t2nRQdvZe7dix1bMtPX2b8vIOqHXrdgoICPQc74/yarFY5Ha7Pe/PydknScrNzVFMTKyCgoI8+5SVHdXhwyXH7OeS9P/3BQAAqG3MeAKoUYmJSbrjjsl66qkHZLMFyOl0qkOHzurevbcSEppp4cL5atgwQqWlR3TXXQ9UO84FF7TQwYM5atWqrWy2ALlcLs2Z84RSUrpoxoz7JUnDh/9NI0eO0vz5M3XBBS1VUVGu6OiYasd8880XlZGxU9nZezV9+iQNGzZSH320WKWlhzVv3lOSpAsv7K6hQ6/TqlUr9Oyzj8jpdKhHj0vUqFFjr2MmJDTTli0/aOXKT3XLLXdpxoypatAgXEFBwerW7SKv+9hsNj3++By9+uocORxVslgsCgwM0vTps+Xn56dLLx2gxYsX6tdff1JBQZ4aNoyUn5+fgoKC9eqrz0uSdu/eqRdffFZZWRmaOHGaIiKiFB0do5demqXw8IaKiPh9qXNycnvNmvWo7r77ITVuHHfqf0AAAAADWNzH/gkdwHlp/eeFahgXrMYXBJsdBadh3rynlJY2UJ06davV45YWO7Tj+0IN+Lv3Eg4AAFAdZjwB+JTiYrsWLXr5hO29e6cpNbXXWY/75ZfLjlsaK0lWq/Wkj1ExKgsAAEB9w4wnAGY8cVqY8QQAAGeLmwsBAAAAAAxF8QQAAAAAGIriCQAAAAAwFMUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDUTwBAAAAAIaieAIAAAAADGU1OwCAuuFQTrkO5ZSbHaPucP+//1pMTQEAAOATKJ4AlJAUZHaEOmfX7l0qLipWamqq2VHqmECzAwAAgHqI4glAie1CzI5Q5+QfdajKdpTPBgAAoAZQPAHAi927d6uoqMjsGAAAAD6B4gkAXrRq1Ur5+flmxwAAAPAJFE8A8CInJ0d2u93sGAAAAD6B4gkAXiQkJMhms5kdAwAAwCfwHE8AAAAAgKGY8QQAL/bu3ctSWwAAgBpC8QQALxITExUcHGx2DAAAAJ/AUlsAAAAAgKEongAAAAAAQ1E8AcCL8vJyVVZWmh0DAADAJ1A8AcCLoKAgBQQEmB0DAADAJ1A8AQAAAACG4jnx0eIAAA0XSURBVK62AOBFenq6ioqKzI4BAADgEyieAOBFcnKy8vPzzY4BAADgE1hqCwAAAAAwFMUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDUTwBwIujR4+qrKzM7BgAAAA+geIJAF6EhIQoODjY7BgAAAA+geIJAAAAADAUxRMAAAAAYCiL2+12mx0CAOqK7du3695771VlZaVcLpcCAgI0adIkDRw40OxoAAAA9RYzngBwjHbt2ik+Pl4lJSU6cuSIgoODKZ0AAADniOIJAH/yl7/8xfN19+7dTUwCAADgGyieAPAnaWlpSklJUWJioq6++mqz4wAAANR7VrMDADg/7PjhsNkRzkAjdWn1Vx0uKZHfkWb1KnvbHg3MjgAAAHACbi4EoFZ8+2GBQqICzY7h03J3H9GVt8abHQMAAOAEzHgCqDURjQMUGUf5NEJluUu5u4+YHQMAAMArrvEEAAAAABiK4gkAAAAAMBTFEwAAAABgKIonAAAAAMBQFE8AAAAAgKEongAAAAAAQ1E8AQAAAACGongCAAAAAAxF8QQAAAAAGIriCQAAAAAwFMUTwHlp9eoVevvtBaYce8yYa0w5LgAAgFkongDqrQUL5ujVV5+vteNddVVfffLJ+57v3W63Ro++WosXL6y1DAAAAPWR1ewAAHA20tO3qWnTC1RaekS//ZapCy5ooYKCPD366ER17dpTxcV2RUZG6+abx2vs2JHq23eADh8uVm5ujh588CnPOJMm3aL77ntccXEJ2rRpvdas+Vp33/2g12M2aZKo7777j4YOvU4Wi0UbN65TZGS05/UFC+bq0KE8HTlyWJdeOlD9+1+plSs/1YYN38lqtclqtWry5Ome9//3vxu1dOl7euSRWbJYLIZ9VgAAAGZjxhNAvfTZZx8qLW2QBgz4qz799ANJkr+/v4qL7Ro16g5NnPiINm5cp+JiuxwOh/r06aexYycqLi5BGzas8YwzePBVWr58qSTpm2+Wa9CgYSc9bkrKhdq0ab0kafnypbr88sGSJKfTqaZNE/XAA0/pgQee0gcf/FuS9O23X2nYsJGaOnWGrr76BjmdTknSrl3b9dFHi/XAA09SOgEAgM+jeAKodw4ePKBfftmiuXNn6KWXZmnz5g0qLrZLkho3jpO/v78kKTIyWnZ7oSQpPr6pJCkqqpEKCws8Y/Xp009r165SWdlRZWfvVZs2HU567CuvHKFPP/1Aubk5CgoKVkRE1HG5XnzxWb399gJVVFRIksaPn6JvvvlSkyffpu+/X+XJ9uyzj6hz51QFBATWyGcCAABQl1E8AdQ7S5e+q0mTHvXMLt5++yR9+umHkqTc3By5XC5JUkHBQUVFNZIk5eTsk/R7OWzUqLFnrICAQHXp0kMvvvis+vTpf8pjN2rUWIGBgVq8+HVdeeUIz/bNmzcoLy9X48dP0ZAh18jt/j3DgQP7ddddD+i55xZo27b/6rffMiVJ//znW1q//lulp2+rgU8EAACgbqN4AqhXXC6Xtmz5Qe3bd/Js69btIq1Z8x9VVVUpIiJSr78+T48/fp969eqr8PCGslgsWrPma82Z84Ty8w+qe/fex415xRVXa9WqFbr88itOK8OVV45QZuYutWvX0bOtZctk5eZm64UXntEPP6xRfHxTffPNcu3c+atmzLhfs2Y9qsjIaDVpkihJCg4O0ZQpj2v+/Jme5bcAAAC+yuJ2u91mhwDg+779sEAJbRsoMs64paV2+yE9+eTvs4vHGjPmGi1c+GG1+2Vk7NTHH7973I1/6pvKcpc2fX5QV94ab3YUAACAE3BXWwDntU8+eV8//rhWEydOkyQVF9u1aNHLJ7yvd+80pab2qu14AAAAPoEZTwC1ojZmPM9nzHgCAIC6jGs8AQAAAACGongCAAAAAAxF8QQAAAAAGIriCQAAAAAwFMUTAAAAAGAoiicAAAAAwFAUTwAAAACAoSieAAAAAABDUTwBAAAAAIaieAIAAAAADGU1OwCA88fP3xSYHeG0VVZWyu12KzAw0Owopy0gkL8lAgCAusnidrvdZocA4PtcTrMTnJnNmzepoKBAAwYMNDvKGfHzNzsBAADAiZjxBFAr6lshsvhJsrjrXW4AAIC6iHVZAOCFw+GQ01nPpmkBAADqKIonAHjhdDopngAAADWE4gkAXgQGBiogIMDsGAAAAD6B4gkAAAAAMBTFEwAAAABgKO5qCwBe7N27V0VFRWbHAAAA8AkUTwDwIjExUcHBwWbHAAAA8AkstQUAAAAAGIoZTwDwIicnR3a73ewYAAAAPoHiCQBeJCQkyGazmR0DAADAJ7DUFgAAAABgKIonAAAAAMBQFE8A8KKwsFAlJSVmxwAAAPAJXOMJAF5ERUXJ6XSaHQMAAMAnMOMJAAAAADAUxRMAAAAAYCiKJwAAAADAUBRPAAAAAIChKJ4AAAAAAENxV1sA8CI9PV1FRUVmxwAAAPAJFE8A8CI5OVn5+flmxwAAAPAJLLUFAAAAABiKGU8AOEZOTo7KysqUk5Mju92ujIwMlZeXq0OHDmZHAwAAqLcsbrfbbXYIAKgrjh49qssvv1xVVVWebe+8847atGljYioAAID6jaW2AHCMkJAQ9ezZ0/N9586dKZ0AAADniOIJAH8yZMgQBQQESJIuvfRSc8MAAAD4AK7xBIA/6dGjh3r06KHCwkJdcsklZscBAACo97jGE4ChKspc+mZJ/XssSVFRkSoqKhQbG2t2lDOW2i9CjZoEmh0DAADAgxlPAIarLHepy4AYs2OcoWizA5yV9B+KzI4AAABwAoongFoREs6PGwAAgPOR2+3m5kIAAAAAAGO43W6KJwAAAADAGMfeTojiCQAAAACoUX+Uzj/+S/EEAAAAANSYP5dOieIJAAAAAKgh1T2tk+IJAAAAADhnx5bOPxdQiicAAAAA4JxUN9P5B4onAAAAAOCs/bl0eiuhFE8AAAAAwFk51UznHyieAAAAAIAz5q10cnMhAD5txoz7z+j9dvshTZ58m0FpAAAAcCyr2QEA4ExMm3aPjh4tVUhIqGfbE0/M1cMPP1Ojx3G73froo3f0ww/fy2YLUFnZUfXseYmuvfYfslgsXveZM+cJjR49QQ0bRpz2cQoK8vT++4s0fvyUmooOAABguNNdYvsHiieAemf8+PvUokXycdtGjRquRYuWavz4G9W3b3+VlBQrMzNdjz02Rw5HlWbNelTBwSEqKMjTbbfdo6ioRic9xhdffKyMjHQ99dQL8vf3l8Ph0LPPPqLPP/9I3br10ty5MzRz5kuSpDFjrtGUKY9rw4Y1cjqd+sc/btdjj01W1649VVxsV2RktG6+ebyee266Bg0arpSUC/XZZx+qoqJCWVm7tWPHVm3cuE6dO6fKZrMZ9rkBAADUhOpK58nKKEttAfiUqqpK9enTX7fccpcaNAhXVtZuFRcXafDg4brvvsc0bNhILV/+ySnHWbt2la666nr5+/tLkqxWq0aMuFHff/+N1/e3adNBCQlNdeut98hms6m42K5Ro+7QxImPaOPGdSoutnvdLy1tkDp3TlVqai9KJwAA8FnMeAKod+bPn+lZatuwYaSmTHn8uNdjYmIlSUFBwaqoqFBUlE1r167Wxo3rVFCQp7CwBqc8hsPh8H4rcL/T+3td48ZxntIaGRktu73wtPYDAACoy85mtlOieAKohyZMmHrCUtuT+fjjd5Wc3F6DBw/XZ599qJ07fz3lPn369NOSJW9p6tQnZbVa5XA49OGHb6tv3wEKCgpSRUW5JKms7KgOHy6RJFksFrndLklSbm6OXC6X/Pz8VFBwUFFRjRQQEOjZ7+DBA4qIiJIkuVyuMzp/AAAAM5zpdZ3HongC8Hldu/bU228v0N69mUpMTNLu3TuUlZVx0n2uuOJqHT5coilTxiokJFTl5eXq1auv+ve/UpIUHR2jl16apfDwhoqIiJQkJSe316xZj2rcuMmKiIjU66/PU25ujnr16qvw8Ia69NIBWrx4oX799ScVFOSpYcNIJSQ005YtP2jVqhW6+OJLFRAQYPjnAQAAUNss7nOprQBwChVlLn35r4PqPSLe7Ci1xm4/pCeffEDPPbeg1o/901cF6nxJAzVqEljrxwYAAL7rVLWRpbYAcBJffrlMO3ZsPW6b1Wrl8SYAAACn6XTmMpnxBGCo83HG00zMeAIAgJp2rrOdEo9TAQAAAAAYjOIJAAAAADAUxRMAAAAA4FVNXZlJ8QQAAAAAnJXTLaYUTwAAAACAoSieAAAAAIAT1OQDUCieAAAAAABDUTwBAAAAAIaieAIAAAAAjlOTy2wlyVqjowFANfL3lpkdAQAAACaheAIwXEzTQNlzKJ61ISSMhSwAAKDusbhreg4VAAAAAFBvnUlF5DmeAAAAAIA6geIJAAAAADAUxRMAAAAAYCiKJwAAAADAUBRPAAAAAIChKJ4AAAAAAENRPAEAAAAAhvq/+s2Vtso5eJMAAAAASUVORK5CYII=)"
      ],
      "metadata": {
        "id": "_OcfqkE3A5uk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "import glob\n",
        "import random\n",
        "from PIL import Image\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "from torch.utils.data import Dataset, DataLoader, random_split\n",
        "import torchvision.transforms as transforms\n",
        "import numpy as np\n",
        "import cv2\n",
        "from tqdm import tqdm\n",
        "import wandb\n",
        "from google.colab import drive\n",
        "import datasets\n",
        "# Configuration\n",
        "config = {\n",
        "    'BATCH_SIZE': 2,\n",
        "    'EPOCHS': 50,\n",
        "    'LEARNING_RATE': 0.001,\n",
        "    'IMAGE_SIZE': 128,\n",
        "    'TRAIN_SPLIT': 0.8,\n",
        "    'num_workers': 2,\n",
        "    'accumulation_steps': 4,\n",
        "    'checkpoint_dir': '/content/drive/MyDrive/checkpoints',\n",
        "    'results_dir': '/content/drive/MyDrive/results',\n",
        "    'model_name': 'caltech256-BIDS-v2'\n",
        "}\n",
        "\n",
        "# Mount Google Drive\n",
        "drive.mount('/content/drive')\n",
        "os.makedirs(config['checkpoint_dir'], exist_ok=True)\n",
        "os.makedirs(config['results_dir'], exist_ok=True)\n",
        "\n",
        "# Dataset Class\n",
        "class Caltech256Dataset(Dataset):\n",
        "    def __init__(self, root_dir, transform=None):\n",
        "        self.root_dir = root_dir\n",
        "        self.transform = transform\n",
        "        self.image_paths = glob.glob(os.path.join(root_dir, \"**/*.jpg\"), recursive=True)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.image_paths)\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        img_path = self.image_paths[idx]\n",
        "        image = Image.open(img_path).convert('RGB')\n",
        "\n",
        "        if self.transform:\n",
        "            image = self.transform(image)\n",
        "\n",
        "        # Corrected dimensions\n",
        "        _, height, width = image.shape\n",
        "        mask = generate_random_mask(height, width)\n",
        "        mask = torch.from_numpy(mask).float().unsqueeze(0)\n",
        "\n",
        "        return image, mask\n",
        "\n",
        "# Mask Generation Functions\n",
        "def generate_random_mask(height, width):\n",
        "    if random.random() > 0.5:\n",
        "        return generate_irregular_mask(height, width)\n",
        "    return generate_large_hole_mask(height, width)\n",
        "\n",
        "def generate_irregular_mask(h, w, max_vertices=8):\n",
        "    mask = np.zeros((h, w), np.float32)\n",
        "    num_vertices = random.randint(3, max_vertices)\n",
        "    vertices = np.random.randint(0, max(h, w), (num_vertices, 2))\n",
        "    vertices = vertices.reshape((-1, 1, 2))\n",
        "    cv2.fillPoly(mask, [vertices], 1.0)\n",
        "    return mask\n",
        "\n",
        "def generate_large_hole_mask(h, w):\n",
        "    mask = np.ones((h, w), np.float32)\n",
        "    cx = np.random.randint(0, w)\n",
        "    cy = np.random.randint(0, h)\n",
        "    radius = np.random.randint(min(h, w)//4, min(h, w)//2)\n",
        "    cv2.circle(mask, (cx, cy), radius, 0, -1)\n",
        "    return 1 - mask\n",
        "\n",
        "# Model Components\n",
        "class PreActResidualBlock(nn.Module):\n",
        "    def __init__(self, in_channels, out_channels, downsample=False):\n",
        "        super().__init__()\n",
        "        stride = 2 if downsample else 1\n",
        "        self.norm1 = nn.BatchNorm2d(in_channels)\n",
        "        self.conv1 = nn.Conv2d(in_channels, out_channels, 3, stride=stride, padding=1, bias=False)\n",
        "        self.norm2 = nn.BatchNorm2d(out_channels)\n",
        "        self.conv2 = nn.Conv2d(out_channels, out_channels, 3, padding=1, bias=False)\n",
        "        self.relu = nn.ReLU(inplace=True)\n",
        "        self.downsample = nn.Sequential(\n",
        "            nn.Conv2d(in_channels, out_channels, 1, stride=stride, bias=False),\n",
        "            nn.BatchNorm2d(out_channels)\n",
        "        ) if in_channels != out_channels or downsample else None\n",
        "\n",
        "    def forward(self, x):\n",
        "        identity = x\n",
        "        if self.downsample is not None:\n",
        "            identity = self.downsample(x)\n",
        "\n",
        "        out = self.norm1(x)\n",
        "        out = self.relu(out)\n",
        "        out = self.conv1(out)\n",
        "\n",
        "        out = self.norm2(out)\n",
        "        out = self.relu(out)\n",
        "        out = self.conv2(out)\n",
        "\n",
        "        return out + identity\n",
        "\n",
        "class FastGlobalSelfAttention(nn.Module):\n",
        "    def __init__(self, dim, num_heads=4, sr_ratio=4):\n",
        "        super().__init__()\n",
        "        self.num_heads = num_heads\n",
        "        self.dim = dim\n",
        "        self.sr_ratio = sr_ratio\n",
        "\n",
        "        if sr_ratio > 1:\n",
        "            self.sr = nn.Conv2d(dim, dim, kernel_size=sr_ratio, stride=sr_ratio)\n",
        "            self.norm = nn.LayerNorm(dim)\n",
        "        else:\n",
        "            self.sr = None\n",
        "\n",
        "        self.q = nn.Linear(dim, dim)\n",
        "        self.kv = nn.Linear(dim, dim*2)\n",
        "        self.proj = nn.Linear(dim, dim)\n",
        "\n",
        "    def forward(self, x):\n",
        "        B, C, H, W = x.shape\n",
        "        q = self.q(x.view(B, C, -1).transpose(1, 2)).reshape(B, -1, self.num_heads, C//self.num_heads).permute(0, 2, 1, 3)\n",
        "\n",
        "        if self.sr is not None:\n",
        "            x_sr = self.sr(x).reshape(B, C, -1).transpose(1, 2)\n",
        "            x_sr = self.norm(x_sr)\n",
        "        else:\n",
        "            x_sr = x.view(B, C, -1).transpose(1, 2)\n",
        "\n",
        "        kv = self.kv(x_sr).reshape(B, -1, 2, self.num_heads, C//self.num_heads).permute(2, 0, 3, 1, 4)\n",
        "        k, v = kv[0], kv[1]\n",
        "\n",
        "        attn = (q @ k.transpose(-2, -1)) * (C ** -0.5)\n",
        "        attn = attn.softmax(-1)\n",
        "        x = (attn @ v).transpose(1, 2).reshape(B, -1, C)\n",
        "        x = self.proj(x).transpose(1, 2).view(B, C, H, W)\n",
        "\n",
        "        return x\n",
        "\n",
        "class BFI(nn.Module):\n",
        "    def __init__(self, channels):\n",
        "        super().__init__()\n",
        "        self.fusion = nn.Sequential(\n",
        "            nn.Conv2d(channels*2, channels, 1),\n",
        "            nn.BatchNorm2d(channels),\n",
        "            nn.ReLU(True)\n",
        "        )\n",
        "\n",
        "    def forward(self, cnn_feat, trans_feat):\n",
        "        return self.fusion(torch.cat([cnn_feat, trans_feat], dim=1))\n",
        "\n",
        "class BIDS_Net(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        # CNN Stream\n",
        "        self.cnn_stream = nn.ModuleList([\n",
        "            PreActResidualBlock(3, 64),\n",
        "            PreActResidualBlock(64, 128, downsample=True),\n",
        "            PreActResidualBlock(128, 256, downsample=True)\n",
        "        ])\n",
        "\n",
        "        # Transformer Stream\n",
        "        self.trans_proj = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3)\n",
        "        self.trans_stream = nn.ModuleList([\n",
        "            FastGlobalSelfAttention(64),\n",
        "            FastGlobalSelfAttention(128),\n",
        "            FastGlobalSelfAttention(256)\n",
        "        ])\n",
        "        self.trans_adapters = nn.ModuleList([\n",
        "            nn.Conv2d(64, 128, 3, stride=2, padding=1),\n",
        "            nn.Conv2d(128, 256, 3, stride=2, padding=1)\n",
        "        ])\n",
        "\n",
        "        # Fusion\n",
        "        self.bfi = nn.ModuleList([BFI(64), BFI(128), BFI(256)])\n",
        "\n",
        "        # Decoder\n",
        "        self.decoder = nn.Sequential(\n",
        "            nn.Conv2d(256, 128, 3, padding=1),\n",
        "            nn.Upsample(scale_factor=2),\n",
        "            nn.Conv2d(128, 64, 3, padding=1),\n",
        "            nn.Upsample(scale_factor=2),\n",
        "            nn.Conv2d(64, 3, 3, padding=1)\n",
        "        )\n",
        "\n",
        "    def forward(self, x, mask):\n",
        "        # Mask input\n",
        "        masked_x = x * (1 - mask)\n",
        "\n",
        "        # CNN features\n",
        "        c1 = self.cnn_stream[0](masked_x)\n",
        "        c2 = self.cnn_stream[1](c1)\n",
        "        c3 = self.cnn_stream[2](c2)\n",
        "\n",
        "        # Transformer features\n",
        "        t = self.trans_proj(masked_x)\n",
        "        t1 = self.trans_stream[0](t)\n",
        "\n",
        "        t = self.trans_adapters[0](t1)\n",
        "        t2 = self.trans_stream[1](t)\n",
        "\n",
        "        t = self.trans_adapters[1](t2)\n",
        "        t3 = self.trans_stream[2](t)\n",
        "\n",
        "        # Fuse features\n",
        "        f1 = self.bfi[0](c1, t1)\n",
        "        f2 = self.bfi[1](c2, t2)\n",
        "        f3 = self.bfi[2](c3, t3)\n",
        "\n",
        "        # Decode\n",
        "        out = self.decoder(f3)\n",
        "        return x * mask + out * (1 - mask)\n",
        "\n",
        "# Add this to verify sample loading\n",
        "sample_img, sample_mask = dataset[0]\n",
        "print(\"Image shape:\", sample_img.shape)\n",
        "print(\"Mask shape:\", sample_mask.shape)\n",
        "# Training Utilities\n",
        "def create_loaders(dataset, config):\n",
        "    train_size = int(config['TRAIN_SPLIT'] * len(dataset))\n",
        "    val_size = len(dataset) - train_size\n",
        "    train_set, val_set = random_split(dataset, [train_size, val_size])\n",
        "\n",
        "    train_loader = DataLoader(\n",
        "        train_set,\n",
        "        batch_size=config['BATCH_SIZE'],\n",
        "        shuffle=True,\n",
        "        num_workers=config['num_workers'],\n",
        "        pin_memory=True\n",
        "    )\n",
        "\n",
        "    val_loader = DataLoader(\n",
        "        val_set,\n",
        "        batch_size=config['BATCH_SIZE'],\n",
        "        shuffle=False,\n",
        "        num_workers=config['num_workers'],\n",
        "        pin_memory=True\n",
        "    )\n",
        "    return train_loader, val_loader\n",
        "\n",
        "def train_epoch(model, loader, optimizer, scaler, device):\n",
        "    model.train()\n",
        "    total_loss = 0\n",
        "    pbar = tqdm(loader, desc='Training')\n",
        "\n",
        "    for i, (images, masks) in enumerate(pbar):\n",
        "        images = images.to(device)\n",
        "        masks = masks.to(device)\n",
        "\n",
        "        with torch.cuda.amp.autocast():\n",
        "            outputs = model(images, masks)\n",
        "            loss = nn.L1Loss()(outputs, images)\n",
        "            loss = loss / config['accumulation_steps']\n",
        "\n",
        "        scaler.scale(loss).backward()\n",
        "\n",
        "        if (i+1) % config['accumulation_steps'] == 0:\n",
        "            scaler.step(optimizer)\n",
        "            scaler.update()\n",
        "            optimizer.zero_grad()\n",
        "\n",
        "        total_loss += loss.item() * config['accumulation_steps']\n",
        "        pbar.set_postfix({'loss': total_loss/(i+1)})\n",
        "\n",
        "    return total_loss / len(loader)\n",
        "\n",
        "@torch.no_grad()\n",
        "def validate(model, loader, device):\n",
        "    model.eval()\n",
        "    total_loss = 0\n",
        "    pbar = tqdm(loader, desc='Validation')\n",
        "\n",
        "    for images, masks in pbar:\n",
        "        images = images.to(device)\n",
        "        masks = masks.to(device)\n",
        "\n",
        "        outputs = model(images, masks)\n",
        "        loss = nn.L1Loss()(outputs, images)\n",
        "        total_loss += loss.item()\n",
        "        pbar.set_postfix({'val_loss': total_loss/(len(pbar)+1)})\n",
        "\n",
        "    return total_loss / len(loader)\n",
        "\n",
        "# Main Training Loop\n",
        "def main():\n",
        "    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
        "\n",
        "    # Dataset and Transforms\n",
        "    transform = transforms.Compose([\n",
        "        transforms.Resize(config['IMAGE_SIZE']),\n",
        "        transforms.ToTensor(),\n",
        "        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
        "    ])\n",
        "\n",
        "    dataset = Caltech256Dataset('/content/caltech256', transform=transform)\n",
        "    # Add this after dataset initialization\n",
        "    print(f\"Found {len(dataset)} images in dataset\")\n",
        "\n",
        "    if len(dataset) == 0:\n",
        "        raise ValueError(\"No images found! Check your dataset path and file extensions.\")\n",
        "\n",
        "    train_loader, val_loader = create_loaders(dataset, config)\n",
        "\n",
        "    # Model and Optimizer\n",
        "    model = BIDS_Net().to(device)\n",
        "    optimizer = optim.AdamW(model.parameters(), lr=config['LEARNING_RATE'])\n",
        "    scaler = torch.cuda.amp.GradScaler()\n",
        "\n",
        "    # WandB Initialization\n",
        "    wandb.init(project=\"BIDS-Net\", config=config)\n",
        "\n",
        "    # Training Loop\n",
        "    best_loss = float('inf')\n",
        "    for epoch in range(config['EPOCHS']):\n",
        "        train_loss = train_epoch(model, train_loader, optimizer, scaler, device)\n",
        "        val_loss = validate(model, val_loader, device)\n",
        "\n",
        "        wandb.log({\n",
        "            'train_loss': train_loss,\n",
        "            'val_loss': val_loss,\n",
        "            'epoch': epoch\n",
        "        })\n",
        "\n",
        "        # Save Checkpoint\n",
        "        if val_loss < best_loss:\n",
        "            best_loss = val_loss\n",
        "            torch.save(model.state_dict(),\n",
        "                      f\"{config['checkpoint_dir']}/{config['model_name']}_best.pth\")\n",
        "\n",
        "        if (epoch+1) % 10 == 0:\n",
        "            torch.save(model.state_dict(),\n",
        "                      f\"{config['checkpoint_dir']}/{config['model_name']}_epoch{epoch+1}.pth\")\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 228
        },
        "id": "qLAaFOoxle62",
        "outputId": "07c99a67-6f4a-420a-f286-6e7c05b88da8"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "'module' object is not subscriptable",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-20-cead3194c6da>\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m    221\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    222\u001b[0m \u001b[0;31m# Add this to verify sample loading\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 223\u001b[0;31m \u001b[0msample_img\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msample_mask\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdataset\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    224\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Image shape:\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msample_img\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    225\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Mask shape:\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msample_mask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: 'module' object is not subscriptable"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 1. Verify all ModuleList declarations\n",
        "class BIDS_Net(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        # CNN Stream (correct ModuleList usage)\n",
        "        self.cnn_stream = nn.ModuleList([\n",
        "            PreActResidualBlock(3, 64),\n",
        "            PreActResidualBlock(64, 128, downsample=True),\n",
        "            PreActResidualBlock(128, 256, downsample=True)\n",
        "        ])\n",
        "\n",
        "        # Transformer Stream (correct initialization)\n",
        "        self.trans_stream = nn.ModuleList([\n",
        "            FastGlobalSelfAttention(64),\n",
        "            FastGlobalSelfAttention(128),\n",
        "            FastGlobalSelfAttention(256)\n",
        "        ])\n",
        "\n",
        "# 2. Fix dataset dimension handling\n",
        "class Caltech256Dataset(Dataset):\n",
        "    def __getitem__(self, idx):\n",
        "        img_path = self.image_paths[idx]\n",
        "        image = Image.open(img_path).convert('RGB')\n",
        "\n",
        "        if self.transform:\n",
        "            image = self.transform(image)  # Should return tensor [C, H, W]\n",
        "\n",
        "        # Correct dimension access\n",
        "        _, height, width = image.shape\n",
        "        mask = generate_random_mask(height, width)\n",
        "        mask = torch.from_numpy(mask).float().unsqueeze(0)\n",
        "\n",
        "        return image, mask\n",
        "\n",
        "# 3. Verify transform pipeline\n",
        "transform = transforms.Compose([\n",
        "    transforms.Resize((config['IMAGE_SIZE'], config['IMAGE_SIZE'])),\n",
        "    transforms.ToTensor(),  # Converts to [C, H, W]\n",
        "    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
        "])"
      ],
      "metadata": {
        "id": "Q6zO_n1PoBv1"
      },
      "execution_count": 22,
      "outputs": []
    }
  ]
}