loading_and_saving_configs.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "loading_and_saving_configs.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/CanopySimulations/canopy-python-examples/blob/master/loading_and_saving_configs.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NtIv7oIVvn5i"
      },
      "source": [
        "# Upgrade Runtime\n",
        "This cell ensures the runtime supports `asyncio` async/await, and is needed on Google Colab. If the runtime is upgraded, you will be prompted to restart it, which you should do before continuing execution."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E-AnefvPvmtR"
      },
      "source": [
        "!pip install ipython ipykernel --upgrade"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_61Bg2_KwIR2"
      },
      "source": [
        "# Set Up Environment"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zX-xTWzWG1bt"
      },
      "source": [
        "### Import required libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gNnEvtkmwL0W"
      },
      "source": [
        "!pip install -q canopy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0bBsqDyhwN_8"
      },
      "source": [
        "import canopy\n",
        "import logging\n",
        "import nest_asyncio\n",
        "\n",
        "logging.basicConfig(level=logging.INFO)\n",
        "nest_asyncio.apply()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nH5-pXaZw5jg"
      },
      "source": [
        "### Authenticate"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SAnwvDofwhw1"
      },
      "source": [
        "authentication_data = canopy.prompt_for_authentication()\n",
        "session = canopy.Session(authentication_data)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JlBY8CwBPG2A"
      },
      "source": [
        "# Set Up Example"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KS9NGfrlPKR_"
      },
      "source": [
        "# Removes sensitive data from configs so we don't publish them in our public repo.\n",
        "def sanitize_config(config: canopy.ConfigResult):\n",
        "    config.document.tenant_id = \\\n",
        "    config.document.user_id = \\\n",
        "    'removed'\n",
        "    \n",
        "    for t in config.user_information.tenants:\n",
        "        t.tenant_id = \\\n",
        "        t.name = \\\n",
        "        t.short_name = \\\n",
        "        'removed'\n",
        "\n",
        "        for u in t.users:\n",
        "            u.user_id = \\\n",
        "            u.username = \\\n",
        "            u.email = \\\n",
        "            'removed'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tnJeZcc0vHif"
      },
      "source": [
        "# Example: Loading and saving configs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "og809ltWzgjP"
      },
      "source": [
        "## User configs and default configs\n",
        "Configs on the platform are split into user configs, which are created by users, and default configs which are supplied by Canopy. When viewing lists of configs in the Canopy Portal (the web UI), the user configs appear first and the default configs are in a separate list below.\n",
        "\n",
        "When using the Canopy Python library, there are separate functions for loading user and default configs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QWxXxaQ40w_l"
      },
      "source": [
        "## Loading a default config"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5j5YbZlUzfRx",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 277
        },
        "outputId": "173c5a76-45c3-46c2-a37d-85733a1e4dda"
      },
      "source": [
        "default_weather = await canopy.load_default_config(session, 'weather', '25 deg, dry')\n",
        "default_car = await canopy.load_default_config(session, 'car', 'Canopy F1 Car 2019')\n",
        "\n",
        "print(default_weather)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"config_type\": \"weather\",\n",
            "  \"name\": \"25 deg, dry\",\n",
            "  \"properties\": null,\n",
            "  \"notes\": null,\n",
            "  \"user_id\": null,\n",
            "  \"config_id\": null,\n",
            "  \"is_edited\": false,\n",
            "  \"is_data_converted\": false,\n",
            "  \"data\": {\n",
            "    \"TAir\": 25,\n",
            "    \"rHumidityRelative\": 0.1,\n",
            "    \"pAirAtmosphericLocal\": 101300\n",
            "  }\n",
            "}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zY26mTm92XHv"
      },
      "source": [
        "## Advanced Usage: The `data` vs `raw_data` properties.\n",
        "\n",
        "The `default_weather` object has two properties for accessing the config data: `default_weather.data` and `default_weather.raw_data`.\n",
        "\n",
        "The reason for this is a compromise between usability and performance. By default, Python deserialises the config data into nested dictionaries. These must be accessed with indexers:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dmK418pKiTFU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "7f865a01-eff8-4ec3-8642-b4011547e698"
      },
      "source": [
        "default_weather.raw_data['TAir']"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "25"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yrgvR2jGjn55",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "9b6a6472-0065-4c64-9953-5d8b8d0079cd"
      },
      "source": [
        "default_car.raw_data['suspension']['front']['internal']['damper']['dampingCoefficient']"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1200"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O0IO1GoFiix8"
      },
      "source": [
        "It would be much nicer to be able to traverse the hierarchy as if it were a set of nested properties. This is what the `data` property allows you to do.\n",
        "\n",
        "When you access the `data` property for the first time it converts the `raw_data` from nested dictionaries to a structure supporting property traversal, allowing you to access the data more concisely:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pf1OPSOCibFy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "dc59c0af-8f52-4c2b-8839-eb7eae4325ae"
      },
      "source": [
        "default_weather.data.TAir"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "25"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fIxkRq7i-Ea6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "ae99ba03-a43d-42ba-9b61-72879800e81b"
      },
      "source": [
        "default_car.data.suspension.front.internal.damper.dampingCoefficient"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1200"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-msUrBTI-RWO"
      },
      "source": [
        "Once this conversion has happened, the `raw_data` property changes to output the same as the `data` property. This is so that if you make any modifications they appear on both properties."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZcamkdSU-mKy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        },
        "outputId": "87950531-5b6e-490d-b687-e485f60ebcbb"
      },
      "source": [
        "default_car.data.suspension.front.internal.damper.dampingCoefficient = 1300\n",
        "\n",
        "print(default_car.data.suspension.front.internal.damper.dampingCoefficient)\n",
        "print(default_car.raw_data['suspension']['front']['internal']['damper']['dampingCoefficient'])\n",
        "\n",
        "# This next line would result in an error if the data property had not already been accessed:\n",
        "print(default_car.raw_data.suspension.front.internal.damper.dampingCoefficient)\n",
        "\n",
        "print(default_car.is_data_converted)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1300\n",
            "1300\n",
            "1300\n",
            "True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tLS8sGLC-9mu"
      },
      "source": [
        "Why not always perform the conversion, and just have one property? If the config is particularly large then the conversion is not cheap. \n",
        "\n",
        "If you have only downloaded a config in order to pass it into a study, performing the conversion would result in unnecessary delay with no benefit. If you are in a performance critical bit of code, you may similarly prefer to use the dictionary notation and avoid the conversion."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1lVYgnEX1LPf"
      },
      "source": [
        "## Saving a user config\n",
        "\n",
        "Before we can show how to load user configs, we should create one:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HRBG7L7cKAvW",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "49b7796d-aca9-4ee9-aaad-f4cdb74fa6c3"
      },
      "source": [
        "user_weather_id_july = await canopy.create_config(\n",
        "    session,\n",
        "    'weather',\n",
        "    'Python Example Weather July',\n",
        "    default_weather.raw_data,\n",
        "    properties={ 'country':'uk', 'month':'july' })\n",
        "\n",
        "default_weather.data.TAir = 10\n",
        "\n",
        "user_weather_id_march = await canopy.create_config(\n",
        "    session,\n",
        "    'weather',\n",
        "    'Python Example Weather March',\n",
        "    default_weather.raw_data,\n",
        "    properties={ 'country':'uk', 'month':'march' })\n",
        "    \n",
        "print(f'July ID: {user_weather_id_july}')\n",
        "print(f'March ID: {user_weather_id_march}')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "July ID: a2d1d73bde0841639b9f25574d5644c3\n",
            "March ID: 822bc12a3df54bf79b87fd84f3731efe\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w9uX3CLqLUAO"
      },
      "source": [
        "## Loading a user config by ID"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QI7kHA55LY0k"
      },
      "source": [
        "The first way to load a config is by its ID:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3glv1wM-LddO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 867
        },
        "outputId": "76069a60-ff89-4c3c-d6f0-155a390d8309"
      },
      "source": [
        "march_weather = await canopy.load_config(session, user_weather_id_march)\n",
        "\n",
        "sanitize_config(march_weather)\n",
        "\n",
        "print(march_weather)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"config_id\": \"822bc12a3df54bf79b87fd84f3731efe\",\n",
            "  \"document\": {\n",
            "    \"document_id\": \"822bc12a3df54bf79b87fd84f3731efe\",\n",
            "    \"tenant_id\": \"removed\",\n",
            "    \"user_id\": \"removed\",\n",
            "    \"name\": \"Python Example Weather March\",\n",
            "    \"type\": \"config\",\n",
            "    \"sub_type\": \"weather\",\n",
            "    \"sim_version\": \"1.3371\",\n",
            "    \"creation_date\": \"2020-04-27 16:36:57.562479+00:00\",\n",
            "    \"modified_date\": \"2020-04-27 16:36:57.562479+00:00\",\n",
            "    \"properties\": {\n",
            "      \"country\": \"uk\",\n",
            "      \"month\": \"march\"\n",
            "    },\n",
            "    \"data\": {\n",
            "      \"TAir\": 10,\n",
            "      \"rHumidityRelative\": 0.1,\n",
            "      \"pAirAtmosphericLocal\": 101300\n",
            "    },\n",
            "    \"support_session\": null,\n",
            "    \"notes\": null,\n",
            "    \"delete_requested\": null,\n",
            "    \"parent_worksheet_id\": null\n",
            "  },\n",
            "  \"user_information\": {\n",
            "    \"tenants\": [\n",
            "      {\n",
            "        \"tenant_id\": \"removed\",\n",
            "        \"name\": \"removed\",\n",
            "        \"short_name\": \"removed\",\n",
            "        \"users\": [\n",
            "          {\n",
            "            \"user_id\": \"removed\",\n",
            "            \"username\": \"removed\",\n",
            "            \"email\": \"removed\"\n",
            "          }\n",
            "        ]\n",
            "      }\n",
            "    ]\n",
            "  },\n",
            "  \"is_data_converted\": false,\n",
            "  \"data\": {\n",
            "    \"TAir\": 10,\n",
            "    \"rHumidityRelative\": 0.1,\n",
            "    \"pAirAtmosphericLocal\": 101300\n",
            "  }\n",
            "}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QWiL2-w4Me_J"
      },
      "source": [
        "This gives us a lot more information than when we loaded the default config. For example the `user_information` section contains the resolved tenant and user information for the document owner, or anyone who has contributed to the support session.\n",
        "\n",
        "If you don't need all this information, we can convert it to the same `canopy.LocalConfig` structure which the default config used:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wp523VkLM5HB",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 329
        },
        "outputId": "f6e3ea35-3849-4b4e-8c29-df867f038648"
      },
      "source": [
        "march_weather_2 = march_weather.to_local_config()\n",
        "\n",
        "print(march_weather_2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"config_type\": \"weather\",\n",
            "  \"name\": \"Python Example Weather March\",\n",
            "  \"properties\": {\n",
            "    \"country\": \"uk\",\n",
            "    \"month\": \"march\"\n",
            "  },\n",
            "  \"notes\": null,\n",
            "  \"user_id\": \"removed\",\n",
            "  \"config_id\": \"822bc12a3df54bf79b87fd84f3731efe\",\n",
            "  \"is_edited\": false,\n",
            "  \"is_data_converted\": false,\n",
            "  \"data\": {\n",
            "    \"TAir\": 10,\n",
            "    \"rHumidityRelative\": 0.1,\n",
            "    \"pAirAtmosphericLocal\": 101300\n",
            "  }\n",
            "}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CwOLU174NG7H"
      },
      "source": [
        "This simpler form can be useful if we want all our functions to use the same config data structure."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "URh_wbC_N8cH"
      },
      "source": [
        "## Getting a Config ID from the Canopy Portal\n",
        "\n",
        "If you want to load an existing config by ID, you can find its ID by opening it in the Canopy Portal (the web UI to the platform).\n",
        "\n",
        "The URL for a config is of the form `/configs/<configType>/<tenantId>/<configId>/edit`. You can therefore simply copy the Config ID from the URL and paste it into your Python code.\n",
        "\n",
        "Similarly for studies the URL is of the form `/studies/<tenantId>/<studyId>`.\n",
        "\n",
        "Failing that, you can load the config by name or other metadata, which we'll do next.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mGdrq7hWNwHr"
      },
      "source": [
        "## Loading a user config by name / metadata"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AvXM5se4N43c"
      },
      "source": [
        "Sometimes it is more convenient to load a config by name:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "43MBs_YaPCVO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "2d528867-7bf8-46bd-f443-04ddec12ad33"
      },
      "source": [
        "july_weather_by_name = await canopy.find_config(\n",
        "    session, \n",
        "    'weather', \n",
        "    name='Python Example Weather July')\n",
        "\n",
        "march_weather_by_name = await canopy.find_config(\n",
        "    session,\n",
        "    'weather',\n",
        "    name='Python Example Weather March')\n",
        "\n",
        "print(july_weather_by_name.data.TAir)\n",
        "print(march_weather_by_name.data.TAir)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "25\n",
            "10\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EK1DgQPYQ8Pz"
      },
      "source": [
        "We can also search by other metadata, such as custom properties:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GmK5VvrvRAJO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "51074198-0534-4bdf-bc66-b7a71163d11a"
      },
      "source": [
        "july_weather_by_property = await canopy.find_config(\n",
        "    session,\n",
        "    'weather',\n",
        "    custom_properties={ 'month': 'july' })\n",
        "\n",
        "march_weather_by_property = await canopy.find_config(\n",
        "    session,\n",
        "    'weather',\n",
        "    custom_properties={ 'month': 'march', 'country': 'uk' })\n",
        "\n",
        "print(july_weather_by_name.data.TAir)\n",
        "print(march_weather_by_name.data.TAir)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "25\n",
            "10\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bmzOgS0pRjQt"
      },
      "source": [
        "When multiple properties match the criteria, it will return the most recently modified:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f_zvfuDvRtvL",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "dee22c93-221a-4778-f8bd-b6544b5439be"
      },
      "source": [
        "most_recently_modified_uk_weather = await canopy.find_config(\n",
        "    session,\n",
        "    'weather',\n",
        "    custom_properties={ 'country': 'uk' })\n",
        "\n",
        "print(most_recently_modified_uk_weather.data.TAir)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "10\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QcsZunkhSW5J"
      },
      "source": [
        "Finally, if your config was in a worksheet, you can also pass in a `parent_worksheet_id` parameter to find the config within the worksheet."
      ]
    }
  ]
}