01_02 TF2_0_NLP_tasks_with_Transformers.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "TF2.0 NLP tasks with Transformers.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6SXrKAeEundU"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1fBiTjumxoFCRQf1zOtZUokC9mJZTw9Y6?usp=sharing)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tmx-6YifWLLQ"
      },
      "source": [
        "**NLP use cases**\n",
        "- Classifying whole sentences\n",
        "- Classifying each word in a sentence\n",
        "- Answering a question\n",
        "- Text summarization\n",
        "- Fill in the blanks\n",
        "- Translating from one language to another"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cd-h9TkLuc_6"
      },
      "source": [
        "%%capture\n",
        "!pip install transformers"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "39XhY1Y-uzMK"
      },
      "source": [
        "from transformers import pipeline\n",
        "import textwrap\n",
        "wrapper = textwrap.TextWrapper(width=80, break_long_words=False, break_on_hyphens=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "93QbwOgisBCK"
      },
      "source": [
        "##Classifying whole sentences"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KcWBiNRru7ta",
        "outputId": "55e61978-919b-4540-9dee-2576f024fe31"
      },
      "source": [
        "sentence = 'The flights were on time both in Sydney and the connecting flight in Singapore. The organisation to cope with the COVID 19 restrictions while in transit was well planned and directions easy to follow, the plane was comfortable with a reasonable selection of in flight entertainment. Crew were pleasant and helpful.'\n",
        "classifier = pipeline('text-classification', model='distilbert-base-uncased-finetuned-sst-2-english')\n",
        "c = classifier(sentence)\n",
        "print('\\nSentence:')\n",
        "print(wrapper.fill(sentence))\n",
        "print(f\"\\nThis sentence is classified with a {c[0]['label']} sentiment\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Sentence:\n",
            "The flights were on time both in Sydney and the connecting flight in Singapore.\n",
            "The organisation to cope with the COVID 19 restrictions while in transit was\n",
            "well planned and directions easy to follow, the plane was comfortable with a\n",
            "reasonable selection of in flight entertainment. Crew were pleasant and helpful.\n",
            "\n",
            "This sentence is classified with a POSITIVE sentiment\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yvvab0H3sfgh"
      },
      "source": [
        "##Classifying each word in a sentence"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uF6vLguV6lkz",
        "outputId": "5156b21b-3901-4ec8-8c8e-06f92a1d0bf7"
      },
      "source": [
        "sentence = \"Singapore Airlines was the first airline to fly the A380. Chew Choon Seng was Singapore Airline's CEO at the time. Singapore Airlines flies to New York daily.\"\n",
        "ner = pipeline('token-classification', model='dbmdz/bert-large-cased-finetuned-conll03-english', grouped_entities=True)\n",
        "ners = ner(sentence)\n",
        "print('\\nSentence:')\n",
        "print(wrapper.fill(sentence))\n",
        "print('\\n')\n",
        "for n in ners:\n",
        "  print(f\"{n['word']} -> {n['entity_group']}\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/pipelines/token_classification.py:129: UserWarning: `grouped_entities` is deprecated and will be removed in version v5.0.0, defaulted to `aggregation_strategy=\"AggregationStrategy.SIMPLE\"` instead.\n",
            "  f'`grouped_entities` is deprecated and will be removed in version v5.0.0, defaulted to `aggregation_strategy=\"{aggregation_strategy}\"` instead.'\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Sentence:\n",
            "Singapore Airlines was the first airline to fly the A380. Chew Choon Seng was\n",
            "Singapore Airline's CEO at the time. Singapore Airlines flies to New York daily.\n",
            "\n",
            "\n",
            "Singapore Airlines -> ORG\n",
            "A380 -> MISC\n",
            "Chew Choon Seng -> PER\n",
            "Singapore Airline -> ORG\n",
            "Singapore Airlines -> ORG\n",
            "New York -> LOC\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XGGSYtXLtF_S"
      },
      "source": [
        "##Answering a question"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QrEtUj8BtF_S",
        "outputId": "6bd13fdc-68b7-47bb-cd96-06f834a723b9"
      },
      "source": [
        "context = '''\n",
        "Singapore Airlines was founded in 1947 and was originally known as Malayan Airways. It is the national airline of Singapore and is based at Singapore Changi Airport. \n",
        "From this hub, the airline flies to more than 60 destinations, with flights to Seoul, Tokyo and Melbourne among the most popular of its routes. \n",
        "It is particularly strong in Southeast Asian and Australian destinations (the so-called Kangaroo Route), but also flies to 6 different continents, covering 35 countries.\n",
        "There are more than 100 planes in the Singapore Airlines fleet, most of which are Airbus aircraft plus a smaller amount Boeings.\n",
        "The company is known for frequently updating the aircraft in its fleet.'''\n",
        "\n",
        "\n",
        "question = 'How many aircrafts does Singapore Airlines have?'\n",
        "\n",
        "print('Text:')\n",
        "print(wrapper.fill(context))\n",
        "print('\\nQuestion:')\n",
        "print(question)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Text:\n",
            " Singapore Airlines was founded in 1947 and was originally known as Malayan\n",
            "Airways. It is the national airline of Singapore and is based at Singapore\n",
            "Changi Airport.  From this hub, the airline flies to more than 60 destinations,\n",
            "with flights to Seoul, Tokyo and Melbourne among the most popular of its routes.\n",
            "It is particularly strong in Southeast Asian and Australian destinations (the\n",
            "so-called Kangaroo Route), but also flies to 6 different continents, covering 35\n",
            "countries. There are more than 100 planes in the Singapore Airlines fleet, most\n",
            "of which are Airbus aircraft plus a smaller amount Boeings. The company is known\n",
            "for frequently updating the aircraft in its fleet.\n",
            "\n",
            "Question:\n",
            "How many aircrafts does Singapore Airlines have?\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DPUX75hiDdlU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        },
        "outputId": "024db06f-8618-44dc-a9ea-66ede64948af"
      },
      "source": [
        "from transformers import pipeline\n",
        "\n",
        "qa = pipeline('question-answering', model='distilbert-base-cased-distilled-squad')\n",
        "\n",
        "print('\\nQuestion:')\n",
        "print(question + '\\n')\n",
        "print('Answer:')\n",
        "a = qa(context=context, question=question)\n",
        "a['answer']"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Question:\n",
            "How many aircrafts does Singapore Airlines have?\n",
            "\n",
            "Answer:\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'more than 100'"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "os4TWQ9XAQgk"
      },
      "source": [
        "##Text summarization"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aVbliCgwtF_T",
        "outputId": "029942bf-c4c7-4081-c702-dcbcaede1158"
      },
      "source": [
        "review = '''\n",
        "Extremely unusual time to fly as we needed an exemption to fly out of Australia from the government. We obtained one as working in Tokyo for the year as teachers.\n",
        "The check in procedure does take a lot longer as more paperwork and phone calls are needed to check if you are allowed to travel. The staff were excellent in explaining the procedure as they are working with very few numbers.\n",
        "The flight had 40 people only, so lots of room and yes we had 3 seats each. The service of meals and beverages was done very quickly and efficiently.\n",
        "Changi airport was like a ghost town with most shops closed and all passengers are walked/transported to a transit zone until your next flight is ready. You are then walked in single file or transported to your next flight, so very strange as at times their seemed be more workers in PPE gear than passengers.\n",
        "The steps we went through at Narita were extensive, downloading apps, fill in paperwork and giving a saliva sample to test for covid 19. \n",
        "It took about 2 hours to get through the steps and we only sat down for maybe 10 minutes at the last stop to get back your covid results. \n",
        "The people involved were fantastic and we were lucky that we were numbers two and three in the initial first line up, but still over 2 hours it took so be aware. We knew we were quick as the people picking us up told us we were first out.'''\n",
        "\n",
        "print('\\nOriginal text:\\n')\n",
        "print(wrapper.fill(review))\n",
        "summarize = pipeline('summarization', model='sshleifer/distilbart-cnn-12-6')\n",
        "summarized_text = summarize(review)[0]['summary_text']\n",
        "print('\\nSummarized text:')\n",
        "print(wrapper.fill(summarized_text))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Original text:\n",
            "\n",
            " Extremely unusual time to fly as we needed an exemption to fly out of Australia\n",
            "from the government. We obtained one as working in Tokyo for the year as\n",
            "teachers. The check in procedure does take a lot longer as more paperwork and\n",
            "phone calls are needed to check if you are allowed to travel. The staff were\n",
            "excellent in explaining the procedure as they are working with very few numbers.\n",
            "The flight had 40 people only, so lots of room and yes we had 3 seats each. The\n",
            "service of meals and beverages was done very quickly and efficiently. Changi\n",
            "airport was like a ghost town with most shops closed and all passengers are\n",
            "walked/transported to a transit zone until your next flight is ready. You are\n",
            "then walked in single file or transported to your next flight, so very strange\n",
            "as at times their seemed be more workers in PPE gear than passengers. The steps\n",
            "we went through at Narita were extensive, downloading apps, fill in paperwork\n",
            "and giving a saliva sample to test for covid 19.  It took about 2 hours to get\n",
            "through the steps and we only sat down for maybe 10 minutes at the last stop to\n",
            "get back your covid results.  The people involved were fantastic and we were\n",
            "lucky that we were numbers two and three in the initial first line up, but still\n",
            "over 2 hours it took so be aware. We knew we were quick as the people picking us\n",
            "up told us we were first out.\n",
            "\n",
            "Summarized text:\n",
            " The flight had 40 people only, so lots of room and yes we had 3 seats each .\n",
            "The check in procedure does take a lot longer as more paperwork and phone calls\n",
            "are needed to check if you are allowed to travel . The staff were excellent in\n",
            "explaining the procedure as they are working with very few numbers .\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g6WQFMF0v1Ts"
      },
      "source": [
        "##Fill in the blanks"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7YgMzrVGxrGq",
        "outputId": "9f3490d3-7c5d-4f6e-d413-0ad6a6130c09"
      },
      "source": [
        "sentence = 'It is the national <mask> of Singapore'\n",
        "mask = pipeline('fill-mask', model='distilroberta-base')\n",
        "masks = mask(sentence)\n",
        "for m in masks:\n",
        "  print(m['sequence'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "It is the national anthem of Singapore\n",
            "It is the national capital of Singapore\n",
            "It is the national pride of Singapore\n",
            "It is the national treasure of Singapore\n",
            "It is the national motto of Singapore\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rj63RtEuNl0Z",
        "outputId": "423c1585-a7ee-440a-d82a-e380187741fa"
      },
      "source": [
        "sentence = 'Singapore Airlines is the national <mask> of Singapore'\n",
        "mask = pipeline('fill-mask', model='distilroberta-base')\n",
        "masks = mask(sentence)\n",
        "for m in masks:\n",
        "  print(m['sequence'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Singapore Airlines is the national airline of Singapore\n",
            "Singapore Airlines is the national carrier of Singapore\n",
            "Singapore Airlines is the national airport of Singapore\n",
            "Singapore Airlines is the national airlines of Singapore\n",
            "Singapore Airlines is the national capital of Singapore\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-7xGy6XkBUSt"
      },
      "source": [
        "##Translation (English to German)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KuylAnUBCQX7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "40c7f21a-5316-411c-de9b-6b4c06c27e94"
      },
      "source": [
        "english = '''It took about 2 hours to get through the steps and we only sat down for maybe 10 minutes at the last stop to get back your covid results. '''\n",
        "\n",
        "translator = pipeline('translation_en_to_de', model='t5-base')\n",
        "german = translator(english)\n",
        "print('\\nEnglish:')\n",
        "print(english)\n",
        "print('\\nGerman:')\n",
        "print(german[0]['translation_text'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "English:\n",
            "It took about 2 hours to get through the steps and we only sat down for maybe 10 minutes at the last stop to get back your covid results. \n",
            "\n",
            "German:\n",
            "Es dauerte ca. 2 Stunden, die Schritte zu durchlaufen und wir saßen nur für etwa 10 Minuten an der letzten Haltestelle, um Ihre Ergebnisse zurückzuholen.\n"
          ]
        }
      ]
    }
  ]
}