Big 5 Text Analysis; Data collection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8f72c052",
   "metadata": {},
   "source": [
    "## Data collection & storage...\n",
    "*Collect updated music track list and lyrics from web...*\n",
    "    \n",
    "#### Data Sources ####\n",
    "*All data has been collected via web-scrapes or developer apis...*\n",
    "- Spotify Api\n",
    "- Genius Api\n",
    "- Genius Website\n",
    "\n",
    "##### Soon-to-be-Sources #####\n",
    "- Facebook Api\n",
    "- Apple Music Api\n",
    "\n",
    "###### Known Data Source Columns ######\n",
    "- Spotify Acoustic Features (11 total, ~2 need normalizing)\n",
    "    - energy\n",
    "    - liveness\n",
    "    - tempo\n",
    "    - speechiness\n",
    "    - acousticness\n",
    "    - instrumentalness\n",
    "    - time_signature\n",
    "    - danceability\n",
    "    - duration_ms\n",
    "    - loudness\n",
    "    - valence\n",
    "- Utils for data_mapping (2)\n",
    "    - utils_spotify_id\n",
    "    - utils_genius_data\n",
    "\n",
    "### NOTE: Will need to re-download all song lyrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b08d302e",
   "metadata": {},
   "source": [
    "### Imports..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03856140",
   "metadata": {},
   "source": [
    "#### Tools... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b674ece0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Figure out if this is the complete list of imports needed...\n",
    "\n",
    "# libraries & tool imports...\n",
    "import os\n",
    "import re\n",
    "import ast\n",
    "import sys\n",
    "import json\n",
    "import pprint\n",
    "import requests\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from os import listdir\n",
    "from tqdm import tqdm\n",
    "from os.path import isfile, join\n",
    "from datetime import datetime\n",
    "from spotify import spotifyApi\n",
    "from text_miner import textMiner\n",
    "from text_miner import geniusApi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2c3b589c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initializing helpers..\n",
    "helper = spotifyApi()\n",
    "txtMiner = textMiner()\n",
    "genius = geniusApi()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5e478d79",
   "metadata": {},
   "outputs": [],
   "source": [
    "# existing lyrics dataset imports...\n",
    "data_path = \"/Users/dayoorigunwa/code_base/music_mapping/data/\"\n",
    "allfiles = [f for f in listdir(data_path) if isfile(join(data_path, f))]\n",
    "\n",
    "gScrapeLyrics = pd.read_csv(data_path + \"SpotifySongLyrics.csv\", encoding=\"utf-8\")\n",
    "gScrapeLyrics.columns = [\"name\", \"lyrics\"]\n",
    "gApiData = pd.read_csv(data_path + \"GeniusAPISearchResults\", encoding=\"utf-8\")\n",
    "# rename cols example -> df2 = df.rename({'a': 'X', 'b': 'Y'}, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a50b844",
   "metadata": {},
   "source": [
    "#### Testing encoded import..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e6a0cdb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # explicit utf-8 import...\n",
    "# utf_scrape_df = pd.read_csv(data_path + \"SpotifySongLyrics.csv\", encoding='utf-8')\n",
    "# utf_scrape_df.columns = ['name', 'lyrics']\n",
    "\n",
    "# # GET unique tracklist\n",
    "# utf_songs = utf_scrape_df.name.unique()\n",
    "\n",
    "# # comparing with non-encoded import\n",
    "\n",
    "# raw_songs = gScrapeLyrics.name.unique()\n",
    "# diff_songs = [song for song in utf_songs if song not in raw_songs]\n",
    "# print(len(diff_songs),diff_songs)\n",
    "\n",
    "# # ...sadness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22b35273",
   "metadata": {},
   "source": [
    "#### Investigating geniusApi breadcrumbs..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f0450fa3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# extracting genius data...\n",
    "names = list(gApiData.columns)\n",
    "genius_data = pd.DataFrame(\n",
    "    [{\"name\": str(s), \"data\": list(gApiData[s].values)[-1]} for s in names]\n",
    ")\n",
    "genius_data.dropna(inplace=True)\n",
    "genius_data[\"name\"] = genius_data[\"name\"].astype(str)\n",
    "genius_data[\"data\"] = genius_data[\"data\"].apply(lambda x: ast.literal_eval(x))\n",
    "name_to_genius = dict(zip(genius_data.name, genius_data.data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5a015f55",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://genius.com/Hammock-then-the-quiet-explosion-lyrics'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sanity check\n",
    "name_to_genius[\"Then the Quiet Explosion\"][\"url\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59574d55",
   "metadata": {},
   "source": [
    "#### This cell converts a string object into json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e22d0c9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # extracting\n",
    "# test = list(gApiData[\"Then the Quiet Explosion\"].values)\n",
    "# test = [val for val in test if val not in [np.nan, 'song', '[]']]\n",
    "\n",
    "# # jsonifying data...\n",
    "# parsed_json = ast.literal_eval(test[0])\n",
    "# parsed_json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3cfbb76",
   "metadata": {},
   "source": [
    "### GET Spotify Data..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64739abe",
   "metadata": {},
   "source": [
    "#### Audio..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8b2e9004",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<spotipy.client.Spotify object at 0x11f6d9750>\n",
      "Name: b'tide pools ', Number of songs: 4, Playlist ID: 7HCAhke3De2jsnXb4ZFXaf \n",
      "Name: b'Unbeknownst Bangers', Number of songs: 27, Playlist ID: 3E684Qjlabk2LmcJjZ2zZQ \n",
      "Name: b'sampleLove', Number of songs: 116, Playlist ID: 0DYobfEIwMJlvVBymPoPCf \n",
      "Name: b'Costa Rica ', Number of songs: 0, Playlist ID: 46CQNWsKAfp3O2dD12aGV6 \n",
      "Name: b'purpleParrots pt.3', Number of songs: 34, Playlist ID: 01RLihOalQPRbNAJdCiwOn \n",
      "Name: b'Study \\xf0\\x9f\\x93\\x9a', Number of songs: 53, Playlist ID: 5c68TrgrsR1COt97GYiHTm \n",
      "Name: b'yeye', Number of songs: 12, Playlist ID: 3YUa8BAtVbKmStMYcUnF5z \n",
      "Name: b\"I'm Just Snacking\", Number of songs: 23, Playlist ID: 7f7xecW1muGd5KamjC3EEM \n",
      "Name: b'la flame', Number of songs: 472, Playlist ID: 0i3CkxalKp9RRW4jf63PU5 \n",
      "Name: b'Fashion Killa', Number of songs: 372, Playlist ID: 6k96myNgO28Lt9lZUPqdiU \n",
      "Name: b'vamos', Number of songs: 8, Playlist ID: 6pS0A7SrSkuxPSZnsaHd54 \n",
      "Name: b'View From A Blue Moon', Number of songs: 13, Playlist ID: 0cOa2UTqNsjWO5AJswRZRA \n",
      "Name: b'Eargasm', Number of songs: 142, Playlist ID: 088Bt2aW9pdozWiIc1t9j3 \n",
      "Name: b'Best Songs of 2023 (Clean)', Number of songs: 57, Playlist ID: 7miiGQXVM6luQXxr1CspvQ \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████| 14/14 [00:20<00:00,  1.48s/it]\n"
     ]
    }
   ],
   "source": [
    "# updating audio feature datasets...\n",
    "helper = spotifyApi()\n",
    "helper.download_spotify_playlist_audio_features()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94d02cae",
   "metadata": {},
   "source": [
    "#### Tracklists..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "19326954",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Getting my spotify playlist data...\n",
      "\n",
      "Name: b'tide pools ', Number of songs: 4, Playlist ID: 7HCAhke3De2jsnXb4ZFXaf \n",
      "Name: b'Unbeknownst Bangers', Number of songs: 27, Playlist ID: 3E684Qjlabk2LmcJjZ2zZQ \n",
      "Name: b'sampleLove', Number of songs: 116, Playlist ID: 0DYobfEIwMJlvVBymPoPCf \n",
      "Name: b'Costa Rica ', Number of songs: 0, Playlist ID: 46CQNWsKAfp3O2dD12aGV6 \n",
      "Name: b'purpleParrots pt.3', Number of songs: 34, Playlist ID: 01RLihOalQPRbNAJdCiwOn \n",
      "Name: b'Study \\xf0\\x9f\\x93\\x9a', Number of songs: 53, Playlist ID: 5c68TrgrsR1COt97GYiHTm \n",
      "Name: b'yeye', Number of songs: 12, Playlist ID: 3YUa8BAtVbKmStMYcUnF5z \n",
      "Name: b\"I'm Just Snacking\", Number of songs: 23, Playlist ID: 7f7xecW1muGd5KamjC3EEM \n",
      "Name: b'la flame', Number of songs: 472, Playlist ID: 0i3CkxalKp9RRW4jf63PU5 \n",
      "Name: b'Fashion Killa', Number of songs: 372, Playlist ID: 6k96myNgO28Lt9lZUPqdiU \n",
      "Name: b'vamos', Number of songs: 8, Playlist ID: 6pS0A7SrSkuxPSZnsaHd54 \n",
      "Name: b'View From A Blue Moon', Number of songs: 13, Playlist ID: 0cOa2UTqNsjWO5AJswRZRA \n",
      "Name: b'Eargasm', Number of songs: 142, Playlist ID: 088Bt2aW9pdozWiIc1t9j3 \n",
      "Name: b'Best Songs of 2023 (Clean)', Number of songs: 57, Playlist ID: 7miiGQXVM6luQXxr1CspvQ \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████| 14/14 [00:08<00:00,  1.71it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Extracted 1333 songs from 14 playlists from your Spotify Account.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# getting most up-to-date songlist from Spotify...\n",
    "print(\"Getting my spotify playlist data...\\n\")\n",
    "songs = []\n",
    "sp = helper.sp\n",
    "username = helper.username\n",
    "playlists = helper.get_user_playlists(username, sp)\n",
    "for playlist in tqdm(playlists):\n",
    "    songs.extend(helper.get_playlist_content(username, playlist, sp, False))\n",
    "\n",
    "print(\n",
    "    f\"\\nExtracted {len(songs)} songs from {len(playlists)} playlists from your Spotify Account.\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "56d2b086",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1333 - 999 = 297 missing songs\n"
     ]
    }
   ],
   "source": [
    "# preparing song lists...\n",
    "sSongs = [val[\"track\"][\"name\"] for val in songs]\n",
    "gSongs = [val for val in list(gApiData.columns)]\n",
    "nSongs = [val for val in songs if val[\"track\"][\"name\"] not in gSongs]\n",
    "\n",
    "# sanity checks...\n",
    "print(len(sSongs), \"-\", len(gSongs), \"=\", len(nSongs), \"missing songs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0088e991",
   "metadata": {},
   "source": [
    "### GET Genius Lyrics..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a6749a4",
   "metadata": {},
   "source": [
    "#### method 1: web-scraping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "15f0987e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████| 774/774 [16:32<00:00,  1.28s/it]\n"
     ]
    }
   ],
   "source": [
    "url_lyrics = dict()\n",
    "for songname in tqdm(name_to_genius):\n",
    "    url = name_to_genius[songname][\"url\"]\n",
    "    songlyrics = txtMiner.scrape_genius_lyrics(\n",
    "        artistname=\"none\", songname=songname, url=url\n",
    "    )\n",
    "    url_lyrics[songname] = songlyrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "40c52097",
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'url_lyrics' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m/var/folders/9f/0jslw5zs3rg1hqt9w92zlbv00000gn/T/ipykernel_59062/574590773.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Sanity check\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0murl_lyrics\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"Then the Quiet Explosion\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'url_lyrics' is not defined"
     ]
    }
   ],
   "source": [
    "# Sanity check\n",
    "url_lyrics[\"Then the Quiet Explosion\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "82e50708",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving scraped song lyric data... \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# saving intermediate results for now...\n",
    "print(\"Saving scraped song lyric data... \\n\")\n",
    "date_piece = datetime.today().strftime(\"%Y-%m-%d\")\n",
    "scraped_df = pd.DataFrame([url_lyrics])\n",
    "scraped_df.to_csv(\n",
    "    data_path + \"scraped_SpotifySongLyrics_\" + date_piece + \".csv\", index=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ba0dc1cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Genius web-scraping accuracy: 0.9573643410852714\n"
     ]
    }
   ],
   "source": [
    "# Sanity check\n",
    "count = 0\n",
    "for key in url_lyrics:\n",
    "    if len(url_lyrics[key]) < 15:\n",
    "        count += 1\n",
    "print(f\"Genius web-scraping accuracy: {1 - count/len(url_lyrics.keys())}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "ae0a288e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# names = [s['track']['name'] for s in nSongs if s['track']['name'] not in list(url_lyrics.keys())]\n",
    "# [n for n in names if n not in a]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "676d424d",
   "metadata": {},
   "source": [
    "#### method 2: querying Genius API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "ce652587",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Querying GENIUS api for track metadata... \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|                                       | 0/85 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: family ties (with Kendrick Lamar), Baby Keem\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  1%|▎                              | 1/85 [00:00<00:35,  2.38it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: trademark usa, Baby Keem\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  2%|▋                              | 2/85 [00:00<00:34,  2.43it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: VALENTINO, 24kGoldn\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  4%|█                              | 3/85 [00:01<00:33,  2.43it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Pepas, Farruko\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  5%|█▍                             | 4/85 [00:01<00:32,  2.52it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Stick (with JID & J. Cole feat. Kenny Mason & Sheck Wes), Dreamville\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  6%|█▊                             | 5/85 [00:02<00:33,  2.39it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: BOP, DaBaby\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  7%|██▏                            | 6/85 [00:02<00:33,  2.38it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: DOLLAZ ON MY HEAD (feat. Young Thug), Gunna\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  8%|██▌                            | 7/85 [00:02<00:30,  2.58it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: RNP (feat. Anderson .Paak), Cordae\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  9%|██▉                            | 8/85 [00:03<00:30,  2.49it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Check, Young Thug\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 11%|███▎                           | 9/85 [00:03<00:35,  2.17it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Diet Coke, Pusha T\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 12%|███▌                          | 10/85 [00:04<00:34,  2.14it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Arya (feat. A$AP Rocky), A$AP Rocky\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 13%|███▉                          | 11/85 [00:04<00:33,  2.24it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: INDUSTRY BABY (feat. Jack Harlow), Lil Nas X\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 14%|████▏                         | 12/85 [00:05<00:31,  2.29it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: The Island, Pt. I (Dawn), Pendulum\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 15%|████▌                         | 13/85 [00:05<00:32,  2.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Woodlawn, Aminé\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 16%|████▉                         | 14/85 [00:06<00:31,  2.23it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Baianá, Bakermat\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 18%|█████▎                        | 15/85 [00:06<00:28,  2.46it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Piece Of Your Heart, MEDUZA\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 19%|█████▋                        | 16/85 [00:06<00:27,  2.49it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: What A Life, Scarlet Pleasure\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 20%|██████                        | 17/85 [00:07<00:34,  1.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: The Bigger Picture, Lil Baby\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 21%|██████▎                       | 18/85 [00:07<00:30,  2.16it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Save Your Tears, The Weeknd\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 22%|██████▋                       | 19/85 [00:08<00:35,  1.83it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Thief, Ookay\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 24%|███████                       | 20/85 [00:09<00:33,  1.95it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Spicy (feat. Fivio Foreign & A$AP Ferg), Nas\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 25%|███████▍                      | 21/85 [00:09<00:35,  1.82it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: WHATS POPPIN (feat. DaBaby, Tory Lanez & Lil Wayne) - Remix, Jack Harlow\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 26%|███████▊                      | 22/85 [00:10<00:30,  2.04it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: New God Flow, Pusha T\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 27%|████████                      | 23/85 [00:10<00:28,  2.15it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: The Box, Roddy Ricch\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 28%|████████▍                     | 24/85 [00:10<00:27,  2.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Marvin & Chardonnay, Big Sean\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 29%|████████▊                     | 25/85 [00:11<00:25,  2.38it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Right Above It, Lil Wayne\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 31%|█████████▏                    | 26/85 [00:11<00:25,  2.34it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Over, Drake\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 32%|█████████▌                    | 27/85 [00:12<00:25,  2.30it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Valley of Peace, TEYMORI\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 33%|█████████▉                    | 28/85 [00:12<00:24,  2.32it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Segrados Do Samba, Souleance\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 34%|██████████▏                   | 29/85 [00:12<00:21,  2.55it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Ani Kuni, Polo & Pan\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 35%|██████████▌                   | 30/85 [00:13<00:27,  2.04it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Rêverie, L. 68, Claude Debussy\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 36%|██████████▉                   | 31/85 [00:13<00:23,  2.30it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Old English (feat. Travi$ Scott), King Chip\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 38%|███████████▎                  | 32/85 [00:14<00:22,  2.34it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Knife, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 39%|███████████▋                  | 33/85 [00:14<00:26,  2.00it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Quintana Pt 2 / Upper Echelon (Mike Dean Live Mix), Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 40%|████████████                  | 34/85 [00:15<00:21,  2.36it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Pablo, Kanye West\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 41%|████████████▎                 | 35/85 [00:15<00:22,  2.24it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: NO BYSTANDERS (remix), Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 42%|████████████▋                 | 36/85 [00:16<00:21,  2.26it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Future Sounds, Kanye West & Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 44%|█████████████                 | 37/85 [00:16<00:24,  1.97it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Hold That Heat (feat. Travis Scott), Southside\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 45%|█████████████▍                | 38/85 [00:17<00:22,  2.06it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: LEVITATE, Don Toliver & Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 46%|█████████████▊                | 39/85 [00:17<00:20,  2.19it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: CHAIN SEX FT. 21 SAVAGE, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 47%|██████████████                | 40/85 [00:18<00:21,  2.10it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: OUTLANDISH, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 48%|██████████████▍               | 41/85 [00:18<00:21,  2.01it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Same Way ft. Quavo, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 49%|██████████████▊               | 42/85 [00:19<00:21,  1.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Gasoline ft. Drake, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 51%|███████████████▏              | 43/85 [00:19<00:17,  2.35it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Last Time, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 52%|███████████████▌              | 44/85 [00:19<00:18,  2.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Can't See That ft. Quavo, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 53%|███████████████▉              | 45/85 [00:20<00:18,  2.16it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Patience ft. Justin Bieber, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 54%|████████████████▏             | 46/85 [00:20<00:19,  2.05it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Never Settle ft. Drake, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 55%|████████████████▌             | 47/85 [00:21<00:17,  2.12it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Every Night ft. Kid Cudi, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 56%|████████████████▉             | 48/85 [00:21<00:18,  2.03it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Help Myself (feat. Don Toliver & Quavo), Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 58%|█████████████████▎            | 49/85 [00:22<00:17,  2.04it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: MAFIA/LOST FOREVER, Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 59%|█████████████████▋            | 50/85 [00:22<00:17,  1.98it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: MAFIA (Live), Travis Scott\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 60%|██████████████████            | 51/85 [00:23<00:16,  2.10it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: About Damn Time, Lizzo\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 61%|██████████████████▎           | 52/85 [00:23<00:15,  2.19it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: First Class, Jack Harlow\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 62%|██████████████████▋           | 53/85 [00:24<00:15,  2.11it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: up at night (feat. justin bieber), Kehlani\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 64%|███████████████████           | 54/85 [00:24<00:13,  2.27it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: MEAN!, Madeline The Person\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 65%|███████████████████▍          | 55/85 [00:25<00:12,  2.35it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Where you are (feat. WILLOW), PinkPantheress\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 66%|███████████████████▊          | 56/85 [00:25<00:12,  2.27it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Birthday Cake, Dylan Conrique\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 67%|████████████████████          | 57/85 [00:26<00:12,  2.17it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: As It Was, Harry Styles\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 68%|████████████████████▍         | 58/85 [00:26<00:12,  2.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Maybe You’re The Problem, Ava Max\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 69%|████████████████████▊         | 59/85 [00:26<00:11,  2.29it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Tamagotchi, Omar Apollo\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 71%|█████████████████████▏        | 60/85 [00:27<00:10,  2.34it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Dua Lipa, Jack Harlow\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 72%|█████████████████████▌        | 61/85 [00:27<00:09,  2.42it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Lola, Maude Latour\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 73%|█████████████████████▉        | 62/85 [00:28<00:09,  2.43it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Slow Song (with Dragonette), The Knocks\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 74%|██████████████████████▏       | 63/85 [00:28<00:09,  2.38it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Sweetest Pie, Megan Thee Stallion\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 75%|██████████████████████▌       | 64/85 [00:28<00:09,  2.23it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Sigue, J Balvin\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 76%|██████████████████████▉       | 65/85 [00:29<00:09,  2.18it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Something in the Orange, Zach Bryan\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 78%|███████████████████████▎      | 66/85 [00:29<00:08,  2.32it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Attention, Omah Lay\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 79%|███████████████████████▋      | 67/85 [00:30<00:08,  2.24it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Beg For You (feat. Rina Sawayama), Charli XCX\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 80%|████████████████████████      | 68/85 [00:30<00:07,  2.30it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: In The Stars, Benson Boone\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 81%|████████████████████████▎     | 69/85 [00:31<00:06,  2.33it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: everything, Kehlani\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 82%|████████████████████████▋     | 70/85 [00:32<00:08,  1.72it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Envolver, Anitta\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 84%|█████████████████████████     | 71/85 [00:32<00:07,  1.91it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Closer (feat. H.E.R.), Saweetie\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 85%|█████████████████████████▍    | 72/85 [00:32<00:06,  1.97it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Finesse, Pheelz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 86%|█████████████████████████▊    | 73/85 [00:33<00:05,  2.19it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Light Switch, Charlie Puth\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 87%|██████████████████████████    | 74/85 [00:33<00:04,  2.33it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Headlights (feat. KIDDO), Alok\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 88%|██████████████████████████▍   | 75/85 [00:34<00:04,  2.17it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Infinity, Jaymes Young\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 89%|██████████████████████████▊   | 76/85 [00:34<00:03,  2.28it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Boyfriend, Dove Cameron\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 91%|███████████████████████████▏  | 77/85 [00:34<00:03,  2.38it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Killing Me, Omar Apollo\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 92%|███████████████████████████▌  | 78/85 [00:35<00:03,  1.86it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: That's Hilarious, Charlie Puth\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 93%|███████████████████████████▉  | 79/85 [00:36<00:03,  1.99it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: May Snow, Sidders\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 94%|████████████████████████████▏ | 80/85 [00:36<00:02,  2.06it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Searching for a match for: Let You Go, Diplo\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 95%|████████████████████████████▌ | 81/85 [00:37<00:02,  1.82it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: What Would You Do?, Joel Corry\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 96%|████████████████████████████▉ | 82/85 [00:37<00:01,  1.84it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Everything But You (feat. A7S), Clean Bandit\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 98%|█████████████████████████████▎| 83/85 [00:38<00:01,  1.68it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: COMPLETE MESS, 5 Seconds of Summer\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 99%|█████████████████████████████▋| 84/85 [00:39<00:00,  1.52it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n",
      "Searching for a match for: Head on Fire, Griff\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████| 85/85 [00:40<00:00,  2.12it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# checking the the geniusApi for song matches...\n",
    "print(\"Querying GENIUS api for track metadata... \\n\")\n",
    "song_set = dict()\n",
    "for song in tqdm(nSongs):\n",
    "    s = song[\"track\"]\n",
    "    artistname = s[\"artists\"][0][\"name\"]\n",
    "    songname = s[\"name\"]\n",
    "    songlyrics = txtMiner.search_genius_song(\n",
    "        track_name=songname, track_artist=artistname\n",
    "    )\n",
    "    song_set[songname] = songlyrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "a419a51d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████| 66/66 [01:17<00:00,  1.18s/it]\n"
     ]
    }
   ],
   "source": [
    "# extracting query results...\n",
    "song_set = dict(\n",
    "    zip(\n",
    "        [val for val in song_set if song_set[val]],\n",
    "        [song_set[val][\"result\"] for val in song_set if song_set[val]],\n",
    "    )\n",
    ")\n",
    "for songname in tqdm(song_set):\n",
    "    url = song_set[songname][\"url\"]\n",
    "    songlyrics = txtMiner.scrape_genius_lyrics(\n",
    "        artistname=\"none\", songname=songname, url=url\n",
    "    )\n",
    "    url_lyrics[songname] = songlyrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "014df1a6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Genius url accuracy: 0.7764705882352941\n"
     ]
    }
   ],
   "source": [
    "missed = [val for val in nSongs if val[\"track\"][\"name\"] not in song_set.keys()]\n",
    "print(f\"Genius url accuracy: {(len(nSongs) - len(missed))/len(nSongs)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c588d726",
   "metadata": {},
   "source": [
    "#### Importing audio datasets..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "279024cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing audio file... (see SpotifySongAudio Wrangling.ipynb for code)\n",
    "data_path = \"/Users/dayoorigunwa/code_base/music_mapping/data/\"\n",
    "allfiles = [f for f in listdir(data_path) if isfile(join(data_path, f))]\n",
    "audio_files = [f for f in allfiles if (\"12178525311\" in f) and (\".csv\" in f)]\n",
    "\n",
    "# sanity check\n",
    "audio_df = pd.DataFrame()\n",
    "for filename in audio_files:\n",
    "    tmp_df = pd.read_csv(data_path + filename)\n",
    "    audio_df = pd.concat([audio_df, tmp_df], axis=0)\n",
    "audio_df.head()\n",
    "audio_df[\"id\"] = audio_df[\"uri\"].apply(lambda x: x.split(\":\")[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "345574d5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "889"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sanity checks...\n",
    "count = 0\n",
    "ids = [s[\"track\"][\"id\"] for s in songs]\n",
    "for val in list(audio_df[\"id\"].values):\n",
    "    if val in ids:\n",
    "        count += 1\n",
    "count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "22ef2cca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# extracting genius data...\n",
    "names = list(gApiData.columns)\n",
    "genius_data = pd.DataFrame(\n",
    "    [{\"name\": str(s), \"data\": list(gApiData[s].values)[-1]} for s in names]\n",
    ")\n",
    "genius_data.dropna(inplace=True)\n",
    "genius_data[\"name\"] = genius_data[\"name\"].astype(str)\n",
    "genius_data[\"data\"] = genius_data[\"data\"].apply(lambda x: ast.literal_eval(x))\n",
    "name_to_genius = dict(zip(genius_data.name, genius_data.data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "7ca73a72",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>lyrics</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Then the Quiet Explosion</td>\n",
       "      <td>I can’t feel you There’s no trace Lights will ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Immunity</td>\n",
       "      <td>You've answered my prayer For a worthless diam...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mumma Don't Tell</td>\n",
       "      <td>My mama don't tell I'm the same My mama don't ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Quick Musical Doodles</td>\n",
       "      <td>[Verse 1] You remember You remember my love Yo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Killing Me To Love You</td>\n",
       "      <td>[Verse 1] Your body is broken but you’re tryin...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       name                                             lyrics\n",
       "0  Then the Quiet Explosion  I can’t feel you There’s no trace Lights will ...\n",
       "1                  Immunity  You've answered my prayer For a worthless diam...\n",
       "2          Mumma Don't Tell  My mama don't tell I'm the same My mama don't ...\n",
       "3     Quick Musical Doodles  [Verse 1] You remember You remember my love Yo...\n",
       "4    Killing Me To Love You  [Verse 1] Your body is broken but you’re tryin..."
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# extracting genius data pt2...\n",
    "url_lyrics = {key: url_lyrics[key] for key in url_lyrics if len(url_lyrics[key]) > 25}\n",
    "names = list(url_lyrics.keys())\n",
    "url_lyric_df = pd.DataFrame([{\"name\": str(s), \"lyrics\": url_lyrics[s]} for s in names])\n",
    "url_lyric_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "25c0b237",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>lyrics</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Then the Quiet Explosion</td>\n",
       "      <td>I can’t feel youThere’s no traceLights will bu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Two Thousand and Seventeen</td>\n",
       "      <td>[Non-Lyrical Vocals]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Immunity</td>\n",
       "      <td>You've answered my prayerFor a worthless diamo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Quick Musical Doodles</td>\n",
       "      <td>[Verse 1]You rememberYou remember my loveYou s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Pretending</td>\n",
       "      <td>[Intrumental]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         name  \\\n",
       "0    Then the Quiet Explosion   \n",
       "1  Two Thousand and Seventeen   \n",
       "2                    Immunity   \n",
       "3       Quick Musical Doodles   \n",
       "4                  Pretending   \n",
       "\n",
       "                                              lyrics  \n",
       "0  I can’t feel youThere’s no traceLights will bu...  \n",
       "1                               [Non-Lyrical Vocals]  \n",
       "2  You've answered my prayerFor a worthless diamo...  \n",
       "3  [Verse 1]You rememberYou remember my loveYou s...  \n",
       "4                                      [Intrumental]  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# collecting lyric data...\n",
    "lyric_df_list = [gScrapeLyrics, url_lyric_df]\n",
    "lyrics_df = pd.DataFrame()\n",
    "for l_df in lyric_df_list:\n",
    "    lyrics_df = pd.concat([lyrics_df, l_df], axis=0)\n",
    "lyrics_dictlist = lyrics_df.to_dict(\"list\")\n",
    "name_to_lyrics = dict(zip(lyrics_dictlist[\"name\"], lyrics_dictlist[\"lyrics\"]))\n",
    "lyrics_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7a7bb5a",
   "metadata": {},
   "source": [
    "#### Unifying Datasets... ####"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "b1fa72fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>artist</th>\n",
       "      <th>lyrics</th>\n",
       "      <th>date_added</th>\n",
       "      <th>energy</th>\n",
       "      <th>liveness</th>\n",
       "      <th>tempo</th>\n",
       "      <th>speechiness</th>\n",
       "      <th>acousticness</th>\n",
       "      <th>instrumentalness</th>\n",
       "      <th>time_signature</th>\n",
       "      <th>danceability</th>\n",
       "      <th>valence</th>\n",
       "      <th>duration_ms</th>\n",
       "      <th>loudness</th>\n",
       "      <th>utils_spotify_id</th>\n",
       "      <th>utils_genius_data</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>family ties (with Kendrick Lamar)</td>\n",
       "      <td>Baby Keem</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2022-04-21T00:18:15Z</td>\n",
       "      <td>0.611</td>\n",
       "      <td>0.230</td>\n",
       "      <td>134.093</td>\n",
       "      <td>0.3300</td>\n",
       "      <td>0.00588</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.711</td>\n",
       "      <td>0.144</td>\n",
       "      <td>252262.0</td>\n",
       "      <td>-5.453</td>\n",
       "      <td>3QFInJAm9eyaho5vBzxInN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>trademark usa</td>\n",
       "      <td>Baby Keem</td>\n",
       "      <td>[Part I] [Intro] I can't help but feel neglect...</td>\n",
       "      <td>2022-04-21T00:18:29Z</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.274</td>\n",
       "      <td>130.732</td>\n",
       "      <td>0.2810</td>\n",
       "      <td>0.10800</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.613</td>\n",
       "      <td>0.067</td>\n",
       "      <td>270671.0</td>\n",
       "      <td>-5.621</td>\n",
       "      <td>6G9aDedv5hYaTgNYDuduqk</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>VALENTINO</td>\n",
       "      <td>24kGoldn</td>\n",
       "      <td>[Chorus] I don't want a valentine,   I just wa...</td>\n",
       "      <td>2022-04-21T00:19:39Z</td>\n",
       "      <td>0.717</td>\n",
       "      <td>0.132</td>\n",
       "      <td>150.964</td>\n",
       "      <td>0.1790</td>\n",
       "      <td>0.19900</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.746</td>\n",
       "      <td>0.523</td>\n",
       "      <td>179133.0</td>\n",
       "      <td>-4.841</td>\n",
       "      <td>6piAUJJQFD8oHDUr0b7l7q</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Pepas</td>\n",
       "      <td>Farruko</td>\n",
       "      <td>[Letra de \"Pepas\"] [Refrán] No me importa lo q...</td>\n",
       "      <td>2022-04-21T00:41:47Z</td>\n",
       "      <td>0.766</td>\n",
       "      <td>0.128</td>\n",
       "      <td>130.001</td>\n",
       "      <td>0.0343</td>\n",
       "      <td>0.00776</td>\n",
       "      <td>0.000070</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.762</td>\n",
       "      <td>0.442</td>\n",
       "      <td>287120.0</td>\n",
       "      <td>-3.955</td>\n",
       "      <td>5fwSHlTEWpluwOM0Sxnh5k</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Stick (with JID &amp; J. Cole feat. Kenny Mason &amp; ...</td>\n",
       "      <td>Dreamville</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2022-04-21T00:42:09Z</td>\n",
       "      <td>0.857</td>\n",
       "      <td>0.668</td>\n",
       "      <td>118.574</td>\n",
       "      <td>0.2920</td>\n",
       "      <td>0.26600</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.671</td>\n",
       "      <td>0.597</td>\n",
       "      <td>309323.0</td>\n",
       "      <td>-5.435</td>\n",
       "      <td>1BzXvBpIFWJgu0P8P6xmP4</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                name      artist  \\\n",
       "0                  family ties (with Kendrick Lamar)   Baby Keem   \n",
       "1                                      trademark usa   Baby Keem   \n",
       "2                                          VALENTINO    24kGoldn   \n",
       "3                                              Pepas     Farruko   \n",
       "4  Stick (with JID & J. Cole feat. Kenny Mason & ...  Dreamville   \n",
       "\n",
       "                                              lyrics            date_added  \\\n",
       "0                                                NaN  2022-04-21T00:18:15Z   \n",
       "1  [Part I] [Intro] I can't help but feel neglect...  2022-04-21T00:18:29Z   \n",
       "2  [Chorus] I don't want a valentine,   I just wa...  2022-04-21T00:19:39Z   \n",
       "3  [Letra de \"Pepas\"] [Refrán] No me importa lo q...  2022-04-21T00:41:47Z   \n",
       "4                                                NaN  2022-04-21T00:42:09Z   \n",
       "\n",
       "   energy  liveness    tempo  speechiness  acousticness  instrumentalness  \\\n",
       "0   0.611     0.230  134.093       0.3300       0.00588          0.000000   \n",
       "1   0.600     0.274  130.732       0.2810       0.10800          0.000002   \n",
       "2   0.717     0.132  150.964       0.1790       0.19900          0.000000   \n",
       "3   0.766     0.128  130.001       0.0343       0.00776          0.000070   \n",
       "4   0.857     0.668  118.574       0.2920       0.26600          0.000000   \n",
       "\n",
       "   time_signature  danceability  valence  duration_ms  loudness  \\\n",
       "0             4.0         0.711    0.144     252262.0    -5.453   \n",
       "1             4.0         0.613    0.067     270671.0    -5.621   \n",
       "2             4.0         0.746    0.523     179133.0    -4.841   \n",
       "3             4.0         0.762    0.442     287120.0    -3.955   \n",
       "4             4.0         0.671    0.597     309323.0    -5.435   \n",
       "\n",
       "         utils_spotify_id utils_genius_data  \n",
       "0  3QFInJAm9eyaho5vBzxInN               NaN  \n",
       "1  6G9aDedv5hYaTgNYDuduqk               NaN  \n",
       "2  6piAUJJQFD8oHDUr0b7l7q               NaN  \n",
       "3  5fwSHlTEWpluwOM0Sxnh5k               NaN  \n",
       "4  1BzXvBpIFWJgu0P8P6xmP4               NaN  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# vars...\n",
    "data = dict()\n",
    "columns = [\n",
    "    \"name\",\n",
    "    \"artist\",\n",
    "    \"lyrics\",\n",
    "    \"date_added\",\n",
    "    \"energy\",\n",
    "    \"liveness\",\n",
    "    \"tempo\",\n",
    "    \"speechiness\",\n",
    "    \"acousticness\",\n",
    "    \"instrumentalness\",\n",
    "    \"time_signature\",\n",
    "    \"danceability\",\n",
    "    \"valence\",\n",
    "    \"duration_ms\",\n",
    "    \"loudness\",\n",
    "    \"utils_spotify_id\",\n",
    "    \"utils_genius_data\",\n",
    "]\n",
    "\n",
    "for s in songs:\n",
    "\n",
    "    # get base fields...\n",
    "    cur_keys = list(data.keys())\n",
    "    name = s[\"track\"][\"name\"]\n",
    "    utils_spotify_id = s[\"track\"][\"id\"]\n",
    "    utils_genius_data = name_to_genius.get(name, np.nan)\n",
    "    artist = s[\"track\"][\"artists\"][0][\n",
    "        \"name\"\n",
    "    ]  # if s['track']['artists'][0]['name'] else np.nan\n",
    "    lyrics = name_to_lyrics.get(name, np.nan)\n",
    "    date_added = s[\"added_at\"]  # if s['added_at'] else np.nan\n",
    "    audio_row = audio_df[audio_df[\"id\"] == utils_spotify_id]\n",
    "\n",
    "    # set spotify fields...\n",
    "    if audio_df[audio_df[\"id\"] == utils_spotify_id].empty == False:\n",
    "        energy = (\n",
    "            list(audio_row.energy.values).pop()\n",
    "            if not audio_row.energy.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        liveness = (\n",
    "            list(audio_row.liveness.values).pop()\n",
    "            if not audio_row.liveness.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        tempo = (\n",
    "            list(audio_row.tempo.values).pop() if not audio_row.tempo.empty else np.nan\n",
    "        )\n",
    "        speechiness = (\n",
    "            list(audio_row.speechiness.values).pop()\n",
    "            if not audio_row.speechiness.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        acousticness = (\n",
    "            list(audio_row.acousticness.values).pop()\n",
    "            if not audio_row.acousticness.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        instrumentalness = (\n",
    "            list(audio_row.instrumentalness.values).pop()\n",
    "            if not audio_row.instrumentalness.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        time_signature = (\n",
    "            list(audio_row.time_signature.values).pop()\n",
    "            if not audio_row.time_signature.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        danceability = (\n",
    "            list(audio_row.danceability.values).pop()\n",
    "            if not audio_row.danceability.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        valence = (\n",
    "            list(audio_row.valence.values).pop()\n",
    "            if not audio_row.valence.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        duration_ms = (\n",
    "            list(audio_row.duration_ms.values).pop()\n",
    "            if not audio_row.duration_ms.empty\n",
    "            else np.nan\n",
    "        )\n",
    "        loudness = (\n",
    "            list(audio_row.loudness.values).pop()\n",
    "            if not audio_row.loudness.empty\n",
    "            else np.nan\n",
    "        )\n",
    "    else:\n",
    "        energy = np.nan\n",
    "        liveness = np.nan\n",
    "        tempo = np.nan\n",
    "        speechiness = np.nan\n",
    "        acousticness = np.nan\n",
    "        instrumentalness = np.nan\n",
    "        time_signature = np.nan\n",
    "        danceability = np.nan\n",
    "        valence = np.nan\n",
    "        duration_ms = np.nan\n",
    "        loudness = np.nan\n",
    "\n",
    "    # wrapping args up...\n",
    "    values = [\n",
    "        name,\n",
    "        artist,\n",
    "        lyrics,\n",
    "        date_added,\n",
    "        energy,\n",
    "        liveness,\n",
    "        tempo,\n",
    "        speechiness,\n",
    "        acousticness,\n",
    "        instrumentalness,\n",
    "        time_signature,\n",
    "        danceability,\n",
    "        valence,\n",
    "        duration_ms,\n",
    "        loudness,\n",
    "        utils_spotify_id,\n",
    "        utils_genius_data,\n",
    "    ]\n",
    "    arg_tups = zip(columns, values)\n",
    "\n",
    "    # build datadict...\n",
    "    for arg_key, arg_val in list(arg_tups):\n",
    "        if arg_key in cur_keys:\n",
    "            data[arg_key].append(arg_val)\n",
    "        else:\n",
    "            data[arg_key] = [arg_val]\n",
    "\n",
    "final_df = pd.DataFrame(columns=columns, data=data)\n",
    "final_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b9deec72",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "282"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_df.lyrics.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9f13959a",
   "metadata": {},
   "outputs": [],
   "source": [
    "date_piece = datetime.today().strftime(\"%Y-%m-%d\")\n",
    "final_df.to_csv(data_path + \"scraped_dataset_\" + date_piece + \".csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4502e543",
   "metadata": {},
   "outputs": [],
   "source": [
    "final_df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49c80b47",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "04aad67b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       3QFInJAm9eyaho5vBzxInN\n",
       "1       6G9aDedv5hYaTgNYDuduqk\n",
       "2       6piAUJJQFD8oHDUr0b7l7q\n",
       "3       5fwSHlTEWpluwOM0Sxnh5k\n",
       "4       1BzXvBpIFWJgu0P8P6xmP4\n",
       "                 ...          \n",
       "1100    0KG7XrgM20KEmJrKsyFDAm\n",
       "1101    1D2ZNECRd7KIlA4pyOUGsv\n",
       "1102    2wVQcA1YCuGL6EW0QbgVoV\n",
       "1103    1Y8COP6Oqef3UxIONvzx6I\n",
       "1104    6QLNcOI6YCqdLAS6GLWXMj\n",
       "Name: utils_spotify_id, Length: 1105, dtype: object"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from utils import get_latest_df\n",
    "scraped_df = get_latest_df()\n",
    "scraped_df.utils_spotify_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "873b2c6c-a6de-4abf-aa69-3e58ac303de8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: b'summer 23', Number of songs: 6, Playlist ID: 1x5oshXpuWGb7TRn1ej1Ow \n",
      "Name: b'none is real', Number of songs: 11, Playlist ID: 3eSJ0jbiLySXUcQsPtMrFO \n",
      "Name: b'Barbie Gorl', Number of songs: 9, Playlist ID: 1COkfyURYUuHXF8S1ezsfh \n",
      "Name: b'june bb', Number of songs: 26, Playlist ID: 2uMUHsn5vwjF9wnM2YDsAf \n",
      "Name: b'breaking bitch ', Number of songs: 12, Playlist ID: 1vzuB380vvaEYWtFsveNqh \n",
      "Name: b'EMPTY DREAMS', Number of songs: 8, Playlist ID: 3UIK2F621zffa53AEuPfBX \n",
      "Name: b'dayo ur a fuckin bitch', Number of songs: 28, Playlist ID: 6JuLwW88QE5FcGRabMZuDN \n",
      "Name: b'new muscles unlocked ', Number of songs: 7, Playlist ID: 2OuWrX2IsFgZnLw0XEzPeF \n",
      "Name: b'la di da - acoustic', Number of songs: 13, Playlist ID: 64sJirLaDYFs8bE9IM8nIy \n",
      "Name: b'mad shower ', Number of songs: 26, Playlist ID: 6uImMNgmnO5Ls6wRZlrPt7 \n",
      "Name: b'late shower ', Number of songs: 13, Playlist ID: 1mZNXPGJHFZujIMt6HSb8g \n",
      "Name: b'stoner works out ', Number of songs: 14, Playlist ID: 0v69rj4zRWesNXlxwBxiGQ \n",
      "Name: b'Your Heart', Number of songs: 27, Playlist ID: 5Gz7Kizr995O9XX4QLEEu8 \n",
      "Name: b'god help us all', Number of songs: 33, Playlist ID: 4ioyY08fTYEbyCM21677Fl \n",
      "Name: b'Miss You', Number of songs: 1, Playlist ID: 3WVyQI7fXOr39PGn6WVGks \n",
      "Name: b\"I Wasn't Made to Fall in Love\", Number of songs: 30, Playlist ID: 4aHtqEEuoaEMz6q087BrD1 \n",
      "Name: b'HIMBO', Number of songs: 24, Playlist ID: 4UO54EWjw4MPFmK6EfzjkI \n",
      "Name: b'Long Day', Number of songs: 9, Playlist ID: 0UXfA16WQYqKykudY7xc5S \n",
      "Name: b'adolescence 2', Number of songs: 6, Playlist ID: 4v0Zs01XXPkjisIvDNcmRC \n",
      "Name: b'real big heart ', Number of songs: 27, Playlist ID: 5x4QesWldLCzZAh7KlTETD \n",
      "Name: b'party', Number of songs: 3, Playlist ID: 6SzqEfI4InZn5bYqKGpYgk \n",
      "Name: b'pigeons > doves', Number of songs: 10, Playlist ID: 5T49C3LWf0DDpIi9e92ILQ \n",
      "Name: b'slow down yo', Number of songs: 46, Playlist ID: 5Ip9V2uG5iP4D76mjLckoz \n",
      "Name: b'fuck these toxic hoes', Number of songs: 13, Playlist ID: 0OBen0ZIv5WQrLoRQq1zxO \n",
      "Name: b'slit ur wrists ', Number of songs: 8, Playlist ID: 4AIvUpX4qSNTrJNeTEXpDE \n",
      "Name: b'Stupid Decisions', Number of songs: 4, Playlist ID: 3vmakAlFg4kt4ywNRK2Ps9 \n",
      "Name: b'24 year old works out ', Number of songs: 36, Playlist ID: 17jXlwg2r1juQ4zznF9ghT \n",
      "Name: b'Dandelion', Number of songs: 16, Playlist ID: 1CdqlA5yTiJ74Aul33kQaF \n",
      "Name: b'Jerk', Number of songs: 27, Playlist ID: 4ConV6qqa7agNr4gqeEhPl \n",
      "Name: b\"DOUBLE D'Z\", Number of songs: 28, Playlist ID: 4XSc5xd920aWWHOBEysDu4 \n",
      "Name: b'nov 17th', Number of songs: 38, Playlist ID: 7jMTHJC372LGiyMIBHZOCD \n",
      "Name: b'Gettin\\xe2\\x80\\x99 Crunk', Number of songs: 13, Playlist ID: 0XmTYqmx5FXoWLEAxKtYHA \n",
      "Name: b'greatest november of ur fucking life', Number of songs: 24, Playlist ID: 3KauOL4lZTwBrS1SoWYCls \n",
      "Name: b'keep it did', Number of songs: 11, Playlist ID: 5HA5h3G4nHMGuEWzSF8IYs \n",
      "Name: b'fresh picks october ', Number of songs: 11, Playlist ID: 0gSjDp7sII7dMOs1UnxuDA \n",
      "Name: b'My Playlist #320', Number of songs: 13, Playlist ID: 3z0hqmahAmtFVbtOu9xgQw \n",
      "Name: b'kate paint \\xf0\\x9f\\x8e\\xa8 ', Number of songs: 31, Playlist ID: 4oR0tXPtpte4aFDJDd0mo7 \n",
      "Name: b'White Mercedes', Number of songs: 10, Playlist ID: 4PNNA1cCUlaStG1p2vvZkA \n",
      "Name: b'September ', Number of songs: 19, Playlist ID: 2L0zKfuo9F2u0yFr6IZOam \n",
      "Name: b'Hard up', Number of songs: 43, Playlist ID: 6eVfJZ9OtSdeM6FImmLdIf \n",
      "Name: b'August 22', Number of songs: 32, Playlist ID: 0CtiF4c37AJJqaTmcLjlnk \n",
      "Name: b'Discounts', Number of songs: 18, Playlist ID: 2rzB2qvBUfd2pB9OiF20mx \n",
      "Name: b'Liquid Courage', Number of songs: 4, Playlist ID: 0vU9mRbPNx2i8JbqRwu28d \n",
      "Name: b'I am ', Number of songs: 4, Playlist ID: 7jSjp8vg8OBIH3mMeAC1md \n",
      "Name: b'depressed as an adult', Number of songs: 16, Playlist ID: 5NjLUvG2qmZx0cBfGE418d \n",
      "Name: b'Bad Girls - Switch Remix', Number of songs: 5, Playlist ID: 0ufOIjaFwDgmWnviFkYPOR \n",
      "Name: b'Summer', Number of songs: 16, Playlist ID: 6DNEqrTIwunlssHiuTMPNK \n",
      "Name: b'Suggestions (Rainer)', Number of songs: 54, Playlist ID: 0949ATgyh2PzWRhdbljAeC \n",
      "Name: b'Miami', Number of songs: 49, Playlist ID: 3EaACudrpQDR3Gc2zgrHFW \n",
      "Name: b'Want U Back', Number of songs: 15, Playlist ID: 70LoYFCXkXvXJ5GqPP3AnV \n"
     ]
    }
   ],
   "source": [
    "from spotify import spotifyApi\n",
    "helper = spotifyApi()\n",
    "sp = helper.sp\n",
    "playlists = helper.get_user_playlists(username=12125327174, sp=sp)\n",
    "playlist = playlists[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f08e23d5-dc6e-41b4-b07e-7cc1db503e74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    spotify:track:0wFd8ei34BCH8R5pkMVNyb\n",
       "1    spotify:track:2IQB5iBapo0Y37Yc0PtIZc\n",
       "2    spotify:track:7Ip8HMNbIvZ1TbB211qAKV\n",
       "3    spotify:track:2HaTi4SDrXRY9bHKVWS8Ty\n",
       "4    spotify:track:1nFusfst42jPWjXa8EAhyE\n",
       "5    spotify:track:2tu7n1Pxm75szYU3ySCpri\n",
       "Name: uri, dtype: object"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tracklist = helper.get_playlist_content(username=12125327174, playlist=playlist, sp=sp, csv=False)\n",
    "tmp_df = helper.get_playlist_audio_features(username=12125327174, playlist=playlist, sp=sp, csv=False)\n",
    "tmp_df['uri']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "552cd7ec-23fd-45d9-b218-a72c89046295",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}