EDA/1_dataset_choice.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data set selection\n",
    "This notebook should serve as a basis for selecting the data sets for our bachelor thesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data sets\n",
    "The OGB benchmark data sets for link prediction and node classification are suitable for us as data sets.\n",
    "\n",
    "Link prediction:\n",
    "* [ogbl-ppa](https://ogb.stanford.edu/docs/linkprop/#ogbl-ppa)\n",
    "* [ogbl-collab](https://ogb.stanford.edu/docs/linkprop/#ogbl-collab)\n",
    "* [ogbl-ddi](https://ogb.stanford.edu/docs/linkprop/#ogbl-ddi)\n",
    "* [ogbl-citation2](https://ogb.stanford.edu/docs/linkprop/#ogbl-citation2)\n",
    "* [ogbl-wikikg2](https://ogb.stanford.edu/docs/linkprop/#ogbl-wikikg2)\n",
    "* [ogbl-biokg](https://ogb.stanford.edu/docs/linkprop/#ogbl-biokg)\n",
    "* [ogbl-vessel](https://ogb.stanford.edu/docs/linkprop/#ogbl-vessel)\n",
    "\n",
    "Node classification:\n",
    "* [ogbn-products](https://ogb.stanford.edu/docs/nodeprop/#ogbn-products)\n",
    "* [ogbn-proteins](https://ogb.stanford.edu/docs/nodeprop/#ogbn-proteins)\n",
    "* [ogbn-arxiv](https://ogb.stanford.edu/docs/nodeprop/#ogbn-arxiv)\n",
    "* [ogbn-papers100M](https://ogb.stanford.edu/docs/nodeprop/#ogbn-papers100M)\n",
    "* [ogbn-mag](https://ogb.stanford.edu/docs/nodeprop/#ogbn-mag)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset Transfer Learning requirement criteria\n",
    "\n",
    "### Same domain\n",
    "Below are our requirement criteria for two datasets for transfer learning within the domain, ordered by priority.\n",
    "1. minimum size (> 100'000 nodes) and maximum size (<= 3'000'000 nodes)\n",
    "2. two **similar** data sets, similarity defined via domain, features\n",
    "3. feature interpretation would be nice to have\n",
    "4. if possible undirected (complexity)\n",
    "\n",
    "### Other domain\n",
    "Below are our requirement criteria for a dataset for transfer learning outside the domain,\n",
    "1. minimum size (> 100'000 nodes) and maximum size (<= 3'000'000 nodes)\n",
    "2. **Different domains** than the datasets from the same domain\n",
    "3. feature interpretation would be nice to have"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Criteria\n",
    "In the next cells, we will evaluate the data sets based on the above criteria and, if necessary, eliminate them from the selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same domain - criterion 1 (size)\n",
    "For the first criterion, we eliminate all data records that have less than **100K** nodes.\n",
    "\n",
    "This concerns the following data sets:\n",
    "* ogbl-ddi\n",
    "* ogbl-biokg\n",
    "\n",
    "These data records are therefore already eliminated from the selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same domain - criterion 2 (similarity)\n",
    "The second criterion is always looked at with data record pairs. If a data record has no partner (same domain), it is eliminated.\n",
    "\n",
    "| ID | dataset | domain | dataset partner | remarks |\n",
    "|---- |---------------- |----------------- |------------------------- |------------------------------------------------------------------------ |\n",
    "| A | ogbn-mag | citations, authors | B, C, D, I | must be converted from two-mode network to one-mode network |\n",
    "| B | ogbn-arxiv | citations | A, D, I | can be a subset of \"obgn-mag\" |\n",
    "| C | ogbl-collab | authors | A | can be a subset of \"obgn-mag\" |\n",
    "| D | ogbl-citation2 | Citations | A, B, I | can be a subset of \"obgn-mag\" |\n",
    "| E | ogbl-ppa | proteins | F | |\n",
    "| F | ogbn-proteins | proteins | E | |\n",
    "| G | ogbn-products | products | |                                                            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same domain - criterion 3 (feature interpretation)\n",
    "The third criterion defines whether the features of the nodes and edges can be interpreted directly or only after processing.\n",
    "An example of a directly interpretable feature would be a continuous variable from everyday life. For example, the age or height of a \"person\" node. Embedding vectors are an example of a feature that cannot be interpreted directly.  \n",
    "\n",
    "\n",
    "| ID | Dataset | Features (Node) | Features (Edges) | Interpretation <br>direct | Interpretation <br>with Processing | Remarks |\n",
    "|---- |---------------- |--------------------------------------------- |------------------------------------- |--------------------------- |------------------------------------- |------------------------------------------------------------------------------------------------------------------------------------- |\n",
    "| A | ogbn-mag | 128-dimensional <br>word2vec feature vector | keine | ❌ | ✅ | 128-dimensional feature vector obtained by <br>averaging the embeddings of words <br>in the paper's title and abstract.             \t|\n",
    "| B | ogbn-arxiv | 128-dimensional <br>word2vec feature vector | keine | ❌ | ✅ | 128-dimensional feature vector obtained by <br>averaging the embeddings of words <br>in the paper's title and abstract.             \t|\n",
    "| C | ogbl-collab | 128-dimensional <br>word2vec feature vector | publication year,<br>Authors | ❌ | ✅ | 128-dimensional feature vector obtained by <br>averaging the word embeddings of papers <br>published by the authors |\n",
    "| D | ogbl-citation2 | 128-dimensional <br>word2vec feature vector | keine | ❌ | ✅ | 128-dimensional feature vector summarize<br> the title and abstract of the paper |\n",
    "| E | ogbl-ppa | Species Vektor | keine | ✅ | ✅ | species vector indicates the species <br>that the corresponding protein comes from.                                                 \t|\n",
    "| F | ogbn-proteins | Species Vektor | keine | ✅ | ✅ | presumably species is not entirely clear,,<br>species vector indicates the species <br>that the corresponding protein comes from. \t| |\n",
    "| I | ogbn-papers100M | 128-dimensional <br>word2vec feature vector | keine | ❌ | ✅ | 128-dimensional feature vector obtained by <br>averaging the embeddings of words <br>in the paper's title and abstract.             \t|\n",
    "\n",
    "There is a clear distinction in the features of the citation datasets (embedding based) and the protein datasets which keep the species as a feature.\n",
    "Since the species features only define the node types, the embedding features are better for us because more information (128 dimensions versus 1 dimension) is available.\n",
    "\n",
    "We therefore eliminate **ogbn-proteins** and **ogbl-ppa** from the selection.\n",
    "However, they remain good candidates for the data set of another domain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same domain - criterion 4 (direction)\n",
    "The fourth criterion determines whether the edges are directed or undirected. Undirected edges reduce the complexity of the prediction and are therefore preferred.\n",
    "| ID | Data set | Undirected |\n",
    "|---- |---------------- |------------- |\n",
    "| A | ogbn-mag | ❌ |\n",
    "| B | ogbn-arxiv | ❌ |\n",
    "| C | ogbl-collab | ✅ |\n",
    "| D | ogbl-citation2 | ❌ |\n",
    "| I | ogbn-papers100M | ❌ |\n",
    "\n",
    "Since we need a few and only **ogbl-collab** is undirected, we can no longer completely eliminate pairs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same domain - final choice\n",
    "\n",
    "Possible pairs are as follows:\n",
    "\n",
    "| data set 1 | data set 2 |\n",
    "|--------|--------|\n",
    "| A | B |\n",
    "| A | C |\n",
    "| A | D |\n",
    "| A | I |\n",
    "| B | D |\n",
    "| **B** | **I** |\n",
    "| D | I |\n",
    "\n",
    "**ogbn-papers100M** (I) is a comprehensive dataset that is highly compatible with other datasets due to its thematic focus. It is particularly suitable for pre-training purposes, as it covers a large number of scientific publications and various disciplines. A broad spectrum of training data can thus be obtained. **ogbn-arxiv** (B) then enables evaluation and fine-tuning on a comparatively smaller data set. In addition to scientific publications, ogbn-arxiv also contains a proportion of financial and economic publications.\n",
    "\n",
    "We therefore select the pair **ogbn-papers100M** and **ogbn-arxiv**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other domain - Criterion 1 (size)\n",
    "For the first criterion, we eliminate all data records that have fewer than **100K** nodes.\n",
    "\n",
    "This also applies to the following datasets:\n",
    "* ogbl-ddi\n",
    "* ogbl-biokg\n",
    "\n",
    "These data records are therefore already eliminated from the selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other domain - Criterion 2 (Other domain)\n",
    "The second criterion determines whether a data set has a domain other than **Citations or authors**.\n",
    "\n",
    "| ID | Data set | Domain | Other domain |\n",
    "|---- |---------------- |----------------- |------------------------- |\n",
    "| A | ogbn-mag | Citations, Authors | ❌ |\n",
    "| B | ogbn-arxiv | Citations | ❌ |\n",
    "| C | ogbl-collab | Authors | ❌ |\n",
    "| D | ogbl-citation2 | Citations | ❌ |\n",
    "| E | ogbl-ppa | Proteins | ✅ |\n",
    "| F | ogbn-proteins | proteins | ✅ |\n",
    "| G | ogbn-products | products | ✅ |\n",
    "| H | ogbl-wiki2g | Wikidata | ✅ |\n",
    "| I | ogbn-papers100M | Citations | ❌                        \t\n",
    "| J | ogbl-vessel | Brain vessels of a mouse | ✅                      \t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Same domain - criterion 3 (feature interpretation)\n",
    "The third criterion defines whether the features of the nodes and edges can be interpreted directly or only after processing.\n",
    "An example of a directly interpretable feature would be a continuous variable from everyday life. For example, the trunk length or the trunk diameter on a \"tree\" node. Embedding vectors are an example of a feature that cannot be interpreted directly.  \n",
    "\n",
    "\n",
    "| ID | Dataset | Features (Nodes) | Features (Edges) | Interpretation <br>direct | Interpretation <br>with Processing | Remarks |\n",
    "|---- |---------------- |--------------------------------------------- |------------------------------------- |--------------------------- |------------------------------------- |------------------------------------------------------------------------------------------------------------------------------------- |\n",
    "| E | ogbl-ppa | Species Vektor | None | ✅ | ✅ | species vector indicates the species <br>that the corresponding protein comes from.                                                 \t|\n",
    "| F | ogbn-proteins | Species Vektor | None | ✅ | ✅ | presumably species is not entirely clear,,<br>species vector indicates the species <br>that the corresponding protein comes from. \t| |\n",
    "| G | ogbn-products | bag-of-words Features | none | ❌ | ✅ |Features are from the product descriptions| |\n",
    "| H | ogbl-wiki2g | none | none | ❌ | ❌ | Features not described in detail, Knowledge Graph und Triple Edge Prediction | |\n",
    "| J | ogbl-vessel | spatial (x,y,z) in Allen Brain atlas reference space | none | ❌ | ❌ | special case for link prediction due to physical conditions in the brain |       \n",
    "\n",
    "\n",
    "Since **ogbl-wiki2g** has not defined any features more precisely, we are eliminating this data set. The bag-of-words features can be interpreted with formatting, but the other two data sets have features that are easier to interpret. We therefore also eliminate **ogbn-products**.\n",
    "**ogbl-vessel** is a special case and uses model architectures that train on physical objects. For our general transfer learning, this dataset is not useful and is also eliminated.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other domain - final choice\n",
    "\n",
    "There is only **ogbl-ppa** (E) or **ogbn-proteins** (F) left to choose from. \n",
    "**ogbn-proteins** is from the *node classification* benchmark and is therefore omitted.\n",
    "\n",
    "We therefore select the **ogbl-ppa** data set."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "link-prediction-in-graphs-V4XDtXd8",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}