examples/DKVMN/DKVMN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dynamic Key-Value Memory Network(DKVMN)\n",
    "\n",
    "This notebook will show you how to train and use the DKVMN.\n",
    "First, we will show how to get the data (here we use assistment-2009-2010-skill as the dataset).\n",
    "Then we will show how to train a DKVMN and perform the parameters persistence.\n",
    "At last, we will show how to load the parameters from the file and evaluate on the test dataset.\n",
    "\n",
    "The script version could be found in [DKVMN.py](DKVMN.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameters Initialization\n",
    "First of all, we need to initialize the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    'max_iter': 50, # 'number of iterations'\n",
    "    'show': True, # 'print progress')\n",
    "    'init_std': 0.1, # 'weight initialization std'\n",
    "    'init_lr': 0.01, # 'initial learning rate'\n",
    "    'lr_decay': 0.75, # 'learning rate decay'\n",
    "    'final_lr': 1E-5, # 'learning rate will not decrease after hitting this threshold'\n",
    "    'momentum': 0.9, # 'momentum rate'\n",
    "    'maxgradnorm': 50.0, # 'maximum gradient norm'\n",
    "    'final_fc_dim': 50, # 'hidden state dim for final fc layer'\n",
    "    'key_embedding_dim': 50, # 'question embedding dimensions')\n",
    "    'batch_size': 64, # 'the batch size')\n",
    "    'value_embedding_dim': 200, # 'answer and question embedding dimensions')\n",
    "    'memory_size': 20, # 'memory size')\n",
    "    'n_question': 123, # 'the number of unique questions in the dataset')\n",
    "    'seqlen': 200, # 'the allowed maximum length of a sequence')\n",
    "    'data_dir': '../../data/2009_skill_builder_data_corrected', # 'data directory')\n",
    "    'data_name': '', # 'data set name')\n",
    "    'load': 'dkvmn.params', # 'model file to load')\n",
    "    'save': 'dkvmn.params' # 'path to save model')\n",
    "}\n",
    "\n",
    "params['lr'] = params['init_lr']\n",
    "params['key_memory_state_dim'] = params['key_embedding_dim']\n",
    "params['value_memory_state_dim'] = params['value_embedding_dim']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation\n",
    "\n",
    "Before we process the data, we need to first acquire the dataset which is shown in [prepare_dataset.ipynb](prepare_dataset.ipynb)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from load_data import Data\n",
    "\n",
    "dat = Data(n_question=params['n_question'], seqlen=params['seqlen'], separate_char=',') \n",
    "\n",
    "train_data_path = params['data_dir'] + \"/\" + params['data_name'] + \"train.txt\"\n",
    "test_data_path = params['data_dir'] + \"/\" + params['data_name'] + \"test.txt\"\n",
    "train_data = dat.load_data(train_data_path)\n",
    "test_data = dat.load_data(test_data_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training and Persistence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 0: 100%|██████████| 54/54 [00:13<00:00,  4.03it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50, loss : 0.59617, auc : 0.68320, accuracy : 0.68844\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 1: 100%|██████████| 54/54 [00:10<00:00,  5.11it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2/50, loss : 0.56427, auc : 0.72989, accuracy : 0.71260\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 2: 100%|██████████| 54/54 [00:10<00:00,  5.00it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3/50, loss : 0.54913, auc : 0.74901, accuracy : 0.72359\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 3: 100%|██████████| 54/54 [00:10<00:00,  4.91it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 4/50, loss : 0.54316, auc : 0.75662, accuracy : 0.72850\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 4: 100%|██████████| 54/54 [00:10<00:00,  4.96it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 5/50, loss : 0.53881, auc : 0.76198, accuracy : 0.73132\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 5: 100%|██████████| 54/54 [00:10<00:00,  4.98it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 6/50, loss : 0.53270, auc : 0.76903, accuracy : 0.73659\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 6: 100%|██████████| 54/54 [00:10<00:00,  5.10it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 7/50, loss : 0.52380, auc : 0.77864, accuracy : 0.74353\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 7: 100%|██████████| 54/54 [00:10<00:00,  5.05it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 8/50, loss : 0.51561, auc : 0.78680, accuracy : 0.74956\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 8: 100%|██████████| 54/54 [00:12<00:00,  4.24it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 9/50, loss : 0.50961, auc : 0.79218, accuracy : 0.75275\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 9: 100%|██████████| 54/54 [00:12<00:00,  4.21it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 10/50, loss : 0.50504, auc : 0.79609, accuracy : 0.75524\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 10: 100%|██████████| 54/54 [00:11<00:00,  4.59it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 11/50, loss : 0.50155, auc : 0.79902, accuracy : 0.75627\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 11: 100%|██████████| 54/54 [00:11<00:00,  4.55it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 12/50, loss : 0.49877, auc : 0.80131, accuracy : 0.75756\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 12: 100%|██████████| 54/54 [00:11<00:00,  4.60it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 13/50, loss : 0.49644, auc : 0.80318, accuracy : 0.75889\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 13: 100%|██████████| 54/54 [00:11<00:00,  4.56it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 14/50, loss : 0.49449, auc : 0.80475, accuracy : 0.76005\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 14: 100%|██████████| 54/54 [00:12<00:00,  4.48it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 15/50, loss : 0.49260, auc : 0.80624, accuracy : 0.76119\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 15: 100%|██████████| 54/54 [00:11<00:00,  4.60it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 16/50, loss : 0.49075, auc : 0.80766, accuracy : 0.76230\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 16: 100%|██████████| 54/54 [00:11<00:00,  4.56it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 17/50, loss : 0.48907, auc : 0.80896, accuracy : 0.76312\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 17: 100%|██████████| 54/54 [00:11<00:00,  4.62it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 18/50, loss : 0.48754, auc : 0.81019, accuracy : 0.76344\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 18: 100%|██████████| 54/54 [00:11<00:00,  4.56it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 19/50, loss : 0.48606, auc : 0.81132, accuracy : 0.76417\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 19: 100%|██████████| 54/54 [00:11<00:00,  4.54it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 20/50, loss : 0.48479, auc : 0.81225, accuracy : 0.76462\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 20: 100%|██████████| 54/54 [00:11<00:00,  4.58it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 21/50, loss : 0.48369, auc : 0.81312, accuracy : 0.76510\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 21: 100%|██████████| 54/54 [00:12<00:00,  4.37it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 22/50, loss : 0.48256, auc : 0.81401, accuracy : 0.76555\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 22: 100%|██████████| 54/54 [00:12<00:00,  4.40it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 23/50, loss : 0.48137, auc : 0.81489, accuracy : 0.76594\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 23: 100%|██████████| 54/54 [00:12<00:00,  4.27it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 24/50, loss : 0.48038, auc : 0.81565, accuracy : 0.76678\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 24: 100%|██████████| 54/54 [00:12<00:00,  4.35it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 25/50, loss : 0.47940, auc : 0.81644, accuracy : 0.76719\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 25: 100%|██████████| 54/54 [00:12<00:00,  4.45it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 26/50, loss : 0.47852, auc : 0.81717, accuracy : 0.76756\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 26: 100%|██████████| 54/54 [00:11<00:00,  4.66it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 27/50, loss : 0.47767, auc : 0.81780, accuracy : 0.76821\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 27: 100%|██████████| 54/54 [00:12<00:00,  4.46it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 28/50, loss : 0.47693, auc : 0.81835, accuracy : 0.76856\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 28: 100%|██████████| 54/54 [00:12<00:00,  4.48it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 29/50, loss : 0.47618, auc : 0.81895, accuracy : 0.76869\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 29: 100%|██████████| 54/54 [00:13<00:00,  4.15it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 30/50, loss : 0.47550, auc : 0.81945, accuracy : 0.76917\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 30: 100%|██████████| 54/54 [00:12<00:00,  4.28it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 31/50, loss : 0.47480, auc : 0.82001, accuracy : 0.76932\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 31: 100%|██████████| 54/54 [00:11<00:00,  4.64it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 32/50, loss : 0.47415, auc : 0.82049, accuracy : 0.76968\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 32: 100%|██████████| 54/54 [00:12<00:00,  4.49it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 33/50, loss : 0.47359, auc : 0.82091, accuracy : 0.77002\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 33: 100%|██████████| 54/54 [00:12<00:00,  4.33it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 34/50, loss : 0.47320, auc : 0.82115, accuracy : 0.77007\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 34: 100%|██████████| 54/54 [00:12<00:00,  4.27it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 35/50, loss : 0.47278, auc : 0.82148, accuracy : 0.77020\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 35: 100%|██████████| 54/54 [00:12<00:00,  4.29it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 36/50, loss : 0.47229, auc : 0.82182, accuracy : 0.77033\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 36: 100%|██████████| 54/54 [00:12<00:00,  4.27it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 37/50, loss : 0.47178, auc : 0.82221, accuracy : 0.77064\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 37: 100%|██████████| 54/54 [00:12<00:00,  4.25it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 38/50, loss : 0.47123, auc : 0.82262, accuracy : 0.77082\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 38: 100%|██████████| 54/54 [00:12<00:00,  4.30it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 39/50, loss : 0.47073, auc : 0.82299, accuracy : 0.77083\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 39: 100%|██████████| 54/54 [00:12<00:00,  4.34it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 40/50, loss : 0.47020, auc : 0.82340, accuracy : 0.77117\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 40: 100%|██████████| 54/54 [00:12<00:00,  4.23it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 41/50, loss : 0.46975, auc : 0.82373, accuracy : 0.77130\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 41: 100%|██████████| 54/54 [00:12<00:00,  4.22it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 42/50, loss : 0.46919, auc : 0.82418, accuracy : 0.77158\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 42: 100%|██████████| 54/54 [00:13<00:00,  3.88it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 43/50, loss : 0.46874, auc : 0.82454, accuracy : 0.77174\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 43: 100%|██████████| 54/54 [00:14<00:00,  3.69it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 44/50, loss : 0.46822, auc : 0.82496, accuracy : 0.77222\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 44: 100%|██████████| 54/54 [00:14<00:00,  3.71it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 45/50, loss : 0.46775, auc : 0.82525, accuracy : 0.77249\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 45: 100%|██████████| 54/54 [00:12<00:00,  4.28it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 46/50, loss : 0.46724, auc : 0.82570, accuracy : 0.77225\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 46: 100%|██████████| 54/54 [00:12<00:00,  4.49it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 47/50, loss : 0.46660, auc : 0.82613, accuracy : 0.77263\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 47: 100%|██████████| 54/54 [00:12<00:00,  4.32it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 48/50, loss : 0.46622, auc : 0.82650, accuracy : 0.77244\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 48: 100%|██████████| 54/54 [00:12<00:00,  4.31it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 49/50, loss : 0.46552, auc : 0.82703, accuracy : 0.77287\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 49: 100%|██████████| 54/54 [00:12<00:00,  4.28it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 50/50, loss : 0.46511, auc : 0.82737, accuracy : 0.77301\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "from EduKTM import DKVMN\n",
    "\n",
    "dkvmn = DKVMN(n_question=params['n_question'],\n",
    "                  batch_size=params['batch_size'],\n",
    "                  key_embedding_dim=params['key_embedding_dim'],\n",
    "                  value_embedding_dim=params['value_embedding_dim'],\n",
    "                  memory_size=params['memory_size'],\n",
    "                  key_memory_state_dim=params['key_memory_state_dim'],\n",
    "                  value_memory_state_dim=params['value_memory_state_dim'],\n",
    "                  final_fc_dim=params['final_fc_dim'])\n",
    "\n",
    "dkvmn.train(params, train_data)\n",
    "dkvmn.save(params['save'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading and Testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Evaluating: 100%|██████████| 23/23 [00:04<00:00,  5.58it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "valid auc : 0.81220, valid accuracy : 0.76446\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(0.4879691173201022, 0.7644589144273288, 0.8122003842542973)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dkvmn.load(params['load'])\n",
    "dkvmn.eval(params, test_data)"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "bb8bd11e89716d57dfd9de1f9b81cc8afd775dceec5e3f1c2f38528fd44f5b25"
  },
  "kernelspec": {
   "display_name": "Python 3.8.12 ('Edu')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}