Parallelization, save to file and exploit symmetries

.gitignore

@@ -111,4 +111,5 @@ ENV/
 # Program specific files
 *bot_algo_time*.json
 *montecarlo_tree*.json
-script.py
+script.py
+montecarlo

Makefile

@@ -56,22 +56,14 @@ lint: lint/flake8 ## check style
 test: ## run tests quickly with the default Python
 	pytest
 
-# This takes a very long time to execute
-# coverage: ## check code coverage quickly with the default Python
-# 	coverage run --source quantikai -m pytest
-# 	coverage report -m
-# 	coverage html
-# 	$(BROWSER) htmlcov/index.html
-
 install: clean ## install the package to the active Python's site-packages ## python setup.py install
-	pip install -r requirements_dev.txt
-	pip install --editable . 
+	pip install --editable .[dev]
 
 cli: # e.g. make cli ARG=bot
-	python src/quantikai/cli.py $(ARG)
+	quantikai $(ARG)
 
 dev: 
 	flask --app src/quantikai/wsgi.py run --debug
 
 devg:
-	gunicorn -w 4 'quantikai.wsgi:create_app()'
+	gunicorn --workers 3 'quantikai.wsgi:create_app()'

README.md

@@ -23,15 +23,13 @@ Tested on Linux with Python 3.12
         make install
 
         # Get help
-        python src/quantikai/cli.py --help
-        # Same as:
-        make cli ARG=--help
+        quantikai --help
 
         # Play against a bot
-        make cli ARG=bot
+        quantikai bot
 
         # Play again yourself
-        make cli ARG=human
+        quantikai human
 
         # Deploy the web interface
         # with gunicorn (recommended)
@@ -52,17 +50,17 @@ FEATURE IN PROGRESS:
 To get a better bot (from the web interface), do:
 
 ```bash
-        make cli ARG=montecarlo
+        quantikai montecarlo --depth=2
         make devg
 ```
 
-`make cli ARG=montecarlo` pre-computes the game tree, going much deeper than the on-the-fly algorithm. The result is then used for the first
-few moves on the board, subsequent moves are calcultaed on the fly.
+`quantikai montecarlo --depth=2` pre-computes the game tree, going much deeper than the on-the-fly algorithm. The result is then used for the first
+2 moves on the board, subsequent moves are calcultaed on the fly.
 
 ## Timing
 
 ```bash
-        make cli ARG=timer
+        quantikai timer
 ```
 
 This writes to a file "bot_algo_time.json".
@@ -158,6 +156,38 @@ Each run is composed of 3 phases:
   Update the score of each node that has been visited during this run. If the node is a move by the current player, add the reward.
   If the node is a move by the opponent, then flip the reward: 0 for a win of the current player, number of moves played for a loss.
 
+#### Montecarlo - pre-compute the game tree
+
+Pre-compute the game tree and save it to files (one file per number of pieces on the board and color to play to keep the files small enough to hold in memory).  
+Expected advantage: compute the tree with way more iterations, getting a better result.
+The issue is that in that case, we compute every state of the board instead of using symmetries for the first 2 moves. Indeed, the opponent may play anything, so if we compute only non-redundant moves we have to map the other board state to these moves. Otherwise, the increased number of moves to compute (eg 16*4 instead of 3 for the first move) negates the advantage of offline computation.
+
+#### Montecarlo - paralellization
+
+See the biography for running parallel computing for the MonteCarlo method, the simplest is also the fastest: `single-run parallelization`, run n iterations in different processes without sharing data and sum the results.
+
+Significant speed-up with the CLI, but the `multiprocess` library interfers with the gunicorn processes (`multiprocess.dummy` is worse than) so it needs more work to use it with the web app.
+
+10'000 iterations, algo time per number of pawns on the board:
+
+```json
+    "0": 16.65,
+    "1": 14.92,
+    "2": 13.11,
+    "3": 2.9,
+    "4": 10.3,
+```
+
+10'000 iterations, 5 processes running 2'000 iterations each:
+
+```json
+    "0": 4.3,
+    "1": 4.96,
+    "2": 4.29,
+    "3": 1.15,
+    "4": 3.07,
+```
+
 ### Speed bottleneck
 
 A Node contains a board and the next move to play.
@@ -173,7 +203,7 @@ How to measure time gains:
     print(min(d.repeat(20, 100000)))
   ```
 
-- for global algos: `make cli ARG=timer`
+- for global algos: `quantikai timer`
 
 #### Board operations: save the board state as a dict of moves vs a list of list
 
@@ -259,11 +289,16 @@ TODO
 
 #### Parallel computation
 
-Instead of `n_iter` sequential runs, we might imagine doing `n` runs in parallel for `n_iter` times. We would introduce a random element in the
-selection steps, so that the `n` runs are different. This is probably getting closer to a reinforcement learning setup, with a Markov decision process.
+For the Monte Carlo tree search, there are 3 methods of parallelization:
+
+- leaf parallelization
+- root (or single-run) parallelization
+- tree parallelization
 
 ## Sources
 
 1. [A Survey of MonteCarlo Search Methods](http://www.incompleteideas.net/609%20dropbox/other%20readings%20and%20resources/MCTS-survey.pdf)
-2. [AlphaZero](https://arxiv.org/pdf/1712.01815)
-3. [MuZero](https://arxiv.org/pdf/1911.08265)
+2. [Tristan Cazenave, Nicolas Jouandeau. On the Parallelization of UCT. Computer Games Workshop, Jun 2007, Amsterdam, Netherlands. ￿hal-02310186￿](https://hal.science/hal-02310186/document)
+3. [Parallel Monte-Carlo Tree Search](https://dke.maastrichtuniversity.nl/m.winands/documents/multithreadedMCTS2.pdf)
+4. [AlphaZero](https://arxiv.org/pdf/1712.01815)
+5. [MuZero](https://arxiv.org/pdf/1911.08265)

pyproject.toml

@@ -18,15 +18,23 @@ classifiers = [
 ]
 license = {text = "MIT license"}
 dependencies = [
-
+    "bump2version==0.5.11",
+    "wheel==0.33.6",
+    "typer==0.15.1",
+    "watchdog==6.0.0",
+    "Flask==3.1.0",
+    "gunicorn==23.0.0",
 ]
+[project.scripts]
+quantikai = "quantikai.cli:app"
 
 [project.optional-dependencies]
 dev = [
-    "coverage",  # testing
-    "black",  # linting
-    "flake8",  # linting
-    "pytest",  # testing
+    "coverage==4.5.4",  # testing
+    "black==24.10.0",  # linting
+    "flake8==7.1.1",  # linting
+    "pytest==6.2.4",  # testing
+    "tox==3.14.0",
 ]
 
 [project.urls]
@@ -55,9 +63,32 @@ strict = true
 warn_unreachable = true
 warn_no_return = true
 
-[[tool.mypy.overrides]]
+[tool.mypy.overrides]
 # Don't require test functions to include types
 module = "tests.*"
 allow_untyped_defs = true
 disable_error_code = "attr-defined"
 
+[tool.black]
+line-length = 79
+include = '\.pyi?$'
+exclude = '''
+/(
+    \.git
+  | \.hg
+  | \.mypy_cache
+  | \.tox
+  | \.venv
+  | _build
+  | buck-out
+  | build
+  | dist
+)/
+'''
+
+[tool.flake8]
+max-line-length = 79
+max-complexity = 18
+
+
+

src/quantikai/bot/__init__.py

@@ -1,5 +1,5 @@
-from quantikai.bot.main import get_best_move
-from quantikai.bot.comparison import get_method_times
 from quantikai.bot import montecarlo
+from quantikai.bot.comparison import get_method_times
+from quantikai.bot.main import get_best_move
 
 __all__ = ["get_best_move", "get_method_times", "montecarlo"]

src/quantikai/bot/comparison.py

@@ -1,13 +1,13 @@
 """Compare the execution time of each method"""
 
-import datetime
 import copy
-import pathlib
+import datetime
 import json
+import pathlib
 import timeit
 
 from quantikai.bot import minmax, montecarlo
-from quantikai.game import Board, Pawns, Colors, Player, Move
+from quantikai.game import Board, Colors, Move, Pawns, Player
 
 
 def init_test_values():
@@ -55,8 +55,8 @@ def init_test_values():
 
 def get_method_times():
 
-    n_iter = 10
-    n_repeat = 5
+    n_iter = 5
+    n_repeat = 1
 
     timestamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
 
@@ -71,7 +71,8 @@ def get_method_times():
     result_file = pathlib.Path("bot_algo_time_" + timestamp + ".json")
     current_color = Colors.BLUE
     times["montecarlo"]["args"] = {
-        "iterations": 10000,
+        "num_process": 5,
+        "iterations": 2000,
         "use_depth": True,
     }
     times["minmax"]["args"] = {}
@@ -88,7 +89,9 @@ def get_method_times():
                     **times["minmax"]["args"]
                 )
             )
-            times["minmax"][idx] = round(min(d.repeat(n_repeat, n_iter)) / n_iter, 2)
+            times["minmax"][idx] = round(
+                min(d.repeat(n_repeat, n_iter)) / n_iter, 2
+            )
 
         d = timeit.Timer(
             lambda: montecarlo.get_best_move(
@@ -98,7 +101,9 @@ def get_method_times():
                 **times["montecarlo"]["args"]
             )
         )
-        times["montecarlo"][idx] = round(min(d.repeat(n_repeat, n_iter)) / n_iter, 2)
+        times["montecarlo"][idx] = round(
+            min(d.repeat(n_repeat, n_iter)) / n_iter, 2
+        )
 
         # Update the measures at every iteration
         result_file.write_text(json.dumps(times, indent=2))

src/quantikai/bot/main.py

@@ -1,19 +1,19 @@
 import pathlib
 
-from quantikai.game import Board, Player, Move
-from quantikai.bot import montecarlo, minmax
+from quantikai.bot import montecarlo
+from quantikai.game import Board, Move, Player
 
 
 def get_best_move(
     board: Board,
     current_player: Player,
     other_player: Player,
-    game_tree_file: pathlib.Path | None,
+    game_tree_folder: pathlib.Path | None = None,
 ) -> Move | None:
 
     return montecarlo.get_best_move(
         board=board,
         current_player=current_player,
         other_player=other_player,
-        game_tree_file=game_tree_file,
+        game_tree_folder=game_tree_folder,
     )

src/quantikai/bot/minmax.py

@@ -1,6 +1,6 @@
 import copy
 
-from quantikai.game import Board, Player, Colors, Move
+from quantikai.game import Board, Colors, Move, Player
 
 
 def get_best_move(

src/quantikai/bot/montecarlo/__init__.py

@@ -1,8 +1,8 @@
 from quantikai.bot.montecarlo.main import (
+    generate_tree,
     get_best_move,
     get_best_play,
     get_move_stats,
-    generate_tree,
 )
 
 __all__ = ["get_best_move", "get_best_play", "get_move_stats", "generate_tree"]