Add group 10 B-Raft project

.gitignore

@@ -169,3 +169,4 @@ tags
 .DS_Store
 
 # End of https://www.gitignore.io/api/vim,git,sbt,java,maven,scala,intellij
+B-Raft/src/.github/workflows/ci-test.yml

B-Raft/.gitignore

@@ -0,0 +1,4 @@
+experiments/resources/
+src/null/
+target/
+experiments/.idea/

B-Raft/README.md

@@ -0,0 +1,15 @@
+# [A Byzantine Fault-Tolerant Raft Algorithm Combined with Schnorr Signature (B-Raft), [1]](https://ieeexplore.ieee.org/document/9377376)
+## Implemented in Scala for IN4391 Distributed Systems (2021/22 Q3)
+Group 10: Julian Biesheuvel, Riley Jense & Pepijn te Marvelde
+
+## General Information
+This repo contains the code for the project of the Distributed Systems course. It is an implementation of the B-Raft with Schnorr Signature [[1]](###-bibliography), based on a prototype Raft implementation by Max Bundscherer [[2]](###-biblography). Akka Actors are used for communication between nodes. 
+
+## File Structure
+Source code for the Raft/B-Raft implementation is found in [`src` (link)](src). Experiments, including result dataset and plotting code, are found in [`experiments` (link)](experiments). More info on each folder is found in their respective `README.md`'s.
+
+
+
+### Bibliography
+1.  S. Tian, Y. Liu, Y. Zhang and Y. Zhao, "A Byzantine Fault-Tolerant Raft Algorithm Combined with Schnorr Signature," 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM), 2021, pp. 1-5, doi: 10.1109/IMCOM51814.2021.9377376.
+2. M. Bundscherer, "(Prototype) Raft Consensus Algorithm in Scala," Github repository, 2020, https://github.com/maxbundscherer/prototype-scala-raft

B-Raft/experiments/README.md

@@ -0,0 +1,46 @@
+# BRaft Experiments
+To quantitatively assess the quality of our BRaft implementation run our cluster under different circumstances with different configurations. 
+
+This is done in run_experiments.py.
+At the bottom of this file one can specify what experiments to run and how many times to replicate these.
+
+The script then creates the different configurations and call BRaft Main with them, this starts our cluster. Logs are captured as they contain an accurate view of our system at any time. These logs are then parsed to collect metrics and draw conclusions.
+
+## Configuring experiments
+Through the `experiment_config` dict all parameters in BRaft's [`application.conf` (link)](../src_/main/resources/application.conf) raftPrototype key can be controlled. These are:
+
+```editorconfig
+raftPrototype {
+	raftType="BRaft"
+
+	electionTimerIntervalMin=3
+
+	electionTimerIntervalMax=4
+
+	heartbeatTimerInterval=1
+
+	nodes=21
+
+	crashIntervalHeartbeats=1000000
+
+	sleepDowntime=8
+
+	maxTerm=9999
+}
+```
+
+### example
+By setting 
+```python
+    num_replications = 4
+    experiment_config = {
+        "raftType": ["Raft", "BRaft"],
+        "nodes": np.linspace(start=3, stop=21, num=9, dtype=int),
+    }
+```
+in `run_experiments.py` 4 runs of experiments are done with every combination of nodes and raftType specified. This particular config will thus result in 2 * 9 * 4 = 54 output logs.
+
+
+## Log parsing and plotting
+Can be found in the [Plotting notebook](DS%20Graphs.ipynb)
+

B-Raft/experiments/julian_run.txt

@@ -0,0 +1 @@
+C:\Users\Jbies\.jdks\azul-13.0.10\bin\java.exe "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.3.2\lib\idea_rt.jar=51173:C:\Program Files\JetBrains\IntelliJ IDEA 2021.3.2\bin" -Dfile.encoding=UTF-8 -classpath "C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\target\scala-2.13\classes;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\github\oshi\oshi-core\5.7.5\oshi-core-5.7.5.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\akka\akka-actor-typed_2.13\2.6.0\akka-actor-typed_2.13-2.6.0.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\akka\akka-actor_2.13\2.6.0\akka-actor_2.13-2.6.0.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\akka\akka-slf4j_2.13\2.6.0\akka-slf4j_2.13-2.6.0.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\config\1.4.1\config-1.4.1.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\io\kamon\kamon-apm-reporter_2.13\2.5.1\kamon-apm-reporter_2.13-2.5.1.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\io\kamon\kamon-bundle_2.13\2.5.1\kamon-bundle_2.13-2.5.1.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\io\kamon\kamon-core_2.13\2.5.1\kamon-core_2.13-2.5.1.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\net\java\dev\jna\jna-platform\5.8.0\jna-platform-5.8.0.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\net\java\dev\jna\jna\5.8.0\jna-5.8.0.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\modules\scala-java8-compat_2.13\0.9.0\scala-java8-compat_2.13-0.9.0.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.13.1\scala-library-2.13.1.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-reflect\2.13.1\scala-reflect-2.13.1.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scalactic\scalactic_2.13\3.0.8\scalactic_2.13-3.0.8.jar;C:\Users\Jbies\OneDrive\Masters\CSE\Q3\Distibuted Sytems\prototype-scala-raft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar" de.maxbundscherer.scala.raft.Main

B-Raft/experiments/run_experiments.py

@@ -0,0 +1,138 @@
+import atexit
+import itertools
+import logging
+import os
+import subprocess
+import time
+
+import numpy as np
+from tqdm import tqdm
+
+with open("run_main_pepijn.txt", "r") as f:
+    run_main_command = f.read()
+
+CURR_POPEN_PROCESS = None
+
+
+def run_experiments(
+        experiment_config,
+        exp_dir=None,
+        max_elections=10,
+        num_replications=10,
+        follow_raft_log=True,
+):
+    global CURR_POPEN_PROCESS
+    if not exp_dir:
+        exp_dir = time.strftime("%Y%m%d-%H%M%S")
+
+    exp_dir = os.path.join(os.path.abspath(os.getcwd()), exp_dir)
+    logging.info(f"Creating experiment directory: {exp_dir}")
+    os.makedirs(exp_dir, exist_ok=True)
+
+    experiments = []
+    for key, values in experiment_config.items():
+        experiments.append([(key, val) for val in values])
+
+    all_exps = sorted(list(itertools.product(*experiments)), key=lambda x: -x[1][1])
+
+    exp_string = "\n".join([str(x) for x in all_exps])
+    logging.info("All experiments: \n" + exp_string)
+    with open(os.path.join(exp_dir, "experiments.txt"), "w+") as f:
+        f.write(exp_string)
+
+    for run in range(num_replications):
+        logging.info(f"Replication run {run + 1}/{num_replications}")
+        for exp_num, value_list in tqdm(enumerate(all_exps)):
+            logging.info(f"Running exp {exp_num}/{len(all_exps)} with values: {value_list}")
+            experiment_vars = {}
+            for (variable, value) in value_list:
+                experiment_vars[variable] = str(value)
+
+            logfile = os.path.join(
+                exp_dir, f"run{run + 1}_" +
+                         "_".join([f"{key}={value}" for key, value in experiment_vars.items()])
+                         + ".log"
+            )
+            # specify logfile
+            logging.info(f"Setting logfile to {logfile}")
+            experiment_vars["LOGFILE"] = logfile
+
+            # run experiment
+            num_elections = 0
+            starttime = time.time()
+            popen = subprocess.Popen(
+                run_main_command,
+                stdout=subprocess.PIPE,
+                env=experiment_vars,
+                bufsize=10,
+                stdin=subprocess.PIPE,
+            )
+
+            CURR_POPEN_PROCESS = popen
+
+            for stdout_line in iter(
+                    popen.stdout.readline,
+                    "",
+            ):
+                stdout_line = str(stdout_line)
+
+                if "[VERIFY APPEND DATA]" in stdout_line:
+                    break
+
+                if "[UNABLE TO VERIFY DUE TO TIMEOUT]" in stdout_line:
+                    break
+
+                if (time.time() - starttime) > 300:
+                    logging.warning(f"Experiment {exp_num} took more than 5 minutes, skipping")
+                    break
+
+            popen.terminate()
+            logging.info(
+                f"Experiment {exp_num} finished in {time.time() - starttime :.2f} seconds"
+            )
+
+
+def cleanup():
+    timeout_sec = 5
+    p = CURR_POPEN_PROCESS
+    if not p:
+        return
+    p_sec = 0
+    for second in range(timeout_sec):
+        if p.poll() == None:
+            time.sleep(1)
+            p_sec += 1
+    if p_sec >= timeout_sec:
+        p.kill()  # supported from python 2.6
+
+    logging.info("Killed all processes")
+
+
+atexit.register(cleanup)
+
+if __name__ == "__main__":
+    # Specify the list of values to use here, see src/main/resources/application.conf for possible variables
+    # Every combination of the values set here will be run 'num_replications' times
+    num_replications = 4
+    experiment_config = {
+        "raftType": ["Raft", "BRaft"],
+        "crashIntervalHeartbeats": [10000],
+        "nodes": np.linspace(start=3, stop=21, num=9, dtype=int),
+    }
+
+    logging.getLogger().setLevel("INFO")
+
+    # output experiment directory, replace with name of experiment(group)
+    experiment_dir = "resources/output/append_data_2"
+
+    # set to true to see Raft logging in console of this process
+    follow_raft_log = False
+
+    # actually run the experiments
+    run_experiments(
+        experiment_config,
+        exp_dir=experiment_dir,
+        max_elections=10,
+        num_replications=num_replications,
+        follow_raft_log=follow_raft_log,
+    )

B-Raft/experiments/run_main_pepijn.txt

@@ -0,0 +1 @@
+"C:\Program Files\Java\jdk-13.0.1\bin\java.exe" -Dfile.encoding=UTF-8 -classpath C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\target\scala-2.13\classes;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scalactic\scalactic_2.13\3.0.8\scalactic_2.13-3.0.8.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-reflect\2.13.1\scala-reflect-2.13.1.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.13.1\scala-library-2.13.1.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\modules\scala-java8-compat_2.13\0.9.0\scala-java8-compat_2.13-0.9.0.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\net\java\dev\jna\jna\5.8.0\jna-5.8.0.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\net\java\dev\jna\jna-platform\5.8.0\jna-platform-5.8.0.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\io\kamon\kamon-core_2.13\2.5.1\kamon-core_2.13-2.5.1.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\io\kamon\kamon-bundle_2.13\2.5.1\kamon-bundle_2.13-2.5.1.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\io\kamon\kamon-apm-reporter_2.13\2.5.1\kamon-apm-reporter_2.13-2.5.1.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\config\1.4.1\config-1.4.1.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\akka\akka-slf4j_2.13\2.6.0\akka-slf4j_2.13-2.6.0.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\akka\akka-actor_2.13\2.6.0\akka-actor_2.13-2.6.0.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\akka\akka-actor-typed_2.13\2.6.0\akka-actor-typed_2.13-2.6.0.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\com\github\oshi\oshi-core\5.7.5\oshi-core-5.7.5.jar;C:\Users\ptema\Documents\uni\y4\IN4391\lab\AkkaRaft\null\Coursier\cache\v1\https\repo1.maven.org\maven2\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar de.maxbundscherer.scala.raft.Main

B-Raft/src/.gitignore

@@ -0,0 +1,9 @@
+../target/
+.idea/
+*.class
+*.log
+.DS_Store
+.java-version
+.idea
+src/null/
+experiments/resources/