toqito-bench

Benchmarking suite for the toqito software package.

Benchmark coverage

The following table lists the functions we have benchmarked across toqito, QuTiPy, and Ket.jl:

toqito	QuTiPy	Ket.jl	QETLAB
basis	ket	ket
is_positive_semidefinite
swap
swap_operator	SWAP
to_density_matrix
permute_systems	syspermute	permute_systems
kraus_to_choi	choi_representation	choi
pauli	generate_nQubit_Pauli	pauli
permutation_operator	permute_tensor_factors	permutation_matrix
apply_channel	apply_channel
partial_transpose	partial_transpose
random_povm		random_povm
partial_trace	partial_trace	partial_trace	PartialTrace
random_density_matrix	random_density_matrix
random_unitary	RandomUnitary	random_unitary
random_psd_operator	random_PSD_operator
trace_distance	norm_trace_dist
trace_norm	trace_norm	trace_norm
log_negativity	log_negativity
von_neumann_entropy	entropy	entropy
natural_representation	natural_representation
amplitude_damping	amplitude_damping_channel	channel_amplitude_damping_generalized
bitflip	bit_flip_channel	channel_bit_flip
dephasing	dephasing_channel

Setup Environments General

Python Environment

# Checks if the Python virtual environment exists at env/python-env.
make check-env-python

# Creates a Python 3.10 virtual environment in env/python-env, upgrades pip.
make setup-python

# Ensures the Python environment exists, creating it if missing.
make ensure-python

# Installs/updates pytest-benchmark in the virtual environment and runs setup/test.py.
make test-python

# Removes the Python virtual environment.
make clean-python

# Cleans and reinstalls the Python environment from scratch.
make reinstall-python

Julia Environment

# Performs a fresh Julia installation, removing previous installations and setting up a new environment with BenchmarkTools.
make install-julia-fresh

# Checks for Julia installation; installs or updates as needed. Ensures BenchmarkTools is available in the project environment.
make setup-julia

# Runs Julia tests via setup/test.jl in the configured environment.
make test-julia

# Removes all Julia installations and environments.
make clean-julia

Toqito Environment Setup & Benchmarking

Setup Environment

# Checks if a Poetry environment for toqito exists at env/toqito-env.
make check-env-toqito

# Initializes a poetry environment for Toqito in env/toqito-env.
make setup-toqito

# Ensures the toqito poetry environment exists, creating it if missing.
make ensure-toqito

# Runs a setup test script (setup/test_toqito.py) inside the toqito environment.
make test-toqito-setup 

# Displays installed packages and environment info for toqito.
make toqito-info

#removes the Toqito Poetry environment.
make clean-toqito

# Reinstalls the toqito poetry environment from scratch.
make reinstall-toqito

Benchmark

1. Runs all the tests in scripts/benchmark_toqito.py with --benchmark-warmup=on and --benchmark-verbose. JSON report is stored in results/toqito/full

make benchmark-full-toqito

2. Runs a simple benchmark with optional filtering and saving. Results are saved in results/toqito/<filter>/<function> with only required columns obtained from --benchmark-columns=mean,median,stddev,ops As an example,

# runs benchmarks only for partial_trace function with varying only dim and displays the results but does not save them.
make benchmark-simple-toqito FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim" SAVE=false

3. Generates histogram visualizations from benchmark results or runs make benchmark-simple-toqito if none exist and then generate them. Histogram SVG files saved to results/toqito/<filter>/<function>/images/. As an example,

# If benchmarks for partial_trace function with varying only dim exist in isolation then constructs histogram through it else runs simple benchmarks for toqito with specified arguments and then construct it.
make benchmark-histogram-toqito FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim"

4. Runs benchmarks with memory profiling using cProfile (currently in development)

make test-toqito-benchmark-memory

QuTIpy Environment Setup & Benchmarking

Setup Environment

# Checks if a Poetry environment for qutipy exists at env/qutipy-env.
make check-env-qutipy

# Initializes a poetry environment for qutipy in env/qutipy-env.
make setup-qutipy

# Ensures the qutipy poetry environment exists, creating it if missing.
make ensure-qutipy

# Runs a setup test script (setup/test_qutipy.py) inside the qutipy environment.
make test-qutipy-setup 

#removes the qutipy Poetry environment.
make clean-qutipy

# Reinstalls the qutipy poetry environment from scratch.
make reinstall-qutipy

Benchmark

1. Runs all the tests in scripts/benchmark_qutipy.py with --benchmark-warmup=on and --benchmark-verbose. JSON report is stored in results/qutipy/full

make benchmark-full-qutipy

2. Runs a simple benchmark with optional filtering and saving. Results are saved in results/qutipy/<filter>/<function> with only required columns obtained from --benchmark-columns=mean,median,stddev,ops As an example,

# runs benchmarks only for partial_trace function with varying only dim and displays the results but does not save them.
make benchmark-simple-qutipy FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim" SAVE=false

3. Generates histogram visualizations from benchmark results or runs make benchmark-simple-qutipy if none exist and then generate them. Histogram SVG files saved to results/qutipy/<filter>/<function>/images/. As an example,

# If benchmarks for partial_trace function with varying only dim exist in isolation then constructs histogram through it else runs simple benchmarks for qutipy with specified arguments and then construct it.
make benchmark-histogram-qutipy FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim"

Ketjl Environment Setup & Benchmarking

Setup Environment

# Checks if an environment for ketjl exists at env/ketjl-env.
make check-env-ketjl

# Initializes an environment for ketjl in env/ketjl-env.
make setup-ketjl

# Ensures the ketjl environment exists, creating it if missing.
make ensure-ketjl

# Runs a setup test script (setup/test_ketjl.jl) inside the ketjl environment.
make test-ketjl-setup 

#removes the ketjl environment.
make clean-ketjl

# Reinstalls the ketjl environment from scratch.
make reinstall-ketjl

Benchmark

1. Runs all the tests in scripts/benchmark_ketjl.jl. JSON report is stored in results/ketjl/full.

make benchmark-full-ketjl

2. Runs a simple benchmark with optional filtering and saving. Results are saved in results/ketjl/<filter>/<function> with all the required data. As an example,

# runs benchmarks only for partial_trace function with varying only dim and saves the results.
make benchmark-simple-ketjl FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim" SAVE=true

3. TODO: Generates histogram visualizations from benchmark results or runs make benchmark-simple-ketjl if none exist and then generate them. Histogram SVG files saved to results/ketjl/<filter>/<function>/images/. As an example,

# If benchmarks for partial_trace function with varying only dim exist in isolation then constructs histogram through it else runs simple benchmarks for ketjl with specified arguments and then construct it.
make benchmark-histogram-ketjlFILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim"