Benchmarking Lumol

[Luthaf]

We need two kind of benchmarks:

• regression benchmarks check the speed of a given algorithm;
• comparison benchmark check the speed of a given simulation for comparison with other codes.

Benchmarks live in benches, and can be run using a nighlty compiler with cargo bench.

We currently have one regression benchmark for Ewald summation here; an,d one comparison benchmark against LAMMPS for the md simulation of Helium.

Here are some additional ideas:

Regression benchmarks

• Energy computation for a Lennard-Jones fluid
• Energy computation for a molecular fluid without charges (butane?)
• Energy computation for a charged system
• Energy computation for a molecular charged system

We already have energy computation for a molecular fluid with charges (water)

Simulation benchmarks

It would be nice to have all the combinations of MD/MC -- Lennard-Jones/butane/water -- NVE/NVT/NPT/μVT. Here is a full list:

• MD -- Lennard-Jones -- NVE
• MD -- Lennard-Jones -- NVT
• MD -- Lennard-Jones -- NPT
• MD -- butane -- NVE
• MD -- butane -- NVT
• MD -- butane -- NPT
• MD -- water -- NVE
• MD -- water -- NVT
• MD -- water -- NPT
• MC -- Lennard-Jones -- NVT
• MC -- Lennard-Jones -- NPT
• MC -- Lennard-Jones -- μVT
• MC -- butane -- NVT
• MC -- butane -- NPT
• MC -- butane -- μVT
• MC -- water -- NVT
• MC -- water -- NPT
• MC -- water -- μVT

That is already 18 different simulations, that we should compare against already existing MD and MC codes.

Maybe we can also have tests for bio-molecules, like a small peptide, DNA strand and a bigger protein.

Please comment with more ideas, and open PR to add the benchmarks!

[g-bauer]

For MC, to go to non-rigid molecules or molecules with more than 3 beads, we need a way to sample intramolecular configurations. See #35.

[odarbelaeze]

I noticed a few long running tests this morning on the tests folder of the project, I think that would discourage a nice test-on-save policy. It would be beneficial to reevaluate those and see if smaller sizes or move counts would do for those integration tests, and move the ones that require longer periods of time into the benches folder. (I apologise if I'm a little bit off topic here)

[Luthaf]

I'd prefer to keep these tests as test, but we can discuss it.

I usually only run the unit tests when developing (cd src/core && cargo test), and only run the integration tests before pushing. I also run the integration tests (the ones in the tests folder) in release mode cargo --test release which cuts a lot of run time here.

My setup is a bit clumsy at the moment, but it will become easier when cargo gets a --all flag (there is a PR for that in cargo repo). At this point, running unit tests will be as simple as cargo test --all --lib from the root directory, and running integration tests will still be cargo test --release.

[g-bauer]

You can assign me to the Lennard-Jones cases for NVT and NPT. Before that, I will implement energy cashing for resize_cell_cost. Maybe I manage to add that to the open PR #58 in time.

A practical question: in #58, I added an example case. Somehow, I couldn't manage to make this a #[test]. I compiles, but is not able to find the configuration file. I placed the config in lumol/tests/data/ and the test in lumol/tests. Using cargo test --test my_test --release fails.

I get the same error for existing tests, like mc-helium.rs:

$ cargo test --test mc-helium
# compiling ...
Finished debug [unoptimized + debuginfo] target(s) in 73.96 secs
Running /usr/.../lumol/target/debug/mc_helium-2217974ed8ac85bc

failures:
---- perfect_gaz stdout ----
        thread 'perfect_gaz' panicked at 'called `Result::unwrap()` on an `Err` value: TrajectoryError(Error { kind: NullPtr, message: "Could not open the file data/helium.xyz" })', ../src/libcore/result.rs:799

Any idea what is going wrong or what I am doing wrong here? Do I have to copy the data/ to the target directory?

[Luthaf]

Any idea what is going wrong or what I am doing wrong here?

Lets keep this issue on topic, I'll answer you in #58.

[Luthaf]

We can use bencher to run the benchmarks using a stable rust compiler, instead of a nightly one.

[antoinewdg]

This is a must have if we are to do any kind of optimization on Lumol. I would really like to help on this one, but have no real idea what a reasonable configuration for molecules looks like :( .

[g-bauer]

I'd be happy to help you with that. I can create configuration files for all systems mentioned in the OP. To compare Lumol to other codes, we have to make sure that the same force fields are used.

For Lennard-Jones (Argon), we can already cover all cases but the grand-canonical ensemble. Maybe we should also include an atomic mixture (Na, Cl) as intermediate step towards water?

For water and butane it's more difficult. For SPC/E water we can do NVT and NPT comparisons for MC simulations right now. For butane, we'd need #35 for MC simulations, MD should work for force fields that use non-fixed bond lengths.

I'd say we start with comparison benchmarks of Argon.

[g-bauer]

I can also perform the simulations in Gromacs, Gromos (maybe DL_Poly) and Cassandra to compare performance.

[antoinewdg]

@g-bauer thanks ! I'm OK for starting with argon.
How are we doing this ? You provide me with the input files and I try to wire it all together ?

[g-bauer]

How are we doing this ? You provide me with the input files and I try to wire it all together ?

For argon, we already have inputs in the examples/data folder. I'd use the example configuration argon.xyz which has 500 atoms. We also have an example for MD (argon.rs) and for NPT MC (mc_npt_argon.rs).

I think it is more convenient to just use input files instead of rust bins? We can set up an argon force field file (argon.toml) only specifying the potential and then several simulation inputs, like mc_nvt_argon.toml, md_nvt_argon.toml, ... that all make use of the force field and configuration file.

We have to use the same cutoff radius for all simulations (say rc = 2.7 * sigma = 9.1935 Angstrom). Also, we should use the same frequencies to write data to files.

Hopefully, I'll get #94 finished tonight, so that we can use it for the MC part.

Does that make sense? If anything is unclear, feel free to ping me (here or gitter).