Skip to content

Add EC2 benchmarking #196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add EC2 benchmarking #196

wants to merge 2 commits into from

Conversation

mkannwischer
Copy link
Contributor

This commit adds benchmarking on

  • Graviton 2 (t4g.small)
  • Graviton 3 (c7g.medium)
  • Graviton 4 (c8g.medium)
  • AMD EPYC 4th gen (c7a.medium)
  • Intel Xeon 4th gen (c7i.metal-24xl)
  • AMD EPYC 3rd gen (c6a.large)
  • Intel Xeon 3rd gen (c6i.large)

With that we have all the platforms that are covered in mlkem-native.

This commit adds benchmarking on
 - Graviton 2 (t4g.small)
 - Graviton 3 (c7g.medium)
 - Graviton 4 (c8g.medium)
 - AMD EPYC 4th gen (c7a.medium)
 - Intel Xeon 4th gen (c7i.metal-24xl)
 - AMD EPYC 3rd gen (c6a.large)
 - Intel Xeon 3rd gen (c6i.large)

With that we have all the platforms that are covered in mlkem-native.

Signed-off-by: Matthias J. Kannwischer <[email protected]>
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arm Cortex-A55 (Snapdragon 888) benchmarks (no-opt)

Benchmark suite Current: a649a36 Previous: 7f1d91a Ratio
ML-DSA-44 keypair 490231 cycles 490071 cycles 1.00
ML-DSA-44 sign 1859719 cycles 1858795 cycles 1.00
ML-DSA-44 verify 563291 cycles 563230 cycles 1.00
ML-DSA-65 keypair 826566 cycles 827394 cycles 1.00
ML-DSA-65 sign 2971718 cycles 2979311 cycles 1.00
ML-DSA-65 verify 881461 cycles 881600 cycles 1.00
ML-DSA-87 keypair 1343939 cycles 1341415 cycles 1.00
ML-DSA-87 sign 3778058 cycles 3794969 cycles 1.00
ML-DSA-87 verify 1433105 cycles 1432983 cycles 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arm Cortex-A76 (Raspberry Pi 5) benchmarks (no-opt)

Benchmark suite Current: a649a36 Previous: 7f1d91a Ratio
ML-DSA-44 keypair 224609 cycles 224767 cycles 1.00
ML-DSA-44 sign 654647 cycles 655094 cycles 1.00
ML-DSA-44 verify 240685 cycles 240724 cycles 1.00
ML-DSA-65 keypair 400234 cycles 399832 cycles 1.00
ML-DSA-65 sign 1074668 cycles 1074680 cycles 1.00
ML-DSA-65 verify 394389 cycles 394226 cycles 1.00
ML-DSA-87 keypair 647577 cycles 646746 cycles 1.00
ML-DSA-87 sign 1394697 cycles 1394467 cycles 1.00
ML-DSA-87 verify 661628 cycles 661491 cycles 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mac Mini (M1, 2020) benchmarks (no-opt)

Benchmark suite Current: a649a36 Previous: 7f1d91a Ratio
ML-DSA-44 keypair 132671 cycles 132382 cycles 1.00
ML-DSA-44 sign 411706 cycles 411575 cycles 1.00
ML-DSA-44 verify 142757 cycles 142560 cycles 1.00
ML-DSA-65 keypair 231361 cycles 231298 cycles 1.00
ML-DSA-65 sign 679330 cycles 679464 cycles 1.00
ML-DSA-65 verify 231665 cycles 231666 cycles 1.00
ML-DSA-87 keypair 381583 cycles 381591 cycles 1.00
ML-DSA-87 sign 878056 cycles 878034 cycles 1.00
ML-DSA-87 verify 391404 cycles 391405 cycles 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SpacemiT K1 8 (Banana Pi F3) benchmarks (no-opt)

Benchmark suite Current: a649a36 Previous: 7f1d91a Ratio
ML-DSA-44 keypair 1025766 cycles 1026707 cycles 1.00
ML-DSA-44 sign 3900854 cycles 3899859 cycles 1.00
ML-DSA-44 verify 1165326 cycles 1164505 cycles 1.00
ML-DSA-65 keypair 1730128 cycles 1730224 cycles 1.00
ML-DSA-65 sign 6388432 cycles 6388724 cycles 1.00
ML-DSA-65 verify 1864624 cycles 1864082 cycles 1.00
ML-DSA-87 keypair 2847597 cycles 2848230 cycles 1.00
ML-DSA-87 sign 7948876 cycles 7955789 cycles 1.00
ML-DSA-87 verify 3035034 cycles 3034947 cycles 1.00

This comment was automatically generated by workflow using github-action-benchmark.

@mkannwischer mkannwischer marked this pull request as ready for review May 2, 2025 09:50
@mkannwischer mkannwischer requested a review from a team as a code owner May 2, 2025 09:50
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arm Cortex-A72 (Raspberry Pi 4) benchmarks (no-opt)

Benchmark suite Current: a649a36 Previous: 7f1d91a Ratio
ML-DSA-44 keypair 309554 cycles 310527 cycles 1.00
ML-DSA-44 sign 975037 cycles 1085241 cycles 0.90
ML-DSA-44 verify 333104 cycles 333278 cycles 1.00
ML-DSA-65 keypair 565318 cycles 569530 cycles 0.99
ML-DSA-65 sign 1585703 cycles 1584312 cycles 1.00
ML-DSA-65 verify 543312 cycles 544235 cycles 1.00
ML-DSA-87 keypair 877774 cycles 878033 cycles 1.00
ML-DSA-87 sign 2053507 cycles 2044306 cycles 1.00
ML-DSA-87 verify 901259 cycles 903051 cycles 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant