Skip to content

Compiler refactor for thumbnail and new introspection system #2856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

adamgerhant
Copy link
Collaborator

This PR is to measure performance with github actions, since I cannot get the benchmarks to work on my computer.

@adamgerhant adamgerhant force-pushed the introspection-compiler-refactor branch from 53125d7 to c6547e1 Compare July 11, 2025 02:54
Copy link

Performance Benchmark Results

compile_demo_art_iai::compile_group::compile_to_proto with_setup_0:load_from_name(isometric-fountain)
Instructions: 25,409,940 (master) -> 16,825,695 (HEAD) : $$\color{lime}-33.78\%$$

Detailed metrics
Baselines:                master|       HEAD
D1mr                     314,372|    164,984         -47.52%
D1mw                      98,901|     61,241         -38.08%
DLmr                      29,415|     19,803         -32.68%
DLmw                      49,442|     25,855         -47.71%
Dr                     6,149,479|  4,570,341         -25.68%
Dw                     4,355,175|  3,297,989         -24.27%
EstimatedCycles       40,156,792| 26,989,185         -32.79%
I1mr                      49,294|      1,455         -97.05%
ILmr                         874|        490         -43.94%
Ir                    25,409,940| 16,825,695         -33.78%
L1hits                35,452,027| 24,466,345         -30.99%
LLhits                   382,836|    181,532         -52.58%
RamHits                   79,731|     46,148         -42.12%
TotalRW               35,914,594| 24,694,025         -31.24%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_1:load_from_name(painted-dreams)
Instructions: 13,139,709 (master) -> 9,005,115 (HEAD) : $$\color{lime}-31.47\%$$

Detailed metrics
Baselines:                master|       HEAD
D1mr                     156,996|     89,727         -42.85%
D1mw                      50,287|     35,935         -28.54%
DLmr                         694|        872         +25.65%
DLmw                      16,435|     13,070         -20.47%
Dr                     3,182,028|  2,545,776         -20.00%
Dw                     2,243,608|  1,865,749         -16.84%
EstimatedCycles       20,029,027| 14,353,496         -28.34%
I1mr                      24,650|      1,002         -95.94%
ILmr                         736|        398         -45.92%
Ir                    13,139,709|  9,005,115         -31.47%
L1hits                18,333,412| 13,289,976         -27.51%
LLhits                   214,068|    112,324         -47.53%
RamHits                   17,865|     14,340         -19.73%
TotalRW               18,565,345| 13,416,640         -27.73%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_2:load_from_name(procedural-string-lights)
Instructions: 2,207,512 (master) -> 1,242,191 (HEAD) : $$\color{lime}-43.73\%$$

Detailed metrics
Baselines:                master|       HEAD
D1mr                      22,567|     10,006         -55.66%
D1mw                      11,025|      6,906         -37.36%
DLmr                          19|        109        +473.68%
DLmw                       1,310|      2,023         +54.43%
Dr                       510,279|    356,544         -30.13%
Dw                       372,836|    281,541         -24.49%
EstimatedCycles        3,307,573|  2,026,092         -38.74%
I1mr                       5,277|        612         -88.40%
ILmr                         720|        392         -45.56%
Ir                     2,207,512|  1,242,191         -43.73%
L1hits                 3,051,758|  1,862,752         -38.96%
LLhits                    36,820|     15,000         -59.26%
RamHits                    2,049|      2,524         +23.18%
TotalRW                3,090,627|  1,880,276         -39.16%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_3:load_from_name(parametric-dunescape)
Instructions: 14,456,952 (master) -> 8,935,805 (HEAD) : $$\color{lime}-38.19\%$$

Detailed metrics
Baselines:                master|       HEAD
D1mr                     186,290|     81,965         -56.00%
D1mw                      65,657|     34,582         -47.33%
DLmr                          44|        212        +381.82%
DLmw                      17,310|     10,896         -37.05%
Dr                     3,464,520|  2,491,458         -28.09%
Dw                     2,482,237|  1,845,547         -25.65%
EstimatedCycles       22,042,691| 14,087,312         -36.09%
I1mr                      21,201|        986         -95.35%
ILmr                         859|        371         -56.81%
Ir                    14,456,952|  8,935,805         -38.19%
L1hits                20,130,561| 13,155,277         -34.65%
LLhits                   254,935|    106,054         -58.40%
RamHits                   18,213|     11,479         -36.97%
TotalRW               20,403,709| 13,272,810         -34.95%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_4:load_from_name(red-dress)
Instructions: 30,719,783 (master) -> 20,381,053 (HEAD) : $$\color{lime}-33.65\%$$

Detailed metrics
Baselines:                master|       HEAD
D1mr                     389,165|    205,976         -47.07%
D1mw                     130,817|     74,736         -42.87%
DLmr                      42,108|     30,206         -28.27%
DLmw                      57,089|     33,997         -40.45%
Dr                     7,377,802|  5,617,013         -23.87%
Dw                     5,235,574|  4,086,243         -21.95%
EstimatedCycles       48,632,001| 33,154,713         -31.83%
I1mr                      54,166|      1,789         -96.70%
ILmr                         878|        477         -45.67%
Ir                    30,719,783| 20,381,053         -33.65%
L1hits                42,759,011| 29,801,808         -30.30%
LLhits                   474,073|    217,821         -54.05%
RamHits                  100,075|     64,680         -35.37%
TotalRW               43,333,159| 30,084,309         -30.57%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_5:load_from_name(valley-of-spires)
Instructions: 19,253,796 (master) -> 12,876,982 (HEAD) : $$\color{lime}-33.12\%$$

Detailed metrics
Baselines:                master|       HEAD
D1mr                     239,318|    130,822         -45.34%
D1mw                      77,428|     50,534         -34.73%
DLmr                      12,731|      7,858         -38.28%
DLmw                      28,711|     21,423         -25.38%
Dr                     4,645,719|  3,730,959         -19.69%
Dw                     3,291,021|  2,765,408         -15.97%
EstimatedCycles       29,873,114| 20,995,625         -29.72%
I1mr                      37,226|      1,403         -96.23%
ILmr                         781|        427         -45.33%
Ir                    19,253,796| 12,876,982         -33.12%
L1hits                26,836,564| 19,190,590         -28.49%
LLhits                   311,749|    153,051         -50.91%
RamHits                   42,223|     29,708         -29.64%
TotalRW               27,190,536| 19,373,349         -28.75%

@adamgerhant adamgerhant force-pushed the introspection-compiler-refactor branch from 89fe867 to 73fcdd4 Compare July 15, 2025 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant