Skip to content

Commit 05804f0

Browse files
d-krupkeclaude
andcommitted
T05: publish Benchmarking & Parameter Tuning slides
Mirror the T05 deck (notes stripped) and link it on the slides index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 60bb522 commit 05804f0

110 files changed

Lines changed: 142940 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: "Benchmarking & Parameter Tuning"
3+
subtitle: "Algorithm Engineering — T05"
4+
author: Dr. Dominik Krupke
5+
format:
6+
revealjs:
7+
slide-number: true
8+
width: 1600
9+
height: 900
10+
chalkboard:
11+
buttons: false
12+
preview-links: auto
13+
css: styles.css
14+
footer: "Algorithm Engineering SS 2026 — T05 Benchmarking & Parameter Tuning"
15+
progress: true
16+
transition: none
17+
background-transition: none
18+
theme: dark
19+
highlight-style: github
20+
code-overflow: wrap
21+
bibliography: references.bib
22+
---
23+
24+
<!-- ============================================================
25+
T05 — Benchmarking & Parameter Tuning
26+
Two halves:
27+
(A) Benchmarking _00 .. _02
28+
(B) Log reading + tuning _03 .. _06
29+
Primary sources (cpsat-primer, ~/Repositories/cpsat-primer):
30+
- chapters/benchmarking.md -> _01, _02
31+
- chapters/understanding_the_log.md -> _03
32+
- chapters/parameters.md -> _03 (subsolver contribution), _04
33+
============================================================ -->
34+
35+
{{< include _00-intro.qmd >}}
36+
{{< include _01-benchmarking-pitfalls.qmd >}}
37+
{{< include _02-benchmarking-visualization.qmd >}}
38+
{{< include _02b-study-design.qmd >}}
39+
{{< include _03-reading-the-log.qmd >}}
40+
{{< include _04-tuning-search.qmd >}}
41+
{{< include _05-optuna-tutorial.qmd >}}
42+
{{< include _06-wrap-up.qmd >}}
43+
44+
## References
45+
46+
::: {#refs}
47+
:::

slides/T05/_00-intro.qmd

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
<!-- Sources for Intro:
2+
- Framing: README.md key message; benchmarking.md intro (§"Benchmarking your Model")
3+
- ASSET TODO: assets/symbol_benchmarking_tuning.png (section-divider background,
4+
dark/atmospheric, match L01/T01 symbol style)
5+
-->
6+
7+
# Benchmarking & Parameter Tuning {background-image="assets/symbol_benchmarking_tuning.png" background-opacity="0.3" background-size="cover" background-color="#2d4059"}
8+
9+
You changed the model and it got faster on your three test instances. Did it actually get better?
10+
11+
## What this tutorial is about
12+
13+
:::: {.columns .viz-row}
14+
::: {.column width="33%"}
15+
![](assets/cactus_plot.png)
16+
:::
17+
::: {.column width="33%"}
18+
::: {.loglet}
19+
```
20+
Starting CP-SAT solver v9.15.6755
21+
Setting number of workers to 24
22+
23+
Initial optimization model 'multiple_knapsack':
24+
#Variables: 560 (#bools: 560)
25+
#kLinearN: 144 (#terms: 1'120)
26+
27+
Starting presolve at 0.00s
28+
[Symmetry] #generators: 7, average support size: 8
29+
presolved: 560 vars, 144 constraints
30+
31+
Starting search at 0.01s with 24 workers
32+
#1 0.01s best:1891 next:[1892,3797] no_lp
33+
#5 0.01s best:1939 next:[1940,2271] core
34+
#10 0.01s best:2226 next:[2227,2271] default_lp
35+
#13 0.02s best:2266 next:[2267,2271] reduced_costs
36+
#Bound 0.05s best:2267 next:[2268,2270] probing_max_lp
37+
#15 0.08s best:2269 next:[2270,2270] default_lp
38+
#16 0.18s best:2270 next:[] graph_var_lns
39+
40+
Solutions (7) Num Rank
41+
'no_lp': 3 [1,7]
42+
'quick_restart': 1 [3,3]
43+
'quick_restart_no_lp': 3 [2,5]
44+
45+
CpSolverResponse summary:
46+
status: OPTIMAL
47+
objective: 2270
48+
best_bound: 2270
49+
gap_integral: 0.44563
50+
```
51+
:::
52+
:::
53+
::: {.column width="33%"}
54+
![](assets/surrogate_acquisition_5.png)
55+
:::
56+
::::
57+
58+
:::: {.columns .col-titles}
59+
::: {.column width="33%"}
60+
### Benchmarking
61+
:::
62+
::: {.column width="33%"}
63+
### Reading the log
64+
:::
65+
::: {.column width="33%"}
66+
### Parameter tuning
67+
:::
68+
::::
69+
70+
:::: {.columns .bullet-row .smaller}
71+
::: {.column width="33%"}
72+
- Is a "win" real, or three lucky instances?
73+
- Which metric and plot fit the goal?
74+
- Timeouts, variance, and the right instance set.
75+
:::
76+
::: {.column width="33%"}
77+
- What did the solver actually do?
78+
- Which strategies moved the bounds?
79+
- Where is the easy tuning win?
80+
:::
81+
::: {.column width="33%"}
82+
- Tuning as a search problem.
83+
- Grid, random, then model-based.
84+
- A hands-on recipe to run it.
85+
:::
86+
::::
87+

0 commit comments

Comments
 (0)