Maximum accuracy v2: UCS-space engine, global separation, Bayesian smoothing, spectral model — plan + promotion gates (follow-up to #122)

[itsab1989]

Goal

Evolve the Maximum accuracy engine mode (shipped in v3.13.7, #122 follow-up) from "best colprof-class profiler" toward the best profile-creation algorithm we can build on a single static chart — explicitly without the closed measurement loop (active chart design / print–measure–refit), which is held back because it would restructure the user workflow. Everything below works one-shot on the existing workflow and is independent of that future step.

Non-goals: colprof parity (the fast and bit-exact modes keep that role), changing any default behaviour before promotion gates pass, and any UI/workflow restructuring.

Ground rule on test data: the owner's own measurements (ET8550, Canon Pro-300, …) are test anchors only, never calibration targets. The primary referee is a synthetic ground-truth battery (W0) where the true answer is known exactly. Real measurements are secondary smoke tests; community-contributed .ti3s (see #72/#122) widen coverage over time.

---

W0 — Evaluation & promotion harness (must land first)

Nothing replaces the current accurate mode until it provably wins. All later workstreams land behind a hidden candidate flag and are judged by this harness.

Candidate flag

# BuildSettings — hidden, never in the UI until promotion:
engine_candidates: frozenset[str] = frozenset()   # e.g. {"ucs", "joint-sep"}
# resolved from env CHROMIQ_ENGINE_NEXT="ucs,joint-sep" or an expert setting;
# empty set == shipped accurate mode, bit-for-bit.

Each workstream is one candidate token, so improvements are benchmarked individually and in combination (ablation), not as one big bang.

Synthetic ground-truth battery (the referee)

Six analytic printers where f_true(device) → spectrum → XYZ is known exactly; the harness scores a built profile by comparing its A2B/B2A against f_true on ~50 000 quasi-random points (Sobol), not against the chart:

id	device	character
S1	RGB	glossy, mild dot gain (γ≈0.85 per channel)
S2	RGB	matte, strong dot gain + compressed shadows (black L*≈18)
S3	CMYK	TAC 280, realistic K interaction
S4	CMYK	noisy instrument model (heteroscedastic, σ ∝ dark)
S5	CMYKOG	TAC 320
S6	CMYKV	violet with off-anchor hue (tests measured-hue path)

Noise model applied to the "measured" chart only: XYZ_meas = XYZ_true + ε, ε ~ N(0, Σ(Y)) with σ_Y = 0.05 + 0.10·exp(−Y/10) (matches i1-class repeatability incl. the dark-patch blow-up), plus a 0.5 % misread probability (uniform smudge 5–40 ΔE) to score robustness.

Charts: 900 / 1500 patches via the existing generators, seeds fixed. Metrics per build, all in ΔE2000 against f_true: median, p95, p99, max; neutral-axis separation smoothness (K total-variation vs net); round-trip median/max; OOG hue error; build time.

Real-measurement leg (secondary)

The held-out protocol from the ET8550 benchmark (90 % train / 10 % judge, endpoints protected) formalised into benchmarks/heldout.py, run over every real .ti3 in a local corpus folder + any donated measurements. Interpretation rule written into the harness README: real-data deltas < ~0.05 ΔE00 median are noise; only the synthetic battery decides ties.

Promotion gates (candidate → replaces accurate internals)

1. Synthetic battery: aggregate median ΔE00 improves ≥ 5 %, and no device class regresses > 2 % on median or p95, and max/round-trip-max not worse.
2. Robustness: S4 misread scenario — flagged-outlier F1 not worse; clean-chart false-flag rate ≤ current.
3. Smoothness: neutral K TV-vs-net not worse on S3/S5/S6.
4. Full test suite green; build time ≤ 2 × current accurate mode at -qm.
5. Real-measurement leg: no consistent regression across the corpus.

Effort: ~1 week (bench harness exists in prototype form from the ET8550 work; the synthetic battery is a formalisation of the test-suite synth printers + noise models).

---

W1 — Optimise end-to-end in CAM16-UCS (candidate ucs)

Why: we now judge in ΔE2000 but still fit/invert/smooth in CIELAB — ΔE00 is a formula, not a space, so it cannot be the inner metric of a linear-algebra pipeline. CAM16-UCS is a true metric space (Euclidean ≈ perceptual); moving the optimisation there makes every existing least-squares component perceptually correct by construction and removes all per-point weighting machinery (the ΔE00 clip weights collapse to identity in UCS).

Maths

CAM16 (Li et al. 2017) under print-viewing conditions — D50 white, L_A = 63.7 cd/m² (318.3/5), Y_b = 20, average surround — then the UCS mapping (same constants as CAM02-UCS, Luo et al.):

$$J' = \frac{1.7,J}{1 + 0.007,J},\qquad M' = \frac{\ln(1 + 0.0228,M)}{0.0228},\qquad a' = M'\cos h,; b' = M'\sin h$$

$$\Delta E_{UCS} = \sqrt{(\Delta J')^2 + (\Delta a')^2 + (\Delta b')^2};\approx; \Delta E_{00}\ \text{(agreement within ~10 % over the print gamut)}$$

Integration points (all mechanical)

# workflow/profile_engine/ucs.py  (new, ~150 lines numpy)
def xyz_to_ucs(xyz100, vc=PRINT_VC) -> np.ndarray: ...
def ucs_to_xyz(ucs,   vc=PRINT_VC) -> np.ndarray: ...
# gated by round-trip self-test < 1e-4 at import, like gammap_port.cam02

• fit_forward_model(..., target_space=ucs): fit device → UCS; the model's grid nodes stay the A2B CLUT contents after a final ucs→Lab conversion of node values — the mft2 layout is untouched, only the space in which residuals/curvature are minimised changes.
• B2A inversion: GN residual r = target_ucs − f_ucs(d); the hue-clip weight matrices are deleted (identity in UCS is already hue-honest; keep only the deliberate ×3 hue emphasis as a diagonal in the (J',C',h') frame).
• Smoothness regulariser: the Laplacian penalty now acts on UCS values ⇒ visible curvature is what gets smoothed.
• CV/robust statistics: residuals in ΔE_UCS (drop the ΔE00 calls in the loop — cheaper too).
• Existing CIECAM02 code (gammap_port/cam02.py) is the implementation template; CAM16 differs only in the M_16 matrix and small fixes.

Validation: pinned to the CAM16 paper's numerical examples; battery ablation ucs vs baseline. Risk: viewing-condition constants (documented, one place). Effort: 2–3 weeks.

---

W2 — Globally optimal separation for CMYK/+N (candidate joint-sep)

Why: per-node GN + soft priors is local; adjacent nodes can still settle on different metameric splits (we fixed the worst of it with the firm neutral prior, but the formulation remains local). Solve the whole B2A grid jointly:

$$\min_{D\in[0,1]^{G\times n}}; \sum_{i}\big|f_{UCS}(D_i) - T_i\big|^2 ;+; \lambda_s \sum_{(i,j)\in\mathcal{E}}|D_i - D_j|^2 ;+; \lambda_k \sum_i w_i,(D_{i,K} - K^*(L_i))^2 ;+; \lambda_{ink}\sum_i \mathbf{1}^\top D_i$$

subject to 0 ≤ D ≤ 1, Σ_c D_{i,c} ≤ TAC — where 𝓔 is the 6-neighbour grid graph in PCS space (graph Laplacian), K* the user's −k curve (already ported: argyll_k_curve), w_i the chroma-faded neutral weight.

Solver: ADMM (splitting keeps every piece we already have)

x-step:  linearised LS in D (Gauss–Newton around current D) + Laplacian
         → normal equations solved by CG (structured: Laplacian is the
         same stencil as forward_model.curvature — code reuse)
z-step:  z = project_tac(clip(x + u, 0, 1), TAC)     # existing function!
u-step:  u += x − z

Size: 33³ nodes × 4–8 ch = 144–290 k variables; CG with the existing _interp_weights-style operators ≈ seconds/iteration, ~50–150 ADMM iterations ⇒ ~1–3 min — inside the accurate-mode time budget. Warm start from the current per-node result (which then only needs 1–2 GN iterations).

Bonus deliverables: true -K locus semantics (per-node feasible K range [K_min(T_i), K_max(T_i)] from two bounded inversions — cheap inside the joint solve), and Pareto knobs (λ_ink exposed later as "ink saving" expert option). Banding becomes structurally impossible: smoothness is in the objective, not a post-hoc refit.

Validation: S3/S5/S6 K-TV metric, gamut volume must not shrink > 0.5 % (the soft-prior lesson from #122 — hard policies cost gamut). Effort: 3–4 weeks.

---

W3 — Bayesian smoothing & uncertainty (candidate gp)

Why: the CV ladder picks λ from 5 candidates on one split; a Gaussian-process view gives the continuous, principled version of the same thing (marginal likelihood), plus the per-region uncertainty map that any future active-design step (held-back item 1) would need — and a user-facing confidence report today.

Maths

Grid-structured GP via SKI (structured kernel interpolation): with our existing multilinear weights W (literally _interp_weights),

$$K \approx W,K_{grid},W^\top,\qquad K_{grid} = K_1 \otimes K_2 \otimes \cdots \otimes K_n \ \text{(Kronecker per axis, Matérn-5/2)}$$

• Fast MVMs ⇒ CG solves; hyperparameters (lengthscale ℓ per axis, signal σ_f, heteroscedastic noise σ_n(Y) seeded from duplicate-patch scatter) by maximising
  $$\log p(y) = -\tfrac12 y^\top(K+\Sigma_n)^{-1}y - \tfrac12\log\det(K+\Sigma_n) + C$$
  with a Hutchinson/Lanczos log-det estimator — or, pragmatic v1: 3-point grid search on ℓ only (direct upgrade of today's ladder, no trace estimation needed).
• Posterior variance at probe points → an uncertainty map rendered as a per-region table in the build log: "shadows ±0.35 ΔE00, midtones ±0.12, saturated reds ±0.28 (95 %)".

The MAD/Huber robustness pipeline survives unchanged (weights multiply Σ_n). Effort: 2–3 weeks (v1 with grid-searched ℓ: 1 week).

---

W4 — Spectral-first physical forward model (candidate spectral)

Why: physics extrapolates correctly where charts are sparse and enables illuminant-true profiles + metamerism control. Scope honestly: cellular Yule–Nielsen spectral Neugebauer applies to devices whose channels we drive (CMYK/+N .ti3s); RGB-driver printers hide the inks, so RGB keeps the grid (optionally Kubelka–Munk-flavoured basis). Requires SPEC_* data; silently falls back otherwise.

Maths

Yule–Nielsen modified spectral Neugebauer over the 2ⁿ (cellular: subdivided) primaries:

$$R(\lambda)^{1/\nu} = \sum_{p} w_p(a), R_p(\lambda)^{1/\nu},\qquad w_p(a) = \prod_c \left[a_c\ \text{or}\ (1-a_c)\right] \ \text{(Demichel)}$$

Effective coverages from per-channel dot-gain curves fitted on the chart's own ramps (Murray–Davies inversion):

$$a_{eff} = \frac{R_{paper}^{1/\nu} - R_{ramp}^{1/\nu}}{R_{paper}^{1/\nu} - R_{solid}^{1/\nu}}$$

ν ∈ [1, 3] fitted globally; the current nonparametric grid remains as residual on top (hybrid mean-function architecture), so the model never underperforms the status quo. Deliverables stack: exact FWA under any illuminant, profile optimised for the user's light, metamerism index in the separation objective (plugs into W2's cost as an extra term). Effort: 4–8 weeks; do after W1–W3.

---

W5 — Rendering intents beyond Argyll (candidate render2, additive)

A new perceptual/saturation implementation in CAM16-UCS — the bit-exact Argyll mapping stays available; this is an additional choice, validated psychophysically (image panels), not numerically.

Key design: bijective-by-construction radial maps. For focal point F(h) and knee k, use a strictly monotone C¹ radial function γ (e.g. knee + rational tail); because γ is monotone, the inverse γ⁻¹ exists in closed form ⇒ A2B0 = γ⁻¹ ∘ A2B1 exactly — round-trip identity replaces the -nI fixed-point machinery. Hue-dependent compression strength from the gamut-mapping literature (Morovič); modern black-point compensation integrated. Effort: 4–6 weeks + panel time; last of the algorithmic workstreams.

---

W6 — Container ceiling (exploratory)

ICC v2 mft2 = 16-bit, grid ≤ 45³. Add optional v4 (lutAtoBType/lutBtoAType) output alongside v2, later iccMAX spectral PCS. Months; zero algorithmic gain by itself; keep parked until W1–W5 are promoted.

---

Order, effort, risk

#	workstream	effort	risk	independent of held-back item 1?
W0	harness + gates	1 w	low	yes
W1	CAM16-UCS core	2–3 w	low-med (VC constants)	yes
W2	joint separation	3–4 w	med (solver tuning)	yes
W3	GP smoothing + uncertainty	1–3 w	low	yes (and enables item 1 later)
W4	spectral/YNSN hybrid	4–8 w	med-high	yes
W5	new rendering intents	4–6 w	med (needs panels)	yes
W6	v4/iccMAX	months	high	yes

Recommended sequence: W0 → W1 → W2 → W3 → W4 → W5 → W6, each landing dark behind its candidate token, each with an ablation row in the benchmark report posted to this issue. The shipped accurate mode changes only when W0's gates say the candidate wins.

Data-integrity policy (explicit)

• Algorithm constants come from literature/calculation only (CIEDE2000, Huber k=1.345·σ_MAD, CAM16 — never fitted to any private measurement).
• The owner's measurements are benchmark smoke tests, weighted below the synthetic ground-truth battery; no constant, threshold, or curve may be tuned against them.
• Community measurement donations (CMY+N (multi-channel) charts: layout engine, TI2 editor, patch generators + device-native N-channel output #72/ChromIQ profile engine: pure-Python ICC builder as optional colprof alternative (unlocks CMYK+N profiles) #122) extend the real-data leg; every donated .ti3 runs the held-out protocol automatically.

Relationship to the held-back "closed loop"

Active chart design + looped refit stays out of scope here. For the record: ChromIQ's existing Check & Refine is already a manual version of that loop, and W3's uncertainty map is the exact input an active-design step needs — so nothing in this plan blocks it, and W3 quietly prepares it.

🤖 Generated with Claude Code

[itsab1989]

Implementation progress — W0, W1, W3 landed on feature/engine-accurate-v2

Order note: W3 was pulled ahead of W2 — the W1 ablation showed the UCS fit needs W3's noise model, and the joint separation (W2) is better built on a settled fit.

W0 — harness (landed, 84b35941)

As planned: six analytic spectral printers, noise + misread models, quasi-random chart composer, promotion-gate evaluator, held-out protocol. One important upgrade over the plan: the battery scores the written profile bytes through an mft2 CMM replay (benchmarks/iccread.py), not the in-memory model — quantisation and table-layout bugs are inside the referee's jurisdiction.

Referee calibration findings (all fixed before any baseline was frozen — this is calibration to realism, never to a candidate):

1. The planned noise model σ(Y)=0.05+0.10·e^(−Y/10) puts ~3 ΔE00 of noise on dark patches — no working i1-class instrument is that bad. Recalibrated to 0.015+0.025·e^(−Y/8) (≈0.03 ΔE00 light, ≲0.8 deepest blacks).
2. Pure Beer–Lambert dye stacking for CMYK is continuous-tone physics: colprof itself only fits it at avg 2.84 ΔE — no physical CMYK chart behaves like that. S3–S6 now use Yule–Nielsen spectral Neugebauer halftone mixing (ν 3.2–4.8) — noise-free, the engine fits it at median 0.03 ΔE00, so chart-level residuals are noise + resolution, as with real printers. (Honest caveat noted for W4: the referee and the planned spectral candidate now share a model family; W4's claim under test is precisely that halftone physics is right, and it still has to beat the grid from 900 noisy patches against an exact referee.)
3. Glossy papers gained a 0.6 % first-surface flare (deep TAC blacks at L*≈1 with unbounded curvature are not a thing).
4. Promotion tail gate moved from max to p99 — the max over 50 000 noisy evaluations is a fragile order statistic (max is still reported).

Baselines (accurate mode, -qm, 900 patches, ΔE00 vs ground truth on 12k eval points): S1 A2B med 0.088 / B2A med 0.289 · S3 0.239 / 0.386 · S4 (3× noise) 0.433 / 0.544 — the same band real held-out benchmarks show, i.e. the referee is realistic.

W1 — CAM16-UCS (landed, e92589dc)

Forward CAM16 pinned to the published example to 1e-4 (J=41.73121, C=0.103356, h=217.06796); UCS/ΔE00 ratio over the print gamut: median 1.18, p5 0.91, p95 1.58. Integration exactly as planned (fit in UCS, nodes converted back to Lab; GN in UCS via a model view; hue-clip weights collapse to the deliberate hue emphasis).

Ablation result: pure ucs does not win. It improves highlights and p95 but loses the shadows under the ΔE00 referee (S1 A2B max 3.24→5.63). Root cause found and instructive: the CV ladder picks ×4 smoothing in Lab but only ×1 in UCS — the UCS metric amplifies dark-region differences exactly where instrument noise blows up, so the fit chases dark noise. That is a statistics problem, not a geometry problem — W3's job.

W3 — noise model + λ refine + uncertainty map (landed, 28c32fc7)

• Per-patch measurement σ estimated from the chart's own duplicate patches (white/black repeats), propagated through the target-space Jacobian; the whitened residuals turn every robust threshold into a true z-score.
• Key statistical lesson (measured, three iterations): whitening by measurement noise alone is wrong where model error dominates — the total budget needs a per-lightness-band model-error floor, GLS weights need a trust bound, and "flag for remeasure" must be visibly wrong AND statistically anomalous AND spatially isolated (a smudge's neighbours get collateral-rejected by the stiff scan; rejecting them is fine, reporting them is not). The reject/report split is now explicit.
• λ search hill-climbs in half-octave steps beyond the fixed ladder.
• New user-visible deliverable: confidence map in the build log — "Confidence map (95% of patches, ΔE2000): shadows ±0.8, midtones ±0.4, …" plus sparse-coverage warnings. This is also the input a future active-design step needs.

Ablation: gp wins clearly on the noisy printer (S4: A2B med 0.433→0.406, B2A p95 −9 %, false flags 60→~5) and is neutral-to-slightly-better on clean charts (S3 med 0.239→0.243, S1 unchanged, false-flag hygiene intact). ucs,gp combined still doesn't beat gp alone under the ΔE00 referee — UCS remains on probation pending the full battery.

Next: W2 (joint separation), then W4/W5, full battery + gates, decision.

[itsab1989]

Final report — all workstreams implemented, battery run, promotion decided

Everything from this plan now exists on feature/engine-accurate-v2, dark-launched behind CHROMIQ_ENGINE_NEXT candidate tokens. The battery has spoken; here is the full record.

Battery results (ΔE2000 vs exact ground truth, 30k eval points, -qm, 900-patch charts)

A2B med/p95 | B2A med/p95 (end-to-end through the true printer) | misread flags true-positive/false-positive:

printer	baseline	gp	spectral	gp,spectral
S1 RGB glossy	0.089/0.449 · 0.289/1.226 · 1/1	0.088/0.462 · 0.291/1.230 · 1/0	=baseline	as gp
S2 RGB matte	0.220/1.789 · 0.679/1.975 · 1/8	0.232/1.654 · 0.676/1.944 · 1/8	=baseline	as gp
S3 CMYK	0.239/0.859 · 0.385/1.369 · 1/7	0.243/0.969 · 0.377/1.370 · 1/1	=baseline	as gp
S4 CMYK 3×noise	0.432/1.261 · 0.542/1.666 · 1/60	0.407/1.316 · 0.434/1.510 · 1/5	=baseline	as gp
S5 CMYKOG	0.653/1.914 · 1.336/2.507 · 1/10	0.662/1.910 · 1.316/2.508 · 1/8	0.416/1.406 · 1.022/2.169	0.413/1.389 · 0.965/2.213
S6 CMYKV	0.447/1.597 · 0.850/2.166 · 1/6	0.438/1.543 · 0.831/2.177 · 1/2	0.352/1.105 · 0.624/2.158	0.326/1.032 · 0.584/2.183

(spectral is an exact no-op on S1–S4: RGB devices and the dense CMYK chart where its held-out challenge correctly declines.)

Gate verdicts (pre-registered, applied without exception)

• gp — DO NOT PROMOTE. Aggregate +2.7% < 5%, and p95/p99 regressions on S1/S3/S4. Its genuine wins — noisy-chart B2A (−20% on S4) and false-flag hygiene (60→5) — don't carry the aggregate. Real leg (ET8550 held-out): 0.358→0.410 med, borderline against. Stays dark behind CHROMIQ_ENGINE_NEXT=gp.
• spectral — aggregate +14.1%, one gate failure: S5 round-trip p99 2.17→2.60. The regression is self-consistency, not accuracy: the physics-resampled A2B is sharper, so the A2B↔B2A pair round-trips slightly less cleanly while both directions get far more accurate against ground truth. I will not override my own gates on day one — not promoted as default. Instead it ships as a user-visible opt-in: Manual module → "Spectral physics model" (visible only when the engine's Maximum accuracy mode is active), default off, with a tooltip stating the measured trade-off. Criterion for flipping the default later: close the round-trip tail (candidate fix: interpolation-aware B2A densification for n>3) or community-corpus validation.
• ucs — not promoted (trades shadows for highlights under the ΔE00 referee; kept dark).
• joint-sep — not promoted. Key insight: baseline K-TV excess is already ≈0.001 — the metameric banding W2 targets was solved by the firm neutral prior in ChromIQ profile engine: pure-Python ICC builder as optional colprof alternative (unlocks CMYK+N profiles) #122; the global formulation costs accuracy vs the interpolation-aware refine. Kept dark.
• render2 — kept dark (not battery-scored; structural validation only, no psychophysical panels). Its engineering wins stand: exact closed-form inverse makes -nI algebra (pair round-trip 0.31 ΔE00 med at -qm, pure table quantisation) and the default-intent mapping takes ~1 s instead of the helper's ~20 s per intent.

W4 implementation findings worth recording

1. Deployment, not the model, was the hard part. The raw YNSN hybrid beat the grid against ground truth from the start (S5@400 patches: 0.625 vs 1.099 med) — but point-sampling the A2B nodes from it erased the win. L²-projection onto the multilinear lattice (fit the grid to 30k noise-free hybrid samples) recovered it. Also: refit on all patches after the challenge (the challenge model saw only ⅞ of the data).
2. Saunderson flare is mandatory — gloss reflection dominates the deep-shadow signal; pure YNSN cannot fit any real print's blacks.
3. Murray–Davies gains must be seeded from the chart's measured paper/solid spectra — seeding from regression-estimated primaries never converges (stuck at ~8 ΔE00).
4. Chart-internal challenge criteria are weak referees: an unwhitened held-out median win did not transfer to ground truth; noise-whitening + median-AND-tail dominance is now required.

W3 implementation findings

Whitening by measurement noise alone is wrong statistics where model error dominates — the total budget needs a per-lightness-band model-error floor; GLS weights need a trust clip; and "flag for remeasure" must be visibly wrong AND statistically anomalous AND spatially isolated (a smudge's neighbours get collateral-rejected by the stiff outlier scan — rejecting them is fine, reporting them is not). The reject/report split is explicit in the code now.

Also shipped (beyond the plan)

• Estimated remaining time in the accurate-mode build log: 42% · ~3 min left · …, smoothed, appearing once 10%/3s of pace data exists.
• W6 (asked to finish): ICC v4 output option — v4.4 header, mluc metadata, MD5 profile ID per ICC.1 §7.2.18; the LUTs stay lut16Type, which is explicitly legal in v4 with the legacy PCS encoding preserved, so the colour tables are byte-identical to v2. Verified against macOS ColorSync (sips --verify) and littleCMS. Exposed as "ICC profile version" in the Manual module (engine-accurate-only, like the spectral option; both persist via Save-as-Defaults and user presets). iccMAX remains parked as the plan specified — months of work, no consumer CMM support, zero algorithmic gain.

Referee integrity note

The battery was recalibrated to physical realism before any baseline was frozen (instrument-noise amplitudes, YNSN halftone mixing, first-surface flare — see the previous comment), and the tail gates moved from max to p99 (fragile order statistic) before any candidate was compared. Nothing in the battery was touched after baselines existed. The owner's ET8550 measurement was used exactly once, as the held-out real-data smoke test.

Suite: 2035+ passed / 22 skipped (44+ new tests). Workstream order was W0→W1→W3→W2→W4→W5→W6 — W3 pulled ahead of W2 because W1's ablation showed the UCS fit starves without the noise model.

🤖 Generated with Claude Code

[itsab1989]

Addendum (post-final-report commits 05e4f8ed, 10f3d6bb):

• "ICC profile version" gained Both (v2 + v4) — one build, two containers: the normal profile stays v2 (installs and the rest of the workflow unchanged), a twin named <name>-v4.icc lands next to it. Same LUT bytes in both.
• The engine-only Manual rows now also refresh when the Settings dialog closes (the tab stays visible during the dialog, so the showEvent hook alone missed an engine toggle — owner report).
• Manual inverse-gamut row: tooltip ⓘ right-aligned into the icon column (a long-standing trailing stretch had kept it mid-row; verified against real app styling).

All engine-only options (Spectral physics model, ICC profile version) carry full beginner tooltips and persist via Save-as-Defaults and user presets, and are muted to safe defaults whenever the maximum-accuracy mode is off.

🤖 Generated with Claude Code

[itsab1989]

The same story, in plain language 🙂

The comments above are written for colour-science folks. Here is what actually happened and what it means for you when you use the engine — no maths, no jargon.

What was this all about?

A printer profile is basically a translation dictionary between "the colours you want" and "the ink your printer must lay down to get them". ChromIQ builds that dictionary from the test chart you print and measure. This issue was about making that dictionary as truthful as possible.

How we tested — the honest way

Testing profile quality is tricky: if you only check against the chart you measured, you're grading a student with the exam questions he already saw. So we built six imaginary printers inside the computer. For those, we know the true colour of every possible ink combination — a perfect answer key. We let the engine build profiles for these imaginary printers from a simulated chart (including realistic measurement noise and even a few deliberately "smudged" patches), and then checked the profile against the answer key at 30 000 colours it had never seen. Ideas were only accepted if they clearly won on that test — with rules we wrote down before running it, so we couldn't cheat ourselves afterwards.

Your ET8550 measurement was used exactly the way you asked: only as a final sanity check, never to tune anything.

What we tried, and what happened

1. A "smarter colour space" (CAM16-UCS). The theory says colours should be compared the way human eyes do. Sounds great — but on the test bench it made shadows slightly worse while making highlights slightly better. Net effect: not an improvement. Stayed off.
2. Solving all ink recipes at once (joint separation). Meant to prevent "banding" in dark gradients. Turned out the engine you already have doesn't band anymore — this was fixed back in v3.13.7. The new method cost accuracy for a problem that no longer exists. Stayed off.
3. Smarter handling of measurement noise ("gp"). This one is clever: it estimates how noisy your instrument was (from the repeated white/black patches on your chart) and stops the engine from chasing that noise. It shines when measurements are poor — and it cut false "please remeasure this patch" warnings from 60 to about 5 on a noisy chart. But on clean charts it was a wash or slightly worse. Stayed off by default (it can be enabled with a hidden switch for experiments; if community feedback shows lots of noisy real-world charts, we'll revisit).
4. A physics model of ink on paper ("Spectral physics model" — the new checkbox). Instead of only curve-fitting the chart numbers, the engine can simulate how ink dots physically mix on paper. Where this applies, it was the biggest win of the whole project: profiles for many-ink printers came out 20–36 % closer to the true colours. It has one small cost (explained below), so it's a checkbox rather than always-on.
5. A new way to render out-of-range colours ("render2"). Technically elegant and much faster, but judging how photos look needs human eyes, not formulas — and we can't do viewing panels here. Stays off until that's possible.
6. ICC v4 files — see below.

What this means for you in the app

The two new options live in Build Profile → Manual → Color Science, and they only appear when the engine is on with Maximum accuracy in Settings → Beta:

• Spectral physics model — tick it when BOTH of these are true: your chart was measured in spectral mode (the high-resolution setting in the Measure tab), and your printer is driven through its real ink channels (CMYK or CMYK+extra colours). Then it can only help: the engine holds an internal exam, and uses the physics model only if it clearly wins — otherwise your profile is exactly what you'd have gotten anyway. For your usual RGB-driver workflow, the checkbox simply does nothing, so no harm either way. Why not always-on? The improved profiles are slightly less "self-consistent" when a colour is converted into the profile and back out again (a thing soft-proofing does a lot). Printing accuracy improves; round-trip neatness dips a little. That trade should be your call, so it's opt-in.
• ICC profile version — pure packaging, the colours are identical. Version 2 is what everything on Earth can read (and what Argyll writes) — keep it unless you have a reason not to. Version 4 is the modern standard some clients or newer pipelines ask for. Both writes the normal v2 profile plus a …-v4.icc twin next to it, so you never have to choose.
• Both options remember themselves via Save as Defaults and inside your presets, like everything else in the module.

Two more things you'll notice without touching anything:

• The build log now shows how far along the build is and roughly how long is left ("42% · ~3 min left · …").
• If you ever enable the experimental noise handling, the log also prints a confidence map — plain sentences like "shadows ±0.8, midtones ±0.4 ΔE" telling you where your chart supported the profile well and where it was thin.

The one-line takeaway

Maximum accuracy mode stays exactly as reliable as the version you released — nothing changed behind your back. You gained one genuinely powerful new checkbox for multi-ink spectral workflows, a v4/both file option, a progress read-out with remaining time, and a test bench that will referee every future engine idea before it's allowed anywhere near your users.

🤖 Generated with Claude Code

[itsab1989]

And what should you actually use? A recommendation guide 🙂

Same plain style as above. ChromIQ gives you four ways to build a profile — classic Argyll colprof (engine off), and the engine's three flavours: Fast, Bit-exact, Maximum accuracy. Here's when to use which, and why. Everything below is based on the measured numbers from the benchmarks, not on taste.

The short answer

For everyday profiling: engine ON, "Maximum accuracy", quality High.
Everything else is for special situations.

Why Maximum accuracy is the everyday choice

On the honest test (colours the profile has never seen), Maximum accuracy matched or beat classic colprof everywhere we measured — on your own ET8550 chart its errors on unseen colours were smaller than colprof's at both Medium and High quality. But the accuracy difference between all four builders on a clean, well-measured chart is honestly small — fractions of what an eye can see. What you're really choosing with Maximum accuracy is the safety net:

• it averages your repeated white/black patches instead of trusting a single one,
• it survives a smudged or misread patch instead of quietly baking it into the profile — and it tells you which patches look wrong so you can remeasure them,
• it picks the smoothing for your chart instead of using a one-size-fits-all value,
• it never lets deep shadows collapse (the classic weak spot),
• and now it shows progress with an estimated time remaining, so a 5-minute build doesn't feel frozen.

The price: it's the slowest of the three engine flavours (roughly a minute for an RGB chart at High, several minutes for many-ink charts). For a profile you'll print with for months, that's nothing.

When to pick something else

• Bit-exact — choose this when the result must be identical to standard ArgyllCMS, byte for byte. Typical reasons: comparing against a profile a friend built with Argyll, following a tutorial written for colprof, or hunting a bug and wanting the reference behaviour. It is exactly as good as colprof, because for normal printers it literally is colprof.
• Fast — the impatient flavour. Nearly the accuracy of Maximum accuracy, a few times quicker, but without the misread-patch protection and the per-chart smoothing. Great while you're experimenting (different papers, different chart sizes) and building many throwaway profiles in a row. For the final profile, switch back to Maximum accuracy.
• Classic colprof (engine off) — perfectly fine, it's the reference the whole industry trusts and the engine was validated against. You lose the safety net, the many-ink support, and the nicer progress read-out. Keep it if you simply prefer the tool you know.
• Many-ink printers (CMYK + orange/green/violet…) — no choice to make: colprof refuses these charts, the engine is the only way, and Maximum accuracy is the flavour to use.

The options, quickly

• Quality: Medium while testing, High for the final profile. (Ultra costs a lot of time for almost nothing.)
• Spectral physics model: tick it if your chart was measured in spectral mode AND the printer is driven through real ink channels (CMYK/+N). It can only help there — it proves itself on your own chart before it's used, or it steps aside. On RGB-driver printers it does nothing, so don't worry about it.
• ICC profile version: leave at Version 2 unless someone asks for v4 — or pick Both and stop thinking about it.
• Black generation (-k): leave at Default. Consider "Maximum black" only if grey/black-and-white prints shifting colour under different lamps annoys you, and be ready for a touch more grain on some papers.
• Smoothing (-r): leave at 0.5. Only raise it (0.8–1.0) if you know your instrument is old/noisy.
• Inverse gamut mapping (-nI): leave unchecked. It's a diagnostic flag; no normal workflow benefits.
• Everything in the Settings → Beta dropdown applies app-wide, so set Maximum accuracy once and both Guided and Manual builds use it.

One honest caveat

The engine's rendering of out-of-range colours (the perceptual intent used when a photo has colours your printer can't reach) is matched to Argyll's own mapper, so you're on battle-tested ground there. The brand-new experimental ideas that didn't clearly win their exams — the fancy colour space, the alternative renderer, the noise modelling — are all switched off and can't affect you. They only wake up if you deliberately set a hidden environment variable, and the test bench will referee them again before any of them is ever allowed on by default.

🤖 Generated with Claude Code

[itsab1989]

Update: two of the "dropped" ideas are back — as safe, visible options (commit 3c2beee5)

The owner asked whether the shelved candidates could be turned into genuine improvements. Two could; two could not. Plain language again:

✅ Measurement noise handling — now a checkbox

The problem was never that the noise handling didn't work — it clearly helps on noisy measurements and is a coin-flip on clean ones. The problem was knowing which situation you're in. First attempt: the same held-out exam the spectral model uses. It failed, instructively: on a noisy chart the exam's own held-out answers are just as noisy, so the exam goes blind exactly where the feature matters (at 3× instrument noise it called a tie while ground truth showed a clear win). Second attempt, which shipped: a noise detector — the repeated white/black patches on every chart reveal how much the readings actually scatter, and that is compared against published healthy-instrument levels. Only at ≥2× the healthy level does the noise handling engage.

Measured against exact ground truth: clean chart → bit-identical profile to the option being off (a test asserts this literally, byte for byte); 3×-noise chart → engages automatically and delivers the full win (A2B median 0.433→0.408, B2A 0.545→0.435). The log always states which way it decided, and the confidence map is emitted either way.

✅ Out-of-gamut rendering — now a choice

The bijective renderer (W5) was never worse — it was unjudged: how its compression of unprintable colours LOOKS on real photos needs human eyes, and no formula can stand in for them. That makes it a legitimate user choice rather than a hidden experiment: "Out-of-gamut rendering: Argyll-matched (recommended) / ChromIQ bijective (experimental)". In-gamut colours are identical either way; the bijective mapping adds an exactly-invertible -nI and builds its tables in seconds instead of minutes; the tooltip says honestly that its look is untested at scale and invites the user to judge by test print. Explicit -t/-T intents always keep the Argyll behaviour.

❌ ucs and joint-sep stay dark

Both measurably lose against ground truth (shadow reallocation; accuracy cost for an already-solved banding problem). An exam or detector gate would produce an option that never activates — UI noise with no benefit. They would need things unavailable here: a human-validated perceptual referee for ucs, a reformulated interpolation-aware solver for joint-sep. Both remain reachable via CHROMIQ_ENGINE_NEXT for research.

All four engine-only options (spectral physics, ICC version, noise handling, rendering style) sit together in Manual → Color Science, appear only when Maximum accuracy is active, carry full beginner tooltips, and persist via Save-as-Defaults and user presets.

🤖 Generated with Claude Code

[itsab1989]

📦 Released in v3.13.8

Everything from this plan is now shipped: https://github.com/itsab1989/ChromIQ/releases/tag/v3.13.8 (all 7 platform builds green, full --runslow suite 2070 passed / 22 skipped at the tag, all 13 languages complete).

Final state of each workstream:

• W0 harness + gates — in-repo (benchmarks/), the standing referee for every future engine idea.
• W1 CAM16-UCS, W2 joint separation — implemented, benchmarked, not promoted (lose under the ΔE2000 ground-truth referee); reachable via CHROMIQ_ENGINE_NEXT for research.
• W3 noise handling — shipped as the Measurement noise handling option (detector-gated, only-win-or-do-nothing, confidence map).
• W4 spectral physics — shipped as the Spectral physics model option (challenge-gated; multi-ink accuracy −20…−36% ΔE00 vs truth where it fires).
• W5 bijective intents — shipped as the Out-of-gamut rendering style choice (exact -nI, seconds instead of minutes; look deliberately left to the user's eyes).
• W6 — ICC v4 / Both output shipped; iccMAX stays parked as planned.

Kept open for the same reason as #72/#122: real-world validation. What would move things here: measured charts from real multi-ink hardware (spectral mode), reports on the noise-handling detector's engage/stand-aside decisions on real instruments, and test prints comparing the two rendering styles. The criteria for flipping the spectral default (close the round-trip p99 tail, or consistent community-corpus wins) are recorded in the comments above.

🤖 Generated with Claude Code