Pin HiGHS_jll below 1.15 and assert solve status in tests#220
Merged
Conversation
This was referenced Jul 16, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #220 +/- ##
==========================================
+ Coverage 92.64% 93.63% +0.99%
==========================================
Files 23 24 +1
Lines 938 1084 +146
==========================================
+ Hits 869 1015 +146
Misses 69 69 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
vtjeng
marked this pull request as draft
July 16, 2026 07:49
A flaky CI failure (#219) reported an objective of 9.32 instead of 6.96 with no way to tell whether the solver claimed optimality at the wrong value or terminated early with a feasible incumbent. Asserting the solve status first makes future occurrences diagnosable.
HiGHS 1.15.0 shipped the first variant of the parallel MIP solver, and 1.15.1 segfaults readily when solving the test-suite MIPs: observed locally at three distinct sites in the new MIP-worker machinery (one with threads=1/parallel=off), and in CI on three platforms within two days of 1.15.1 arriving — the #218 segfaults on ubuntu (2026-07-13) and windows (2026-07-14), and the #219 wrong objective on macOS. It is also nondeterministic run to run on identical models. The wrong objective is attributed to the same instability: the formulation with the loosest valid bounds still solves to the expected optimum, and HiGHS 1.14 shows none of this behavior. Unpin once upstream stabilizes the parallel MIP solver.
vtjeng
force-pushed
the
fix/highs-1.15-instability
branch
from
July 16, 2026 22:09
047f47e to
d4508b1
Compare
vtjeng
marked this pull request as ready for review
July 16, 2026 22:23
vtjeng
added a commit
that referenced
this pull request
Jul 16, 2026
## Why The initial log (#240) carried its numbers in free text, so rows could not be scanned or compared without reading each Notes cell. ## What changed - Verification rows gain **Total (base → cand)** — pooled end-to-end wall clock over the paired 500-sample WK17a LP benchmark — and **Median ratio** (per-sample candidate ÷ baseline). The medians are ratios because the benchmark reports publish per-sample ratio distributions rather than absolute medians. - CI rows gain **Total (before → after)** — runner time summed over the jobs the row changes — and **Long pole** — the longest single job of a full CI run, "—" where a row does not move it. Column semantics are defined in each section intro, along with the caveat that same-day rows overlap in their before/after runs. - The queue-aggregate footer is removed; its numbers survive inside the #237 and #227 rows. #220 (HiGHS_jll pin) is deliberately not added: it plausibly shifts solve time but has no paired measurement; the 2026-07-17 nightly reflects #209 and #220 combined. Formatted with the pinned prettier (3.9.5). Left for review rather than auto-merged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this matters
MIPVerify's repository jobs became unreliable after the Julia workspace began resolving HiGHS.jl 1.24.1 with HiGHS_jll 1.15.1. Issues #218 and #219 record native solver crashes on Ubuntu, Windows, and macOS, plus repeated unexpected objective values for the same macOS test.
A standalone reproducer for that test workload crashed in 3 of 5 process runs with 1.15.1. The observed crashes occurred with
threads = 1andparallel = "off", so those settings did not prevent the reproduced failure. The same script completed 3 of 3 runs with 1.14.0, and every full solve finishedOPTIMALat the expected objective. These small samples support temporarily excluding 1.15.x from this repository; they do not estimate a stable failure rate.The objective-mismatch CI logs did not record termination status. They therefore cannot show whether HiGHS claimed optimality at the unexpected value or stopped early with a feasible incumbent. A separately checked feasible point at 6.959316 proves that the reported 9.323265 value cannot be the optimum, but it does not recover the missing status.
HiGHS 1.15.0 was the first release of its parallel mixed-integer programming (MIP) solver. A relevant upstream fix, ERGO-Code/HiGHS#3141, removes non-thread-safe C API scheduler resets that can cause random Julia-test segfaults. It merged after HiGHS 1.15.1; no released HiGHS version contained the fix as of July 16, 2026. The exact MIPVerify reproducer has not been tested against that fix, so the pin should remain until a released binary includes it and this workload passes local and CI validation.
What changes
HiGHS_jll = "~1.14"to the test project's dependencies and compatibility bounds. Because the repository uses a Julia workspace, this constrains tests, benchmarks, and examples. Current registry metadata resolves HiGHS.jl 1.23.0 with HiGHS_jll 1.14.0. MIPVerify's package dependencies are unchanged; downstream users can resolve other HiGHS versions.SolveStatus == OPTIMALassertion before the existing objective comparison. A future log will show whether a mismatch followed an optimal or non-optimal termination. The existing objective checks continue after the assertion; infeasible cases are unchanged.Mitigates #218 and #219.
Validation
julia --project=. -e 'using Pkg; Pkg.test(; coverage=false)'on a local merge of047f47einto105b11e: 613 passed, 1 expected broken, 614 total. The resolved solver pair was HiGHS.jl 1.23.0 with HiGHS_jll 1.14.0.047f47e, including all six OS/version package-test jobs. That run used base8a455e2; the local merge test above covers the later base.git diff --checkpasses.For the solver-version comparison, three matched full-suite runs used Linux x64, Julia 1.12.6, and the same 521-test selection:
The aggregate wall time was 3.9% lower with the pin. One machine and three pairs are insufficient to characterize general solver performance or the failure rate.