Skip to content

Commit bbb80ad

Browse files
ChrisRackauckas-ClaudeChrisRackauckasclaude
authored
Use SciMLTesting v1.1.0 (run_tests harness) (#615)
Replace the hand-written GROUP if-ladder in test/runtests.jl with a single declarative SciMLTesting.run_tests call. Behavior-equivalent: the set of tests run under every GROUP value (bare/All, Core, QA, SymbolicIndexingInterface, Downstream, GPU, NoPre, and each lib/ sublibrary name + _<group>) is identical to the pre-conversion runtests.jl. v1.1.0 specifics used (this corrected pass fixes two v1.0.0-style bugs): - sublib_env = "RECURSIVEARRAYTOOLS_TEST_GROUP". The monorepo root reads GROUP to pick which lib/<Sublibrary> to Pkg.test, but the sublibraries read their own RECURSIVEARRAYTOOLS_TEST_GROUP for the sub-group. The first pass relied on the default sublib_env=GROUP, which left RECURSIVEARRAYTOOLS_TEST_GROUP unset in the sublibrary subprocess (it only coincidentally ran Core via the sublib's own default). Setting sublib_env restores the original withenv handoff exactly. - all = ["Core", "SII_Main"]. "All" runs exactly the root Core body plus the in-main-env SymbolicIndexingInterface API safetestset, matching the original `GROUP == "Core" || "All"` and `GROUP == "SymbolicIndexingInterface" || "All"` branches. It deliberately excludes QA, Downstream, GPU and NoPre, which the original All branch never ran. - umbrellas: GROUP=SymbolicIndexingInterface expands to the SII API safetestset (main env) then the DiffEqArray symbol-indexing test (Downstream env), in that order, preserving the original two-stage env handoff. The Downstream group runs its four functional safetestsets followed by the same symbol_indexing test, as before. - lib_dir = lib routes a bare sublibrary GROUP (or <sublib>_<group>) to Pkg.test(sublib) with the sub-group exported via sublib_env. - per-group env = (Downstream / GPU / NoPre / QA) replaces the activate_*_env helpers; GPU keeps its explicit parent develop of root + the ArrayPartitionAnyAll sublibrary. Add SciMLTesting (1) to [extras] + [compat] + targets.test. Pkg is retained in the test deps because the Core body still Pkg.develops the ShorthandConstructors sublibrary into the main env. qa.jl is unchanged and SciMLTesting is not added to test/QA/Project.toml (qa.jl uses Aqua directly, not run_qa). Verified locally: GROUP=Core ran the full Core suite to green via the new harness (RecursiveArrayTools tests passed), and a routing simulation against the real SciMLTesting source confirmed the recorded action sequence per GROUP matches the pre-conversion runtests.jl, including RECURSIVEARRAYTOOLS_TEST_GROUP handoff. Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2138d08 commit bbb80ad

2 files changed

Lines changed: 91 additions & 160 deletions

File tree

Project.toml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@ RecipesBase = "1.3.4"
6969
RecursiveArrayToolsShorthandConstructors = "1"
7070
ReverseDiff = "1.15"
7171
SafeTestsets = "0.1"
72+
SciMLTesting = "1"
7273
SparseArrays = "1.10"
7374
StaticArrays = "1.6"
7475
StaticArraysCore = "1.4.2"
@@ -94,6 +95,7 @@ Polyester = "f517fe37-dbe3-4b94-8317-1923a5111588"
9495
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
9596
RecursiveArrayToolsShorthandConstructors = "39fb7555-b4ad-4efd-8abe-30331df017d3"
9697
SafeTestsets = "1bc83da4-3b8d-516f-aca4-4fe02f6d838f"
98+
SciMLTesting = "09d9d899-5365-40a9-917a-5f67fddea283"
9799
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
98100
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
99101
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
@@ -104,4 +106,4 @@ Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d"
104106
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
105107

106108
[targets]
107-
test = ["FastBroadcast", "ForwardDiff", "KernelAbstractions", "Measurements", "Mooncake", "Pkg", "Polyester", "Random", "RecursiveArrayToolsShorthandConstructors", "ReverseDiff", "SafeTestsets", "SparseArrays", "StaticArrays", "Statistics", "StructArrays", "Tables", "Test", "Unitful", "Zygote"]
109+
test = ["FastBroadcast", "ForwardDiff", "KernelAbstractions", "Measurements", "Mooncake", "Pkg", "Polyester", "Random", "RecursiveArrayToolsShorthandConstructors", "ReverseDiff", "SafeTestsets", "SciMLTesting", "SparseArrays", "StaticArrays", "Statistics", "StructArrays", "Tables", "Test", "Unitful", "Zygote"]

test/runtests.jl

Lines changed: 88 additions & 159 deletions
Original file line numberDiff line numberDiff line change
@@ -1,163 +1,92 @@
1-
using Pkg
21
using RecursiveArrayTools
3-
using Test
42
using SafeTestsets
3+
using SciMLTesting
4+
using Pkg
55

6-
const GROUP = get(ENV, "GROUP", "All")
7-
8-
function activate_downstream_env()
9-
Pkg.activate(joinpath(@__DIR__, "Downstream"))
10-
Pkg.develop(PackageSpec(path = dirname(@__DIR__)))
11-
return Pkg.instantiate()
12-
end
13-
14-
function activate_gpu_env()
15-
Pkg.activate(joinpath(@__DIR__, "GPU"))
16-
Pkg.develop(PackageSpec(path = dirname(@__DIR__)))
17-
Pkg.develop(
18-
PackageSpec(
19-
path = joinpath(dirname(@__DIR__), "lib", "RecursiveArrayToolsArrayPartitionAnyAll")
20-
)
21-
)
22-
return Pkg.instantiate()
23-
end
24-
25-
function activate_nopre_env()
26-
Pkg.activate(joinpath(@__DIR__, "NoPre"))
27-
Pkg.develop(PackageSpec(path = dirname(@__DIR__)))
28-
return Pkg.instantiate()
29-
end
30-
31-
function activate_qa_env()
32-
Pkg.activate(joinpath(@__DIR__, "QA"))
33-
# On Julia < 1.11, the [sources] section in the QA Project.toml is not
34-
# honored, so Pkg.develop the umbrella root path explicitly.
35-
if VERSION < v"1.11.0-DEV.0"
36-
Pkg.develop(PackageSpec(path = dirname(@__DIR__)))
37-
end
38-
return Pkg.instantiate()
39-
end
40-
41-
@time begin
42-
# Detect sublibrary test groups.
43-
# GROUP can be a bare sublibrary name (Core test group) or
44-
# "{sublibrary}_{TEST_GROUP}" for any custom group (e.g., QA, GPU, etc.).
45-
# Sublibraries declare their groups in test/test_groups.toml.
46-
lib_dir = joinpath(dirname(@__DIR__), "lib")
47-
48-
# Check if GROUP matches a sublibrary, possibly with a _SUFFIX for the test group.
49-
# Scan underscores right-to-left to find the longest matching sublibrary prefix.
50-
function _detect_sublibrary_group(group, lib_dir)
51-
isdir(joinpath(lib_dir, group)) && return (group, "Core")
52-
for i in length(group):-1:1
53-
if group[i] == '_' && isdir(joinpath(lib_dir, group[1:(i - 1)]))
54-
return (group[1:(i - 1)], group[(i + 1):end])
55-
end
56-
end
57-
return (group, "Core")
58-
end
59-
base_group, test_group = _detect_sublibrary_group(GROUP, lib_dir)
60-
61-
if isdir(joinpath(lib_dir, base_group))
62-
Pkg.activate(joinpath(lib_dir, base_group))
63-
# On Julia < 1.11, the [sources] section in Project.toml is not supported.
64-
# Manually Pkg.develop local path dependencies so CI tests the PR branch code.
65-
# The sublibraries depend on the umbrella root via [sources] (../..); develop
66-
# those local paths, skipping the active project itself.
67-
if VERSION < v"1.11.0-DEV.0"
68-
developed = Set{String}()
69-
push!(developed, normpath(joinpath(lib_dir, base_group)))
70-
specs = Pkg.PackageSpec[]
71-
queue = [joinpath(lib_dir, base_group)]
72-
while !isempty(queue)
73-
pkg_dir = popfirst!(queue)
74-
toml_path = joinpath(pkg_dir, "Project.toml")
75-
isfile(toml_path) || continue
76-
toml = Pkg.TOML.parsefile(toml_path)
77-
if haskey(toml, "sources")
78-
for (dep_name, source_spec) in toml["sources"]
79-
if source_spec isa Dict && haskey(source_spec, "path")
80-
dep_path = normpath(joinpath(pkg_dir, source_spec["path"]))
81-
if isdir(dep_path) && !(dep_path in developed)
82-
push!(developed, dep_path)
83-
push!(specs, Pkg.PackageSpec(path = dep_path))
84-
push!(queue, dep_path)
85-
end
86-
end
87-
end
88-
end
89-
end
90-
isempty(specs) || Pkg.develop(specs)
91-
end
92-
withenv("RECURSIVEARRAYTOOLS_TEST_GROUP" => test_group) do
93-
Pkg.test(base_group, julia_args = ["--check-bounds=auto"], force_latest_compatible_version = false, allow_reresolve = true)
94-
end
95-
else
96-
# Root package's own test groups.
97-
# Captured so that, in a local "All" run, the QA group's isolated env does
98-
# not leak into the subsequent functional Core tests run in the main env.
99-
main_test_project = Base.active_project()
100-
101-
if GROUP == "QA"
102-
# QA (Aqua) runs in its own dep-adding environment (test/QA/Project.toml)
103-
# so the QA tooling is not part of the main test dependency set.
104-
activate_qa_env()
105-
@time @safetestset "Quality Assurance" include("QA/qa.jl")
106-
Pkg.activate(main_test_project)
107-
end
108-
109-
if GROUP == "Core" || GROUP == "All"
110-
# The root Core tests use the VA[...]/AP[...] shorthand syntax, which
111-
# lives in the RecursiveArrayToolsShorthandConstructors sublibrary.
112-
Pkg.develop(
113-
PackageSpec(
114-
path = joinpath(dirname(@__DIR__), "lib", "RecursiveArrayToolsShorthandConstructors")
115-
)
6+
run_tests(;
7+
core = function ()
8+
# The root Core tests use the VA[...]/AP[...] shorthand syntax, which lives
9+
# in the RecursiveArrayToolsShorthandConstructors sublibrary; develop it into
10+
# the active main env before running them.
11+
Pkg.develop(
12+
Pkg.PackageSpec(
13+
path = joinpath(dirname(@__DIR__), "lib", "RecursiveArrayToolsShorthandConstructors")
11614
)
117-
@time @safetestset "Utils Tests" include("Core/utils_test.jl")
118-
@time @safetestset "NamedArrayPartition Tests" include("Core/named_array_partition_tests.jl")
119-
@time @safetestset "Partitions Tests" include("Core/partitions_test.jl")
120-
@time @safetestset "Partitions and StaticArrays Tests" include("Core/partitions_and_static_arrays.jl")
121-
@time @safetestset "VecOfArr Indexing Tests" include("Core/basic_indexing.jl")
122-
@time @safetestset "VecOfArr Interface Tests" include("Core/interface_tests.jl")
123-
@time @safetestset "Table traits" include("Core/tabletraits.jl")
124-
@time @safetestset "StaticArrays Tests" include("Core/copy_static_array_test.jl")
125-
@time @safetestset "Linear Algebra Tests" include("Core/linalg.jl")
126-
@time @safetestset "Adjoint Tests" include("Core/adjoints.jl")
127-
@time @safetestset "Mooncake Tests" include("Core/mooncake.jl")
128-
@time @safetestset "Measurement Tests" include("Core/measurements.jl")
129-
end
130-
131-
if GROUP == "SymbolicIndexingInterface" || GROUP == "All"
132-
@time @safetestset "SymbolicIndexingInterface API test" include("SymbolicIndexingInterface/symbolic_indexing_interface_test.jl")
133-
end
134-
135-
if GROUP == "Downstream"
136-
activate_downstream_env()
137-
@time @safetestset "ODE Solve Tests" include("Downstream/odesolve.jl")
138-
@time @safetestset "Event Tests with ArrayPartition" include("Downstream/downstream_events.jl")
139-
@time @safetestset "Measurements and Units" include("Downstream/measurements_and_units.jl")
140-
@time @safetestset "TrackerExt" include("Downstream/TrackerExt.jl")
141-
# TODO: re-enable after SciMLBase compat bump for RAT v4 (SciML/SciMLBase.jl#1297)
142-
# @time @safetestset "Downstream Adjoint Tests" include("Downstream/adjoints.jl")
143-
end
144-
145-
if GROUP == "SymbolicIndexingInterface" || GROUP == "Downstream"
146-
if GROUP == "SymbolicIndexingInterface"
147-
activate_downstream_env()
148-
end
149-
@time @safetestset "DiffEqArray Indexing Tests" include("Downstream/symbol_indexing.jl")
150-
end
151-
152-
if GROUP == "GPU"
153-
activate_gpu_env()
154-
@time @safetestset "VectorOfArray GPU" include("GPU/vectorofarray_gpu.jl")
155-
@time @safetestset "ArrayPartition GPU" include("GPU/arraypartition_gpu.jl")
156-
end
157-
158-
if GROUP == "NoPre"
159-
activate_nopre_env()
160-
@time @safetestset "JET Tests" include("NoPre/jet_tests.jl")
161-
end
162-
end
163-
end
15+
)
16+
@time @safetestset "Utils Tests" include("Core/utils_test.jl")
17+
@time @safetestset "NamedArrayPartition Tests" include("Core/named_array_partition_tests.jl")
18+
@time @safetestset "Partitions Tests" include("Core/partitions_test.jl")
19+
@time @safetestset "Partitions and StaticArrays Tests" include("Core/partitions_and_static_arrays.jl")
20+
@time @safetestset "VecOfArr Indexing Tests" include("Core/basic_indexing.jl")
21+
@time @safetestset "VecOfArr Interface Tests" include("Core/interface_tests.jl")
22+
@time @safetestset "Table traits" include("Core/tabletraits.jl")
23+
@time @safetestset "StaticArrays Tests" include("Core/copy_static_array_test.jl")
24+
@time @safetestset "Linear Algebra Tests" include("Core/linalg.jl")
25+
@time @safetestset "Adjoint Tests" include("Core/adjoints.jl")
26+
@time @safetestset "Mooncake Tests" include("Core/mooncake.jl")
27+
return @time @safetestset "Measurement Tests" include("Core/measurements.jl")
28+
end,
29+
groups = Dict(
30+
# The SII API safetestset runs in the main test env. It is the in-main-env
31+
# half shared between the "All" group (Core + this) and the
32+
# "SymbolicIndexingInterface" umbrella.
33+
"SII_Main" => function ()
34+
return @time @safetestset "SymbolicIndexingInterface API test" include("SymbolicIndexingInterface/symbolic_indexing_interface_test.jl")
35+
end,
36+
# The DiffEqArray symbol-indexing test runs in the Downstream env. It is the
37+
# second half of both the "SymbolicIndexingInterface" and "Downstream" paths.
38+
"SII_Downstream" => (;
39+
env = joinpath(@__DIR__, "Downstream"),
40+
body = function ()
41+
return @time @safetestset "DiffEqArray Indexing Tests" include("Downstream/symbol_indexing.jl")
42+
end,
43+
),
44+
"Downstream" => (;
45+
env = joinpath(@__DIR__, "Downstream"),
46+
body = function ()
47+
@time @safetestset "ODE Solve Tests" include("Downstream/odesolve.jl")
48+
@time @safetestset "Event Tests with ArrayPartition" include("Downstream/downstream_events.jl")
49+
@time @safetestset "Measurements and Units" include("Downstream/measurements_and_units.jl")
50+
@time @safetestset "TrackerExt" include("Downstream/TrackerExt.jl")
51+
# TODO: re-enable after SciMLBase compat bump for RAT v4 (SciML/SciMLBase.jl#1297)
52+
# @time @safetestset "Downstream Adjoint Tests" include("Downstream/adjoints.jl")
53+
return @time @safetestset "DiffEqArray Indexing Tests" include("Downstream/symbol_indexing.jl")
54+
end,
55+
),
56+
"GPU" => (;
57+
env = joinpath(@__DIR__, "GPU"),
58+
parent = [
59+
dirname(@__DIR__),
60+
joinpath(dirname(@__DIR__), "lib", "RecursiveArrayToolsArrayPartitionAnyAll"),
61+
],
62+
body = function ()
63+
@time @safetestset "VectorOfArray GPU" include("GPU/vectorofarray_gpu.jl")
64+
return @time @safetestset "ArrayPartition GPU" include("GPU/arraypartition_gpu.jl")
65+
end,
66+
),
67+
"NoPre" => (;
68+
env = joinpath(@__DIR__, "NoPre"),
69+
body = function ()
70+
return @time @safetestset "JET Tests" include("NoPre/jet_tests.jl")
71+
end,
72+
),
73+
),
74+
qa = (;
75+
env = joinpath(@__DIR__, "QA"),
76+
body = joinpath(@__DIR__, "QA", "qa.jl"),
77+
),
78+
# "All" runs exactly Core plus the in-main-env SII API safetestset (matching the
79+
# original `GROUP == "Core" || "All"` and `GROUP == "SymbolicIndexingInterface"
80+
# || "All"` branches). It deliberately excludes QA, Downstream, GPU and NoPre.
81+
all = ["Core", "SII_Main"],
82+
umbrellas = Dict(
83+
# GROUP=SymbolicIndexingInterface runs the SII API safetestset (main env) and
84+
# then the DiffEqArray symbol-indexing test (Downstream env), in that order.
85+
"SymbolicIndexingInterface" => ["SII_Main", "SII_Downstream"],
86+
),
87+
lib_dir = joinpath(dirname(@__DIR__), "lib"),
88+
# Root reads GROUP to pick the sublibrary, but the sublibraries read
89+
# RECURSIVEARRAYTOOLS_TEST_GROUP for their own sub-group; hand it off via that
90+
# variable (matching the original withenv around Pkg.test(sublib)).
91+
sublib_env = "RECURSIVEARRAYTOOLS_TEST_GROUP",
92+
)

0 commit comments

Comments
 (0)