Track devices via a typed device inventory in the StorageScheduler by Flix6x · Pull Request #2321 · FlexMeasures/flexmeasures

Flix6x · 2026-07-15T07:24:49Z

Description

Device tracking in the StorageScheduler was index-based and fragile: every per-device property lived in a parallel list, each code site re-derived an entry's kind (device / stock-only) from the raw dicts, and a local flex_model variable silently flipped between the filtered and unfiltered entry list inside _prepare(). Several in-flight PRs (#2310, #2295, #2276, #2289) each add more entry kinds and more index-keyed bookkeeping on top of this, and two of them (#2276 and #2319) independently fixed the same misalignment bug.

This PR sets a more robust basis for those works: a typed device inventory (planning/devices.py) that classifies every flex-model entry (and the flex-context's inflexible devices) exactly once, right after flex-config deserialization, and serves as the single source of truth for:

entry roles (DEVICE / STOCK_ONLY / INFLEXIBLE, with GROUP and CONVERTER_PORT as documented extension points),
canonical solver device indices (flexible devices first, then top-level inflexible devices, then per-commodity-context inflexible devices),
stock groups and their SoC parameters (sharing one key space), and
sensor-id and commodity lookups.

The raw deserialized flex-model dicts are kept as-is on each FlexDevice, so downstream code and new flex-model fields need no dataclass changes. _prepare()'s return value and device_scheduler's interface are unchanged (linear_optimization.py is untouched).

Bugs fixed by construction

All of these are locked by new solver-level regression tests (added first as strict xfails, then un-xfailed by the fix commit):

Stock-only entries shifted device properties. Per-device property lists (efficiencies, capacities, output references, and the sensors/assets pair) were partly built from the unfiltered flex-model list, so a stock-only entry listed before a device entry made devices inherit their neighbor's properties (e.g. another device's power capacity).
Devices with only a nested output reference were dropped. An entry referencing its power sensor only via {"consumption": {"sensor": N}} (or "production") was misclassified as stock-only and silently unscheduled — one of the two bugs in Fix device misalignment when flex-model entries are filtered from multi-device schedules #2319, which this PR subsumes.
Flex-context commitments bound the wrong devices. convert_to_commitments enumerated the unfiltered list, assigning commitments to stock-only entries (and to out-of-range device indices).
Devices without a SoC sensor lost their SoC parameters. The synthetic stock keys diverged between _prepare (based on the full list length) and _build_stock_groups (based on the filtered list length) whenever a stock-only entry was present, so soc-at-start/soc-min/… of devices without a state-of-charge sensor were silently dropped.
Two mirrored device enumerations. _prepare() and _reconstruct_commodity_to_devices() each maintained a ~40-line commodity/device-index enumeration that had to stay in sync byte for byte; both now read the inventory.

Relation to in-flight PRs (rebase guide)

Fix device misalignment when flex-model entries are filtered from multi-device schedules #2319: subsumed — both fixes and the regression test are carried here (the test now runs on its own parent site to avoid shared-fixture pollution). Fix device misalignment when flex-model entries are filtered from multi-device schedules #2319 can be closed.
Value the state of charge at the end of the scheduling horizon #2310 (soc-value-at-end): near-zero conflict; its soc_value_at_end list and per-device StockCommitment slot into blocks this PR preserves.
Commitments can be scoped to specific sensors, binding their aggregate flow #2295 (sensor-scoped commitments): replace its enumerate(flex_model) scan with inventory.by_sensor_id(sensor_id).
Support intermediate power constraints on groups of devices (flex-model group field) #2276 (intermediate power constraints): add DeviceRole.GROUP as the first classification step; its device_sensor_id_to_index map is inventory.by_sensor_id, its _group_models/_group_to_devices become inventory fields; its own copy of fix (1) drops out.
Combined Heat and Power (CHP) #2218/Balance internal commodity nodes (heat/steam networks) via first-class balance groups #2289 (CHP / balance groups): commodity is already first-class on FlexDevice; converter ports extend it (coupling, coefficient or a CONVERTER_PORT role), and coupling_groups/balance_groups derive from the inventory.

How to test

pytest flexmeasures/data/models/planning/tests/test_device_inventory.py
pytest flexmeasures/data/models/planning/tests/test_solver.py -k "alignment or stock_only or without_soc_sensor or commitments_target"

The full planning suite (flexmeasures/data/models/planning/tests/) passes; the four new regression tests fail on main.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fkpk8kwbjHreWaCJN4Mncp

Add strict-xfail regression tests for four confirmed device-alignment bugs in multi-device storage scheduling: - A stock-only entry listed before device entries shifts every device's properties (power capacity, efficiencies) by one, because per-device properties are read from the unfiltered flex-model list. - A device without a state-of-charge sensor silently loses its own SoC parameters when a stock-only entry is present, because the synthetic stock keys diverge between _prepare and _build_stock_groups. - Flex-context commitments are assigned device indices by enumerating the unfiltered flex-model list, so stock-only entries receive commitments meant for devices. - A device referencing its power sensor only via a nested consumption/ production output reference is misclassified as stock-only and dropped (carried over from PR #2319, which this branch subsumes). The xfail markers are removed in the commits that fix each bug. Signed-off-by: F.N. Claessen <felix@seita.nl>

Introduce planning/devices.py: every flex-model entry is classified exactly once into a DeviceInventory, which becomes the single source of truth for - entry roles (device / stock-only, extensible to group entries and converter ports), - canonical solver device indices (flexible devices first, then top-level inflexible devices, then per-commodity-context inflexible devices), - stock groups and their SoC parameters (sharing one key space, so parameters can no longer be lost to diverging synthetic keys), and - sensor-id and commodity lookups. Power sensors are resolved from the top-level sensor field or, failing that, from a nested consumption/production output reference, so such devices are no longer misclassified as stock-only entries. The raw deserialized flex-model dicts are kept as-is on each FlexDevice, so downstream code and new flex-model fields need no dataclass changes. Not yet wired into the StorageScheduler; see subsequent commits. Signed-off-by: F.N. Claessen <felix@seita.nl>

Scheduling on the shared module-scoped building fixture is fragile: earlier tests create sensors on it whose legacy attributes (capacity_in_mw, max_soc_in_mwh, ...) get moved into the building's flex-model, which collect_flex_config then injects into the test's flex-model as an extra asset-only device entry, changing the optimization problem. Each alignment test now schedules on its own fresh parent site. Signed-off-by: F.N. Claessen <felix@seita.nl>

Build the DeviceInventory once after flex-config deserialization (with a lazy rebuild for tests that bypass deserialization), and replace _prepare()'s ad-hoc entry classification with it. The local flex_model variable now holds (copies of) the classified device flex-models instead of reverting to the unfiltered flex-model list. This fixes, by construction: - Devices no longer inherit a neighboring entry's properties (power capacity, efficiencies, output sensors) when a stock-only entry precedes them in the flex-model list: all per-device property lists, including the sensors/assets pair, now derive from the same filtered, index-aligned device list. - Devices without a state-of-charge sensor no longer lose their own SoC parameters when a stock-only entry is present (the synthetic stock keys of stock parameters and stock groups now come from one counter). - Flex-context commitments now bind the scheduled devices instead of enumerating the unfiltered flex-model list. - Devices referencing their power sensor only via a nested consumption/ production output reference are scheduled (previously misclassified as stock-only and silently dropped) — subsumes PR #2319. - The dead (and misaligned) stock-groups build in _deserialize_flex_model and the duplicated stock-only detection are gone. The strict-xfail alignment regression tests now pass; their markers are removed. _prepare()'s return value and device_scheduler's interface are unchanged. Signed-off-by: F.N. Claessen <felix@seita.nl>

Both _prepare() and the compute()-side result mapping now take the commodity -> device indices mapping (flexible devices, then top-level inflexible devices, then per-commodity-context inflexible devices) from the device inventory, instead of maintaining two mirrored ~40-line enumerations that had to stay in sync byte for byte. The device constraints for inflexible devices are likewise sized and filled from the inventory's canonical order. _reconstruct_commodity_to_devices keeps a fallback for bare schedulers (e.g. in tests) that classifies the stored device models on the fly. Signed-off-by: F.N. Claessen <felix@seita.nl>

…stock Only a sensor reference can link devices into a shared stock. A state-of-charge field holding timed values (a list of specs) is not a stock identity: classify such devices under their own synthetic stock key, keeping their SoC parameters. On main, this shape crashed _prepare (unhashable stock_models key) if it carried SoC parameters; it only went unnoticed because such flex-models were exercised up to deserialization, never scheduling. Signed-off-by: F.N. Claessen <felix@seita.nl>

Two latent order-dependence bugs, surfaced by running the planning test modules in a new order: - GenericAsset.get_flex_context crashed (TypeError) when an ancestor's flex_context was still None, e.g. a pending asset created without one, before its column default is applied on flush. - The charging station test fixture never flushed, so its assets had no IDs unless an earlier test happened to trigger a flush; scheduler config collection then silently failed to match the flex-model to its asset. The affected test (test_deserialize_storage_soc_at_start_from_ state_of_charge_time_series) failed when run in isolation. Both are now fixed; the test passes in any order. Signed-off-by: F.N. Claessen <felix@seita.nl>

Signed-off-by: F.N. Claessen <felix@seita.nl>

…tory Signed-off-by: F.N. Claessen <felix@seita.nl> # Conflicts: # documentation/changelog.rst

read-the-docs-community · 2026-07-15T07:32:57Z

Documentation build overview

📚 flexmeasures | 🛠️ Build #33596985 | 📁 Comparing d8fce81 against latest (b74cb05)

🔍 Preview build

7 files changed · + 1 added · ± 6 modified

+ Added

_autosummary/flexmeasures.data.models.planning.devices.html

± Modified

Copilot

Pull request overview

This PR introduces a typed DeviceInventory as a single source of truth for classifying flex-model entries (device vs stock-only vs inflexible) and establishing canonical solver device indices, then refactors StorageScheduler to use that inventory to eliminate several index-alignment bugs in multi-device scheduling.

Changes:

Add planning/devices.py implementing DeviceInventory, FlexDevice, and role classification (including nested output-sensor power references).
Refactor StorageScheduler to derive device lists, stock groups/params, and commodity→device mappings from the inventory (removing duplicated enumeration logic).
Add solver/inventory regression tests and update changelog to document the bug fixes and infrastructure change.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
flexmeasures/data/models/planning/tests/test_solver.py	Adds solver-level regression tests covering alignment, stock-only ordering, SoC-param retention, and commitment targeting
flexmeasures/data/models/planning/tests/test_device_inventory.py	Adds unit tests for inventory classification and canonical enumeration
flexmeasures/data/models/planning/storage.py	Uses `DeviceInventory` as the canonical source for device/stock/inflexible enumeration and mappings
flexmeasures/data/models/planning/devices.py	New typed inventory module for flex-config classification and device index mapping
flexmeasures/data/models/planning/init.py	Adds `device_inventory` as derived scheduler state and deprecates `_build_stock_groups` in favor of the inventory
flexmeasures/data/models/generic_assets.py	Makes flex-context inheritance robust to `None` flex_context on ancestors
flexmeasures/conftest.py	Ensures charging-station asset IDs are assigned deterministically via a flush
documentation/changelog.rst	Adds infrastructure/support entry for the inventory and a bugfix entry for the alignment issues

The fixed alignment bugs only affect flex-model shapes introduced during v1 development (entries without a top-level sensor were rejected outright in v0.33.1), so no released behavior regressed: reference the PR from the multiple-feeders feature entry instead of a separate bugfix entry. Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x

Requesting changes myself.

Address review comments on PR #2321: - Raise a ValidationError when more than one flex-model entry defines state-of-charge parameters for the same stock, instead of silently letting the last entry win (stricter than the historical behavior). - Reflow attribute comments to natural clause breaks, clarify that the flex_model None check is a precaution for inflexible devices, and note why inflexible devices always have a power sensor. - Word choice: scheduling features (not code sites) re-derived entry kinds; pass the sensor name as a keyword argument in the test helper. Signed-off-by: F.N. Claessen <felix@seita.nl>

Adapt the sensor-scoped commitment branch to the typed DeviceInventory introduced in PR #2321: resolve the scoped devices' canonical solver indices via inventory.by_sensor_id() instead of re-enumerating the raw flex-model entry list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxM7UZ5wHTz3ftz1Mf9yy Signed-off-by: F.N. Claessen <felix@seita.nl>

Adapt the CHP converter-port concept to the device inventory: FlexDevice entries carry their coupling name and signed coupling coefficient, and DeviceInventory.coupling_groups replaces Scheduler._build_coupling_groups as the single source of truth for the solver's coupling groups, keyed by canonical device indices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxM7UZ5wHTz3ftz1Mf9yy Signed-off-by: F.N. Claessen <felix@seita.nl>

…feat/internal-commodity-balance Balance groups now derive from the inventory's canonical commodity enumeration: the internal-node detection runs in the loop over inventory.commodity_to_devices, so each balance group lists the commodity's flexible and inflexible devices by their canonical solver indices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxM7UZ5wHTz3ftz1Mf9yy Signed-off-by: F.N. Claessen <felix@seita.nl>

Adapt the group feature to the typed DeviceInventory introduced in PR #2321: - Classify group entries in the inventory (new DeviceRole.GROUP), as the first classification step, so a group entry's own power sensor or output sensors cannot make it pass for a device. - Replace the scheduler's ad-hoc _group_models/_group_to_devices derivation with the inventory's group_entries field and group_to_devices resolution (transitive leaf-device resolution with cycle detection). - Drop this branch's own copy of the stock-only misalignment fix (device_sensor_id_to_index and the device_models zip fix), which main now solves by construction via the inventory's canonical device indices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxM7UZ5wHTz3ftz1Mf9yy Signed-off-by: F.N. Claessen <felix@seita.nl>

…oup (#2324) - storage-efficiency may now be defined on the entry holding a shared stock's SoC parameters (or on exactly one member device) and applies to all members; conflicting definitions fail fast, mirroring how #2321 treats the other SoC parameters. - device_scheduler now models one stock variable and one balance recursion per stock group instead of per device, dropping the redundant secondary recursions, and validates that grouped devices share their storage efficiency and initial stock. - Fix a latent key collision in device_to_group: an ungrouped device whose index equalled a state-of-charge sensor id was silently merged into that stock group, feeding its flows into the wrong stock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Flix6x added 9 commits July 14, 2026 20:45

docs: changelog entries and _build_stock_groups deprecation note

1b0ff35

Signed-off-by: F.N. Claessen <felix@seita.nl>

Merge remote-tracking branch 'origin/main' into refactor/device-inven…

c06527b

…tory Signed-off-by: F.N. Claessen <felix@seita.nl> # Conflicts: # documentation/changelog.rst

Flix6x requested a review from Copilot July 15, 2026 09:03

Copilot started reviewing on behalf of Flix6x July 15, 2026 09:03 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Comment thread flexmeasures/data/models/planning/tests/test_device_inventory.py

Comment thread flexmeasures/data/models/planning/devices.py

Flix6x commented Jul 15, 2026

View reviewed changes

Flix6x merged commit 97a5ef1 into main Jul 15, 2026
13 checks passed

Flix6x deleted the refactor/device-inventory branch July 15, 2026 10:17

Flix6x self-assigned this Jul 15, 2026

Flix6x added Scheduling refactoring labels Jul 15, 2026

Flix6x added this to the 1.0.0 milestone Jul 15, 2026

This was referenced Jul 15, 2026

Fix device misalignment when flex-model entries are filtered from multi-device schedules #2319

Closed

Fix: couple stock constraints for multi-device batteries whose SoC comes from a sensor #2322

Merged

Flix6x mentioned this pull request Jul 17, 2026

Share storage-efficiency per stock, then index solver stock variables by stock group #2324

Open

This was referenced Jul 17, 2026

Share storage-efficiency per stock group and index solver stock by group #2325

Open

Feat/offtick soc projection #2194

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Track devices via a typed device inventory in the StorageScheduler#2321

Track devices via a typed device inventory in the StorageScheduler#2321
Flix6x merged 11 commits into
mainfrom
refactor/device-inventory

Flix6x commented Jul 15, 2026

Uh oh!

read-the-docs-community Bot commented Jul 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Flix6x left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Flix6x commented Jul 15, 2026

Description

Bugs fixed by construction

Relation to in-flight PRs (rebase guide)

How to test

Uh oh!

read-the-docs-community Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Flix6x left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

read-the-docs-community Bot commented Jul 15, 2026 •

edited

Loading