diff --git a/documentation/changelog.rst b/documentation/changelog.rst index 7f020f3647..39ec7663e5 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -19,6 +19,11 @@ New features * Let storage scheduling infer missing ``power-capacity`` from directional device capacities before falling back to site capacity, and default the missing opposite capacity to zero when only a non-zero ``consumption-capacity`` or ``production-capacity`` is configured [see `PR #2222 `_] * Support multiple feeders to a shared storage [see `PR #2001 `_ ] * The flex-context can now define multiple commodities, each specifying their own prices and grid capacities [see `PR #1946 `_, `PR #2172 `_, `PR #2235 `_ and `PR #2271 `_] +* Support commodity-converting devices, such as a CHP, e-boiler or heat pump, by describing each of the device's commodity ports as a flex-model entry sharing one ``coupling`` group with fixed flow ratios (``coupling-coefficient``) [see `PR #2218 `_] +* Commodities without energy prices in the flex-context (e.g. a heat or steam network without a grid connection) are scheduled as internal nodes whose devices must balance each other at every time step [see `PR #2279 `_] +* Commodity contexts that omit grid-connection fields (prices and site capacities) now get smart defaults instead of failing or silently leaving the grid unconstrained — for instance, a bare ``{"commodity": "gas"}`` is treated as having no grid connection; see :ref:`commodity_context_defaults` for the full rules [see `PR #2272 `_] +* In the UI, the flex-context editor supports editing commitments (name, commodity, baseline and deviation prices, each accepting a fixed value or a sensor), also within each commodity context [see `PR #2287 `_] +* Commitments in the flex-context support a ``commodity`` field, defaulting to electricity, to bind a commitment to that commodity's devices [see `PR #2287 `_] * CLI support for adding/editing account attributes [see `PR #2242 `_] * Extended ``GET /api/v3_0/jobs/`` with a ``result`` field containing ``unresolved`` and ``resolved`` soft state-of-charge constraint analysis (``soc-minima``/``soc-maxima`` violations or satisfied constraints, keyed by asset ID) for scheduling jobs; both arrays are empty when no SoC constraints were defined [see `PR #2072 `_] @@ -36,6 +41,7 @@ Infrastructure / Support Bugfixes ----------- +* Namespace user-given commitment names with a ``custom:`` prefix, so they cannot shadow the commitments the scheduler sets up internally (whose costs are reported in the same name-keyed dict of the scheduling results) [see `PR #2285 `_] * Let storage scheduling treat missing constant SoC bounds as unconstrained lower or upper bounds [see `PR #2221 `_] * Allow root assets belonging to different accounts to share the same name, while keeping asset names unique among root assets within the same account and among children of the same parent [see `PR #2226 `_] * Fix queued train-predict forecasting jobs losing their resolved forecast window or failing on detached database objects in workers [see `PR #2035 `_] diff --git a/documentation/features/scheduling.rst b/documentation/features/scheduling.rst index aec2ce3397..18ae3159a4 100644 --- a/documentation/features/scheduling.rst +++ b/documentation/features/scheduling.rst @@ -41,6 +41,10 @@ The ``flex-context`` is independent of the type of flexible device that is optim With the flexibility context, we aim to describe the system in which the flexible assets operate, such as its physical and contractual limitations. For multi-commodity scheduling problems, the flex-context can be defined separately per commodity (e.g. electricity and gas). See :ref:`tut_multi_commodity` for a hands-on example. +A commodity that defines no energy prices in the flex-context (e.g. a heat or steam network without a grid connection) is treated as an internal node: +its devices must balance each other at every time step, so everything produced into the node is consumed from it within the same time step. +Devices that convert between commodities (such as a CHP unit, gas boiler or electric heater) are described in the flex-model, one entry per commodity port, tied together by a ``coupling`` group. + Fields can have fixed values, but some fields can also point to sensors, so they will always represent the dynamics of the asset's environment (as long as that sensor has current data). The full list of flex-context fields follows below. For more details on the possible formats for field values, see :ref:`variable_quantities`. @@ -62,6 +66,9 @@ And if the asset belongs to a larger system (a hierarchy of assets), the schedul * - ``commodity`` - |COMMODITY_FLEX_CONTEXT.example| - .. include:: ../_autodoc/COMMODITY_FLEX_CONTEXT.rst + * - ``commodities`` + - |COMMODITIES.example| + - .. include:: ../_autodoc/COMMODITIES.rst * - ``inflexible-device-sensors`` - |INFLEXIBLE_DEVICE_SENSORS.example| - .. include:: ../_autodoc/INFLEXIBLE_DEVICE_SENSORS.rst @@ -149,6 +156,33 @@ And if the asset belongs to a larger system (a hierarchy of assets), the schedul The flexible device can still have its own power limit defined in its flex-model. +.. _commodity_context_defaults: + +Smart defaults for commodity-context grid connections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For multi-commodity scheduling problems, each entry of the top-level ``commodities`` list is itself a flex-context (a "commodity context") describing the grid connection for that commodity. +A commodity context that leaves out some or all of its grid-connection fields (``consumption-price``, ``production-price``, ``site-consumption-capacity``, ``site-production-capacity`` and ``site-power-capacity``) gets sensible defaults for the missing fields, rather than failing or silently leaving the grid unconstrained. + +As a rule of thumb, a price given for a direction (consumption or production) implies a grid connection in that direction, with an unlimited capacity unless a capacity is also given; a capacity given for a direction (without a price) implies a zero ``consumption-price`` or ``production-price`` (respectively) in that direction; and anything not implied by a given field defaults to "no connection" (a zero capacity, as a soft constraint). +The exception is ``site-power-capacity`` given on its own, which sets a *hard* (symmetric) capacity limit instead. + +This leads to the following defaults, depending on which fields are explicitly given: + +- **Nothing given** (e.g. just ``{"commodity": "gas"}``): both ``site-consumption-capacity`` and ``site-production-capacity`` default to zero, as soft constraints (a breach is possible, but penalized). ``site-power-capacity`` stays unlimited. +- **Only** ``consumption-price``: Then, ``site-power-capacity`` and ``site-consumption-capacity`` stay unlimited; ``site-production-capacity`` defaults to zero (soft). +- **Only** ``production-price``: the mirror image, for production. +- **Only** ``site-consumption-capacity``: Then, ``site-power-capacity`` stays unlimited; ``consumption-price`` defaults to zero; ``site-production-capacity`` (and, transitively, ``production-price``) default to zero. +- **Only** ``site-production-capacity``: the mirror image, for production. +- **Only** ``site-power-capacity``: Then, a *hard* constraint applies at that capacity, with ``site-consumption-capacity`` and ``site-production-capacity`` both set equal to it, and ``consumption-price``/``production-price`` defaulting to zero. + +When several fields are given, each rule only fills in the fields not already determined by a given field, per direction (consumption/production) independently. +Giving all capacity fields is perfectly valid, too: the directional capacities then act as soft constraints, within the hard ``site-power-capacity`` limit. +As a safety net, ``consumption-price`` still defaults to zero if it remains unset after applying the rules above, since the scheduler requires a resolvable consumption price. + +.. note:: Setting ``relax-constraints`` to ``False`` on a commodity context that ends up with a smart-defaulted 0 hard capacity can make the schedule infeasible; FlexMeasures logs a warning in that case. + + .. _flex_models_and_schedulers: The flex-models & corresponding schedulers @@ -196,6 +230,12 @@ For more details on the possible formats for field values, see :ref:`variable_qu * - ``commodity`` - |COMMODITY_FLEX_MODEL.example| - .. include:: ../_autodoc/COMMODITY_FLEX_MODEL.rst + * - ``coupling`` + - |COUPLING.example| + - .. include:: ../_autodoc/COUPLING.rst + * - ``coupling-coefficient`` + - |COUPLING_COEFFICIENT.example| + - .. include:: ../_autodoc/COUPLING_COEFFICIENT.rst * - ``consumption`` - |CONSUMPTION.example| - .. include:: ../_autodoc/CONSUMPTION.rst diff --git a/flexmeasures/data/models/planning/__init__.py b/flexmeasures/data/models/planning/__init__.py index 0f8c606d2f..849f08a07a 100644 --- a/flexmeasures/data/models/planning/__init__.py +++ b/flexmeasures/data/models/planning/__init__.py @@ -100,6 +100,70 @@ def _build_stock_groups(flex_model: list[dict]) -> dict: return dict(groups) + @staticmethod + def _build_coupling_groups( + flex_model: list[dict], + ) -> dict[str, list[tuple[int, float]]]: + """Build coupling groups from the 'coupling' and 'coupling_coefficient' fields + of each device model. + + Devices sharing the same coupling name form a coupling group. + The optimization model introduces a decision variable ``alpha`` per group per time + step, and constrains every device by ``P[d] == coeff_d * alpha``. + + Coupling coefficients in flex-models are user-facing positive magnitudes. + The internal sign is inferred from directional capacities: + + - ``consumption_capacity == 0`` -> output device -> internally negative coefficient + - ``production_capacity == 0`` -> input device -> internally positive coefficient + + If neither direction is explicitly blocked, the coefficient stays positive. + + Example — a CHP with 50% heat efficiency and 30% power efficiency: + + [ + {"coupling": "chp", "coupling_coefficient": 1.0}, # gas input (alpha = P_gas) + {"coupling": "chp", "coupling_coefficient": 0.5}, # heat output (50% of gas) + {"coupling": "chp", "coupling_coefficient": 0.3}, # power output (30% of gas) + ] + + :param flex_model: List of deserialized device flex-model dicts. + :returns: Mapping from coupling-group name to a list of + ``(device_index, internal_signed_coefficient)`` tuples suitable for + passing to ``device_scheduler(coupling_groups=...)``. Returns an empty dict + when no device defines a ``coupling`` field. + """ + + def _is_zero_capacity(value: Any) -> bool: + """Return True if the capacity value is numerically zero.""" + + if value is None: + return False + + # Pint quantities expose ``magnitude``. + magnitude = getattr(value, "magnitude", value) + try: + return bool(np.isclose(float(magnitude), 0.0)) + except (TypeError, ValueError): + return False + + groups: dict[str, list[tuple[int, float]]] = defaultdict(list) + for d, fm in enumerate(flex_model): + coupling_name = fm.get("coupling") + if coupling_name is None: + continue + coefficient = abs(float(fm.get("coupling_coefficient", 1.0))) + + is_output = _is_zero_capacity(fm.get("consumption_capacity")) + is_input = _is_zero_capacity(fm.get("production_capacity")) + + if is_output and not is_input: + coefficient = -coefficient + + groups[coupling_name].append((d, coefficient)) + + return dict(groups) + def __init__( self, sensor: Sensor | None = None, # deprecated diff --git a/flexmeasures/data/models/planning/linear_optimization.py b/flexmeasures/data/models/planning/linear_optimization.py index 12b33fd70c..6e734eda48 100644 --- a/flexmeasures/data/models/planning/linear_optimization.py +++ b/flexmeasures/data/models/planning/linear_optimization.py @@ -42,6 +42,8 @@ def device_scheduler( # noqa C901 commitments: list[pd.DataFrame] | list[Commitment] | None = None, initial_stock: float | list[float] = 0, stock_groups: dict[int, list[int]] | None = None, + coupling_groups: dict[str, list[tuple[int, float]]] | None = None, + balance_groups: dict[str, list[int]] | None = None, ems_constraint_groups: list[list[int]] | None = None, ) -> tuple[list[pd.Series], float, SolverResults, ConcreteModel]: """This generic device scheduler is able to handle an EMS with multiple devices, @@ -80,6 +82,24 @@ def device_scheduler( # noqa C901 device: 0 (corresponds to device d; if not set, commitment is on an EMS level) :param initial_stock: initial stock for each device. Use a list with the same number of devices as device_constraints, or use a single value to set the initial stock to be the same for all devices. + :param coupling_groups: Hard flow-coupling constraints between devices. Each entry maps a group name to a list of + ``(device_index, coefficient)`` tuples. A decision variable ``alpha`` is introduced per group + per time step and every device ``d`` in the group is constrained by ``P[d, j] == coeff_d * alpha[group, j]``. + Sign convention: positive coefficient for input devices (consuming, positive ``ems_power``), + negative coefficient for output devices (producing, negative ``ems_power``). + Example — a CHP with gas input (d=0, coeff 1.0), heat output (d=1, coeff −0.5) and + power output (d=2, coeff −0.3):: + + coupling_groups={"chp": [(0, 1.0), (1, -0.5), (2, -0.3)]} + + :param balance_groups: Flow-balance constraints for internal commodity nodes (e.g. a heat or steam network + without a grid connection). Each entry maps a node name to a list of device indices + whose stock-side flows must balance at every time step: + ``sum_d(P_up[d, j] * eff_up[d, j] + P_down[d, j] / eff_down[d, j] + stock_delta[d, j]) == 0``. + In other words, everything produced into the node is consumed from it within the + same time step; the node itself stores nothing. To add storage to a node, include + a storage device in the group (its flow absorbs the imbalance and its stock is + bounded by its own device constraints). Potentially deprecated arguments: commitment_quantities: amounts of flow specified in commitments (both previously ordered and newly requested) @@ -131,19 +151,43 @@ def device_scheduler( # noqa C901 # map device -> primary stock group (used for per-device stock bounds) # and map stock group -> all member devices (used for stock accumulation). device_to_group = {} + group_to_devices = {} if stock_groups: for g, devices in stock_groups.items(): + group_to_devices[g] = list(devices) for d in devices: - device_to_group[d] = g - # For devices not in any stock group (e.g., inflexible devices), - # map them to themselves so they're treated as individual groups + # Keep first assignment as the primary group. A device can still + # participate in multiple groups via ``group_to_devices``. + if d not in device_to_group: + device_to_group[d] = g + # Devices not in any stock group are treated as single-device groups. for d in range(len(device_constraints)): if d not in device_to_group: - device_to_group[d] = d + g = f"_device_{d}" + device_to_group[d] = g + group_to_devices[g] = [d] else: for d in range(len(device_constraints)): - device_to_group[d] = d + g = f"_device_{d}" + device_to_group[d] = g + group_to_devices[g] = [d] + + # Collect (group_index, device_index, coefficient) triples for coupling constraints. + # Each device in each group will be constrained: P[d, j] == coeff * alpha[group, j] + # where alpha is a free variable representing the common normalised flow. + coupling_device_specs: list[tuple[int, int, float]] = [] + if coupling_groups: + for g_idx, (_group_name, members) in enumerate(coupling_groups.items()): + for d_idx, coeff in members: + coupling_device_specs.append((g_idx, d_idx, coeff)) + + # Collect the device lists of the balance groups (internal commodity nodes). + balance_group_specs: list[list[int]] = [] + if balance_groups: + balance_group_specs = [ + list(devices) for devices in balance_groups.values() if devices + ] # Move commitments from old structure to new if commitments is None: @@ -550,6 +594,35 @@ def grouped_commitment_equalities(m, c, j, g): ) model.commitment_sign = Var(model.c, domain=Binary, initialize=0) + # def _get_stock_change(m, d, j): + # """Determine final stock change of device d until time j. + # + # Apply conversion efficiencies to conversion from flow to stock change and vice versa, + # and apply storage efficiencies to stock levels from one datetime to the next. + # """ + # if isinstance(initial_stock, list): + # # No initial stock defined for inflexible device + # initial_stock_d = initial_stock[d] if d < len(initial_stock) else 0 + # else: + # initial_stock_d = initial_stock + # + # stock_changes = [ + # ( + # m.device_power_down[d, k] / m.device_derivative_down_efficiency[d, k] + # + m.device_power_up[d, k] * m.device_derivative_up_efficiency[d, k] + # + m.stock_delta[d, k] + # ) + # for k in range(0, j + 1) + # ] + # efficiencies = [m.device_efficiency[d, k] for k in range(0, j + 1)] + # final_stock_change = [ + # stock - initial_stock_d + # for stock in apply_stock_changes_and_losses( + # initial_stock_d, stock_changes, efficiencies + # ) + # ][-1] + # return final_stock_change + def _get_stock_change(m, d, j): """Determine final stock change of the stock group of device d until time j. @@ -583,7 +656,7 @@ def _get_stock_change(m, d, j): group = device_to_group[d] # all devices belonging to this stock - devices = [dev for dev, g in device_to_group.items() if g == group] + devices = group_to_devices[group] # initial stock if isinstance(initial_stock, list): @@ -778,6 +851,51 @@ def device_derivative_equalities(m, d, j): model.d, model.j, rule=device_derivative_equalities ) + if coupling_device_specs: + n_coupling_groups = len(coupling_groups) + + # One free variable per group per time step: the common normalised flow. + model.coupling_group_range = RangeSet(0, n_coupling_groups - 1) + model.coupling_alpha = Var(model.coupling_group_range, model.j, domain=Reals) + + model.coupling_device_range = RangeSet(0, len(coupling_device_specs) - 1) + + def flow_coupling_rule(m, c, j): + """Enforce P[d, j] == coeff * alpha[group, j] for each coupled device. + + This pins every device's flow to the same normalised level ``alpha``, + scaled by its coupling coefficient. The coefficient sign indicates direction: + positive for inputs (consuming), negative for outputs (producing). + """ + g, d, coeff = coupling_device_specs[c] + return m.ems_power[d, j] == coeff * m.coupling_alpha[g, j] + + model.flow_coupling_constraints = Constraint( + model.coupling_device_range, model.j, rule=flow_coupling_rule + ) + + if balance_group_specs: + model.balance_group_range = RangeSet(0, len(balance_group_specs) - 1) + + def node_balance_rule(m, b, j): + """Balance the power flows of an internal commodity node at every time step. + + Everything produced into the node must be consumed from it within the same + time step. The balance sums the devices' commodity-side flows (ems_power); + derivative efficiencies and stock deltas describe each device's own + stock-side conversion (e.g. of a shared buffer) and do not enter the + commodity balance. + """ + return ( + 0, + sum(m.ems_power[d, j] for d in balance_group_specs[b]), + 0, + ) + + model.node_balance_constraints = Constraint( + model.balance_group_range, model.j, rule=node_balance_rule + ) + # Add objective def cost_function(m): costs = 0 diff --git a/flexmeasures/data/models/planning/storage.py b/flexmeasures/data/models/planning/storage.py index dc0c02b7f7..fc22a40203 100644 --- a/flexmeasures/data/models/planning/storage.py +++ b/flexmeasures/data/models/planning/storage.py @@ -35,6 +35,7 @@ CommodityFlexContextSchema, FlexContextSchema, MultiSensorFlexModelSchema, + SharedSchema, ) from flexmeasures.data.schemas.sensors import SensorReference, VariableQuantityField from flexmeasures.data.services.scheduling_result import SchedulingJobResult @@ -216,6 +217,15 @@ def _prepare(self, skip_validation: bool = False) -> tuple: # noqa: C901 # This ensures the mapping aligns with the device indices self.stock_groups = self._build_stock_groups(device_models) + # Build coupling_groups from the 'coupling' and 'coupling_coefficient' fields + # of each device model. Devices sharing the same coupling name form a group. + self.coupling_groups = self._build_coupling_groups(device_models) + + # Balance groups for internal commodity nodes (commodities without energy + # prices, i.e. without a grid connection) are derived further below, once + # the devices of each commodity are enumerated. + self.balance_groups: dict[str, list[int]] = {} + # List the asset(s) and sensor(s) being scheduled if self.asset is not None: if not isinstance(self.flex_model, list): @@ -238,12 +248,6 @@ def _prepare(self, skip_validation: bool = False) -> tuple: # noqa: C901 asset = self.sensor.generic_asset assets = [asset] # noqa: F841 - # For backwards compatibility with the single asset scheduler - flex_model = self.flex_model.copy() - if not isinstance(flex_model, list): - flex_model = [flex_model] - else: - flex_model = [flex_model_d.copy() for flex_model_d in flex_model] for flex_model_d in flex_model: self._default_missing_directional_capacity_to_zero(flex_model_d) num_flexible_devices = len(device_models) @@ -410,10 +414,33 @@ def device_list_series( if production_price is None: production_price = consumption_price - if consumption_price is None: - raise ValueError( - f"Missing consumption price for commodity '{commodity}'." - ) + # A context without user-given price fields may still carry smart-defaulted + # zero prices (see CommodityFlexContextSchema.fill_grid_connection_defaults), + # in which case it is flagged with prices_are_defaulted. + has_no_user_given_prices = consumption_price is None or ( + commodity_context.get("prices_are_defaulted", False) + and consumption_price_sensor is None + and production_price_sensor is None + ) + + if has_no_user_given_prices: + if commodity == "electricity": + # Electricity is assumed to be grid-connected, so a missing + # price is treated as a configuration error rather than as + # an internal node. + raise ValueError( + f"Missing consumption price for commodity '{commodity}'." + ) + # A non-electricity commodity without energy prices is treated as an + # internal node (e.g. a heat or steam network without a grid + # connection): its devices must balance each other at every time + # step, and it needs no commitments or EMS-level capacity constraints. + current_app.logger.info( + f"Commodity '{commodity}' has no energy prices; treating it as an " + f"internal node whose devices (indices {devices}) balance each other." + ) + self.balance_groups[commodity] = list(devices) + continue # Energy prices for this commodity. up_deviation_prices = get_continuous_series_sensor_or_quantity( @@ -1315,7 +1342,13 @@ def convert_to_commitments( flex_model, **timing_kwargs, ) -> list[FlowCommitment | StockCommitment]: - """Convert list of commitment specifications (dicts) to a list of FlowCommitments.""" + """Convert list of commitment specifications (dicts) to a list of FlowCommitments. + + User-given commitment names are namespaced with a "custom:" prefix, so users can + pick any name without colliding with the commitments the scheduler sets up + internally (e.g. "electricity net energy" or "any soc minima"). The prefixed + name is what shows up in the commitment costs of the scheduling results. + """ commitment_specs = self.flex_context.get("commitments", []) if len(commitment_specs) == 0: return [] @@ -1324,6 +1357,11 @@ def convert_to_commitments( price_unit = self.flex_context["shared_currency_unit"] + "/MW" commitments = [] for commitment_spec in commitment_specs: + # Namespace the user-given name (guarding against double prefixing, + # in case the specs are converted more than once). + if not commitment_spec["name"].startswith("custom:"): + commitment_spec["name"] = f"custom:{commitment_spec['name']}" + # Convert baseline, up_price and down_price to pd.Series, then create FlowCommitment if "up_price" in commitment_spec: commitment_spec["upwards_deviation_price"] = ( @@ -1351,6 +1389,43 @@ def convert_to_commitments( start, end, timing_kwargs["resolution"] ) commitment_commodity = commitment_spec.get("commodity", "electricity") + + # A commitment scoped to specific sensors binds the *aggregate* flow + # of those devices as one commitment, rather than each device separately. + scoped_sensors = commitment_spec.pop("sensors", None) + if scoped_sensors is not None: + scoped_sensor_ids = { + sensor.id if hasattr(sensor, "id") else sensor + for sensor in scoped_sensors + } + scoped_devices = [ + d + for d, flex_model_d in enumerate(flex_model) + if getattr(flex_model_d.get("sensor"), "id", None) + in scoped_sensor_ids + ] + if not scoped_devices: + current_app.logger.warning( + f"Commitment '{commitment_spec.get('name')}' is scoped to" + f" sensors {sorted(scoped_sensor_ids)}, none of which appear" + " in the flex-model. This commitment will not bind any device." + ) + continue + index = commitment_spec["index"] + group_label = commitment_spec.get("name", "scoped commitment") + commitment = FlowCommitment( + device=pd.Series([scoped_devices] * len(index), index=index), + # device_group maps device index -> group label; one shared + # label makes the engine bind the aggregate flow. + device_group=pd.Series( + {d: group_label for d in scoped_devices} + ), + **commitment_spec, + ) + commitments.append(commitment) + continue + + bound_device_count = 0 for d, flex_model_d in enumerate(flex_model): device_commodity = flex_model_d.get("commodity", "electricity") if device_commodity != commitment_commodity: @@ -1361,6 +1436,15 @@ def convert_to_commitments( **commitment_spec, ) commitments.append(commitment) + bound_device_count += 1 + if bound_device_count == 0: + current_app.logger.warning( + f"Commitment '{commitment_spec.get('name')}' has commodity" + f" '{commitment_commodity}', which matches none of the devices" + " in the flex-model. This commitment will not bind any device" + " (check for a typo in the commitment's `commodity` field, or in" + " a device's `commodity` field in the flex-model)." + ) return commitments @@ -1408,9 +1492,17 @@ def _deserialize_flex_context(self): commodity_flex_context ) - # Ensure all flex-contexts share the same currency unit + # Ensure all flex-contexts share the same currency unit. Contexts with + # no user-given price fields at all (shared_currency_unit_is_default) + # only carry a fallback "EUR" currency, which isn't a real constraint, + # so they're skipped here and instead backfilled below, once a real + # portfolio currency is known. shared_currency_unit = None + default_currency_contexts = [] for commodity_flex_context in self.flex_context: + if commodity_flex_context.get("shared_currency_unit_is_default"): + default_currency_contexts.append(commodity_flex_context) + continue context_currency_unit = commodity_flex_context["shared_currency_unit"] if shared_currency_unit is None: shared_currency_unit = context_currency_unit @@ -1421,6 +1513,20 @@ def _deserialize_flex_context(self): f"All prices in the flex-context must share the same currency unit (in this case: '{shared_currency_unit}')." ) + # Let price-free contexts inherit the portfolio's actual currency, + # where determinable (i.e. when at least one other context set one). + if shared_currency_unit is not None: + for commodity_flex_context in default_currency_contexts: + SharedSchema._rebase_default_context_currency( + commodity_flex_context, shared_currency_unit + ) + elif default_currency_contexts: + # No context anywhere gave an explicit price: fall back to the + # (shared) default currency already stamped on each of them. + shared_currency_unit = default_currency_contexts[0][ + "shared_currency_unit" + ] + # Nest the flex-contexts per commodity under the commodity_contexts field self.flex_context = dict( commodity_contexts=self.flex_context, @@ -2651,6 +2757,8 @@ def compute(self, skip_validation: bool = False) -> SchedulerOutputType: commitments=commitments, initial_stock=initial_stock, stock_groups=self.stock_groups, + coupling_groups=self.coupling_groups if self.coupling_groups else None, + balance_groups=getattr(self, "balance_groups", None) or None, ) if "infeasible" in (tc := scheduler_results.solver.termination_condition): raise InfeasibleProblemException(tc) diff --git a/flexmeasures/data/models/planning/tests/test_commitments.py b/flexmeasures/data/models/planning/tests/test_commitments.py index 4275892c53..a03b402c01 100644 --- a/flexmeasures/data/models/planning/tests/test_commitments.py +++ b/flexmeasures/data/models/planning/tests/test_commitments.py @@ -3,6 +3,7 @@ import numpy as np from flexmeasures.data.services.utils import get_or_create_model +from flexmeasures.utils.unit_utils import ur from flexmeasures.data.models.planning import ( Commitment, StockCommitment, @@ -17,6 +18,7 @@ from flexmeasures.data.models.planning.linear_optimization import device_scheduler from flexmeasures.data.models.generic_assets import GenericAsset, GenericAssetType from flexmeasures.data.utils import save_to_db +from flexmeasures.utils.unit_utils import ur def test_multi_feed_device_scheduler_shared_buffer(): @@ -1556,6 +1558,693 @@ def test_simulation_with_dynamic_consumption_capacity(app, db): ), "Electric heater should have one expected partial 80 kW dispatch step before the first cheap-electricity window." +def test_chp_coupling(): + """Test that coupling_groups enforces fixed flow ratios between CHP devices. + + Models a Combined Heat and Power unit with three pure flow devices: + + - d=0 gas input: can only consume gas (derivative_min=0) + - d=1 heat output: can only produce heat (derivative_max=0) + - d=2 power output: can only produce electricity (derivative_max=0) + + The coupling group ``"chp"`` is specified with coefficients + ``[(0, 1.0), (1, -0.5), (2, -0.3)]``, introducing a decision variable ``alpha`` + and enforcing ``P[d] == coeff * alpha`` for each device: + + P_gas = 1.0 * alpha (input, coeff = 1.0) + P_heat = -0.5 * alpha (output, coeff = -0.5, heat efficiency 50%) + P_power = -0.3 * alpha (output, coeff = -0.3, power efficiency 30%) + + Heat production is forced to exactly 10 kW via ``derivative equals = -10`` + on device 1. Substituting ``P_heat = -10`` gives ``alpha = 20``, so: + + P_gas = 20 kW (gas consumed) + P_heat = -10 kW (heat produced, forced) + P_power = 20 kW * -0.3 + ≈ -6 kW (electricity produced) + + """ + start = pd.Timestamp("2026-01-01T00:00+01:00") + end = pd.Timestamp("2026-01-01T04:00+01:00") + resolution = pd.Timedelta("1h") + index = initialize_index(start=start, end=end, resolution=resolution) + + # d=0: gas input — can only consume (derivative_min=0), capacity 100 kW. + # NaN stock bounds mean no cumulative-stock constraint (pure flow device). + gas_constraints = pd.DataFrame( + { + "min": np.nan, + "max": np.nan, + "equals": np.nan, + "derivative min": 0.0, + "derivative max": 100.0, + "derivative equals": np.nan, + "derivative down efficiency": 1.0, + "derivative up efficiency": 1.0, + }, + index=index, + ) + + # d=1: heat output — can only produce (derivative_max=0). + # Forced to exactly -10 kW via derivative equals. + heat_constraints = pd.DataFrame( + { + "min": np.nan, + "max": np.nan, + "equals": np.nan, + "derivative min": -100.0, + "derivative max": 0.0, + "derivative equals": -10.0, + "derivative down efficiency": 1.0, + "derivative up efficiency": 1.0, + }, + index=index, + ) + + # d=2: power output — can only produce (derivative_max=0), capacity 100 kW. + # Flow is free; the coupling constraint will determine its value. + power_constraints = pd.DataFrame( + { + "min": np.nan, + "max": np.nan, + "equals": np.nan, + "derivative min": -100.0, + "derivative max": 0.0, + "derivative equals": np.nan, + "derivative down efficiency": 1.0, + "derivative up efficiency": 1.0, + }, + index=index, + ) + + ems_constraints = pd.DataFrame( + {"derivative min": -200.0, "derivative max": 200.0}, + index=index, + ) + + # Coupling group: one reference device (gas, coeff 1.0) and two coupled + # devices (heat with coeff -0.5, power with coeff -0.3). + coupling_groups = {"chp": [(0, 1.0), (1, -0.5), (2, -0.3)]} + + # Gas-price commitment gives the objective a finite value and models the + # cost of consuming gas. With quantity=0 and both prices set the + # commitment acts as a two-sided soft equality: any upward deviation + # (gas consumption) incurs a cost of 1 EUR/kW. + gas_price_commitment = FlowCommitment( + name="gas cost", + index=index, + quantity=pd.Series(0.0, index=index), + upwards_deviation_price=pd.Series(1.0, index=index), + downwards_deviation_price=pd.Series(0.0, index=index), + device=pd.Series(0, index=index), + ) + + schedules, planned_costs, results, model = device_scheduler( + device_constraints=[gas_constraints, heat_constraints, power_constraints], + ems_constraints=ems_constraints, + commitments=[gas_price_commitment], + coupling_groups=coupling_groups, + ) + + assert ( + results.solver.termination_condition == "optimal" + ), "Solver did not find an optimal solution." + + # Heat is fixed to -10 kW by derivative_equals. + pd.testing.assert_series_equal( + schedules[1], + pd.Series(-10.0, index=index), + check_names=False, + rtol=1e-4, + obj="heat output forced to -10 kW by derivative_equals", + ) + + # Coupling: P_gas / 1.0 == P_heat / -0.5 → P_gas = -10 / -0.5 = 20 kW + pd.testing.assert_series_equal( + schedules[0], + pd.Series(20.0, index=index), + check_names=False, + rtol=1e-4, + obj="gas consumption determined by coupling (20 kW from 10 kW heat at coeff -0.5)", + ) + + # Coupling: P_gas / 1.0 == P_power / -0.3 → P_power = 20 / -0.3 = -6 kW + pd.testing.assert_series_equal( + schedules[2], + pd.Series(-6.0, index=index), + check_names=False, + rtol=1e-4, + obj="power output determined by coupling (-0.3 * alpha = -0.3 * 20 = -6 kW)", + ) + + +def test_dual_fuel_chp_coupling(): + """Test coupling_groups with two input devices (dual-fuel CHP). + + Models a CHP unit that consumes equal parts natural gas and hydrogen, + producing heat and electricity: + + - d=0 gas input: can only consume gas (derivative_min=0) + - d=1 hydrogen input: can only consume hydrogen (derivative_min=0) + - d=2 heat output: can only produce heat (derivative_max=0) + - d=3 power output: can only produce electricity (derivative_max=0) + + Coupling group ``"chp"`` with coefficients + ``[(0, 0.5), (1, 0.5), (2, -0.5), (3, -0.3)]`` introduces a free variable + ``alpha`` and enforces ``P[d] == coeff * alpha``: + + P_gas = 0.5 * alpha (50% of total fuel from gas) + P_hydrogen = 0.5 * alpha (50% of total fuel from hydrogen) + P_heat = -0.5 * alpha (heat efficiency 50% of total fuel) + P_power = -0.3 * alpha (power efficiency 30% of total fuel) + + Because gas and hydrogen share the same coefficient the two fuel flows are + always equal, confirming that device order does not affect the result. + + Heat production is forced to exactly 10 kW via ``derivative equals = -10`` on device 2. + Substituting ``P_heat = -10`` gives ``alpha = 20``, so: + + P_gas = 10 kW (equal gas input) + P_hydrogen = 10 kW (equal hydrogen input) + P_heat = -10 kW (heat produced, forced) + P_power = -6 kW (electricity produced) + """ + start = pd.Timestamp("2026-01-01T00:00+01:00") + end = pd.Timestamp("2026-01-01T04:00+01:00") + resolution = pd.Timedelta("1h") + index = initialize_index(start=start, end=end, resolution=resolution) + + def _flow_df(**kwargs) -> pd.DataFrame: + defaults = { + "min": np.nan, + "max": np.nan, + "equals": np.nan, + "derivative min": 0.0, + "derivative max": 0.0, + "derivative equals": np.nan, + "derivative down efficiency": 1.0, + "derivative up efficiency": 1.0, + } + defaults.update(kwargs) + return pd.DataFrame(defaults, index=index) + + # d=0: gas input — can only consume, capacity 100 kW + gas_constraints = _flow_df(**{"derivative max": 100.0}) + # d=1: hydrogen input — can only consume, capacity 100 kW + hydrogen_constraints = _flow_df(**{"derivative max": 100.0}) + # d=2: heat output — can only produce, forced to -10 kW + heat_constraints = _flow_df( + **{"derivative min": -100.0, "derivative equals": -10.0} + ) + # d=3: power output — can only produce, free (coupling determines value) + power_constraints = _flow_df(**{"derivative min": -100.0}) + + ems_constraints = pd.DataFrame( + {"derivative min": -200.0, "derivative max": 200.0}, + index=index, + ) + + # Both fuel inputs share coefficient 0.5, so they receive identical flows. + # Outputs have negative coefficients equal to their efficiency fractions. + coupling_groups = {"chp": [(0, 0.5), (1, 0.5), (2, -0.5), (3, -0.3)]} + + # Gas-price commitment for device 0 just to give the objective a finite value + # Even though hydrogen is free, it will still be used because its consumption is coupled to gas. + fuel_cost_commitment = FlowCommitment( + name="fuel cost", + index=index, + quantity=pd.Series(0.0, index=index), + upwards_deviation_price=pd.Series(1.0, index=index), + downwards_deviation_price=pd.Series(0.0, index=index), + device=pd.Series(0, index=index), + ) + + schedules, _costs, results, _model = device_scheduler( + device_constraints=[ + gas_constraints, + hydrogen_constraints, + heat_constraints, + power_constraints, + ], + ems_constraints=ems_constraints, + commitments=[fuel_cost_commitment], + coupling_groups=coupling_groups, + ) + + assert ( + results.solver.termination_condition == "optimal" + ), "Solver did not find an optimal solution." + + # Heat is fixed to -10 kW; alpha = -10 / -0.5 = 20. + pd.testing.assert_series_equal( + schedules[2], + pd.Series(-10.0, index=index), + check_names=False, + rtol=1e-4, + obj="heat output forced to -10 kW by derivative_equals", + ) + + # Coupling: P_gas = 0.5 * alpha = 0.5 * 20 = 10 kW + pd.testing.assert_series_equal( + schedules[0], + pd.Series(10.0, index=index), + check_names=False, + rtol=1e-4, + obj="gas input = 0.5 * alpha = 10 kW", + ) + + # Coupling: P_hydrogen = 0.5 * alpha = 10 kW (equal to gas) + pd.testing.assert_series_equal( + schedules[1], + pd.Series(10.0, index=index), + check_names=False, + rtol=1e-4, + obj="hydrogen input = 0.5 * alpha = 10 kW (equal to gas input)", + ) + + # Coupling: P_power = -0.3 * alpha = -0.3 * 20 = -6 kW + pd.testing.assert_series_equal( + schedules[3], + pd.Series(-6.0, index=index), + check_names=False, + rtol=1e-4, + obj="power output = -0.3 * alpha = -6 kW", + ) + + +def _run_factory_scenario( + gas_price: float, + elec_price: float, + use_balance_groups: bool = False, +) -> tuple: + """Run the simplified factory scenario and return the 7 device schedules. + + With ``use_balance_groups=False``, the heat and steam nodes are balanced via + shared stock groups whose first ("reference") device carries min=max=0 stock + bounds. With ``use_balance_groups=True``, the same nodes are expressed directly + as ``balance_groups``, needing neither stock groups nor reference-device bounds. + + Devices + ~~~~~~~ + d=0 e-heater electricity → heat coupling (ems_power ≥ 0, i.e. consumes electricity) + d=1 gas boiler gas → heat coupling (ems_power ≥ 0, i.e. consumes gas) + d=2 steamer heat coupling → steam (ems_power ≤ 0, i.e. produces steam) + d=3 CHP gas input gas → chp coupling (ems_power ≥ 0, i.e. consumes gas, coupling member = alpha) + d=4 CHP heat out chp coupling → steam (ems_power ≤ 0, i.e. produces steam, coupling member = -0.5 alpha) + d=5 CHP power out chp coupling → electricity (ems_power ≤ 0, i.e. produces electricity, coupling member = -0.3 alpha) + d=6 steam demand steam → fixed flow (ems_power = 15, i.e. consumes steam) + + CHP coupling coefficients + ~~~~~~~~~~~~~~~~~~~~~~~~~ + The coupling constraint introduces a free variable ``alpha`` (the normalised gas flow) + and enforces ``P[d_i] == coeff_i * alpha`` for every device in the group. + Choosing thermal efficiency η_heat = 0.5 and power efficiency η_power = 0.3, + the coefficients simply become the signed efficiency fractions:: + + P_gas = 1.0 * alpha (input, coeff = 1.0) + P_heat = -0.5 * alpha (output, coeff = η_heat = -0.5) + P_power = -0.3 * alpha (output, coeff = η_power = −0.3) + + """ + ETA_HEAT = 0.5 # fraction of CHP gas input that becomes heat + ETA_POWER = 0.3 # fraction of CHP gas input that becomes power + STEAM_DEMAND = 15.0 # kW, constant heat drain representing steam production + CHP_GAS_MAX = 20.0 # kW, maximum gas input to CHP + BOILER_GAS_MAX = 10.0 # kW, maximum gas input to gas boiler + HEATER_POWER_MAX = 100.0 # kW, maximum electricity input to e-heater + + start = pd.Timestamp("2026-01-01T00:00+01:00") + end = pd.Timestamp("2026-01-01T04:00+01:00") + resolution = pd.Timedelta("1h") + index = initialize_index(start=start, end=end, resolution=resolution) + + def _df(**kwargs) -> pd.DataFrame: + """Build a device-constraints DataFrame with defaults for unused columns.""" + defaults = { + "min": np.nan, + "max": np.nan, + "equals": np.nan, + "derivative min": 0.0, + "derivative max": 0.0, + "derivative equals": np.nan, + "derivative down efficiency": 1.0, + "derivative up efficiency": 1.0, + "stock delta": 0.0, + } + defaults.update(kwargs) + return pd.DataFrame(defaults, index=index) + + # With balance groups, no reference device needs min=max=0 stock bounds. + node_bounds = {} if use_balance_groups else {"min": 0.0, "max": 0.0} + + device_constraints = [ + # d=0 e-heater: heat-node reference device. The min=max=0 forces the heat + # node to balance at every step (zero-capacity flow node), making + # the per-step dispatch deterministic despite flat prices. + _df(**node_bounds, **{"derivative max": HEATER_POWER_MAX}), + # d=1 gas boiler: up to 100 kW gas → 100 kW heat (efficiency 1 for clean maths in test) + _df(**{"derivative max": BOILER_GAS_MAX, "commodity": "gas"}), + # d=2 steamer: can only produce steam (negative ems_power). + # The lower bound is finite to avoid unbounded model messages while still + # being looser than the upstream heat-supply limits. + _df( + **{ + "derivative min": -(HEATER_POWER_MAX + BOILER_GAS_MAX), + "derivative max": 0.0, + "commodity": "steam", + } + ), + # d=3 CHP gas input: up to CHP_GAS_MAX kW gas + _df(**{"derivative max": CHP_GAS_MAX, "commodity": "gas"}), + # d=4 CHP heat output: positive ems_power adds heat to the steam node. + # The min=max=0 forces the steam node to balance at every step. + _df( + **node_bounds, + **{ + "derivative min": -CHP_GAS_MAX * ETA_HEAT, + "derivative max": 0.0, + "commodity": "steam", + }, + ), + # d=5 CHP power output: negative ems_power only (production) + _df(**{"derivative min": -CHP_GAS_MAX * ETA_POWER, "derivative max": 0.0}), + # d=6 steam demand: fixed steam consumption at STEAM_DEMAND kW. + _df( + **{ + "derivative min": STEAM_DEMAND, + "derivative max": STEAM_DEMAND, + "commodity": "steam", + } + ), + ] + + ems_constraints = pd.DataFrame( + {"derivative min": -300.0, "derivative max": 300.0}, + index=index, + ) + + # Node membership: the steamer (d=2) converts heat to steam, so it belongs + # to both nodes (its single flow drains heat and feeds steam). + heat_node = [0, 1, 2] + steam_node = [2, 4, 6] + if use_balance_groups: + stock_groups = None + balance_groups = {"heat": heat_node, "steam": steam_node} + else: + # stock group: all heat-buffer devices share the same stock + # (keys 0 and 1 are arbitrary group ids, not device indices) + stock_groups = {0: heat_node, 1: steam_node} + balance_groups = None + + # CHP coupling: coefficients are signed efficiency fractions. + # coeff_heat = -η_heat = -0.5 → P_heat = -0.5 * alpha = -0.5 * P_gas + # coeff_power = -η_power = -0.3 → P_power = -0.3 * alpha = -0.3 * P_gas + coupling_groups = { + "chp": [ + (3, 1.0), + (4, -ETA_HEAT), # = -0.5 + (5, -ETA_POWER), # = -0.3 + ] + } + + # --- energy-price commitments ------------------------------------------- + # Gas price applies to gas boiler (d=1) and CHP gas input (d=3). + # Electricity price applies to e-heater (d=0) and CHP power output (d=5). + # Using both upwards and downwards prices makes each commitment a two-sided + # soft equality (quantity = 0): + # • upward deviation = consuming more than 0 → positive cost + # • downward deviation = producing (negative flow) → negative cost (revenue) + gas_p = pd.Series(gas_price, index=index) + elec_p = pd.Series(elec_price, index=index) + + commitments = [] + for d, price in [(1, gas_p), (3, gas_p), (0, elec_p), (5, elec_p)]: + commitments.append( + FlowCommitment( + name="gas cost" if d in (1, 3) else "electricity cost", + index=index, + quantity=pd.Series(0.0, index=index), + upwards_deviation_price=price, + downwards_deviation_price=price, + device=pd.Series(d, index=index), + ) + ) + + schedules, _costs, results, _model = device_scheduler( + device_constraints=device_constraints, + ems_constraints=ems_constraints, + commitments=commitments, + stock_groups=stock_groups, + coupling_groups=coupling_groups, + balance_groups=balance_groups, + ) + + assert results.solver.termination_condition == "optimal", ( + f"Solver did not find an optimal solution " + f"(gas_price={gas_price}, elec_price={elec_price})" + ) + return tuple(schedules) + + +@pytest.mark.parametrize("use_balance_groups", [False, True]) +def test_factory_chp_dispatch(use_balance_groups): + """Factory: CHP + gas boiler + e-heater competing to meet a fixed steam demand. + + The heat and steam nodes are balanced either via shared stock groups with + a min=max=0 reference device (``use_balance_groups=False``) or via explicit + ``balance_groups`` (``use_balance_groups=True``) — both must yield the same + dispatch. The steam node is drained at a + constant rate of 15 kW by the steam demand device. Two price scenarios + verify that the optimizer correctly chooses the cheapest heat source. + + Scenario A — gas cheaper than electricity + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Prices: gas = 20 EUR/kW, electricity = 50 EUR/kW. + + Effective cost per kW of heat delivered: + - CHP: gas_cost − power_revenue = (20·20 − 50·6) / 10 = 10 EUR/kW + - gas boiler: 20 EUR/kW (efficiency = 1) + - e-heater: 50 EUR/kW (efficiency = 1) + + Merit order: CHP ≪ gas boiler ≪ e-heater. + + With CHP at maximum (20 kW gas → 10 kW heat + 6 kW power): + - remaining heat demand = 15 − 10 = 5 kW → gas boiler + - e-heater not needed + + Scenario B — electricity cheaper than gas + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Prices: gas = 100 EUR/kW, electricity = 10 EUR/kW. + + Effective cost per kW of heat: + - CHP: (100·20 − 10·6) / 10 = 194 EUR/kW + - gas boiler: 100 EUR/kW + - e-heater: 10 EUR/kW + + Merit order: e-heater ≪ gas boiler ≪ CHP. + + All 15 kW steam demand is met by the e-heater; CHP and gas boiler are off. + + Scenario C — gas slightly cheaper + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Prices: gas = 50 EUR/kW, electricity = 55 EUR/kW. + + Effective cost per kW of heat delivered: + - CHP: gas_cost − power_revenue = (50·20 − 55·6) / 10 = 67 EUR/kW + - gas boiler: 50 EUR/kW + - e-heater: 55 EUR/kW + + Merit order: gas boiler ≪ e-heater ≪ CHP. + + With gas boiler at maximum (10 kW gas → 10 kW heat): + - remaining heat demand = 15 − 10 = 5 kW → e-heater + - CHP not needed + """ + # ------------------------------------------------------------------ # + # Scenario A: gas cheaper — CHP at max, gas boiler fills the rest # + # ------------------------------------------------------------------ # + (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = ( + _run_factory_scenario( + gas_price=20.0, elec_price=50.0, use_balance_groups=use_balance_groups + ) + ) + + expected_chp_gas = pd.Series(20.0, index=e_heater.index) + expected_chp_heat = pd.Series(-10.0, index=e_heater.index) # -0.5 * 20 + expected_chp_power = pd.Series(-6.0, index=e_heater.index) # -0.3 * 20 + expected_boiler = pd.Series(5.0, index=e_heater.index) # fills 15-10 kW gap + expected_steamer = pd.Series(-5.0, index=e_heater.index) + expected_demand = pd.Series(15.0, index=e_heater.index) + expected_eheater = pd.Series(0.0, index=e_heater.index) + + pd.testing.assert_series_equal( + chp_gas, + expected_chp_gas, + check_names=False, + rtol=1e-4, + obj="Scenario A: CHP gas input at maximum (20 kW)", + ) + pd.testing.assert_series_equal( + chp_heat, + expected_chp_heat, + check_names=False, + rtol=1e-4, + obj="Scenario A: CHP heat output = 0.5 × gas input (10 kW)", + ) + pd.testing.assert_series_equal( + chp_power, + expected_chp_power, + check_names=False, + rtol=1e-4, + obj="Scenario A: CHP power output = −0.3 × gas input (−6 kW)", + ) + pd.testing.assert_series_equal( + gas_boiler, + expected_boiler, + check_names=False, + rtol=1e-4, + obj="Scenario A: gas boiler fills remaining 5 kW heat demand", + ) + pd.testing.assert_series_equal( + steamer, + expected_steamer, + check_names=False, + rtol=1e-4, + obj="Scenario A: steamer supplies remaining 5 kW steam", + ) + pd.testing.assert_series_equal( + demand, + expected_demand, + check_names=False, + rtol=1e-4, + obj="Scenario A: steam demand fixed at 15 kW", + ) + pd.testing.assert_series_equal( + e_heater, + expected_eheater, + check_names=False, + atol=1e-4, + obj="Scenario A: e-heater not used (gas is cheapest)", + ) + + # ------------------------------------------------------------------ # + # Scenario B: electricity cheaper — e-heater meets all demand # + # ------------------------------------------------------------------ # + (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = ( + _run_factory_scenario( + gas_price=100.0, elec_price=10.0, use_balance_groups=use_balance_groups + ) + ) + + expected_eheater_b = pd.Series(15.0, index=e_heater.index) + expected_zero = pd.Series(0.0, index=e_heater.index) + expected_steamer_b = pd.Series(-15.0, index=e_heater.index) + expected_demand_b = pd.Series(15.0, index=e_heater.index) + + pd.testing.assert_series_equal( + e_heater, + expected_eheater_b, + check_names=False, + rtol=1e-4, + obj="Scenario B: e-heater meets all 15 kW steam demand", + ) + pd.testing.assert_series_equal( + chp_gas, + expected_zero, + check_names=False, + atol=1e-4, + obj="Scenario B: CHP not used (electricity is cheapest)", + ) + pd.testing.assert_series_equal( + gas_boiler, + expected_zero, + check_names=False, + atol=1e-4, + obj="Scenario B: gas boiler not used (electricity is cheapest)", + ) + pd.testing.assert_series_equal( + steamer, + expected_steamer_b, + check_names=False, + rtol=1e-4, + obj="Scenario B: steamer supplies all 15 kW steam", + ) + pd.testing.assert_series_equal( + demand, + expected_demand_b, + check_names=False, + rtol=1e-4, + obj="Scenario B: steam demand fixed at 15 kW", + ) + + # --------------------------------------------------------------------------------- # + # Scenario C: gas slightly cheaper — gas boiler at max, e-heater fills the rest # + # --------------------------------------------------------------------------------- # + (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = ( + _run_factory_scenario( + gas_price=50.0, elec_price=55.0, use_balance_groups=use_balance_groups + ) + ) + + expected_chp_gas = pd.Series(0.0, index=e_heater.index) + expected_chp_heat = pd.Series(0.0, index=e_heater.index) + expected_chp_power = pd.Series(0.0, index=e_heater.index) + expected_boiler = pd.Series(10.0, index=e_heater.index) + expected_steamer = pd.Series(-15.0, index=e_heater.index) + expected_demand = pd.Series(15.0, index=e_heater.index) + expected_eheater = pd.Series(5.0, index=e_heater.index) # fills 15-10 kW gap + + pd.testing.assert_series_equal( + chp_gas, + expected_chp_gas, + check_names=False, + rtol=1e-4, + obj="Scenario C: CHP not used", + ) + pd.testing.assert_series_equal( + chp_heat, + expected_chp_heat, + check_names=False, + rtol=1e-4, + obj="Scenario C: CHP not used", + ) + pd.testing.assert_series_equal( + chp_power, + expected_chp_power, + check_names=False, + rtol=1e-4, + obj="Scenario C: CHP not used", + ) + pd.testing.assert_series_equal( + gas_boiler, + expected_boiler, + check_names=False, + rtol=1e-4, + obj="Scenario C: gas boiler at maximum (10 kW)", + ) + pd.testing.assert_series_equal( + steamer, + expected_steamer, + check_names=False, + rtol=1e-4, + obj="Scenario C: steamer supplies all 15 kW steam", + ) + pd.testing.assert_series_equal( + demand, + expected_demand, + check_names=False, + rtol=1e-4, + obj="Scenario C: steam demand fixed at 15 kW", + ) + pd.testing.assert_series_equal( + e_heater, + expected_eheater, + check_names=False, + atol=1e-4, + obj="Scenario C: e-heater fills remaining 5 kW heat demand", + ) + + def test_all_gas_flex_model_without_electricity_device(app, db): """test_all_gas_flex_model_without_electricity_device: a flex-model with only gas devices (no electricity device at all) should not raise a KeyError, now that @@ -1782,3 +2471,182 @@ def test_electricity_device_indices_exclude_other_commodities(): assert mapping["electricity"] == [0, 2, 3, 4] assert mapping["gas"] == [1, 5] assert scheduler._electricity_device_indices() == [0, 2, 3, 4] + + +def test_commitment_commodity_does_not_bind_other_commodity_devices(): + """A commitment + listed under the flex-context's `commitments` should only bind devices of its own + `commodity` (defaulting to "electricity", like devices do). A gas commitment + should therefore not create a FlowCommitment against an electricity device, and + vice versa. + + This is a DB-free, unit-level test of StorageScheduler.convert_to_commitments. + """ + scheduler = object.__new__(StorageScheduler) + scheduler.flex_context = { + "shared_currency_unit": "EUR", + "commitments": [ + { + "name": "gas commitment", + "commodity": "gas", + "baseline": ur.Quantity("1 MW"), + }, + { + # No `commodity` given: defaults to "electricity", like devices do. + "name": "electricity commitment", + "baseline": ur.Quantity("2 MW"), + }, + ], + } + # Flexible devices: 0 = electricity, 1 = gas. + flex_model = [ + {"commodity": "electricity"}, + {"commodity": "gas"}, + ] + + start = pd.Timestamp("2024-01-01T00:00:00+01:00") + end = pd.Timestamp("2024-01-01T03:00:00+01:00") + resolution = pd.Timedelta("1h") + + commitments = scheduler.convert_to_commitments( + flex_model=flex_model, + query_window=(start, end), + resolution=resolution, + beliefs_before=None, + ) + + assert len(commitments) == 2 + + gas_commitment = next(c for c in commitments if c.name == "custom:gas commitment") + electricity_commitment = next( + c for c in commitments if c.name == "custom:electricity commitment" + ) + + # The gas commitment binds only the gas device (index 1), not the electricity + # device (index 0). + assert (gas_commitment.device == 1).all() + assert set(gas_commitment.device_group.unique()) == {"gas"} + + # The electricity commitment (commodity defaulting to "electricity") binds only + # the electricity device (index 0), not the gas device (index 1). + assert (electricity_commitment.device == 0).all() + assert set(electricity_commitment.device_group.unique()) == {"electricity"} + + +def test_sensor_scoped_commitment_binds_aggregate_of_selected_devices(app, db): + """A commitment scoped to specific sensors (here: two e-heaters) binds their + aggregate flow as one commitment: a baseline of 10 MW with a steep penalty on + downward deviation keeps their combined consumption at 10 MW even though a + cheaper allocation (0 MW) exists, while an unscoped battery stays unaffected. + """ + heater_type = get_or_create_model(GenericAssetType, name="e-heater") + site = GenericAsset( + name="Band site (scoped commitment test)", generic_asset_type=heater_type + ) + db.session.add(site) + db.session.flush() + + resolution = pd.Timedelta("1h") + start = pd.Timestamp("2026-02-01T00:00:00+01:00") + end = pd.Timestamp("2026-02-01T04:00:00+01:00") + + def sensor(name): + s = Sensor(name=name, unit="MW", event_resolution=resolution, generic_asset=site) + db.session.add(s) + return s + + heater_1 = sensor("band heater 1") + heater_2 = sensor("band heater 2") + db.session.flush() + + flex_model = [ + { + # Heaters burn money at the consumption price; without the band + # commitment the optimum is to stay off. + "sensor": heater_1.id, + "power-capacity": "8 MW", + "consumption-capacity": "8 MW", + "production-capacity": "0 kW", + }, + { + "sensor": heater_2.id, + "power-capacity": "8 MW", + "consumption-capacity": "8 MW", + "production-capacity": "0 kW", + }, + ] + flex_context = { + "consumption-price": "50 EUR/MWh", + "production-price": "50 EUR/MWh", + "site-power-capacity": "1 GW", + "commitments": [ + { + "name": "reserved band", + "sensors": [heater_1.id, heater_2.id], + "baseline": "10 MW", + # Steep penalty for consuming less than the band (negative price + # penalizes downward deviation); consuming more is free. + "down-price": "-10000 EUR/MWh", + } + ], + } + + scheduler = StorageScheduler( + asset_or_sensor=site, + start=start, + end=end, + resolution=resolution, + belief_time=start, + flex_model=flex_model, + flex_context=flex_context, + return_multiple=True, + ) + results = scheduler.compute(skip_validation=True) + schedules = { + r["sensor"]: r["data"] for r in results if r.get("name") == "storage_schedule" + } + combined = schedules[heater_1] + schedules[heater_2] + # The band keeps the aggregate at 10 MW (cheapest way to avoid the penalty), + # even though each heater alone (8 MW max) could not carry it. + np.testing.assert_allclose(combined.iloc[:-1], 10.0, rtol=1e-4) + + +def test_user_commitment_names_are_namespaced(app): + """User-given commitment names get a "custom:" prefix, so they cannot shadow + the commitments the scheduler sets up internally (e.g. "electricity net energy"), + whose costs are reported in the same name-keyed dict. + """ + scheduler = object.__new__(StorageScheduler) + start = pd.Timestamp("2024-01-01T00:00:00+01:00") + end = pd.Timestamp("2024-01-01T04:00:00+01:00") + resolution = pd.Timedelta("1h") + scheduler.flex_context = { + "shared_currency_unit": "EUR", + "commitments": [ + { + # Deliberately shadowing an internal commitment name + "name": "electricity net energy", + "baseline": ur.Quantity("0 MW"), + "up_price": ur.Quantity("100 EUR/MWh"), + }, + ], + } + flex_model = [{"commodity": "electricity"}] + + commitments = scheduler.convert_to_commitments( + flex_model, + query_window=(start, end), + resolution=resolution, + beliefs_before=start, + ) + assert len(commitments) == 1 + assert commitments[0].name == "custom:electricity net energy" + + # Converting the same specs again must not double the prefix. + commitments = scheduler.convert_to_commitments( + flex_model, + query_window=(start, end), + resolution=resolution, + beliefs_before=start, + ) + assert commitments[0].name == "custom:electricity net energy" diff --git a/flexmeasures/data/models/planning/tests/test_storage.py b/flexmeasures/data/models/planning/tests/test_storage.py index a03d09e05d..c4a5cb3144 100644 --- a/flexmeasures/data/models/planning/tests/test_storage.py +++ b/flexmeasures/data/models/planning/tests/test_storage.py @@ -7,15 +7,18 @@ import numpy as np import pandas as pd +from flexmeasures.data.models.generic_assets import GenericAsset, GenericAssetType from flexmeasures.data.models.planning import Scheduler from flexmeasures.data.models.planning.storage import StorageScheduler from flexmeasures.data.models.planning.utils import initialize_index +from flexmeasures.data.models.data_sources import DataSource from flexmeasures.data.models.time_series import Sensor, TimedBelief from flexmeasures.data.models.planning.tests.utils import ( check_constraints, get_sensors_from_db, series_to_ts_specs, ) +from flexmeasures.data.services.utils import get_or_create_model from flexmeasures.data.services.scheduling_result import SchedulingJobResult @@ -149,12 +152,12 @@ def test_battery_solver_multi_commitment(add_battery_assets, db): # No production peak np.testing.assert_almost_equal(costs["electricity production peak"], 0) - # Sample commitments + # Sample commitments (user-given names are namespaced with a "custom:" prefix) np.testing.assert_almost_equal( - costs["a sample commitment penalizing peaks"], 4 * (1 - 0.4) + costs["custom:a sample commitment penalizing peaks"], 4 * (1 - 0.4) ) np.testing.assert_almost_equal( - costs["a sample commitment penalizing demand/supply"], 1 * (1 - 0.4) + costs["custom:a sample commitment penalizing demand/supply"], 1 * (1 - 0.4) ) # Check consumption/production output sensor schedules. @@ -1223,3 +1226,354 @@ def test_resolve_soc_at_start_from_percent_sensor_uses_device_sensor_fallback( ) == 2.5 ) + + +def test_storage_scheduler_chp_coupling(app, db): + """Test that the StorageScheduler enforces CHP coupling constraints between devices. + + Models a Combined Heat and Power unit with three sensors. + + In the flex-model, the coupling coefficients are entered as positive magnitudes:: + + gas input -> 1.0 + heat output -> 0.5 + power output -> 0.3 + + Internally, the CHP is interpreted with the signed commodity-flow coefficients:: + + P_gas -> 1.0 + P_heat -> -0.5 + P_power -> -0.3 + + The returned storage schedule for the heat buffer is still positive, because this + test uses the storage sign convention for buffer charging. + + - d=0 gas input: CHP gas consumption + - d=1 heat output: CHP heat -> heat buffer + - d=2 power output: CHP electricity production + + The heat output is forced to exactly 5 kW per step by combining: + - ``production-capacity: "0 kW"`` (hard lower bound: derivative_min = 0) + - ``consumption-capacity: "5 kW"`` (hard upper bound: derivative_max = 0.005 MW) + - ``soc-targets`` requiring 20 kWh at the end of the 4-hour window + + With soc_at_start = 0 and max 5 kW over 4 × 1-hour steps the only feasible + solution is P_heat = 5 kW every step. Substituting P_heat = 5 kW gives + alpha = 5 / 0.5 = 10 kW, so: + + P_gas = 1.0 × 10 kW = 10 kW + P_power = −0.3 × 10 kW = −3 kW + """ + # ---- asset type + asset + chp_type = get_or_create_model(GenericAssetType, name="chp-plant") + chp = GenericAsset(name="CHP plant (coupling test)", generic_asset_type=chp_type) + db.session.add(chp) + db.session.flush() + + # ---- schedule window + start = pd.Timestamp("2026-01-01T00:00:00+01:00") + end = pd.Timestamp("2026-01-01T04:00:00+01:00") + resolution = timedelta(hours=1) + + # CHP efficiencies (same values as the factory scenario in test_commitments.py) + ETA_HEAT = 0.5 # fraction of gas input that becomes heat + ETA_POWER = 0.3 # fraction of gas input that becomes electricity + + # ---- sensors + gas_input_sensor = Sensor( + name="CHP gas input (coupling test)", + generic_asset=chp, + unit="MW", + event_resolution=resolution, + ) + heat_output_sensor = Sensor( + name="CHP heat output (coupling test)", + generic_asset=chp, + unit="MW", + event_resolution=resolution, + ) + power_output_sensor = Sensor( + name="CHP power output (coupling test)", + generic_asset=chp, + unit="MW", + event_resolution=resolution, + ) + db.session.add_all([gas_input_sensor, heat_output_sensor, power_output_sensor]) + db.session.flush() + + # ---- flex model + # Flex-model coupling-coefficients are user-facing positive magnitudes. + # The intended internal CHP coefficients are +1.0 for gas, -0.5 for heat, + # and -0.3 for power. + flex_model = [ + { + # d=0: gas input — pure flow device (no SoC), can only consume gas. + "sensor": gas_input_sensor.id, + "power-capacity": "20 kW", + "production-capacity": "0 kW", # derivative_min = 0 + "coupling": "chp", + "coupling-coefficient": 1.0, + }, + { + # d=1: heat output — tracks heat-buffer SoC, positive ems_power = heat + # added to buffer. The SoC target forces P_heat = 5 kW per step. + "sensor": heat_output_sensor.id, + "soc-at-start": "0 MWh", + "soc-min": "0 MWh", + "soc-max": "0.02 MWh", # 20 kWh — matches the SoC target + "soc-targets": [ + { + # Single target at the schedule end: cumulative heat = 20 kWh. + # With max 5 kW and 4 × 1 h steps the only feasible solution + # is 5 kW every step. + "start": "2026-01-01T04:00:00+01:00", + "duration": "PT1H", + "value": "0.02 MWh", + } + ], + "power-capacity": "5 kW", + "consumption-capacity": "5 kW", + "production-capacity": "0 kW", # can only add heat, not extract + "prefer-charging-sooner": True, + "coupling": "chp", + "coupling-coefficient": ETA_HEAT, # = 0.5 + }, + { + # d=2: power output — pure flow device (no SoC), can only produce + # electricity (negative ems_power). + "sensor": power_output_sensor.id, + "power-capacity": "6 kW", + "consumption-capacity": "0 kW", # derivative_max = 0 + "coupling": "chp", + "coupling-coefficient": ETA_POWER, # = 0.3 (sign inferred from capacities) + }, + ] + + flex_context = { + "consumption-price": "50 EUR/MWh", + "production-price": "50 EUR/MWh", + "site-power-capacity": "1 MW", # large enough to avoid EMS constraints + } + + scheduler = StorageScheduler( + asset_or_sensor=chp, + start=start, + end=end, + resolution=resolution, + flex_model=flex_model, + flex_context=flex_context, + return_multiple=True, + ) + + results = scheduler.compute(skip_validation=True) + + # ---- extract storage schedules per sensor + storage_schedules = { + r["sensor"]: r["data"] for r in results if r.get("name") == "storage_schedule" + } + + assert gas_input_sensor in storage_schedules, "Gas input schedule missing" + assert heat_output_sensor in storage_schedules, "Heat output schedule missing" + assert power_output_sensor in storage_schedules, "Power output schedule missing" + + gas_schedule = storage_schedules[gas_input_sensor] + heat_schedule = storage_schedules[heat_output_sensor] + power_schedule = storage_schedules[power_output_sensor] + + # The SoC target of 20 kWh is met after 4 × 1-hour steps at 5 kW. + # The schedule index runs from ``start`` to ``end`` inclusive (5 time slots), + # so the last slot has no binding SoC constraint and the CHP is idle there. + # All assertions therefore apply to the first four active slots only. + active_steps = slice(None, -1) # exclude the final trailing idle slot + + # Heat output is forced to exactly 5 kW per step by the SoC target. + # alpha = P_heat / ETA_HEAT = 0.005 / 0.5 = 0.010 MW + np.testing.assert_allclose( + heat_schedule.iloc[active_steps], + 0.005, # 5 kW expressed in MW + rtol=1e-4, + err_msg="Heat output should be exactly 5 kW per step (forced by SoC target)", + ) + + # Coupling: P_gas = 1.0 * alpha = 0.010 MW = 10 kW + np.testing.assert_allclose( + gas_schedule.iloc[active_steps], + 0.010, # 10 kW expressed in MW + rtol=1e-4, + err_msg="Gas input must be 10 kW — determined by coupling (1.0 * alpha)", + ) + + # Coupling: P_power = -ETA_POWER * alpha = -0.3 * 0.010 MW = -0.003 MW = -3 kW + np.testing.assert_allclose( + power_schedule.iloc[active_steps], + -0.003, # -3 kW expressed in MW + rtol=1e-4, + err_msg="Power output must be -3 kW — determined by coupling (-0.3 * alpha)", + ) + + +def test_factory_chp_dispatch_through_storage_scheduler(app, db): + """The full factory scenario (CHP + gas boiler + e-heater meeting a fixed steam + demand) scheduled end-to-end through ``StorageScheduler.compute()``. + + Unlike the engine-level ``test_factory_chp_dispatch`` (which passes balance groups + to ``device_scheduler`` directly), this test only supplies a flex-model and a + flex-context. Each converter is described as one device per commodity port, tied + together by a coupling group. The heat and steam commodities have no energy prices + in the flex-context, so the scheduler derives internal-node balance groups for them. + + Topology (flex-model device indices):: + + electricity (grid) --0--> [e-heater] --1--> heat + gas (grid) --2--> [boiler] --3--> heat + heat --4--> [steamer] --5--> steam + gas (grid) --6--> [CHP] --7--> steam + --8--> electricity (grid) + steam --9--> fixed 15 kW demand (inflexible sensor) + + Prices: gas 20 EUR/MWh, electricity 50 EUR/MWh. Marginal cost per kW of steam: + CHP (20·20 − 50·6) / 10 = 10, boiler-via-steamer 20, e-heater-via-steamer 50. + So the CHP runs at maximum (20 kW gas → 10 kW steam + 6 kW power) and the boiler + covers the remaining 5 kW of steam via the steamer; the e-heater stays off. + """ + factory_type = get_or_create_model(GenericAssetType, name="factory") + factory = GenericAsset( + name="Factory (end-to-end CHP dispatch)", generic_asset_type=factory_type + ) + db.session.add(factory) + db.session.flush() + + start = pd.Timestamp("2026-01-01T00:00:00+01:00") + end = pd.Timestamp("2026-01-01T04:00:00+01:00") + resolution = timedelta(hours=1) + + def make_sensor(name: str) -> Sensor: + sensor = Sensor( + name=name, generic_asset=factory, unit="MW", event_resolution=resolution + ) + db.session.add(sensor) + return sensor + + eheater_elec_in = make_sensor("e-heater electricity input") + eheater_heat_out = make_sensor("e-heater heat output") + boiler_gas_in = make_sensor("boiler gas input") + boiler_heat_out = make_sensor("boiler heat output") + steamer_heat_in = make_sensor("steamer heat input") + steamer_steam_out = make_sensor("steamer steam output") + chp_gas_in = make_sensor("CHP gas input") + chp_steam_out = make_sensor("CHP steam output") + chp_power_out = make_sensor("CHP power output") + steam_demand = make_sensor("steam demand") + db.session.flush() + + # A constant 15 kW steam demand, recorded as beliefs. + # By default, power sensors store consumption as negative values + # (get_power_values flips the sign to the scheduler's consumption-positive convention). + index = initialize_index(start, end, resolution) + source = get_or_create_model(DataSource, name="test source", type="forecaster") + db.session.add_all( + TimedBelief( + sensor=steam_demand, + source=source, + event_start=dt, + belief_time=start, + event_value=-15e-3, # 15 kW in MW + ) + for dt in index + ) + db.session.commit() + + def input_port(sensor: Sensor, commodity: str, coupling: str, max_power: str): + return { + "sensor": sensor.id, + "commodity": commodity, + "coupling": coupling, + "coupling-coefficient": 1.0, + "power-capacity": max_power, + "production-capacity": "0 kW", + } + + def output_port( + sensor: Sensor, commodity: str, coupling: str, coefficient: float = 1.0 + ): + return { + "sensor": sensor.id, + "commodity": commodity, + "coupling": coupling, + "coupling-coefficient": coefficient, + "power-capacity": "1 MW", + "consumption-capacity": "0 kW", + } + + flex_model = [ + input_port(eheater_elec_in, "electricity", "eheater", "100 kW"), + output_port(eheater_heat_out, "heat", "eheater"), + input_port(boiler_gas_in, "gas", "boiler", "10 kW"), + output_port(boiler_heat_out, "heat", "boiler"), + input_port(steamer_heat_in, "heat", "steamer", "1 MW"), + output_port(steamer_steam_out, "steam", "steamer"), + input_port(chp_gas_in, "gas", "chp", "20 kW"), + output_port(chp_steam_out, "steam", "chp", coefficient=0.5), + output_port(chp_power_out, "electricity", "chp", coefficient=0.3), + ] + + flex_context = [ + { + "commodity": "electricity", + "consumption-price": "50 EUR/MWh", + "production-price": "50 EUR/MWh", + }, + { + "commodity": "gas", + "consumption-price": "20 EUR/MWh", + "production-price": "20 EUR/MWh", + }, + { + # No prices: steam is an internal node. Its fixed demand is inflexible. + "commodity": "steam", + "inflexible-device-sensors": [steam_demand.id], + }, + # The heat commodity has no context at all: also an internal node. + ] + + scheduler = StorageScheduler( + asset_or_sensor=factory, + start=start, + end=end, + resolution=resolution, + belief_time=start, + flex_model=flex_model, + flex_context=flex_context, + return_multiple=True, + ) + + results = scheduler.compute(skip_validation=True) + + # The scheduler derived one balance group per priceless commodity: + # heat: e-heater out (1), boiler out (3), steamer in (4) + # steam: steamer out (5), CHP out (7), inflexible demand (9) + assert scheduler.balance_groups == {"heat": [1, 3, 4], "steam": [5, 7, 9]} + + schedules = { + r["sensor"]: r["data"] for r in results if r.get("name") == "storage_schedule" + } + + expected_mw = { + eheater_elec_in: 0.0, + eheater_heat_out: 0.0, + boiler_gas_in: 0.005, + boiler_heat_out: -0.005, + steamer_heat_in: 0.005, + steamer_steam_out: -0.005, + chp_gas_in: 0.020, + chp_steam_out: -0.010, + chp_power_out: -0.006, + } + for sensor, expected_value in expected_mw.items(): + np.testing.assert_allclose( + schedules[sensor], + expected_value, + rtol=1e-4, + atol=1e-9, + err_msg=f"Unexpected schedule for {sensor.name}", + ) diff --git a/flexmeasures/data/schemas/scheduling/__init__.py b/flexmeasures/data/schemas/scheduling/__init__.py index 93c86ac555..b577aff97d 100644 --- a/flexmeasures/data/schemas/scheduling/__init__.py +++ b/flexmeasures/data/schemas/scheduling/__init__.py @@ -4,6 +4,8 @@ from datetime import timedelta from typing import Any, Callable, Dict +from flask import current_app + from marshmallow import ( Schema, fields, @@ -70,7 +72,25 @@ def forbid_time_series_specs(self, data: dict, **kwargs): class CommitmentSchema(Schema): - name = fields.Str(required=True, data_key="name") + name = fields.Str(required=True, data_key="name", validate=validate.Length(min=1)) + commodity = fields.Str( + required=False, + load_default="electricity", + data_key="commodity", + metadata=dict( + description="Commodity whose devices this commitment binds. Defaults to electricity.", + example="electricity", + ), + ) + # Optional scoping: bind this commitment to the aggregate flow of the + # devices whose power sensors are listed, rather than binding each device + # of the matching commodity separately. Useful to commit a band on a + # subset of devices (e.g. an aFRR band on a site's e-heaters). + sensors = fields.List( + SensorIdField(), + required=False, + data_key="sensors", + ) baseline = VariableQuantityField("MW", required=False, data_key="baseline") up_price = VariableQuantityField("/MW", required=False, data_key="up-price") down_price = VariableQuantityField( @@ -370,9 +390,57 @@ def _try_to_convert_price_units(self, data: dict, original_data: dict, **kwargs) elif sensor := data.get("production_price_sensor"): data["shared_currency_unit"] = self._to_currency_per_mwh(sensor.unit) else: + # No user-given price fields at all: fall back to "EUR", but flag this + # as a default (not user-given), so cross-context/cross-schema currency + # comparisons can skip it (a price-free context should never trip a + # currency-mismatch error against the rest of a differently-currencied + # portfolio; see CommodityFlexContextSchema.fill_grid_connection_defaults + # and FlexContextSchema.validate_commodity_contexts_shared_currency). data["shared_currency_unit"] = "EUR" + data["shared_currency_unit_is_default"] = True return data + # Currency-denominated fields that CommodityFlexContextSchema's smart defaults + # (fill_grid_connection_defaults) may fill with a fallback "EUR" price/breach + # price when a context has no user-given price fields at all. + _CURRENCY_DENOMINATED_FIELDS = ( + "consumption_price", + "production_price", + "ems_consumption_breach_price", + "ems_production_breach_price", + "consumption_breach_price", + "production_breach_price", + "soc_minima_breach_price", + "soc_maxima_breach_price", + "ems_peak_consumption_price", + "ems_peak_production_price", + ) + + @classmethod + def _rebase_default_context_currency(cls, context: dict, new_currency: str): + """Re-express a price-free context's fallback-currency fields in another currency. + + Only called for a commodity context that had no user-given price fields + (``shared_currency_unit_is_default`` is True), once a real portfolio + currency becomes known (e.g. from the top-level flex-context, or from a + sibling commodity context). All of that context's currency-denominated + fields were filled with plain quantities in a fallback "EUR", so their + magnitudes carry over unchanged under the new currency label (no FX + conversion is implied or attempted). + """ + for field in cls._CURRENCY_DENOMINATED_FIELDS: + value = context.get(field) + if not isinstance(value, ur.Quantity): + continue + old_units = str(value.units) + denominator = old_units.split("/", 1)[1] if "/" in old_units else None + new_unit = ( + new_currency if denominator is None else f"{new_currency}/{denominator}" + ) + context[field] = ur.Quantity(value.magnitude, new_unit) + context["shared_currency_unit"] = new_currency + context["shared_currency_unit_is_default"] = False + @staticmethod def _to_currency_per_mwh(price_unit: str) -> str: """Convert a price unit to a base currency used to express that price per MWh. @@ -402,6 +470,172 @@ def __init__(self, *args, **kwargs): [("commodity", commodity_field), *self.fields.items()] ) + @post_load(pass_original=True) + def fill_grid_connection_defaults(self, data: dict, original_data: dict, **kwargs): + """Fill in smarter defaults for a commodity context's grid-connection fields. + + A commodity context (an entry of the top-level `commodities` list) may omit + some or all of the grid-connection fields (`consumption-price`, + `production-price`, `site-consumption-capacity`, `site-production-capacity`, + `site-power-capacity`). Rather than leaving those simply unset (which, for + `consumption-price`, would make the scheduler fail, since it requires one), + we derive sensible defaults from *which* of those five fields were explicitly + given (inspecting the original input, not post-default-fill presence). + + A price given for a direction (consumption or production) implies a grid + connection in that direction, with an unlimited capacity unless a capacity + is also given; a capacity given for a direction (without a price) implies a + 0 price in that direction; and anything not implied by a given field + defaults to "no connection" (0 capacity, as a soft constraint). The + exception is `site-power-capacity` given on its own, which sets a *hard* + (symmetric) capacity limit instead. See :ref:`commodity_context_defaults` + for the full user-facing explanation, including worked examples. + + Precedence (single-field triggers): + + 1. None of the five given (e.g. just `{"commodity": "gas"}`): no grid + connection at all. `site-consumption-capacity` and + `site-production-capacity` default to 0, as *soft* constraints (a default + breach price is filled in, so breaching is possible but penalized -- this + relies on `relax-constraints`/`relax-site-capacity-constraints`, which + default to True). `site-power-capacity` is left unlimited (unset). + 2. Only `consumption-price` given: assume a grid connection for consumption. + `site-power-capacity` and `site-consumption-capacity` stay unlimited. + `site-production-capacity` defaults to 0 (soft). + 3. Only `production-price` given: the mirror image of (2), for production. + 4. Only `site-consumption-capacity` given: `site-power-capacity` stays + unlimited; `consumption-price` defaults to 0; `site-production-capacity` + (and, transitively, `production-price`) default to 0. + 5. Only `site-production-capacity` given: the mirror image of (4). + 6. Only `site-power-capacity` given: a *hard* constraint at that capacity. + `site-consumption-capacity` and `site-production-capacity` are both set + equal to it (no breach price is filled in, so the constraint stays hard); + `consumption-price` and `production-price` default to 0. + + As a safety net (since the scheduler requires a resolvable consumption + price), `consumption-price` defaults to 0 if still unset after applying the + rules above (`production-price` already falls back to `consumption-price` + at the scheduler level, so no separate safety net is needed for it). + + A commodity context with no user-given price fields does not trip a spurious cross-currency error against a differently-currencied portfolio; + its 0-price/breach-price fields instead inherit the portfolio's real currency where determinable (from a top-level price or a sibling commodity context). + """ + + has_consumption_price = "consumption-price" in original_data + has_production_price = "production-price" in original_data + has_consumption_capacity = "site-consumption-capacity" in original_data + has_production_capacity = "site-production-capacity" in original_data + has_power_capacity = "site-power-capacity" in original_data + + any_given = ( + has_consumption_price + or has_production_price + or has_consumption_capacity + or has_production_capacity + or has_power_capacity + ) + + # Record durably whether any price field was user-given, so the scheduler + # can distinguish a smart-defaulted zero price from a deliberate one + # (e.g. to treat a priceless commodity as an internal node). + data["prices_are_defaulted"] = not ( + has_consumption_price or has_production_price + ) + + currency = data.get("shared_currency_unit") or "EUR" + zero_price = ur.Quantity(f"0 {currency}/MWh") + zero_capacity = ur.Quantity("0 MW") + + # Case 6: site-power-capacity is the only field given -> hard constraint. + if has_power_capacity and not ( + has_consumption_price + or has_production_price + or has_consumption_capacity + or has_production_capacity + ): + power_capacity = data["ems_power_capacity_in_mw"] + data.setdefault("ems_consumption_capacity_in_mw", power_capacity) + data.setdefault("ems_production_capacity_in_mw", power_capacity) + data.setdefault("consumption_price", zero_price) + data.setdefault("production_price", zero_price) + return data + + # Case 1: nothing given at all -> fully disconnected commodity. + if not any_given: + self._default_zero_capacity_as_soft_constraint( + data, "ems_consumption_capacity_in_mw", zero_capacity + ) + self._default_zero_capacity_as_soft_constraint( + data, "ems_production_capacity_in_mw", zero_capacity + ) + data.setdefault("consumption_price", zero_price) + return data + + # Cases 2-5 and combinations thereof: fill in what's still missing, per + # direction (consumption/production), independently. + if not has_consumption_price and not has_consumption_capacity: + self._default_zero_capacity_as_soft_constraint( + data, "ems_consumption_capacity_in_mw", zero_capacity + ) + if has_consumption_capacity and not has_consumption_price: + data.setdefault("consumption_price", zero_price) + + if not has_production_price and not has_production_capacity: + self._default_zero_capacity_as_soft_constraint( + data, "ems_production_capacity_in_mw", zero_capacity + ) + if has_production_capacity and not has_production_price: + data.setdefault("production_price", zero_price) + + # Safety net: the scheduler requires a resolvable consumption price. + data.setdefault("consumption_price", zero_price) + + return data + + def _default_zero_capacity_as_soft_constraint( + self, data: dict, field: str, zero_capacity: ur.Quantity + ): + """Default a site capacity field to 0, as a *soft* constraint. + + Also fills in a default breach price for that direction (unless one was + already set), so the 0 capacity is enforced as a soft constraint (breaching + is possible, but penalized) rather than a hard, potentially infeasible, one. + This mirrors FlexContextSchema.check_prices, but scoped to a single + commodity context, and only fired for capacities defaulted here (not for + capacities the caller explicitly set to 0). + """ + if field in data: + # Already set (e.g. by an earlier rule in this method); leave it as-is. + return + data[field] = zero_capacity + + breach_price_field = { + "ems_consumption_capacity_in_mw": "ems_consumption_breach_price", + "ems_production_capacity_in_mw": "ems_production_breach_price", + }[field] + if data.get("relax_site_capacity_constraints") or data.get("relax_constraints"): + if not data.get(breach_price_field): + currency = data.get("shared_currency_unit") or "EUR" + shared_currency = ur.Quantity(currency) + self.set_default_breach_prices( + data, + fields=[breach_price_field], + price=10000 * shared_currency / ur.Quantity("kW"), + ) + elif data.get("relax_constraints") is False: + # relax-constraints defaults to True, so False here can only be an + # explicit user choice. Since relax-site-capacity-constraints is also + # not set/true, this 0 capacity ends up as a *hard* constraint, which + # is likely infeasible for any commodity with actual devices/flow. + current_app.logger.warning( + f"Commodity context '{data.get('commodity', 'electricity')}' has" + f" its '{field}' defaulted to a 0 capacity, but" + " 'relax-constraints' was explicitly set to False (and" + " 'relax-site-capacity-constraints' was not set to True), so this" + " ends up as a hard 0-capacity constraint, which is likely" + " infeasible." + ) + class FlexContextSchema(SharedSchema): """This schema defines fields that provide context to the portfolio to be optimized.""" @@ -496,6 +730,11 @@ def validate_commodity_contexts_shared_currency( shared_currency_unit = None for context in commodity_contexts: + if context.get("shared_currency_unit_is_default"): + # No user-given price fields in this context: its "EUR" currency is + # just a fallback, not a real constraint, so don't let it clash with + # a differently-currencied portfolio. + continue context_currency_unit = context.get("shared_currency_unit") if context_currency_unit is None: continue @@ -516,25 +755,44 @@ def validate_commodity_contexts_shared_currency( # serve as the electricity context only when the commodities list has no # electricity entry (see _get_commodity_contexts in storage.py). - @validates_schema(pass_original=True) - def check_prices(self, data: dict, original_data: dict, **kwargs): - """Check assumptions about prices. + def _reconcile_commodity_context_currencies(self, data: dict) -> str: + """Backfill price-free contexts' currency with the portfolio's real currency. - 1. The flex-context must contain at most 1 consumption price and at most 1 production price field. - 2. All prices must share the same currency. + Determines the portfolio's real (user-given) shared currency, if any: the + top-level one, unless it's itself just a fallback (no user-given price + fields at the top level), in which case falls back to the first + non-default commodity context's currency, if any. Then rebases any + price-free ("default currency") commodity context onto that real currency, + so their 0-price/breach-price fills inherit it. Returns the (possibly + just-updated) top-level `shared_currency_unit`. """ + commodity_contexts = data.get("commodity_contexts", []) or [] + real_shared_currency_unit = None + if not data.get("shared_currency_unit_is_default"): + real_shared_currency_unit = data["shared_currency_unit"] + else: + for context in commodity_contexts: + if not context.get("shared_currency_unit_is_default"): + real_shared_currency_unit = context["shared_currency_unit"] + break - # The flex-context must contain at most 1 consumption price and at most 1 production price field - if "consumption_price_sensor" in data and "consumption_price" in data: - raise ValidationError( - "Must pass either consumption-price or consumption-price-sensor." - ) - if "production_price_sensor" in data and "production_price" in data: - raise ValidationError( - "Must pass either production-price or production-price-sensor." - ) + if real_shared_currency_unit is not None and data.get( + "shared_currency_unit_is_default" + ): + data["shared_currency_unit"] = real_shared_currency_unit + data["shared_currency_unit_is_default"] = False + + if real_shared_currency_unit is not None: + for context in commodity_contexts: + if context.get("shared_currency_unit_is_default"): + self._rebase_default_context_currency( + context, real_shared_currency_unit + ) + + return data["shared_currency_unit"] - # New price fields can only be used after updating to the new consumption-price and production-price fields + def _check_deprecated_price_sensor_migration(self, data: dict, original_data: dict): + """New price fields can only be used after updating to consumption-price/production-price.""" field_map = { field.data_key: field_var for field_var, field in self.declared_fields.items() @@ -567,14 +825,41 @@ def check_prices(self, data: dict, original_data: dict, **kwargs): f"""Please switch to using `production-price: {{"sensor": {data[field_map["production-price-sensor"]].id}}}`.""" ) + @validates_schema(pass_original=True) + def check_prices(self, data: dict, original_data: dict, **kwargs): + """Check assumptions about prices. + + 1. The flex-context must contain at most 1 consumption price and at most 1 production price field. + 2. All prices must share the same currency. + """ + + # The flex-context must contain at most 1 consumption price and at most 1 production price field + if "consumption_price_sensor" in data and "consumption_price" in data: + raise ValidationError( + "Must pass either consumption-price or consumption-price-sensor." + ) + if "production_price_sensor" in data and "production_price" in data: + raise ValidationError( + "Must pass either production-price or production-price-sensor." + ) + + self._check_deprecated_price_sensor_migration(data, original_data) + # make sure that the prices fields are valid price units # All prices must share the same unit data = self._try_to_convert_price_units(data, original_data) - shared_currency = ur.Quantity(data["shared_currency_unit"]) + shared_currency = ur.Quantity( + self._reconcile_commodity_context_currencies(data) + ) # Also check that top-level prices share their currency with any per-commodity contexts for context in data.get("commodity_contexts", []) or []: + if context.get("shared_currency_unit_is_default"): + # Already reconciled (or left as a harmless fallback, if no real + # currency was determinable anywhere) by + # _reconcile_commodity_context_currencies above. + continue context_currency_unit = context.get("shared_currency_unit") if context_currency_unit is None: continue @@ -658,21 +943,11 @@ def check_prices(self, data: dict, original_data: dict, **kwargs): "aggregate-consumption": { "default": None, "description": rst_to_openapi(metadata.AGGREGATE_CONSUMPTION.description), - # todo: the field type is defined in asset_context.html in 3 places? - # "types": { - # "backend": "typeTwo", - # "ui": "A sensor which records the scheduled aggregate consumption.", - # }, "example-units": EXAMPLE_UNIT_TYPES["power"], }, "aggregate-production": { "default": None, "description": rst_to_openapi(metadata.AGGREGATE_PRODUCTION.description), - # todo: the field type is defined in asset_context.html in 3 places? - # "types": { - # "backend": "typeTwo", - # "ui": "A sensor which records the scheduled aggregate production.", - # }, "example-units": EXAMPLE_UNIT_TYPES["power"], }, "consumption-price": { @@ -769,8 +1044,25 @@ def check_prices(self, data: dict, original_data: dict, **kwargs): "description": rst_to_openapi(metadata.AGGREGATE_POWER.description), "example-units": EXAMPLE_UNIT_TYPES["power"], }, + # The commodities list is not offered as an addable field in the editor; + # the UI manages it through a commodity tab bar instead. + "commodities": { + "default": None, + "description": rst_to_openapi(metadata.COMMODITIES.description), + "example-units": EXAMPLE_UNIT_TYPES["commodity"], + }, } +# Mark which flex-context fields can also be set within each entry of the +# commodities list (i.e. which fields CommodityFlexContextSchema shares), +# so the UI editor can offer the right fields per commodity tab. +_COMMODITY_CONTEXT_DATA_KEYS = { + schema_field.data_key or field_name + for field_name, schema_field in CommodityFlexContextSchema().fields.items() +} +for _field_name, _entry in UI_FLEX_CONTEXT_SCHEMA.items(): + _entry["per-commodity"] = _field_name in _COMMODITY_CONTEXT_DATA_KEYS + UI_FLEX_MODEL_SCHEMA: Dict[str, Dict[str, Any]] = { "consumption": { "default": None, @@ -1064,6 +1356,55 @@ def _forbid_fixed_prices(self, data: dict, **kwargs): ) +# One UI description per backend type token (same tokens as UI_FLEX_MODEL_SCHEMA uses). +UI_TYPE_DESCRIPTIONS: Dict[str, str] = { + "typeOne": "One fixed value only.", + "typeTwo": "One dynamic signal (via a sensor) only.", + "typeThree": "One fixed value or a dynamic signal (via a sensor).", + "typeFour": "A list of sensors.", + "typeFive": "One fixed string value only.", + "typeSix": "A list of structured entries.", +} + + +def _derive_backend_type(schema_field: fields.Field) -> str: + """Derive the UI editor's backend type token from a marshmallow field.""" + if isinstance(schema_field, fields.Bool): + return "typeOne" + if isinstance(schema_field, fields.List): + return "typeFour" + if isinstance(schema_field, fields.Nested): + return "typeSix" if schema_field.many else "typeTwo" + if isinstance(schema_field, VariableQuantityField): + return "typeThree" + if isinstance(schema_field, fields.Str): + return "typeFive" + raise NotImplementedError( + f"Cannot derive a UI type for field {schema_field.data_key}." + ) + + +# Fill in the "types" of each UI flex-context schema entry by deriving them +# from the corresponding DBFlexContextSchema field, so the UI editor stays in +# sync with what the DB schema actually accepts. +_db_flex_context_fields: Dict[str, fields.Field] = { + schema_field.data_key or field_name: schema_field + for field_name, schema_field in DBFlexContextSchema().fields.items() +} +for _field_name, _entry in UI_FLEX_CONTEXT_SCHEMA.items(): + _backend_type = _derive_backend_type(_db_flex_context_fields[_field_name]) + if _field_name in ("consumption-price", "production-price", "aggregate-power"): + # Fixed prices are forbidden when storing the flex-context in the DB + # (see DBFlexContextSchema._forbid_fixed_prices), and aggregate-power + # must reference a sensor (see validate_aggregate_power_is_sensor), + # so the editor only offers a sensor for these fields. + _backend_type = "typeTwo" + _entry["types"] = { + "backend": _backend_type, + "ui": UI_TYPE_DESCRIPTIONS[_backend_type], + } + + class MultiSensorFlexModelSchema(Schema): """ diff --git a/flexmeasures/data/schemas/scheduling/metadata.py b/flexmeasures/data/schemas/scheduling/metadata.py index 4f6f9d4298..f3576a92b3 100644 --- a/flexmeasures/data/schemas/scheduling/metadata.py +++ b/flexmeasures/data/schemas/scheduling/metadata.py @@ -33,6 +33,13 @@ def to_dict(self): """, examples=["electricity", "gas"], ) +COMMODITIES = MetaData( + description="""List of per-commodity flex-contexts (one for each commodity, e.g. electricity and gas), each holding the prices and grid-connection fields for that commodity. +The fields given at the top level of the flex-context describe the electricity commodity. +See :ref:`tut_multi_commodity` for a hands-on example. +""", + example=[{"commodity": "gas", "consumption-price": {"sensor": 5}}], +) INFLEXIBLE_DEVICE_SENSORS = MetaData( description="""Power sensors representing devices that are relevant, but not flexible in the timing of their demand/supply. For example, a sensor recording rooftop solar power that is connected behind the main meter, and whose production falls under the same contract as the flexible device(s) being scheduled. @@ -74,8 +81,16 @@ def to_dict(self): example={"sensor": 11}, ) COMMITMENTS = MetaData( - description="Prior commitments. Support for this field in the UI is still under further development, but you can find more information in :ref:`commitments`.", - example=[], + description="""Prior commitments. Each commitment needs a ``name`` and a ``baseline``, plus at least one deviation price (``up-price`` and/or ``down-price``); its ``commodity`` defaults to electricity. +You can find more information in :ref:`commitments`. +""", + example=[ + { + "name": "capacity contract", + "baseline": "100 kW", + "up-price": {"sensor": 5}, + } + ], ) CONSUMPTION_PRICE = MetaData( description="The commodity price (e.g. electricity price) applied to the site's aggregate consumption. Can be (a sensor recording) market prices, but also CO₂ intensity—whatever fits your optimization problem. [#old_consumption_price_field]_", @@ -223,6 +238,25 @@ def to_dict(self): """, examples=["electricity", "gas"], ) +COUPLING = MetaData( + description="""Name of the coupling group this device belongs to. +Devices sharing the same coupling name are constrained to have proportionally related power flows, via a hard equality constraint. +Use this to model a device that converts one commodity into another, by describing each of its commodity ports as a separate device. +For example, a combined heat and power (CHP) unit is described as a gas input device, a heat output device and an electricity output device, all sharing one coupling name. +Use together with ``coupling-coefficient`` to set the flow ratios. +""", + example="chp", +) +COUPLING_COEFFICIENT = MetaData( + description="""Positive coupling magnitude for this device within its coupling group. +The scheduler couples the power flows of all devices in the group: each device's power is its coupling coefficient times the group's common flow level. +The flow direction of each device is inferred from its directional capacities: a device with ``consumption-capacity: "0 kW"`` is an output (producing) device, and a device with ``production-capacity: "0 kW"`` is an input (consuming) device. +Exactly one of the two directional capacities must be set to a fixed zero for each coupled device. +For example, a CHP unit with 50% thermal and 30% electrical efficiency uses a gas input device (coefficient 1), a heat output device (coefficient 0.5) and an electricity output device (coefficient 0.3). +Defaults to 1. +""", + example=0.5, +) CONSUMPTION = MetaData( description="""Sensor used to record the scheduled power as seen from a consumption perspective. diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py index 60d9da3448..4ecc9aae4b 100644 --- a/flexmeasures/data/schemas/scheduling/storage.py +++ b/flexmeasures/data/schemas/scheduling/storage.py @@ -245,6 +245,25 @@ class StorageFlexModelSchema(Schema): validate=validate.Length(min=1), metadata=metadata.SOC_USAGE.to_dict(), ) + commodity = fields.Str( + data_key="commodity", + load_default="electricity", + metadata=metadata.COMMODITY_FLEX_MODEL.to_dict(), + ) + + coupling = fields.Str( + data_key="coupling", + required=False, + load_default=None, + metadata=metadata.COUPLING.to_dict(), + ) + coupling_coefficient = fields.Float( + data_key="coupling-coefficient", + required=False, + load_default=1.0, + validate=validate.Range(min=0, min_inclusive=False), + metadata=metadata.COUPLING_COEFFICIENT.to_dict(), + ) def __init__( self, @@ -394,6 +413,33 @@ def validate_commodity(self, commodity: str, **kwargs): if not isinstance(commodity, str) or not commodity.strip(): raise ValidationError("commodity must be a non-empty string.") + @validates_schema + def validate_coupling_direction_is_unambiguous(self, data: dict, **kwargs): + """A coupled device must have an inferable flow direction. + + The sign of the coupling coefficient is inferred from directional capacities: + a fixed zero consumption-capacity marks an output device, a fixed zero + production-capacity marks an input device. When both directional capacities + allow flow (or are given in a form whose value cannot be checked statically, + such as a sensor reference), the direction is ambiguous, so we reject the + flex-model rather than silently treating the device as an input. + """ + if data.get("coupling") is None: + return + + def _is_fixed_zero(value) -> bool: + return isinstance(value, ur.Quantity) and float(value.magnitude) == 0.0 + + consumption_blocked = _is_fixed_zero(data.get("consumption_capacity")) + production_blocked = _is_fixed_zero(data.get("production_capacity")) + if consumption_blocked == production_blocked: + raise ValidationError( + "A device with a 'coupling' field must have an unambiguous flow direction: " + "set either its consumption-capacity (for an output device) or its " + "production-capacity (for an input device) to a fixed 0, but not both.", + field_name="coupling", + ) + @post_load def post_load_sequence(self, data: dict, **kwargs) -> dict: """Perform some checks and corrections after we loaded.""" diff --git a/flexmeasures/data/schemas/tests/test_scheduling.py b/flexmeasures/data/schemas/tests/test_scheduling.py index 90842d2cf9..034b1594dc 100644 --- a/flexmeasures/data/schemas/tests/test_scheduling.py +++ b/flexmeasures/data/schemas/tests/test_scheduling.py @@ -16,6 +16,7 @@ ) from flexmeasures.data.models.time_series import Sensor from flexmeasures.data.schemas.sensors import TimedEventSchema, VariableQuantityField +from flexmeasures.utils.unit_utils import ur @pytest.mark.parametrize( @@ -1053,6 +1054,150 @@ def test_commodity_flex_context_defaults( assert loaded.get("relax_constraints", True) is True +def _assert_quantity_or_none(actual, expected): + """Compare an (optionally None) ur.Quantity against an expected ur.Quantity or None.""" + if expected is None: + assert actual is None + else: + assert actual is not None + assert actual.to(expected.units).magnitude == pytest.approx(expected.magnitude) + + +@pytest.mark.parametrize( + ["context_input", "expected"], + [ + # Case 1: none of the 5 grid-connection fields given -> fully disconnected + # commodity. Both site capacities default to 0 as *soft* constraints (a + # default breach price is filled in); site-power-capacity stays unlimited. + ( + {"commodity": "gas"}, + { + "ems_consumption_capacity_in_mw": ur.Quantity("0 MW"), + "ems_production_capacity_in_mw": ur.Quantity("0 MW"), + "ems_power_capacity_in_mw": None, + "consumption_price": ur.Quantity("0 EUR/MWh"), + "ems_consumption_breach_price_set": True, + "ems_production_breach_price_set": True, + }, + ), + # Case 2: only consumption-price given -> assume a grid connection for + # consumption (unlimited site-power/consumption-capacity); 0 + # site-production-capacity (soft). + ( + {"commodity": "gas", "consumption-price": "10 EUR/MWh"}, + { + "ems_consumption_capacity_in_mw": None, + "ems_production_capacity_in_mw": ur.Quantity("0 MW"), + "ems_power_capacity_in_mw": None, + "consumption_price": ur.Quantity("10 EUR/MWh"), + "ems_production_breach_price_set": True, + }, + ), + # Case 3: only production-price given -> mirror image of case 2. + ( + {"commodity": "gas", "production-price": "10 EUR/MWh"}, + { + "ems_consumption_capacity_in_mw": ur.Quantity("0 MW"), + "ems_production_capacity_in_mw": None, + "ems_power_capacity_in_mw": None, + "consumption_price": ur.Quantity("0 EUR/MWh"), + "production_price": ur.Quantity("10 EUR/MWh"), + "ems_consumption_breach_price_set": True, + }, + ), + # Case 4: only site-consumption-capacity given -> unlimited + # site-power-capacity, 0 consumption-price, 0 site-production-capacity + # (soft), (and thereby 0 production-price). + ( + {"commodity": "gas", "site-consumption-capacity": "5 MW"}, + { + "ems_consumption_capacity_in_mw": ur.Quantity("5 MW"), + "ems_production_capacity_in_mw": ur.Quantity("0 MW"), + "ems_power_capacity_in_mw": None, + "consumption_price": ur.Quantity("0 EUR/MWh"), + "ems_production_breach_price_set": True, + }, + ), + # Case 5: only site-production-capacity given -> mirror image of case 4. + ( + {"commodity": "gas", "site-production-capacity": "5 MW"}, + { + "ems_consumption_capacity_in_mw": ur.Quantity("0 MW"), + "ems_production_capacity_in_mw": ur.Quantity("5 MW"), + "ems_power_capacity_in_mw": None, + "consumption_price": ur.Quantity("0 EUR/MWh"), + "production_price": ur.Quantity("0 EUR/MWh"), + "ems_consumption_breach_price_set": True, + }, + ), + # Case 6: only site-power-capacity given -> a *hard* constraint at that + # capacity (both site capacities set equal to it; no breach price filled + # in); 0 consumption- and production-price. + ( + {"commodity": "gas", "site-power-capacity": "5 MW"}, + { + "ems_consumption_capacity_in_mw": ur.Quantity("5 MW"), + "ems_production_capacity_in_mw": ur.Quantity("5 MW"), + "ems_power_capacity_in_mw": ur.Quantity("5 MW"), + "consumption_price": ur.Quantity("0 EUR/MWh"), + "production_price": ur.Quantity("0 EUR/MWh"), + "ems_consumption_breach_price_set": False, + "ems_production_breach_price_set": False, + }, + ), + # A multi-field combination: consumption-price given together with an + # explicit site-power-capacity. The site-power-capacity is not the *sole* + # field given, so it does not trigger the hard-constraint case; instead, + # each direction is filled in independently: consumption-price given -> + # site-consumption-capacity stays unlimited (implicitly bounded by + # site-power-capacity at the scheduler level); production side untouched + # -> 0 site-production-capacity (soft). + ( + { + "commodity": "gas", + "consumption-price": "10 EUR/MWh", + "site-power-capacity": "5 MW", + }, + { + "ems_consumption_capacity_in_mw": None, + "ems_production_capacity_in_mw": ur.Quantity("0 MW"), + "ems_power_capacity_in_mw": ur.Quantity("5 MW"), + "consumption_price": ur.Quantity("10 EUR/MWh"), + "ems_production_breach_price_set": True, + }, + ), + ], +) +def test_commodity_flex_context_smart_defaults(context_input, expected): + """Test the smarter defaults for commodity contexts (see + CommodityFlexContextSchema.fill_grid_connection_defaults). + + These are DB-free, direct schema loads (no sensors involved). + """ + from flexmeasures.data.schemas.scheduling import CommodityFlexContextSchema + + loaded = CommodityFlexContextSchema().load(context_input) + + for field in ( + "ems_consumption_capacity_in_mw", + "ems_production_capacity_in_mw", + "ems_power_capacity_in_mw", + "consumption_price", + "production_price", + ): + if field in expected: + _assert_quantity_or_none(loaded.get(field), expected[field]) + + if "ems_consumption_breach_price_set" in expected: + assert (loaded.get("ems_consumption_breach_price") is not None) == expected[ + "ems_consumption_breach_price_set" + ] + if "ems_production_breach_price_set" in expected: + assert (loaded.get("ems_production_breach_price") is not None) == expected[ + "ems_production_breach_price_set" + ] + + @pytest.mark.parametrize( ["flex_context_listing", "fails"], [ @@ -1137,6 +1282,71 @@ def test_flex_context_listing_shared_currency( check_schema_loads_data(schema=schema, data=flex_context_listing, fails=fails) +def test_flex_context_listing_tolerates_price_free_context_in_other_currency(): + """test_flex_context_listing_tolerates_price_free_context_in_other_currency: + a bare (price-free) commodity context must not trip the shared-currency check + against a differently-currencied portfolio, since it has no user-given prices + of its own -- its 0-price/breach-price fills should just inherit the + portfolio's real currency. + """ + schema = FlexContextSchema() + + # Case A: top-level price sets the portfolio currency. + loaded = schema.load( + { + "consumption-price": "10 USD/MWh", + "commodities": [ + {"commodity": "electricity", "consumption-price": "10 USD/MWh"}, + {"commodity": "gas"}, + ], + } + ) + assert loaded["shared_currency_unit"] == "USD" + gas_context = next( + c for c in loaded["commodity_contexts"] if c["commodity"] == "gas" + ) + assert gas_context["shared_currency_unit"] == "USD" + assert str(gas_context["consumption_price"].units) == "USD/MWh" + + # Case B: no top-level price; a sibling commodity context sets the currency. + loaded = schema.load( + { + "commodities": [ + {"commodity": "electricity", "consumption-price": "10 USD/MWh"}, + {"commodity": "gas"}, + ], + } + ) + assert loaded["shared_currency_unit"] == "USD" + gas_context = next( + c for c in loaded["commodity_contexts"] if c["commodity"] == "gas" + ) + assert gas_context["shared_currency_unit"] == "USD" + assert str(gas_context["consumption_price"].units) == "USD/MWh" + + # Case C: no price given anywhere -> falls back to EUR everywhere. + loaded = schema.load({"commodities": [{"commodity": "gas"}]}) + assert loaded["shared_currency_unit"] == "EUR" + gas_context = loaded["commodity_contexts"][0] + assert gas_context["shared_currency_unit"] == "EUR" + + # A genuine mismatch (both contexts have explicit, different currencies) must + # still be rejected. + check_schema_loads_data( + schema=schema, + data={ + "consumption-price": "10 USD/MWh", + "commodities": [ + {"commodity": "electricity", "consumption-price": "10 USD/MWh"}, + {"commodity": "gas", "consumption-price": "10 EUR/MWh"}, + ], + }, + fails={ + "commodities": "all prices in the flex-context must share the same currency unit" + }, + ) + + def test_flex_context_listing_rejects_duplicate_commodities(db, app): """test_flex_context_listing_rejects_duplicate_commodities: a commodity listed twice must be rejected.""" schema = FlexContextSchema() @@ -1197,3 +1407,46 @@ def test_asset_trigger_schema_rejects_malformed_flex_context(app): with pytest.raises(ValidationError) as e_info: schema.normalize_flex_context_format({"flex-context": "not-a-dict-or-list"}) assert "flex-context" in str(e_info.value) + + +@pytest.mark.parametrize( + "capacity_fields, fails", + [ + # Input device: production blocked, direction is unambiguous + ({"production-capacity": "0 kW"}, False), + # Output device: consumption blocked, direction is unambiguous + ({"consumption-capacity": "0 kW"}, False), + # Output device with a bounded input side still has one blocked direction + ({"consumption-capacity": "5 kW", "production-capacity": "0 kW"}, False), + # Neither direction blocked: ambiguous + ({}, True), + # Both directions open: ambiguous + ({"consumption-capacity": "5 kW", "production-capacity": "5 kW"}, True), + # Both directions blocked: degenerate (device pinned to zero flow) + ({"consumption-capacity": "0 kW", "production-capacity": "0 kW"}, True), + ], +) +def test_coupling_direction_must_be_unambiguous(app, capacity_fields, fails): + """test_coupling_direction_must_be_unambiguous: a device with a `coupling` field must + have exactly one directional capacity fixed to zero, so the sign of its coupling + coefficient can be inferred.""" + schema = StorageFlexModelSchema(start=datetime(2026, 6, 1), sensor=None) + flex_model = { + "power-capacity": "20 kW", + "coupling": "chp", + "coupling-coefficient": 0.5, + **capacity_fields, + } + if fails: + with pytest.raises(ValidationError) as e_info: + schema.load(flex_model) + assert "unambiguous flow direction" in str(e_info.value) + else: + schema.load(flex_model) + + +def test_uncoupled_device_needs_no_directional_capacities(app): + """test_uncoupled_device_needs_no_directional_capacities: the coupling-direction check + only applies to devices that define a `coupling` field.""" + schema = StorageFlexModelSchema(start=datetime(2026, 6, 1), sensor=None) + schema.load({"power-capacity": "20 kW"}) diff --git a/flexmeasures/ui/static/openapi-specs.json b/flexmeasures/ui/static/openapi-specs.json index be45760153..28fedaf781 100644 --- a/flexmeasures/ui/static/openapi-specs.json +++ b/flexmeasures/ui/static/openapi-specs.json @@ -4573,7 +4573,18 @@ "type": "object", "properties": { "name": { - "type": "string" + "type": "string", + "minLength": 1 + }, + "commodity": { + "type": "string", + "default": "electricity", + "description": "Commodity whose devices this commitment binds. Defaults to electricity.", + "example": "electricity" + }, + "commodity": { + "type": "string", + "default": "electricity" }, "baseline": {}, "up-price": {}, @@ -4718,8 +4729,16 @@ "example": true }, "commitments": { - "description": "Prior commitments. Support for this field in the UI is still under further development, but you can find more information in the docs.", - "example": [], + "description": "Prior commitments. Each commitment needs a name and a baseline, plus at least one deviation price (up-price and/or down-price); its commodity defaults to electricity.\nYou can find more information in the docs.\n", + "example": [ + { + "name": "capacity contract", + "baseline": "100 kW", + "up-price": { + "sensor": 5 + } + } + ], "type": "array", "items": { "$ref": "#/components/schemas/Commitment" @@ -6288,6 +6307,22 @@ ], "items": {} }, + "coupling": { + "type": [ + "string", + "null" + ], + "default": null, + "description": "Name of the coupling group this device belongs to.\nDevices sharing the same coupling name are constrained to have proportionally related power flows, via a hard equality constraint.\nUse this to model a device that converts one commodity into another, by describing each of its commodity ports as a separate device.\nFor example, a combined heat and power (CHP) unit is described as a gas input device, a heat output device and an electricity output device, all sharing one coupling name.\nUse together with coupling-coefficient to set the flow ratios.\n", + "example": "chp" + }, + "coupling-coefficient": { + "type": "number", + "default": 1.0, + "minimum": 0.0, + "description": "Positive coupling magnitude for this device within its coupling group.\nThe scheduler couples the power flows of all devices in the group: each device's power is its coupling coefficient times the group's common flow level.\nThe flow direction of each device is inferred from its directional capacities: a device with consumption-capacity: \"0 kW\" is an output (producing) device, and a device with production-capacity: \"0 kW\" is an input (consuming) device.\nExactly one of the two directional capacities must be set to a fixed zero for each coupled device.\nFor example, a CHP unit with 50% thermal and 30% electrical efficiency uses a gas input device (coefficient 1), a heat output device (coefficient 0.5) and an electricity output device (coefficient 0.3).\nDefaults to 1.\n", + "example": 0.5 + }, "sensor": { "type": "integer", "description": "ID of the device's power sensor." diff --git a/flexmeasures/ui/templates/assets/asset_context.html b/flexmeasures/ui/templates/assets/asset_context.html index 09a1348dfb..2d7752cae8 100644 --- a/flexmeasures/ui/templates/assets/asset_context.html +++ b/flexmeasures/ui/templates/assets/asset_context.html @@ -159,6 +159,7 @@
Edit {{ asset.name }}'s flex-context
+
@@ -335,6 +336,118 @@
Edit {{ asset.name }}'s flex-context
for (const [key, value] of Object.entries(schemaSpecs)) { assetFlexContextSchema[key] = value["default"]; } + + // Sensor-only fields offer no "Fixed value" tab in the editor. + // typeTwo = a single sensor reference, typeFour = a list of sensors. + function isSensorOnlyField(fieldName) { + const backendType = schemaSpecs[fieldName]?.["types"]?.["backend"]; + return backendType === "typeTwo" || backendType === "typeFour"; + } + + // ============== Commodity scope BEGIN ============== // + // The editor either works on the top-level flex-context (the electricity + // commodity; activeCommodityIndex === null) or on one entry of the + // "commodities" list (activeCommodityIndex is that entry's index). + let activeCommodityIndex = null; + + // Fields managed by the commodity tab bar rather than as cards. + const structuralFields = ["commodities", "commodity"]; + + function getActiveContext(assetFlexContext) { + if (activeCommodityIndex === null) { + return assetFlexContext; + } + return assetFlexContext["commodities"][activeCommodityIndex]; + } + + function isFieldAvailableInScope(fieldName) { + if (structuralFields.includes(fieldName)) { + return false; + } + if (activeCommodityIndex === null) { + return true; + } + return schemaSpecs[fieldName]?.["per-commodity"] === true; + } + + function switchCommodityScope(newIndex) { + activeCommodityIndex = newIndex; + setActiveCard(null); + localStorage.removeItem('activeCard'); + flexOptionsContainer.innerHTML = ""; + senSearchResEle.innerHTML = ""; + handleFlexSelectChange(null); + renderFlexContextForm(); + } + + function renderCommodityTabs() { + const commodityTabsEle = document.getElementById("commodityTabs"); + commodityTabsEle.innerHTML = ""; + const assetFlexContext = getFlexContext(); + const commodityContexts = assetFlexContext["commodities"] || []; + + function addPill(label, isActive, onClick, removeHandler = null) { + const li = document.createElement("li"); + li.className = "nav-item"; + const a = document.createElement("a"); + a.className = `nav-link py-1 px-2 ${isActive ? "active" : ""}`; + a.role = "button"; + a.textContent = label; + a.onclick = onClick; + if (removeHandler) { + const closeIcon = document.createElement('i'); + closeIcon.className = 'fa fa-times ps-2'; + closeIcon.setAttribute('data-bs-toggle', 'tooltip'); + closeIcon.setAttribute('title', 'Remove this commodity'); + closeIcon.style.cursor = 'pointer'; + closeIcon.onclick = (event) => { + event.stopPropagation(); + removeHandler(); + }; + a.appendChild(closeIcon); + } + li.appendChild(a); + commodityTabsEle.appendChild(li); + } + + // The top-level flex-context describes the electricity commodity. + addPill("electricity", activeCommodityIndex === null, () => switchCommodityScope(null)); + + commodityContexts.forEach((commodityContext, index) => { + addPill( + commodityContext["commodity"] || `commodity ${index + 1}`, + activeCommodityIndex === index, + () => switchCommodityScope(index), + () => { + const flexContext = getFlexContext(); + flexContext["commodities"].splice(index, 1); + if (flexContext["commodities"].length === 0) { + flexContext["commodities"] = null; + } + switchCommodityScope(null); + setFlexContext(flexContext); + }, + ); + }); + + addPill("+ commodity", false, () => { + const name = window.prompt("Name of the commodity to add (e.g. gas, heat):"); + if (!name || !name.trim()) { + return; + } + const commodityName = name.trim().toLowerCase(); + const flexContext = getFlexContext(); + const existingContexts = flexContext["commodities"] || []; + if (commodityName === "electricity" || existingContexts.some(c => c["commodity"] === commodityName)) { + showToast(`A flex-context for '${commodityName}' already exists`, "error"); + return; + } + flexContext["commodities"] = [...existingContexts, { "commodity": commodityName }]; + switchCommodityScope(flexContext["commodities"].length - 1); + setFlexContext(flexContext); + }); + } + // ============== Commodity scope END ============== // const [assetFlexContextRawJSON, extraFields] = processResourceRawJSON({ ...assetFlexContextSchema }, "{{ asset.flex_context | safe }}", true); const spinnerElement = document.getElementById('spinnerElement'); @@ -371,7 +484,7 @@
Edit {{ asset.name }}'s flex-context
return; } const assetFlexContext = getFlexContext(); - assetFlexContext[fieldName] = ""; + getActiveContext(assetFlexContext)[fieldName] = ""; flexOptionsContainer.innerHTML = ""; setFlexContext(assetFlexContext); @@ -380,9 +493,8 @@
Edit {{ asset.name }}'s flex-context
setTimeout(() => { const card = document.getElementById(`${fieldName}-control`); - const isOnlySensorField = (fieldName === "inflexible-device-sensors" || fieldName === "consumption-price" || fieldName === "production-price" || fieldName === "aggregate-power" || fieldName === "aggregate-consumption" || fieldName === "aggregate-production"); card.classList.add('border-on-click'); // Add border to the clicked card - flexOptionsContainer.appendChild(renderFlexInputOptions(fieldName, isOnlySensorField)); + flexOptionsContainer.appendChild(renderFlexInputOptions(fieldName, isSensorOnlyField(fieldName))); handleFlexSelectChange(fieldName); setActiveCard(card); @@ -395,22 +507,33 @@
Edit {{ asset.name }}'s flex-context
const apiURL = apiBasePath + "/api/v3_0/assets/{{ asset.id }}"; let data = getFlexContext(); - // Take out keys thats are not in the specs - for (const key in data) { - if (!assetFlexContextSchema.hasOwnProperty(key)) { - delete data[key]; + function cleanContext(context, isCommodityContext) { + // Take out keys that are not in the specs + for (const key in context) { + if (isCommodityContext && key === "commodity") { + continue; + } + if (!assetFlexContextSchema.hasOwnProperty(key)) { + delete context[key]; + } } - } - // Clean null values - for (const [key, value] of Object.entries(data)) { + // Clean null values + for (const [key, value] of Object.entries(context)) { + if ( + value === null || + value === "" || + (Array.isArray(value) && value.length === 0) + ) { + delete context[key]; + } + } + } - if ( - value === null || - value === "" || - (Array.isArray(value) && value.length === 0) - ) { - delete data[key]; + cleanContext(data, false); + if (data["commodities"]) { + for (const commodityContext of data["commodities"]) { + cleanContext(commodityContext, true); } } @@ -487,35 +610,35 @@
Edit {{ asset.name }}'s flex-context
async function removeFlexContextField(fieldName) { const assetFlexContext = getFlexContext(); - getFlexContext()[fieldName] = null; + getActiveContext(assetFlexContext)[fieldName] = null; setFlexContext(assetFlexContext); console.log(`Removed field ${fieldName} from flex context`); } async function removeFlexContextSensorValue(fieldName, sensorId) { - const assetFlexContext = getFlexContext(); + const activeContext = getActiveContext(getFlexContext()); if (sensorId) { - const fieldValue = assetFlexContext[fieldName]; + const fieldValue = activeContext[fieldName]; const isArray = Array.isArray(fieldValue); const isKeyValueObject = typeof fieldValue === "object" && fieldValue !== null && !Array.isArray(fieldValue) && // Exclude arrays Object.keys(fieldValue).length > 0; if (isArray) { - const sensorIndex = assetFlexContext[fieldName].indexOf(sensorId); - assetFlexContext[fieldName].splice(sensorIndex, 1); + const sensorIndex = activeContext[fieldName].indexOf(sensorId); + activeContext[fieldName].splice(sensorIndex, 1); } else if (isKeyValueObject) { - assetFlexContext[fieldName] = ""; + activeContext[fieldName] = ""; } } else { - assetFlexContext[fieldName] = null; + activeContext[fieldName] = null; } } async function renderFlexContextCard(fieldName) { - const assetFlexContext = getFlexContext(); + const activeContext = getActiveContext(getFlexContext()); senSearchResEle.innerHTML = ""; - let fieldValue = assetFlexContext[fieldName] + let fieldValue = activeContext[fieldName] let newFieldValue; let isSensorValue = false; const sensorsContent = []; @@ -541,7 +664,15 @@
Edit {{ asset.name }}'s flex-context
!Array.isArray(fieldValue) && // Exclude arrays Object.keys(fieldValue).length > 0; - if (isKeyValueObject) { + if (fieldName === "commitments" && isArray) { + // Commitments are structured entries; summarize them by name. + fieldValue.forEach((commitment) => { + const item = document.createElement('div'); + item.className = 'pb-1 fw-light'; + item.textContent = `${commitment["name"] || "(unnamed)"} (${commitment["commodity"] || "electricity"})`; + sensorsContent.push(item); + }); + } else if (isKeyValueObject) { const item = convertHtmlToElement(await renderSensor(fieldValue["sensor"])); sensorsContent.push(item); } else if (isArray) { @@ -590,7 +721,7 @@
Edit {{ asset.name }}'s flex-context
card.onclick = function () { if (activeCard() && activeCard().id === card.id) { - if (fieldName !== "inflexible-device-sensors") { + if (fieldName !== "inflexible-device-sensors" && fieldName !== "commitments") { setActiveCard(null); // Deselect if the same card is clicked again // remove active card from localstorage localStorage.removeItem('activeCard'); @@ -608,8 +739,7 @@
Edit {{ asset.name }}'s flex-context
document.querySelectorAll(".card-highlight").forEach(el => el.classList.remove("border-on-click")); // Remove border from all cards card.classList.add("border-on-click"); // Add border to the clicked card handleFlexSelectChange(fieldName); - const isOnlySensorField = (fieldName === "inflexible-device-sensors" || fieldName === "consumption-price" || fieldName === "production-price" || fieldName === "aggregate-power" || fieldName === "aggregate-consumption" || fieldName === "aggregate-production"); - flexOptionsContainer.appendChild(renderFlexInputOptions(fieldName, (isOnlySensorField))); + flexOptionsContainer.appendChild(renderFlexInputOptions(fieldName, isSensorOnlyField(fieldName))); setActiveCard(card); // Store active card in local storage localStorage.setItem('activeCard', fieldName); @@ -632,7 +762,8 @@
Edit {{ asset.name }}'s flex-context
closeIcon.onclick = (event) => { event.stopPropagation(); // prevent click from reaching the card - assetFlexContext[fieldName] = null; + const assetFlexContext = getFlexContext(); + getActiveContext(assetFlexContext)[fieldName] = null; card.remove(); if (activeCard() && activeCard().id === card.id) { flexOptionsContainer.innerHTML = ""; @@ -709,9 +840,11 @@
Edit {{ asset.name }}'s flex-context
const storedActiveCard = localStorage.getItem("activeCard") const assetFlexContext = getFlexContext(); + renderCommodityTabs(); + const activeContext = getActiveContext(assetFlexContext); - for (const [key, value] of Object.entries(assetFlexContext)) { - if (value !== null) { + for (const [key, value] of Object.entries(activeContext)) { + if (value !== null && !structuralFields.includes(key)) { const fieldCard = await renderFlexContextCard(key); flexContextFormEle.appendChild(fieldCard); } @@ -719,11 +852,10 @@
Edit {{ asset.name }}'s flex-context
if (storedActiveCard && activeCard()) { const flexId = (activeCard() ? activeCard().id : null).slice(0, -8); - if (assetFlexContext[storedActiveCard]) { - const isOnlySensorField = (flexId === "inflexible-device-sensors" || flexId === "consumption-price" || flexId === "production-price" || flexId === "aggregate-power" || flexId === "aggregate-consumption" || flexId === "aggregate-production"); + if (activeContext[storedActiveCard]) { setTimeout(() => { flexOptionsContainer.innerHTML = ""; - flexOptionsContainer.appendChild(renderFlexInputOptions(flexId, isOnlySensorField)); + flexOptionsContainer.appendChild(renderFlexInputOptions(flexId, isSensorOnlyField(flexId))); handleFlexSelectChange(flexId); activeCard().classList.add('border-on-click'); // Add border to the clicked card const renderedActiveCard = document.getElementById(`${flexId}-control`); @@ -755,6 +887,7 @@
Edit {{ asset.name }}'s flex-context
// get card id and remove '-control' at the end const cardId = highlightedCard.id; const flexId = cardId.slice(0, -8); + const activeContext = getActiveContext(assetFlexContext); if (dataType === "fixedValue") { const powerCapacityInputValue = document.getElementById("fixed-value").value; // check if value is valid @@ -762,16 +895,25 @@
Edit {{ asset.name }}'s flex-context
showToast("Please enter a value", "error"); return; } - assetFlexContext[flexId] = powerCapacityInputValue; + activeContext[flexId] = powerCapacityInputValue; } else if (dataType === "sensor" && sensorId) { + if (flexId === "commitments" && pendingCommitmentTarget) { + setCommitmentField( + pendingCommitmentTarget.index, + pendingCommitmentTarget.key, + { sensor: sensorId }, + ); + pendingCommitmentTarget = null; + return; + } if (flexId === "inflexible-device-sensors") { - if (assetFlexContext[flexId]) { - assetFlexContext[flexId].push(sensorId); + if (activeContext[flexId]) { + activeContext[flexId].push(sensorId); } else { - assetFlexContext[flexId] = [sensorId]; + activeContext[flexId] = [sensorId]; } } else { - assetFlexContext[flexId] = { sensor: sensorId }; + activeContext[flexId] = { sensor: sensorId }; } } @@ -784,22 +926,288 @@
Edit {{ asset.name }}'s flex-context
} function renderFlexSelectOptions() { - // comapare the assetFlexContext with the template and get the fields that are not set - const assetFlexContextTemplate = Object.assign({}, assetFlexContextSchema, getFlexContext()); + // compare the active context with the template and get the fields that are not set + const assetFlexContextTemplate = Object.assign({}, assetFlexContextSchema, getActiveContext(getFlexContext())); flexSelect.innerHTML = ``; for (const [key, value] of Object.entries(assetFlexContextTemplate)) { - if (value === null || value.length === 0) { + if ((value === null || value.length === 0) && isFieldAvailableInScope(key)) { flexSelect.innerHTML += ``; } } } + function createTooltipIcon(title) { + const icon = document.createElement("i"); + icon.className = "fa fa-info-circle ps-2"; + icon.setAttribute("data-bs-toggle", "tooltip"); + icon.setAttribute("data-bs-placement", "bottom"); + icon.title = title; + return icon; + } + + // Build the sensor search block (search input, units filter, public-assets + // checkbox), labeled for the given target. + function createSensorSearchBlockFor(targetLabel) { + const group = document.createElement("div"); + group.id = "select-sensor-input"; + group.className = "flex-input-group p-2"; + + // Label + const label = document.createElement("label"); + label.htmlFor = "sensor"; + label.className = "form-label"; + label.innerHTML = `Look up Sensor for ${targetLabel} `; + label.appendChild(createTooltipIcon( + `Search for sensor(s) to add here. All sensors on the same site as this asset (including parent- and sibling assets) are included by default. +Make sure that the sensor data will be available consistently, though, so scheduling will work.` + )); + group.appendChild(label); + + // Checkbox group + const formCheck = document.createElement("div"); + formCheck.className = "form-check"; + + const checkbox = document.createElement("input"); + checkbox.className = "form-check-input"; + checkbox.type = "checkbox"; + checkbox.value = ""; + checkbox.id = "flexCheckBox"; + checkbox.onchange = searchSensorsFlexContextForm; + + + const checkLabel = document.createElement("label"); + checkLabel.className = "form-check-label"; + checkLabel.htmlFor = "flexCheckBox"; + checkLabel.textContent = "Include public assets"; + + formCheck.appendChild(checkbox); + formCheck.appendChild(checkLabel); + group.appendChild(formCheck); + + // Row with search + units + const row = document.createElement("div"); + row.className = "row"; + + const col8 = document.createElement("div"); + col8.className = "col-8 pe-1"; + const searchInput = document.createElement("input"); + searchInput.type = "text"; + searchInput.id = "flexContextSensorSearch"; + searchInput.className = "form-control"; + searchInput.placeholder = "Search sensors..."; + searchInput.oninput = searchSensorsFlexContextForm; + searchInput.onkeydown = function (event) { + if (event.key === 'Enter') { + event.preventDefault(); // Prevent form submission + searchSensorsFlexContextForm(); + } + }; + + col8.appendChild(searchInput); + + const col4 = document.createElement("div"); + col4.className = "col-4 ps-0"; + + // Units select - START + const select = document.createElement("select"); + select.id = "flexUnitsSelect"; + select.className = "form-select"; + select.onchange = searchSensorsFlexContextForm; + + const defaultOption = document.createElement("option"); + defaultOption.value = null; + defaultOption.selected = true; + defaultOption.textContent = "Units"; + select.appendChild(defaultOption); + + availableUnits.forEach(unit => { + const option = document.createElement("option"); + option.value = unit; + option.textContent = unit; + select.appendChild(option); + }); + col4.appendChild(select); + // Units select - END + + + row.appendChild(col8); + row.appendChild(col4); + group.appendChild(row); + + return group; + } + + // ============== Commitments editor BEGIN ============== // + // When a sensor is picked while this target is set, the sensor reference is + // written into that commitment's field instead of replacing the whole card value. + let pendingCommitmentTarget = null; + + function setCommitmentField(commitmentIndex, key, value) { + const assetFlexContext = getFlexContext(); + const activeContext = getActiveContext(assetFlexContext); + activeContext["commitments"][commitmentIndex][key] = value; + localStorage.setItem('activeCard', 'commitments'); + setFlexContext(assetFlexContext); + } + + function describeCommitmentValue(value) { + if (value === undefined || value === null || value === "") { + return "not set"; + } + if (typeof value === "object" && value["sensor"]) { + return `sensor ${value["sensor"]}`; + } + return String(value); + } + + function renderCommitmentsEditor() { + const container = document.createElement("div"); + container.id = "flex-input-options"; + container.className = "card-header"; + + const activeContext = getActiveContext(getFlexContext()); + const commitments = Array.isArray(activeContext["commitments"]) ? activeContext["commitments"] : []; + + const intro = document.createElement("p"); + intro.className = "form-label"; + intro.innerHTML = "Each commitment needs a name and a baseline, plus at least one deviation price. " + + "Values can be fixed quantities (e.g. '100 kW', '35 EUR/MWh') or a sensor. Further checks happen on save."; + container.appendChild(intro); + + commitments.forEach((commitment, index) => { + const group = document.createElement("div"); + group.className = "card p-2 mb-2"; + + // Header row: name + commodity + remove + const headerRow = document.createElement("div"); + headerRow.className = "row g-2 align-items-end pb-2"; + + function textInput(labelText, key, placeholder, colClass) { + const col = document.createElement("div"); + col.className = colClass; + const label = document.createElement("label"); + label.className = "form-label mb-0"; + label.textContent = labelText; + const input = document.createElement("input"); + input.type = "text"; + input.className = "form-control form-control-sm"; + input.placeholder = placeholder; + input.value = commitment[key] || ""; + input.onchange = () => setCommitmentField(index, key, input.value); + col.appendChild(label); + col.appendChild(input); + return col; + } + + headerRow.appendChild(textInput("Name", "name", "e.g. capacity contract", "col-5")); + headerRow.appendChild(textInput("Commodity", "commodity", "electricity", "col-4")); + + const removeCol = document.createElement("div"); + removeCol.className = "col-3 text-end"; + const removeBtn = document.createElement("button"); + removeBtn.className = "btn btn-sm btn-outline-danger"; + removeBtn.textContent = "Remove"; + removeBtn.onclick = () => { + const assetFlexContext = getFlexContext(); + const context = getActiveContext(assetFlexContext); + context["commitments"].splice(index, 1); + if (context["commitments"].length === 0) { + context["commitments"] = null; + } + localStorage.setItem('activeCard', 'commitments'); + setFlexContext(assetFlexContext); + }; + removeCol.appendChild(removeBtn); + headerRow.appendChild(removeCol); + group.appendChild(headerRow); + + // Quantity fields: fixed value or sensor + ["baseline", "up-price", "down-price"].forEach((key) => { + const row = document.createElement("div"); + row.className = "row g-2 align-items-center pb-1"; + + const labelCol = document.createElement("div"); + labelCol.className = "col-3"; + labelCol.innerHTML = `
${describeCommitmentValue(commitment[key])}
`; + row.appendChild(labelCol); + + const inputCol = document.createElement("div"); + inputCol.className = "col-4"; + const input = document.createElement("input"); + input.type = "text"; + input.className = "form-control form-control-sm"; + input.placeholder = key === "baseline" ? "e.g. 100 kW" : "e.g. 35 EUR/MWh"; + input.onkeydown = (event) => { + if (event.key === 'Enter') { + event.preventDefault(); + if (input.value !== "") setCommitmentField(index, key, input.value); + } + }; + inputCol.appendChild(input); + row.appendChild(inputCol); + + const buttonsCol = document.createElement("div"); + buttonsCol.className = "col-5"; + const useBtn = document.createElement("button"); + useBtn.className = "btn btn-sm btn-secondary me-1"; + useBtn.textContent = "Use value"; + useBtn.onclick = () => { + if (input.value === "") { + showToast("Please enter a value", "error"); + return; + } + setCommitmentField(index, key, input.value); + }; + const sensorBtn = document.createElement("button"); + sensorBtn.className = "btn btn-sm btn-outline-secondary"; + sensorBtn.textContent = "Pick sensor"; + sensorBtn.onclick = () => { + pendingCommitmentTarget = { index: index, key: key }; + const existingSearch = document.getElementById("select-sensor-input"); + if (existingSearch) { + existingSearch.remove(); + } + container.appendChild(createSensorSearchBlockFor(`commitment '${commitment["name"] || index}' ${key}`)); + }; + buttonsCol.appendChild(useBtn); + buttonsCol.appendChild(sensorBtn); + row.appendChild(buttonsCol); + + group.appendChild(row); + }); + + container.appendChild(group); + }); + + const addBtn = document.createElement("button"); + addBtn.className = "btn btn-secondary btn-sm"; + addBtn.textContent = "Add commitment"; + addBtn.onclick = () => { + const assetFlexContext = getFlexContext(); + const context = getActiveContext(assetFlexContext); + const existing = Array.isArray(context["commitments"]) ? context["commitments"] : []; + context["commitments"] = [...existing, { "name": "", "commodity": "electricity" }]; + localStorage.setItem('activeCard', 'commitments'); + setFlexContext(assetFlexContext); + }; + container.appendChild(addBtn); + + return container; + } + // ============== Commitments editor END ============== // + function renderFlexInputOptions(contextKey, sensorsOnly = false) { if (!contextKey) { flexInfoContainer.innerHTML = ""; return; } + if (contextKey === "commitments") { + pendingCommitmentTarget = null; + return renderCommitmentsEditor(); + } + pendingCommitmentTarget = null; + const tabsContainer = document.createElement("div"); tabsContainer.id = "flex-input-options" tabsContainer.className = "card-header"; @@ -809,15 +1217,6 @@
Edit {{ asset.name }}'s flex-context
ul.id = "pills-tab"; ul.setAttribute("role", "tablist"); - function createTooltipIcon(title) { - const icon = document.createElement("i"); - icon.className = "fa fa-info-circle ps-2"; - icon.setAttribute("data-bs-toggle", "tooltip"); - icon.setAttribute("data-bs-placement", "bottom"); - icon.title = title; - return icon; - } - if (sensorsOnly) { // Only the dynamic value tab const li = document.createElement("li"); @@ -879,95 +1278,9 @@
Edit {{ asset.name }}'s flex-context
tabsContainer.appendChild(ul); - // A helper function to create the sensor search block + // A helper function to create the sensor search block for this field function createSensorSearchBlock() { - const group = document.createElement("div"); - group.id = "select-sensor-input"; - group.className = "flex-input-group p-2"; - - // Label - const label = document.createElement("label"); - label.htmlFor = "sensor"; - label.className = "form-label"; - label.innerHTML = `Look up Sensor for ${getFlexFieldTitle(contextKey)} `; - label.appendChild(createTooltipIcon( - `Search for sensor(s) to add here. All sensors on the same site as this asset (including parent- and sibling assets) are included by default. -Make sure that the sensor data will be available consistently, though, so scheduling will work.` - )); - group.appendChild(label); - - // Checkbox group - const formCheck = document.createElement("div"); - formCheck.className = "form-check"; - - const checkbox = document.createElement("input"); - checkbox.className = "form-check-input"; - checkbox.type = "checkbox"; - checkbox.value = ""; - checkbox.id = "flexCheckBox"; - checkbox.onchange = searchSensorsFlexContextForm; - - - const checkLabel = document.createElement("label"); - checkLabel.className = "form-check-label"; - checkLabel.htmlFor = "flexCheckBox"; - checkLabel.textContent = "Include public assets"; - - formCheck.appendChild(checkbox); - formCheck.appendChild(checkLabel); - group.appendChild(formCheck); - - // Row with search + units - const row = document.createElement("div"); - row.className = "row"; - - const col8 = document.createElement("div"); - col8.className = "col-8 pe-1"; - const searchInput = document.createElement("input"); - searchInput.type = "text"; - searchInput.id = "flexContextSensorSearch"; - searchInput.className = "form-control"; - searchInput.placeholder = "Search sensors..."; - searchInput.oninput = searchSensorsFlexContextForm; - searchInput.onkeydown = function (event) { - if (event.key === 'Enter') { - event.preventDefault(); // Prevent form submission - searchSensorsFlexContextForm(); - } - }; - - col8.appendChild(searchInput); - - const col4 = document.createElement("div"); - col4.className = "col-4 ps-0"; - - // Units select - START - const select = document.createElement("select"); - select.id = "flexUnitsSelect"; - select.className = "form-select"; - select.onchange = searchSensorsFlexContextForm; - - const defaultOption = document.createElement("option"); - defaultOption.value = null; - defaultOption.selected = true; - defaultOption.textContent = "Units"; - select.appendChild(defaultOption); - - availableUnits.forEach(unit => { - const option = document.createElement("option"); - option.value = unit; - option.textContent = unit; - select.appendChild(option); - }); - col4.appendChild(select); - // Units select - END - - - row.appendChild(col8); - row.appendChild(col4); - group.appendChild(row); - - return group; + return createSensorSearchBlockFor(getFlexFieldTitle(contextKey)); } // Create the main tab-content wrapper @@ -1011,7 +1324,7 @@
Edit {{ asset.name }}'s flex-context
fixedInput.type = "text"; fixedInput.id = "fixed-value"; fixedInput.className = "form-control"; - fixedInput.value = (typeof getFlexContext()[contextKey] === "string") ? getFlexContext()[contextKey] : ""; + fixedInput.value = (typeof getActiveContext(getFlexContext())[contextKey] === "string") ? getActiveContext(getFlexContext())[contextKey] : ""; fixedInput.placeholder = "e.g. 15000 kW"; fixedInput.onkeydown = function (event) { if (event.key === 'Enter') { @@ -1062,6 +1375,7 @@
Edit {{ asset.name }}'s flex-context
flexContexFormModal.addEventListener('hidden.bs.modal', function () { localStorage.removeItem('activeCard'); + activeCommodityIndex = null; flexOptionsContainer.innerHTML = ""; handleFlexSelectChange(null); }); diff --git a/flexmeasures/ui/tests/test_utils.py b/flexmeasures/ui/tests/test_utils.py index b9925823b7..acdb742710 100644 --- a/flexmeasures/ui/tests/test_utils.py +++ b/flexmeasures/ui/tests/test_utils.py @@ -79,7 +79,6 @@ def test_ui_flexcontext_schema(): "relax-site-capacity-constraints", "consumption-price-sensor", "production-price-sensor", - "commodities", # todo: https://github.com/FlexMeasures/flexmeasures/issues/2230 "commodity", # single-dict form is electricity-only; not exposed in the UI ] @@ -99,6 +98,25 @@ def test_ui_flexcontext_schema(): ), "If this fails, you may have added UI support for a new flex-context field, but forgot to remove it from exclude_fields." +def test_ui_flexcontext_schema_per_commodity_flags(): + """The context editor relies on each UI schema entry telling whether the field + can also be set within a commodity context (an entry of the commodities list).""" + from flexmeasures.data.schemas.scheduling import CommodityFlexContextSchema + + commodity_context_keys = { + schema_field.data_key or field_name + for field_name, schema_field in CommodityFlexContextSchema().fields.items() + } + for field_name, entry in UI_FLEX_CONTEXT_SCHEMA.items(): + assert entry["per-commodity"] == (field_name in commodity_context_keys), ( + f"UI schema entry '{field_name}' has a per-commodity flag that contradicts " + "CommodityFlexContextSchema." + ) + # The commodities list itself cannot be nested inside a commodity context, + # and the editor manages it through the commodity tab bar. + assert UI_FLEX_CONTEXT_SCHEMA["commodities"]["per-commodity"] is False + + def test_ui_flexmodel_schema(): """ This test ensures that all fields in the DBStorageFlexModelSchema are also in the UI schema and vice versa. diff --git a/tests/documentation/test_schemas.py b/tests/documentation/test_schemas.py index fab2946647..b6599b3975 100644 --- a/tests/documentation/test_schemas.py +++ b/tests/documentation/test_schemas.py @@ -10,6 +10,8 @@ # Metadata constants that intentionally do not appear in the documentation EXCLUDED_METADATA = { + "COMMODITY_FLEX_CONTEXT", # appears as `commodity` in the flex-context listing in scheduling.rst + "COMMODITY_FLEX_MODEL", # appears as `commodity` in the flex-model listing in scheduling.rst "RELAX_CAPACITY_CONSTRAINTS", "RELAX_SITE_CAPACITY_CONSTRAINTS", "RELAX_SOC_CONSTRAINTS",