Add measures to metrics (temporary) & fix metric deduplication (#436)

Towards #387

### Description

This PR contains a few small changes and improvements, all grouped up in the process of getting to a point where metricflow can onboard onto the new dsi version immediately.

At a high level, these changes take three forms.  I've tried to detail in comments throughout the PR, but here as well:
* general clean up or improvements that came up along the way
* fixing the equality checking when searching for matching metrics (to avoid creating duplicates).
  * this is a correctness issue here in this repo.  It can arise when we have a metric created from a measure with create_metric=true and then also create one for inputs.  (This was found in some of the "parse-the-entire-project" tests in metricflow.)
* adding `measure`s and sometimes `input_measures` to some of the new metrics made in transformations.  This is only necessary because the upgrades in MF are not yet complete and some operations still throw errors when encountering metrics without input measures (especially in tests :/ ), and these new lines can be removed in a few weeks when everything is ready.

I've also tested this by pointing the metricflow repo at the top commit here to make sure `make test` is running alright since the latest dsi version would not work in MF before.

### Checklist

- [x] I have read [the contributing guide](https://github.com/dbt-labs/dbt-semantic-interfaces/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [x] I have signed the [CLA](https://docs.getdbt.com/docs/contributor-license-agreements)
- [x] This PR includes tests, or tests are not required/relevant for this PR
- [x] I have run `changie new` to [create a changelog entry](https://github.com/dbt-labs/dbt-semantic-interfaces/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)

Author: theyostalservice

.changes/unreleased/Fixes-20251002-112637.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Fix transformation metric de-duplication and add measures to new metrics to improve metricflow compatibility.
+time: 2025-10-02T11:26:37.069417-07:00
+custom:
+    Author: theyostalservice
+    Issue: "387"

dbt_semantic_interfaces/implementations/metric.py

@@ -347,10 +347,14 @@ def build_metric_aggregation_params(
 
         It lives here instead of measures to avoid circular import issues.
         """
+        agg_params = measure.agg_params.copy(deep=True) if measure.agg_params is not None else None
+        non_additive_dimension = (
+            measure.non_additive_dimension.copy(deep=True) if measure.non_additive_dimension is not None else None
+        )
         return PydanticMetricAggregationParams(
             semantic_model=semantic_model_name,
             agg=measure.agg,
-            agg_params=measure.agg_params,
+            agg_params=agg_params,
             agg_time_dimension=measure.agg_time_dimension,
-            non_additive_dimension=measure.non_additive_dimension,
+            non_additive_dimension=non_additive_dimension,
         )

dbt_semantic_interfaces/transformations/add_input_metric_measures.py

@@ -31,9 +31,7 @@ def _get_measures_for_metric(
     ) -> Set[PydanticMetricInputMeasure]:
         """Returns a unique set of input measures for a given metric."""
         measures: Set = set()
-        matched_metric = next(
-            iter((metric for metric in semantic_manifest.metrics if metric.name == metric_name)), None
-        )
+        matched_metric = next((metric for metric in semantic_manifest.metrics if metric.name == metric_name), None)
         if matched_metric:
             if matched_metric.type is MetricType.SIMPLE or matched_metric.type is MetricType.CUMULATIVE:
                 if matched_metric.type_params.measure is not None:

dbt_semantic_interfaces/transformations/measure_to_metric_transformation_pieces/measure_features_to_metric_name.py

@@ -3,6 +3,7 @@
 from dbt_semantic_interfaces.implementations.elements.measure import PydanticMeasure
 from dbt_semantic_interfaces.implementations.metric import (
     PydanticMetric,
+    PydanticMetricInputMeasure,
     PydanticMetricTypeParams,
 )
 from dbt_semantic_interfaces.implementations.semantic_manifest import (
@@ -52,7 +53,7 @@ def _store_metric_name(
         key = (measure_name, fill_nulls_with, join_to_timespine)
         self._metric_name_dict[key] = metric_name
 
-    def _find_metric_clone_in_manifest(
+    def _find_simple_metric_functional_clone_in_manifest(
         self,
         metric: PydanticMetric,
         manifest: PydanticSemanticManifest,
@@ -61,20 +62,36 @@ def _find_metric_clone_in_manifest(
 
         returns the metric if it exists, otherwise None
 
-        Note: this can be further optimized by pre-caching metrics based on features,
-        but let's not prematurely optimize.
+        Note: this is appropriate for SIMPLE metrics that would **replace a measure** in
+        the new YAML.  This code would require updates and expansion to handle anything beyond that.
         """
-        search_metric = metric.copy(deep=True)
+
+        def _metrics_equivalent(search_metric: PydanticMetric, manifest_metric: PydanticMetric) -> bool:
+            """Check if the given metric and manifest_metric are equivalent based on selected fields."""
+            fields_match = (
+                search_metric.type == manifest_metric.type
+                and search_metric.type_params.window == manifest_metric.type_params.window
+                and search_metric.type_params.grain_to_date == manifest_metric.type_params.grain_to_date
+                and search_metric.type_params.metric_aggregation_params
+                == manifest_metric.type_params.metric_aggregation_params
+                and search_metric.type_params.join_to_timespine == manifest_metric.type_params.join_to_timespine
+                and search_metric.type_params.fill_nulls_with == manifest_metric.type_params.fill_nulls_with
+                and search_metric.type_params.expr == manifest_metric.type_params.expr
+                and search_metric.filter == manifest_metric.filter
+                and search_metric.time_granularity == manifest_metric.time_granularity
+            )
+            if not fields_match:
+                return False
+            if (
+                manifest_metric.type_params.measure is not None
+                and search_metric.type_params.measure != manifest_metric.type_params.measure
+            ):
+                return False
+            return True
+
         for existing_metric in manifest.metrics:
-            # this allows us to a straight equality comparison, which is safer in the future
-            # than implementing a custom comparison function.
-            search_metric.name = existing_metric.name
-            search_metric.metadata = existing_metric.metadata
-            search_metric.type_params.is_private = existing_metric.type_params.is_private
-            if search_metric == existing_metric:
+            if _metrics_equivalent(search_metric=metric, manifest_metric=existing_metric):
                 return existing_metric
-            print("provided metric", search_metric)
-            print("existing metric", existing_metric)
         return None
 
     @staticmethod
@@ -83,7 +100,7 @@ def build_metric_from_measure_configuration(
         semantic_model_name: str,
         fill_nulls_with: Optional[int],
         join_to_timespine: Optional[bool],
-        is_private: bool = True,
+        is_private: bool,
     ) -> PydanticMetric:
         """Build a metric from the measure configuration.
 
@@ -176,8 +193,14 @@ def get_or_create_metric_for_measure(
             semantic_model_name=model_name,
             fill_nulls_with=fill_nulls_with,
             join_to_timespine=join_to_timespine,
+            is_private=True,
         )
-        metric = self._find_metric_clone_in_manifest(
+        # TODO SL-4257: this is supporting legacy cases in MF until work there is complete,
+        # and should be removeable long before the rest of the backward-compatibility work.
+        built_metric.type_params.measure = PydanticMetricInputMeasure(name=measure.name)
+        built_metric.type_params.input_measures = [PydanticMetricInputMeasure(name=measure.name)]
+
+        metric = self._find_simple_metric_functional_clone_in_manifest(
             metric=built_metric,
             manifest=manifest,
         )

tests/parsing/test_metric_parsing_with_custom_grain.py

@@ -48,6 +48,7 @@ def test_cumulative_metric_with_custom_grain_to_date() -> None:  # noqa: D
           type_params:
             measure:
               name: bookings
+              fill_nulls_with: 15
             cumulative_type_params:
               grain_to_date: martian_week
         """
@@ -121,9 +122,9 @@ def test_cumulative_metric_with_custom_window() -> None:  # noqa: D
     )
     assert not model.issues.has_blocking_issues
     semantic_manifest = model.semantic_manifest
-    # 2 explicit ones and one that is created for the measure input for the
-    # cumulative metric's params
-    assert len(semantic_manifest.metrics) == 3
+    # 2 explicit metrics.  The cumulative metric's input metric should be deduplicated
+    # so it will match.
+    assert len(semantic_manifest.metrics) == 2
 
     metric = next((m for m in semantic_manifest.metrics if m.name == "test_cumulative_metric_with_custom_window"), None)
     assert metric is not None, "Can't find metric"
@@ -185,9 +186,9 @@ def test_conversion_metric_with_custom_grain_window() -> None:  # noqa: D
     )
     assert not model.issues.has_blocking_issues
     semantic_manifest = model.semantic_manifest
-    # 2 explicit ones and one that is created for the measure input for the
-    # cumulative metric's params
-    assert len(semantic_manifest.metrics) == 3
+    # 2 explicitly created metrics.  The conversion measure -> metric transformation
+    # should not need to create a new metric since the existing one already matches.
+    assert len(semantic_manifest.metrics) == 2
 
     metric = next(
         (m for m in semantic_manifest.metrics if m.name == "test_conversion_metric_with_custom_grain_window"), None

tests/transformations/test_e2e_measure_to_model_transformations.py

@@ -0,0 +1,221 @@
+from dbt_semantic_interfaces.implementations.elements.dimension import (
+    PydanticDimension,
+    PydanticDimensionTypeParams,
+)
+from dbt_semantic_interfaces.implementations.elements.entity import PydanticEntity
+from dbt_semantic_interfaces.implementations.elements.measure import PydanticMeasure
+from dbt_semantic_interfaces.implementations.metric import (
+    PydanticCumulativeTypeParams,
+    PydanticMetric,
+    PydanticMetricInput,
+    PydanticMetricInputMeasure,
+    PydanticMetricTimeWindow,
+    PydanticMetricTypeParams,
+)
+from dbt_semantic_interfaces.implementations.node_relation import PydanticNodeRelation
+from dbt_semantic_interfaces.implementations.project_configuration import (
+    PydanticProjectConfiguration,
+)
+from dbt_semantic_interfaces.implementations.semantic_manifest import (
+    PydanticSemanticManifest,
+)
+from dbt_semantic_interfaces.implementations.semantic_model import (
+    PydanticSemanticModel,
+    PydanticSemanticModelDefaults,
+)
+from dbt_semantic_interfaces.transformations.semantic_manifest_transformer import (
+    PydanticSemanticManifestTransformer,
+)
+from dbt_semantic_interfaces.type_enums import (
+    AggregationType,
+    DimensionType,
+    EntityType,
+    MetricType,
+    TimeGranularity,
+)
+from dbt_semantic_interfaces.validations.semantic_manifest_validator import (
+    SemanticManifestValidator,
+)
+
+
+def _project_config() -> PydanticProjectConfiguration:
+    return PydanticProjectConfiguration()
+
+
+def test_e2e_measure_create_metric_then_cumulative_uses_metric_input() -> None:
+    """End-to-end: measure create_metric=True, cumulative references created metric by name."""
+    sm_name = "sm"
+    time_dim_name = "ds"
+    sm = PydanticSemanticModel(
+        name=sm_name,
+        defaults=PydanticSemanticModelDefaults(agg_time_dimension=time_dim_name),
+        node_relation=PydanticNodeRelation(alias=sm_name, schema_name="schema"),
+        entities=[
+            PydanticEntity(
+                name="user",
+                type=EntityType.PRIMARY,
+                expr="user_id",
+            ),
+        ],
+        dimensions=[
+            PydanticDimension(
+                name="ds",
+                type=DimensionType.TIME,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.DAY),
+            ),
+            PydanticDimension(
+                name="created_at",
+                type=DimensionType.TIME,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.DAY),
+            ),
+            PydanticDimension(
+                name="ds_partitioned",
+                type=DimensionType.TIME,
+                is_partition=True,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.DAY),
+            ),
+            PydanticDimension(
+                name="home_state",
+                type=DimensionType.CATEGORICAL,
+            ),
+            PydanticDimension(
+                name="last_profile_edit_ts",
+                type=DimensionType.TIME,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.MILLISECOND),
+            ),
+            PydanticDimension(
+                name="bio_added_ts",
+                type=DimensionType.TIME,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.SECOND),
+            ),
+            PydanticDimension(
+                name="last_login_ts",
+                type=DimensionType.TIME,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.MINUTE),
+            ),
+            PydanticDimension(
+                name="archived_at",
+                type=DimensionType.TIME,
+                type_params=PydanticDimensionTypeParams(time_granularity=TimeGranularity.HOUR),
+            ),
+        ],
+        measures=[
+            PydanticMeasure(
+                name="archived_users",
+                agg=AggregationType.SUM,
+                expr="1",
+                create_metric=True,
+            )
+        ],
+    )
+
+    metrics = [
+        PydanticMetric(
+            name="subdaily_cumulative_window_metric",
+            type=MetricType.CUMULATIVE,
+            description="m1_cumulative_1 description",
+            type_params=PydanticMetricTypeParams(
+                measure=PydanticMetricInputMeasure(name="archived_users"),
+                cumulative_type_params=PydanticCumulativeTypeParams(
+                    window=PydanticMetricTimeWindow(count=3, granularity="hour"),
+                ),
+            ),
+        ),
+        PydanticMetric(
+            name="subdaily_cumulative_grain_to_date_metric",
+            type=MetricType.CUMULATIVE,
+            description="m1_cumulative_2 description",
+            type_params=PydanticMetricTypeParams(
+                measure=PydanticMetricInputMeasure(name="archived_users"),
+                cumulative_type_params=PydanticCumulativeTypeParams(
+                    grain_to_date="hour",
+                ),
+            ),
+        ),
+        PydanticMetric(
+            name="subdaily_offset_window_metric",
+            type=MetricType.DERIVED,
+            description="archived_users_offset_window description",
+            type_params=PydanticMetricTypeParams(
+                expr="archived_users",
+                metrics=[
+                    PydanticMetricInput(
+                        name="archived_users",
+                        offset_window=PydanticMetricTimeWindow(count=1, granularity="hour"),
+                    )
+                ],
+            ),
+        ),
+        PydanticMetric(
+            name="subdaily_offset_grain_to_date_metric",
+            type=MetricType.DERIVED,
+            description="offset grain to date metric with a sub-daily agg time dim",
+            type_params=PydanticMetricTypeParams(
+                expr="archived_users",
+                metrics=[
+                    PydanticMetricInput(
+                        name="archived_users",
+                        offset_to_grain="hour",
+                    )
+                ],
+            ),
+        ),
+        PydanticMetric(
+            name="subdaily_join_to_time_spine_metric",
+            type=MetricType.SIMPLE,
+            description="simple metric with sub-daily agg time dim that joins to time spine",
+            type_params=PydanticMetricTypeParams(
+                measure=PydanticMetricInputMeasure(
+                    name="archived_users",
+                    join_to_timespine=True,
+                ),
+            ),
+        ),
+        PydanticMetric(
+            name="simple_subdaily_metric_default_day",
+            type=MetricType.SIMPLE,
+            description="simple metric with sub-daily agg time dim that doesn't specify default granularity",
+            type_params=PydanticMetricTypeParams(
+                measure=PydanticMetricInputMeasure(
+                    name="archived_users",
+                ),
+            ),
+        ),
+        PydanticMetric(
+            name="simple_subdaily_metric_default_hour",
+            type=MetricType.SIMPLE,
+            description="simple metric with sub-daily agg time dim that has an explicit default granularity",
+            type_params=PydanticMetricTypeParams(
+                measure=PydanticMetricInputMeasure(
+                    name="archived_users",
+                ),
+            ),
+            time_granularity="hour",
+        ),
+        PydanticMetric(
+            name="archived_users_join_to_time_spine",
+            type=MetricType.SIMPLE,
+            description="subdaily metric joining to time spine",
+            type_params=PydanticMetricTypeParams(
+                measure=PydanticMetricInputMeasure(
+                    name="archived_users",
+                    join_to_timespine=True,
+                ),
+            ),
+        ),
+    ]
+
+    manifest = PydanticSemanticManifest(
+        semantic_models=[sm],
+        metrics=metrics,
+        project_configuration=_project_config(),
+    )
+
+    transformed = PydanticSemanticManifestTransformer.transform(model=manifest)
+
+    model_validator = SemanticManifestValidator[PydanticSemanticManifest]()
+    model_validator.checked_validations(transformed)
+
+    # Expect exactly 1 new metric - the proxy simple metric created for the measure
+    assert len(transformed.metrics) == len(metrics) + 1
+    assert any(m for m in transformed.metrics if m.type == MetricType.SIMPLE and m.name == "archived_users")