[FR] Custom Metrics in Scenario Analysis

[Lucasmogsan]

Proposal Summary

Custom EvaluationMetric plugin aggregates should be recomputed when a Scenario Analysis is created in the App.

Motivation

Custom metrics (e.g. pixel-level keypoint distance) correctly populate the model evaluation, but is not adjusting to new Scenario Analysis. This makes Scenario Analysis (in app) unusable for custom metrics.

What areas of FiftyOne does this feature affect?

• App: FiftyOne application
• Core: Core fiftyone Python library
• Server: FiftyOne server

Willingness to contribute

The FiftyOne Community welcomes contributions! Would you or another member of your organization be willing to contribute an implementation of this feature?

• Yes. I can contribute this feature independently
• Yes. I would be willing to contribute this feature with guidance from the FiftyOne community
• No. I cannot contribute this feature at this time

[avithal]

Yes. I would be willing to contribute this feature with guidance from the FiftyOne community