Skip to content

feat: add metadata field to scores #1153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

feat: add metadata field to scores #1153

wants to merge 7 commits into from

Conversation

davidmc971
Copy link

@davidmc971 davidmc971 commented Mar 25, 2025

Important

Add optional metadata field to score-related models and methods for enhanced contextual data support.

  • Models:
    • Add metadata field to ScoreBody in score_body.py.
    • Add metadata field to CreateScoreRequest in create_score_request.py and BaseScore in base_score.py.
    • Add metadata field to Score_Numeric, Score_Categorical, and Score_Boolean in score.py.
    • Add metadata field to responses in get_scores_response_data.py.
  • Client:
    • Update score() function in client.py to include metadata parameter.
  • Tests:
    • Update tests in test_core_sdk.py to include metadata in score creation and validation.

This description was created by Ellipsis for 876ecec. It will automatically update as commits are pushed.


Greptile Summary

Disclaimer: Experimental PR review

This PR uniformly introduces an optional metadata field across score-related models and methods for enhanced contextual data support.

  • Added metadata field to ScoreBody in /langfuse/api/resources/ingestion/types/score_body.py.
  • Integrated metadata into CreateScoreRequest in /langfuse/api/resources/score/types/create_score_request.py and BaseScore in /langfuse/api/resources/commons/types/base_score.py.
  • Updated Score_Numeric, Score_Categorical, and Score_Boolean in /langfuse/api/resources/commons/types/score.py and responses in /langfuse/api/resources/score/types/get_scores_response_data.py.
  • Modified score creation in /langfuse/client.py and corresponding tests in /tests/test_core_sdk.py to reflect the changes.

💡 (2/5) Greptile learns from your feedback when you react with 👍/👎!

@davidmc971 davidmc971 requested a review from maxdeichmann April 3, 2025 14:19
@davidmc971 davidmc971 marked this pull request as ready for review April 3, 2025 14:19
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

8 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

@davidmc971 davidmc971 requested a review from hassiebp April 8, 2025 11:38
Copy link
Contributor

@hassiebp hassiebp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @davidmc971 - looks good to me, left to minor comments :)

Copy link
Author

Great, thank you! Will implement those shortly :)

@davidmc971
Copy link
Author

Just FYI, these failing tests are expected:

FAILED tests/test_core_sdk.py::test_metadata[Test Metadata-expected_metadata0] - KeyError: 'metadata'
FAILED tests/test_core_sdk.py::test_metadata[1-expected_metadata1] - KeyError: 'metadata'
FAILED tests/test_core_sdk.py::test_metadata[1.0-expected_metadata2] - KeyError: 'metadata'
FAILED tests/test_core_sdk.py::test_metadata[input_metadata3-expected_metadata3] - KeyError: 'metadata'
FAILED tests/test_core_sdk.py::test_metadata[input_metadata4-expected_metadata4] - KeyError: 'metadata'
FAILED tests/test_core_sdk.py::test_score_trace - KeyError: 'metadata'
FAILED tests/test_core_sdk.py::test_score_trace_nested_trace - AssertionError: assert None == {'key': 'value'}
 +  where None = Score_Numeric(value=0.5, id='796a90fc-c719-4b5e-b4e4-4fbafb9553ce', trace_id='e804e291-f432-42ab-a597-bba48c47c846', name='valuation', source=<ScoreSource.API: 'API'>, observation_id=None, timestamp=datetime.datetime(2025, 4, 9, 9, 59, 7, 63000, tzinfo=datetime.timezone.utc), created_at=datetime.datetime(2025, 4, 9, 9, 59, 7, 570000, tzinfo=datetime.timezone.utc), updated_at=datetime.datetime(2025, 4, 9, 9, 59, 7, 617000, tzinfo=datetime.timezone.utc), author_user_id=None, comment='tests/test_core_sdk.py::test_score_trace_nested_trace', metadata=None, config_id=None, queue_id=None, environment='default', data_type='NUMERIC', projectId='7a88fb47-b4e2-43b8-a06c-a5ce950dc53a', stringValue=None).metadata
FAILED tests/test_core_sdk.py::test_score_trace_nested_observation - AssertionError: assert None == {'key': 'value'}
 +  where None = Score_Numeric(value=0.5, id='edc36e38-10ed-4376-ac17-814ca860652a', trace_id='4845b182-6b4d-44a9-a66a-ca9a05cfc703', name='valuation', source=<ScoreSource.API: 'API'>, observation_id='ad689c74-e951-4844-884f-403a1f35cc8e', timestamp=datetime.datetime(2025, 4, 9, 9, 59, 9, 712000, tzinfo=datetime.timezone.utc), created_at=datetime.datetime(2025, 4, 9, 9, 59, 10, 221000, tzinfo=datetime.timezone.utc), updated_at=datetime.datetime(2025, 4, 9, 9, 59, 10, 359000, tzinfo=datetime.timezone.utc), author_user_id=None, comment='tests/test_core_sdk.py::test_score_trace_nested_observation', metadata=None, config_id=None, queue_id=None, environment='default', data_type='NUMERIC', projectId='7a88fb47-b4e2-43b8-a06c-a5ce950dc53a', stringValue=None).metadata
FAILED tests/test_core_sdk.py::test_score_span - KeyError: 'metadata'

There was no OSS release of the server changes yet, such that these test suites are failing because the backend doesn't handle metadata for them. Locally the tests were confirmed to be working.

@davidmc971 davidmc971 enabled auto-merge (squash) April 9, 2025 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: definition for score() method is missing metadata property
2 participants