Skip to content

Conversation

@cademack
Copy link
Member

@cademack cademack commented Nov 11, 2024

This PR adds a new detail to #react: "boundary_values_context_features". This detail allows users to ask questions about how much context values would have to change to observe differences in the resulting predictions.

The API changes are as follows:

  • #react (and #single_react) now accept 2 additional indices within the details (assoc) parameter.
  • The first new index is "boundary_values_context_features" which takes a list of feature names to compute boundary values for.
  • The second new index is boundary_values_action_outcome, which accepts a map of conditions for each action feature. This allows users to specify a condition that indicates the boundary in which they are interested in. For example, a user could specify {"feat1" [5 15]} which would then find the boundary values that would cause the prediction of "feat1" to be within the range of 5 to 15. (This condition formatting matches that of the standard Condition schema)
  • The ReactResponse object has a new key, "boundary_values", which holds a map of feature names to their computed boundary values.
    • For continuous and ordinal features, a tuple of boundary values is given allowing for a boundary to be found on either side of the given context value.
    • For nominal features, a single value is given. This value is the "most similar" class to the given context value that crosses the boundary or maximizes the change in the action.

When boundary_values_action_outcome is specified, the boundary is considered "well defined", and the algorithm will attempt to find the closest values to the given context values that cross the boundary. When it is unspecified, there are two possibilities. If all action features are nominal, then the boundary is still considered "well defined" and any change in the action values will indicate a crossed boundary and the algorithm still searches for the closest values that cross this boundary. In the case that boundary_values_action_outcome is unspecified and there are non-nominal action features, then the algorithm will attempt to find the closest local maxima of the gradient in the change of action values with respect to the change in context values.

With the current implementation, boundary values can be computed for continuous, nominal, and ordinal (numeric and string) features. It is not supported for edit-distance features (JSON, YAML, Amalgam).

@cademack cademack changed the title 13693: Adds "boundary_values" support as a detail to #react, MINOR 13693: Adds "boundary_values" as a detail to #react, MINOR Nov 12, 2024
@cademack cademack marked this pull request as ready for review November 13, 2024 16:23
@cademack cademack requested a review from a team as a code owner November 13, 2024 16:23
@cademack cademack merged commit 3e4ce20 into main Nov 20, 2024
10 checks passed
@cademack cademack deleted the 13693-boundary-values branch November 20, 2024 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants