Skip to content

Document maximum allowed input lengths for each metric #85

@yosukehigashi

Description

@yosukehigashi

Because many of our metrics rely on local models with a limited context length, they can fail when an input string is too long. E.g. here are a subset of our Japanese metrics (local version) on a very long input string.

langcheck.metrics.ja.toxicity(long_str)  # Fails
langcheck.metrics.ja.fluency(long_str)  # Fails
langcheck.metrics.ja.sentiment(long_str)  # Fails
langcheck.metrics.ja.tateishi_ono_yamada_reading_ease(long_str)  # Succeeds
langcheck.metrics.ja.semantic_similarity(long_str, long_str)  # Succeeds
langcheck.metrics.ja.rougeL(long_str, long_str)  # Succeeds

We should document this, and gracefully handle cases where the input string is too long.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions