Because many of our metrics rely on local models with a limited context length, they can fail when an input string is too long. E.g. here are a subset of our Japanese metrics (local version) on a very long input string.
langcheck.metrics.ja.toxicity(long_str) # Fails
langcheck.metrics.ja.fluency(long_str) # Fails
langcheck.metrics.ja.sentiment(long_str) # Fails
langcheck.metrics.ja.tateishi_ono_yamada_reading_ease(long_str) # Succeeds
langcheck.metrics.ja.semantic_similarity(long_str, long_str) # Succeeds
langcheck.metrics.ja.rougeL(long_str, long_str) # Succeeds
We should document this, and gracefully handle cases where the input string is too long.
Because many of our metrics rely on local models with a limited context length, they can fail when an input string is too long. E.g. here are a subset of our Japanese metrics (local version) on a very long input string.
We should document this, and gracefully handle cases where the input string is too long.