Skip to content

Strange behavior with bayesfactor_ttest for weak effects with one-sided test #473

@dsuplica

Description

@dsuplica

The documentation for bayesfactor_ttest states that

One-sided Bayes Factor (BF) are simply obtained by doubling the two-sided BF, which is not the same behavior as R or JASP. Be extra careful when interpretating one-sided BF, and if you can, always double-check your results."

While this is generally correct, it does not completely follow for t values translating to bayes factors below BF = 1. Essentially, for a two-sided test which corresponds to a BF < 0.5, the BF for alternative="greater" and alternative="less" will be identical. I have included a curve of the outputs of bayesfactor_ttest corresponding to different alternative hypotheses:

Image

Essentially, within a certain range of weak effects, the one-sided BF appears to be uninterpretable as a result. I believe it is due to this code in bayesian.py, where it will only invert the BF if it is greater than 1

tail_binary = "two-sided" if alternative == "two-sided" else "one-sided"

bf10 = bf10 * (1 / 0.5) if tail_binary == "one-sided" else bf10

# Now check the direction of the test

if ((alternative == "greater" and t < 0) or (alternative == "less" and t > 0)) and bf10 > 1:

        bf10 = 1 / bf10

return bf10

I am not familiar enough with bayesian statistics to necessarily know what the "correct" output should be. However, I would have at least expected to see a monotonic curve for the one-sided BF as the t value changes. This makes it quite hard to see evidence in favor of a null when there is an a priori prediction about the data. Should I be only using a two-sided test, even where this would be the case? If that's true, it might be good to have an explicit warning about this beahavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions