approx equal can be more restricted than ==

[yuyichao]

This is similar to #13047 but is broader than that. In fact, the current PR that fixes that issue will only cause similar issues between other types. I'm very surprised that such major behavioral change may be introduced as a bug fix in a patch release.....

While I think it's wrong to make such change in a patch release, I do agree with using more exact comparison for boolean types. However, it seems over restricted when comparison using normal == produces True.

In [2]: 1 == pytest.approx(True)
Out[2]: False

In [3]: 1.0 == pytest.approx(True)
Out[3]: False

In [4]: 1.0 == True
Out[4]: True

In [5]: 1 == True
Out[5]: True

This makes the behavior of approx a bit difficult to reason about. I doubt if anyone would be asking for a more restricted comparison by adding approx...

[Zac-HD]

That changed recently - I actually think that the change is good, because I think it's much worse when tests pass-when-the-shouldn't than fail-when-they-shouldn't: in the latter case you find out and can do something about it.

This should definitely be documented though, perhaps by adding a sentence to the top of the pytest.approx() docstring: "Note that unlike builtin equality, this function considers booleans unequal to numeric zero or one."

[yuyichao]

I know that is is changed recently and as I said, I agree using exact comparisons for Boolean is reasonable (I think its borderline unacceptable to do so in a patch release or even a minor release but that’s a different issue).

However, type exact comparisons is way more strict than exact equality which I believe was what the original issue was asking about. This introduce a significant inconsistency between Boolean type and other types that’s way more significant their corresponding value range would suggest. It’s unclear where this should end. Should integer get the same treatment? After all, they also take discrete values and not continuous ones.

The argument that raising more error is good also doesn’t make any sense. If raising more error / returning false is seen as a benefit, then why not make everything that’s not floating point fail when using approx. After all, someone calling approx on integer or boolean may be doing so unintentionally so the safe thing to do here should be simply erroring out for all cases there any non floating point values and just require the user to do something about it instead. Also note that the “do something” could involve replicating approx’s logic with nested data structures e.g. for comparing [True] and [1].

And fwiw, the fix #13132 for the linked issue shows how wrong the exact type comparison is. Unless there’s a standard way to query if a type should be treated as bool, adding special cases for third party types simply doesn’t make sense. I’m sure there are other libraries that implement such types (numba seems to have it previously). Are those types going to be added as special cases as well? If not, I don’t see why numpy should be singled out if it was not a direct dependency.

[TTsangSC]

Hi, sorry for barging in, but here's my two cents as a fellow user.

• Another somewhat counterintuitive edge case is how

  >>> 1 == pytest.approx(True)
  False
  >>> True == pytest.approx(1)
  True

  Approximate equality being in a sense not commutative sounds a bit wrong to me.
• On the topic of booleans, I've done this in a personal project:

  def is_bool(x) -> bool:
      """Test boolean-like-ness via `str()` round-tripping."""
      as_str = str(x)
      values = {'True': True, 'False': False}
      try:
          return bool(x == values[as_str])
      except Exception:
          return False

  which does catch both builtins.bool and numpy.bool_ in a kinda logically consistent and dependency-agnostic way. (Of course it falls apart if someone implemented their booleans like def __str__(self) -> str: return 'T' if self else 'F', but still.)
• While I do agree on principle that it doesn't make sense for numpy booleans to receive special treatment as a non-direct dep, FWIW the ship has sailed and much of pytest.approx is already engineered specifically with numpy compatibility in mind (e.g. _pytest.python_api.ApproxNumpy). So maybe special-casing numpy.bool_ isn't as bad (nor as much of) a precedent as it sounds, but a more general implementation (that is also more robust than what I have above) would be much welcome.