Skip to content

Comparing float arguments with limited precision #698

@cbmarini

Description

@cbmarini

What should the correct output be for the following program?

#StartFloat 20
CFunction f;
Local F = f(1.00000000000000001) + f(1.0);
Print;
.end

Currently, this produces:

   F =
      f(1.0e+00) + f(1.0e+00);

This happens because the arguments are compared in their raw internal representation: although 1.00000000000000001 and 1.0 should be indistinguishable at 20 bits of precision, the comparison occurs before rounding, and the extra limbs make them appear distinct.

This is closely related to #695 and #697.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions