fix: cast dd/mm/yyyy dates day-first (was silently swapping day and month)

[estellebertrand]

Problem

date._is() accepts dd/mm/yyyy values through jjmmaaaa_pattern, which is day-first
by construction.

But date_casting() then converted those values with dateutil.parser.parse(val),
falling back on dateparser.parse(val) - both default to month-first, and neither
was given a dayfirst / DATE_ORDER argument.

The casting function is used in Hydra (to be confirmed), and every value where both components are
≤ 12 was silently inverted:

"07/03/2024"  _is=True  ->  2024-07-03   # 7 March read as 3 July
"01/02/2024"  _is=True  ->  2024-01-02   # 1 February read as 2 January

The inversion is silent: no error, no warning, and the output is a perfectly valid date.

This issue has been uncovered through this discussion on the platform.

Proposal

Pass dayfirst to the parsers only for values that match jjmmaaaa_pattern from the detection step. Because this pattern requires the second component to be 01–12 and the first to be 01–31, a matching value can only be read as DD then MM.

date_casting moves below the pattern definitions so it can use jjmmaaaa_pattern.

Note : Why not just dayfirst=True everywhere

Tried first, and it breaks two things:

• dateutil applies dayfirst to the day/month pair regardless of position in regard to year digits, so ISO
  2024-03-07 was parsed as 3 July.
• DATE_ORDER: DMY makes the dateparser fallback accept 12152003, 20031512 and
  02052003, which date._test_values[False] explicitly requires to be rejected.

Scoping the flag to the pattern avoids both.

Impact on existing data

Behaviour-changing for downstream consumers. Dates already stored by hydra for CSV resources with ambiguous dd/mm/yyyy values are wrong and would need reprocessing.

[ThibaudDauce]

I think this solution will break for US dates in files, (it will take french format for dates that match french format and US format for dates with US format during casting, no?). I've already worked on this in #245 for other reasons, but I can try to incorporate a fix for this problem too (getting the format at detection time from the full file and just using it at casting time without redetecting)

[estellebertrand]

I think this solution will break for US dates in files, (it will take french format for dates that match french format and US format for dates with US format during casting, no?). I've already worked on this in #245 for other reasons, but I can try to incorporate a fix for this problem too (getting the format at detection time from the full file and just using it at casting time without redetecting)

Yes it will break any US dates that could be matched as a French date. But as of today, from my understanding, the same is true about French dates being interpreted as US dates during casting - and I imagine French dates may be more common in our datasets (?).

Though I totally agree that is an issue and should be improved, I opened this issue #269. It looks like your PR #245 is actually implementing what I had in mind ?

[bolinocroustibat]

LGTM, thank you for spotting and solving this!

(Tempted to open an issue on date parsers about this, defaulting to month-first is just the worst)

[bolinocroustibat]

Closing, replaced by #245