coerce_int96_timestamp_unit warning

[mycaule]

When reading Parquet with wr.s3.read_parquet I have multiple warnings of this form,

How do I resolve this issue thank you ?

_read_parquet.py:47
coerce_int96_timestamp_unit is not supported in pyarrow 2 and below

[malachi-constant]

Hi @mycaule are you able to upgrade pyarrow to >=3.x.x?

[malachi-constant]

I can confirm that this is replicated with pyarrow 4 as a dependency. So the warning is incorrect, however this is still raising a TypeError and not sure there is anything else you can do outside of upgrading to a version of airflow that supports pyarrow 6 or greater.

[mycaule]

MWAA currently runs airflow=2.2.2 maximum

[malachi-constant]

Understood, unfortunately there isn't anything we can do here until support is added for libraries that work with pyarrow 6 and above.

[mycaule]

Do you think I can change the coerce_int96_timestamp_unit parameter to something else?

[malachi-constant]

Is the question if you can change its value or if there's a different parameter that can be used to set how timestamps are inferred when creating a the parquet file? For the latter i'd have to do some investigation into what pyarrow supported at 4.0.1