GH-48214: [C++][Parquet] Fix Arrow Reader Internal logic to enable Parquet DB support on s390x

[Vishwanatha-HD]

Rationale for this change

This PR is intended to enable Parquet DB support on Big-endian (s390x) systems. The fix in this PR fixes the "Arrow/Reader_internal" logic.

What changes are included in this PR?

The fix includes changes to following file:
cpp/src/parquet/arrow/reader_internal.cc

Are these changes tested?

Yes. The changes are tested on s390x arch to make sure things are working fine. The fix is also tested on x86 arch, to make sure there is no new regression introduced.

Are there any user-facing changes?

No.

GitHub main Issue link: #48151

• GitHub Issue: [C++][Parquet] Fix Arrow Reader Internal logic to enable Parquet DB support on Big-Endian (s390x) systems #48214

[github-actions]

⚠️ GitHub issue #48214 has been automatically assigned in GitHub to PR creator.

[k8ika0s]

@Vishwanatha-HD

These little corners of the Arrow/Parquet bridge tend to hide the more “surprising” BE behaviors, so it’s always nice to see them getting attention.

My own s390x work didn’t touch reader_internal.cc, so I’m mostly reading this with the lens of “does this match the patterns I’ve seen on hardware.” The decimal min/max extraction you added looks straightforward, and from what I’ve observed, normalizing those integer-backed stats before they’re handed downstream makes a real difference on BE.

Same with the half-float swap: I’ve noticed that half-floats are one of the places where BE architectures drift quickest if the reader doesn’t explicitly re-LE them, so calling FromLittleEndian here feels like the safer side of the fence.

I don’t see any conflicts with what I’ve been doing in encode/decode land — just wanted to chime in and confirm the behavior you’re targeting here lines up with what I’ve seen when running the full Parquet → Arrow → Parquet round-trip paths on BE.

[pitrou]

I would favor a different approach: turn TransferBinary into:

Status TransferBinary(RecordReader* reader, MemoryPool* pool,
                      const std::shared_ptr<Field>& logical_type_field,
                      std::function<Result<std::shared_ptr<Array>>(std::shared_ptr<Array>)> array_process,
                      std::shared_ptr<ChunkedArray>* out) {

such that the optional array_process is called for each chunk. If carefully coded, this will help limit memory consumption by disposing of old chunks while creating the new ones.

[Vishwanatha-HD]

@pitrou.. Thanks for your comments here.. Can I plan to implement this change in the next pass, since the present code change is working.. We want to enable the Apache/Arrow support on s390x as soon as possible since its blocking many of the OCP AI tests on IBM Z.. Thanks.. !!

[Vishwanatha-HD]

I have addressed all the review comments.. Thanks..