Fix: Preserve CreatedAt timestamp during compaction

[corylanou]

Problem

Fixes #771

In v0.5.0, LTX files became unavailable for timestamp-based restoration after compaction. When level 0 files were compacted into level 1, the compacted file was assigned the current time as its CreatedAt timestamp rather than preserving the earliest timestamp from the source files.

This caused restoration to fail for any timestamp that fell between:

• The original L0 file creation time (earlier)
• The compaction time (later)

After compaction deleted the L0 files, these timestamps became "unavailable" because the L1 file's CreatedAt was too late to match the requested restoration point.

Solution

Per author's suggestion, this PR implements a cleaner approach: reading timestamps from LTX file headers rather than file system or object metadata.

LTX files contain an authoritative Timestamp field in their headers (milliseconds since epoch). The issue was that LTXFiles() implementations were reading timestamps from:

• File modification times (file, sftp backends)
• Object metadata LastModified/CreationTime (S3, GCS, Azure backends)
• Or not setting timestamps at all (NATS backend)

This fix modifies LTXFiles() in all storage backends to use ltx.PeekHeader() to read the authoritative timestamp from each LTX file's header.

Benefits of this approach:

• No interface changes required
• More architecturally correct (LTX headers are source of truth)
• Works across all storage backends consistently
• Timestamps are preserved through compaction automatically

Changes

• file: Read header timestamp using ltx.PeekHeader(), add time import
• sftp: Read header timestamp via SFTP client
• s3: Read header timestamp in file iterator
• gs: Read header timestamp in file iterator, pass client to iterator
• abs: Read header timestamp in file iterator
• nats: Add CreatedAt field and read from header
• test: Added TestCompaction_PreservesEarliestTimestamp verifying timestamp preservation

All implementations use a fallback to file/object metadata if header reading fails, ensuring robustness.

Testing

• ✅ New test TestCompaction_PreservesEarliestTimestamp passes
• ✅ All existing tests pass (full test suite with -short)
• ✅ Pre-commit hooks pass (goimports, go-vet, staticcheck)
• ✅ Clean build with no errors

Example

Before this fix:

L0 files: TXID 1-3, header.Timestamp: 10:00:00, file.ModTime: 10:00:00
Compaction at: 10:01:00
L1 file: TXID 1-3, header.Timestamp: 10:00:00, file.ModTime: 10:01:00
LTXFiles() reads: file.ModTime (10:01:00)  ← Problem!
Restore to 10:00:30 → FAILS (no files before 10:01:00)

After this fix:

L0 files: TXID 1-3, header.Timestamp: 10:00:00
Compaction at: 10:01:00
L1 file: TXID 1-3, header.Timestamp: 10:00:00
LTXFiles() reads: header.Timestamp (10:00:00)  ← Fixed!
Restore to 10:00:30 → SUCCESS

Performance Consideration

Reading headers adds one network/disk request per file when listing files. This is necessary for correctness and only occurs during list operations, not during normal replication.

[corylanou]

Verification Testing Complete ✅

I've tested the exact scenario from the issue description and confirmed both the bug and the fix.

Test Setup

• Insert 10 rows over 20 seconds
• Wait 35 seconds for L1 compaction (30s interval)
• L0 files (TXID 1-10) get deleted and compacted into L1
• Attempt to restore to timestamp from early in the sequence

Main Branch (v0.5.0) - Bug Confirmed ✗

Error:

time=2025-10-01T13:19:27.876-05:00 level=ERROR msg="failed to run" 
error="cannot calc restore plan: transaction not available"

Root Cause: L1 file has CreatedAt: 18:18:50Z (compaction time), but we're trying to restore to 18:18:34Z (before compaction). The L1 file appears "too new" so restoration fails.

Fix Branch - Verified Working ✓

Result: ✓ RESTORE SUCCESS - found 10 rows

How it works: The fix preserves the earliest CreatedAt timestamp from all source L0 files when creating the L1 compacted file. This maintains temporal granularity for point-in-time restoration.

Full test results saved in my local testing at /tmp/test-results-771.md.

[corylanou]

Closing in favor of #778