Skip to content

Missing collection boundaries for zero-granularity data causes incorrect collection range #264

@pskrbasu

Description

@pskrbasu

Problem

When a data set has zero granularity (e.g., no explicit time-based segmentation), Tailpipe does not persist the collection’s lower and upper boundaries in its collection state.

As a result:
• The collection state is effectively ignored on subsequent runs.
• If no --from or --to flags are provided, Tailpipe defaults to collecting data from the last 7 days, regardless of what was previously collected.

This leads to unintended data re-collection or missed data, particularly in workflows where collections are expected to resume incrementally or from a known state.

Expected Behavior

Tailpipe should persist and honour the collection range (lower and upper) even when granularity is zero, so that future runs can correctly determine the collection window when --from/--to are not explicitly provided.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions