Add a Lookback Window on Cursor Based Postgres Connection #52667
tmas-definitive
started this conversation in
Connector Ideas and Features
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In the current design of the postgres source, a lookback window cannot be specified which would be extremely helpful in a cursor based replication strategy if a specific window of data needed to be replayed. I realize WAL or Xmin can probably handle this already, but if there are constraints to using cursor based (ie. pulling data from a read replica)
So now the query roughly generated on a cursor based run would be:
SELECT <columns> FROM <stream_source> WHERE cursor_field > {cursor_value}
Would like it to look something like
SELECT <columns> FROM <stream_source> WHERE cursor_field > {cursor_value} - {lookback}
Beta Was this translation helpful? Give feedback.
All reactions