Skip to content

Conversation

chpache
Copy link

@chpache chpache commented May 1, 2025

This PR logs on the spark worker when there is a difference between the number of rows of a scan and the number of rows returned due to the use of a TTL field.

@chpache chpache force-pushed the scylla-5/log-tll-skipped branch from 6516b7e to e8f4c83 Compare May 5, 2025 08:45
Copy link

@dkropachev dkropachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise looks good

@chpache
Copy link
Author

chpache commented May 8, 2025

For information, those logs won't appear during tests. you need to navigate, using the Spark UI, to the worker to find this log.

@chpache chpache force-pushed the scylla-5/log-tll-skipped branch from 3ef8add to 8c982ef Compare May 9, 2025 07:42
chpache and others added 2 commits May 9, 2025 15:59
@chpache
Copy link
Author

chpache commented May 9, 2025

concerns scylladb/scylla-migrator#232

@chpache chpache marked this pull request as ready for review May 14, 2025 05:43
@chpache chpache changed the title WIP: Log skipped line count due to TTL Log skipped line count due to TTL May 14, 2025
@chpache chpache requested review from dkropachev and tarzanek May 14, 2025 08:49
@chpache chpache force-pushed the scylla-5/log-tll-skipped branch from 9170076 to b7ed223 Compare May 14, 2025 13:04
@chpache chpache force-pushed the scylla-5/log-tll-skipped branch from b7ed223 to 26fae73 Compare May 14, 2025 13:05
@chpache chpache requested a review from dkropachev May 14, 2025 13:05
@tarzanek
Copy link
Collaborator

thnx, looks good @chpache
will you also open related migrator issue which uses this and prints it out?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants