Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YQ-4168 added test on YT import #15572

Merged
merged 18 commits into from
Mar 20, 2025

Conversation

GrigoriyPA
Copy link
Collaborator

@GrigoriyPA GrigoriyPA commented Mar 11, 2025

Changelog entry

Added test on YT import

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

Copy link

🟢 2025-03-11 10:46:25 UTC The validation of the Pull Request description is successful.

@GrigoriyPA GrigoriyPA force-pushed the YQ-4168-add-test-on-ut-import branch from 13695df to 7e5df53 Compare March 11, 2025 10:46
Copy link

github-actions bot commented Mar 11, 2025

2025-03-11 10:50:05 UTC Pre-commit check linux-x86_64-relwithdebinfo for 57e4f1b has started.
2025-03-11 10:50:19 UTC Artifacts will be uploaded here
2025-03-11 10:53:05 UTC ya make is running...
🟡 2025-03-11 11:50:08 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26587 23998 0 3 2474 112

2025-03-11 11:52:47 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 12:03:38 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
165 (only retried tests) 62 0 1 0 102

2025-03-11 12:03:48 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-11 12:16:06 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
142 (only retried tests) 41 0 1 0 100

🟢 2025-03-11 12:16:16 UTC Build successful.
🟢 2025-03-11 12:16:36 UTC ydbd size 2.1 GiB changed* by +1.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4880f2d merge: 57e4f1b diff diff %
ydbd size 2 296 125 704 Bytes 2 296 127 032 Bytes +1.3 KiB +0.000%
ydbd stripped size 480 730 208 Bytes 480 730 400 Bytes +192 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 11, 2025

2025-03-11 10:50:05 UTC Pre-commit check linux-x86_64-release-asan for 57e4f1b has started.
2025-03-11 10:50:19 UTC Artifacts will be uploaded here
2025-03-11 10:53:06 UTC ya make is running...
🟡 2025-03-11 11:58:28 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12221 11968 0 201 20 32

2025-03-11 11:59:48 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 12:14:28 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
340 (only retried tests) 247 0 62 1 30

2025-03-11 12:14:41 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-11 12:36:26 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
174 (only retried tests) 126 0 19 1 28

🟢 2025-03-11 12:36:33 UTC Build successful.
🟢 2025-03-11 12:37:05 UTC ydbd size 3.7 GiB changed* by +2.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4880f2d merge: 57e4f1b diff diff %
ydbd size 3 998 881 392 Bytes 3 998 884 128 Bytes +2.7 KiB +0.000%
ydbd stripped size 1 389 593 896 Bytes 1 389 594 536 Bytes +640 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

gridnevvvit
gridnevvvit previously approved these changes Mar 11, 2025
Copy link

github-actions bot commented Mar 11, 2025

2025-03-11 14:19:44 UTC Pre-commit check linux-x86_64-release-asan for 8ba9d75 has started.
2025-03-11 14:20:10 UTC Artifacts will be uploaded here
2025-03-11 14:23:30 UTC ya make is running...
🟡 2025-03-11 15:31:21 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12220 12057 0 118 9 36

2025-03-11 15:32:43 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 15:47:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
254 (only retried tests) 161 0 59 2 32

2025-03-11 15:47:15 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-11 16:01:24 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
169 (only retried tests) 105 0 35 2 27

🟢 2025-03-11 16:01:36 UTC Build successful.
🟢 2025-03-11 16:02:09 UTC ydbd size 3.7 GiB changed* by -97.3 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 5ed8c5f merge: 8ba9d75 diff diff %
ydbd size 3 999 006 576 Bytes 3 998 906 928 Bytes -97.3 KiB -0.002%
ydbd stripped size 1 389 625 224 Bytes 1 389 629 480 Bytes +4.2 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 11, 2025

2025-03-11 14:19:47 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8ba9d75 has started.
2025-03-11 14:20:21 UTC Artifacts will be uploaded here
2025-03-11 14:23:51 UTC ya make is running...
🟡 2025-03-11 15:25:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26593 23999 0 1 2473 120

2025-03-11 15:28:08 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-11 15:42:26 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
174 (only retried tests) 70 0 0 0 104

🟢 2025-03-11 15:42:38 UTC Build successful.
🟢 2025-03-11 15:42:59 UTC ydbd size 2.1 GiB changed* by +1.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: eedc02f merge: 8ba9d75 diff diff %
ydbd size 2 296 077 216 Bytes 2 296 078 544 Bytes +1.3 KiB +0.000%
ydbd stripped size 480 710 240 Bytes 480 710 432 Bytes +192 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA requested a review from gridnevvvit March 11, 2025 15:56
@GrigoriyPA GrigoriyPA force-pushed the YQ-4168-add-test-on-ut-import branch from 15e28bf to 1ca48b3 Compare March 12, 2025 10:01
Copy link

github-actions bot commented Mar 12, 2025

2025-03-12 10:02:24 UTC Pre-commit check linux-x86_64-release-asan for 5866c22 has started.
2025-03-12 10:02:39 UTC Artifacts will be uploaded here
2025-03-12 10:05:33 UTC ya make is running...
🟡 2025-03-12 11:42:17 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12348 12279 0 24 8 37

2025-03-12 11:43:23 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-12 11:55:43 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
111 (only retried tests) 76 0 0 4 31

🟢 2025-03-12 11:55:52 UTC Build successful.
🟢 2025-03-12 11:56:19 UTC ydbd size 3.7 GiB changed* by +4.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: add786e merge: 5866c22 diff diff %
ydbd size 4 001 765 184 Bytes 4 001 770 168 Bytes +4.9 KiB +0.000%
ydbd stripped size 1 390 233 704 Bytes 1 390 234 664 Bytes +960 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 12, 2025

2025-03-12 10:05:24 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5866c22 has started.
2025-03-12 10:05:39 UTC Artifacts will be uploaded here
2025-03-12 10:08:32 UTC ya make is running...
🟡 2025-03-12 11:34:55 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26722 24138 0 2 2471 111

2025-03-12 11:37:22 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-12 11:49:17 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
161 (only retried tests) 60 0 0 0 101

🟢 2025-03-12 11:49:26 UTC Build successful.
🟢 2025-03-12 11:49:45 UTC ydbd size 2.1 GiB changed* by +2.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: add786e merge: 5866c22 diff diff %
ydbd size 2 297 773 344 Bytes 2 297 775 960 Bytes +2.6 KiB +0.000%
ydbd stripped size 480 910 144 Bytes 480 910 208 Bytes +64 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA added the rebase-and-check Rebase PR with the current base branch and check label Mar 15, 2025
@github-actions github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Mar 15, 2025
Copy link

github-actions bot commented Mar 15, 2025

2025-03-15 13:57:55 UTC Pre-commit check linux-x86_64-release-asan for a0e0fe0 has started.
2025-03-15 13:58:10 UTC Artifacts will be uploaded here
2025-03-15 14:01:03 UTC ya make is running...
🟡 2025-03-15 15:12:26 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12434 12290 0 95 9 40

2025-03-15 15:13:29 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-15 15:31:44 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
198 (only retried tests) 154 0 2 5 37

2025-03-15 15:31:55 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-15 15:48:46 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74 (only retried tests) 34 0 0 6 34

🟢 2025-03-15 15:48:55 UTC Build successful.
🟢 2025-03-15 15:49:25 UTC ydbd size 3.8 GiB changed* by +2.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: f1e6120 merge: a0e0fe0 diff diff %
ydbd size 4 046 901 144 Bytes 4 046 903 944 Bytes +2.7 KiB +0.000%
ydbd stripped size 1 400 127 368 Bytes 1 400 128 072 Bytes +704 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 15, 2025

2025-03-15 13:58:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for a0e0fe0 has started.
2025-03-15 13:59:25 UTC Artifacts will be uploaded here
2025-03-15 14:02:37 UTC ya make is running...
🟡 2025-03-15 15:01:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26834 24221 0 2 2492 119

2025-03-15 15:04:02 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-15 15:16:13 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
183 (only retried tests) 71 0 0 3 109

🟢 2025-03-15 15:16:20 UTC Build successful.
🟢 2025-03-15 15:16:39 UTC ydbd size 2.1 GiB changed* by +1.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: f1e6120 merge: a0e0fe0 diff diff %
ydbd size 2 298 367 064 Bytes 2 298 368 392 Bytes +1.3 KiB +0.000%
ydbd stripped size 481 107 008 Bytes 481 107 200 Bytes +192 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

add_sample_table(kqp_run)

kqp_run.add_query("""
PRAGMA ydb.DataSizePerJob = "1";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

плохое название у прагмы =\

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Стоит переименовать в DataSizePerPartition (это не PerTask, так как одна таска может читать много партиций)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

в названии не упоминается YT и это смущает

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

В будущем эту прагму можно и для других провайдеров использовать, для настройки размера партиций (например отдавать в generic provider, чтобы настраивать размер плитов)

@GrigoriyPA GrigoriyPA force-pushed the YQ-4168-add-test-on-ut-import branch from 1ca48b3 to 2ca4a8d Compare March 18, 2025 07:42
Copy link

github-actions bot commented Mar 18, 2025

2025-03-18 07:45:20 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3964155 has started.
2025-03-18 07:45:35 UTC Artifacts will be uploaded here
2025-03-18 07:48:21 UTC ya make is running...
🟡 2025-03-18 08:51:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26853 24235 0 7 2491 120

2025-03-18 08:53:53 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-18 09:22:26 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
191 (only retried tests) 81 0 0 0 110

🟢 2025-03-18 09:22:33 UTC Build successful.
🟢 2025-03-18 09:22:54 UTC ydbd size 2.1 GiB changed* by +5.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d4b7722 merge: 3964155 diff diff %
ydbd size 2 299 114 728 Bytes 2 299 120 152 Bytes +5.3 KiB +0.000%
ydbd stripped size 481 241 728 Bytes 481 246 016 Bytes +4.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 18, 2025

2025-03-18 07:47:13 UTC Pre-commit check linux-x86_64-release-asan for 3964155 has started.
2025-03-18 07:47:28 UTC Artifacts will be uploaded here
2025-03-18 07:50:15 UTC ya make is running...
🟡 2025-03-18 08:59:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12440 12356 0 34 11 39

2025-03-18 09:00:46 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-18 09:20:39 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
122 (only retried tests) 79 0 1 5 37

2025-03-18 09:20:50 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-18 09:47:12 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
72 (only retried tests) 32 0 0 2 38

🟢 2025-03-18 09:47:19 UTC Build successful.
🟢 2025-03-18 09:47:51 UTC ydbd size 3.8 GiB changed* by +2.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: d4b7722 merge: 3964155 diff diff %
ydbd size 4 048 166 400 Bytes 4 048 169 200 Bytes +2.7 KiB +0.000%
ydbd stripped size 1 400 490 600 Bytes 1 400 491 304 Bytes +704 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA merged commit ca9587c into ydb-platform:main Mar 20, 2025
12 checks passed
@GrigoriyPA GrigoriyPA deleted the YQ-4168-add-test-on-ut-import branch March 20, 2025 10:32
adameat pushed a commit to adameat/ydb that referenced this pull request Mar 20, 2025
blinkov pushed a commit that referenced this pull request Mar 21, 2025
Emgariko pushed a commit to Emgariko/ydb that referenced this pull request Mar 24, 2025
lberserq pushed a commit to lberserq/ydb that referenced this pull request Mar 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants