Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YQ-4186 refactor errors in s3 write actor #16020

Conversation

GrigoriyPA
Copy link
Collaborator

Changelog entry

Refactored errors in s3 write actor

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

Copy link

🟢 2025-03-20 13:40:58 UTC The validation of the Pull Request description is successful.

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 13:48:59 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9f28c4b has started.
2025-03-20 13:49:30 UTC Artifacts will be uploaded here
2025-03-20 13:52:57 UTC ya make is running...
🔴 2025-03-20 13:55:30 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 13:57:36 UTC Pre-commit check linux-x86_64-release-asan for 9f28c4b has started.
2025-03-20 13:58:02 UTC Artifacts will be uploaded here
2025-03-20 14:01:14 UTC ya make is running...
🔴 2025-03-20 14:03:46 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 14:57:06 UTC Pre-commit check linux-x86_64-release-asan for 058ba0f has started.
2025-03-20 14:57:10 UTC Artifacts will be uploaded here
2025-03-20 14:59:52 UTC ya make is running...
🔴 2025-03-20 15:05:30 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 15:27:53 UTC Pre-commit check linux-x86_64-release-asan for de0db10 has started.
2025-03-20 15:28:29 UTC Artifacts will be uploaded here
2025-03-20 15:31:43 UTC ya make is running...
🟡 2025-03-20 16:32:16 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12242 12109 0 78 19 36

2025-03-20 16:33:57 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-20 16:46:33 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
181 (only retried tests) 137 0 2 7 35

2025-03-20 16:46:44 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-20 16:58:54 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
72 (only retried tests) 33 0 1 5 33

🟢 2025-03-20 16:59:05 UTC Build successful.
🟢 2025-03-20 16:59:36 UTC ydbd size 3.8 GiB changed* by +34.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 00995cf merge: de0db10 diff diff %
ydbd size 4 071 958 816 Bytes 4 071 994 416 Bytes +34.8 KiB +0.001%
ydbd stripped size 1 408 718 408 Bytes 1 408 721 544 Bytes +3.1 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 15:28:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for de0db10 has started.
2025-03-20 15:29:15 UTC Artifacts will be uploaded here
2025-03-20 15:32:29 UTC ya make is running...
🟡 2025-03-20 16:27:17 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26668 23833 0 197 2524 114

2025-03-20 16:29:37 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-20 16:56:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
404 (only retried tests) 104 0 189 3 108

2025-03-20 16:56:46 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-20 17:08:49 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
350 (only retried tests) 55 0 190 3 102

🟢 2025-03-20 17:09:07 UTC Build successful.
🟢 2025-03-20 17:09:29 UTC ydbd size 2.2 GiB changed* by +26.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 00995cf merge: de0db10 diff diff %
ydbd size 2 313 317 144 Bytes 2 313 343 808 Bytes +26.0 KiB +0.001%
ydbd stripped size 484 266 592 Bytes 484 268 512 Bytes +1.9 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 17:46:29 UTC Pre-commit check linux-x86_64-release-asan for 5443a56 has started.
2025-03-20 17:48:31 UTC Artifacts will be uploaded here
2025-03-20 17:51:15 UTC ya make is running...
🟡 2025-03-20 18:53:38 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12251 12129 0 65 20 37

2025-03-20 18:55:07 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-20 19:19:18 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
174 (only retried tests) 129 0 4 5 36

2025-03-20 19:19:27 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-20 19:31:04 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
75 (only retried tests) 33 0 2 6 34

🟢 2025-03-20 19:31:11 UTC Build successful.
🟢 2025-03-20 19:31:40 UTC ydbd size 3.8 GiB changed* by +36.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: ae498a7 merge: 5443a56 diff diff %
ydbd size 4 072 391 680 Bytes 4 072 428 736 Bytes +36.2 KiB +0.001%
ydbd stripped size 1 408 860 552 Bytes 1 408 866 504 Bytes +5.8 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 20, 2025

2025-03-20 17:47:34 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5443a56 has started.
2025-03-20 17:47:51 UTC Artifacts will be uploaded here
2025-03-20 17:50:47 UTC ya make is running...
🟡 2025-03-20 18:48:59 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26677 24037 0 5 2519 116

2025-03-20 18:51:13 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-20 19:04:06 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
197 (only retried tests) 56 0 3 28 110

2025-03-20 19:04:15 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-20 19:15:42 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
180 (only retried tests) 70 0 0 3 107

🟢 2025-03-20 19:15:51 UTC Build successful.
🟢 2025-03-20 19:16:16 UTC ydbd size 2.2 GiB changed* by +24.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: ae498a7 merge: 5443a56 diff diff %
ydbd size 2 313 521 936 Bytes 2 313 547 352 Bytes +24.8 KiB +0.001%
ydbd stripped size 484 306 336 Bytes 484 307 808 Bytes +1.4 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 21, 2025

2025-03-21 06:52:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for 41f4a9a has started.
2025-03-21 06:52:57 UTC Artifacts will be uploaded here
2025-03-21 06:55:49 UTC ya make is running...
2025-03-21 07:22:40 UTC Check cancelled

Copy link

github-actions bot commented Mar 21, 2025

2025-03-21 06:56:43 UTC Pre-commit check linux-x86_64-release-asan for 41f4a9a has started.
2025-03-21 06:56:58 UTC Artifacts will be uploaded here
2025-03-21 06:59:50 UTC ya make is running...
2025-03-21 07:22:41 UTC Check cancelled

Copy link

github-actions bot commented Mar 21, 2025

2025-03-21 07:26:35 UTC Pre-commit check linux-x86_64-relwithdebinfo for 2031b39 has started.
2025-03-21 07:26:50 UTC Artifacts will be uploaded here
2025-03-21 07:29:35 UTC ya make is running...
🟡 2025-03-21 08:23:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26677 24053 0 3 2502 119

2025-03-21 08:25:21 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-21 08:50:29 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
184 (only retried tests) 76 0 0 4 104

🟢 2025-03-21 08:50:38 UTC Build successful.
🟢 2025-03-21 08:50:55 UTC ydbd size 2.2 GiB changed* by +25.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 5f13bfb merge: 2031b39 diff diff %
ydbd size 2 313 765 584 Bytes 2 313 791 368 Bytes +25.2 KiB +0.001%
ydbd stripped size 484 329 888 Bytes 484 334 624 Bytes +4.6 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 21, 2025

2025-03-21 07:26:59 UTC Pre-commit check linux-x86_64-release-asan for 2031b39 has started.
2025-03-21 07:27:14 UTC Artifacts will be uploaded here
2025-03-21 07:29:59 UTC ya make is running...
🟡 2025-03-21 08:30:10 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12252 12146 0 42 27 37

2025-03-21 08:31:26 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-21 08:51:07 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
154 (only retried tests) 111 0 3 5 35

2025-03-21 08:51:15 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-21 09:03:24 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
73 (only retried tests) 33 0 2 4 34

🟢 2025-03-21 09:03:30 UTC Build successful.
🟢 2025-03-21 09:04:00 UTC ydbd size 3.8 GiB changed* by +30.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 5f13bfb merge: 2031b39 diff diff %
ydbd size 4 072 829 160 Bytes 4 072 860 088 Bytes +30.2 KiB +0.001%
ydbd stripped size 1 408 946 216 Bytes 1 408 946 920 Bytes +704 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 21, 2025

2025-03-21 09:48:47 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1086d73 has started.
2025-03-21 09:49:01 UTC Artifacts will be uploaded here
2025-03-21 09:51:43 UTC ya make is running...
🟡 2025-03-21 10:43:42 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26685 24047 0 6 2521 111

2025-03-21 10:46:06 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-21 11:10:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
192 (only retried tests) 58 0 3 26 105

2025-03-21 11:11:02 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-21 11:28:57 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
173 (only retried tests) 68 0 0 0 105

🟢 2025-03-21 11:29:03 UTC Build successful.
🟡 2025-03-21 11:29:25 UTC ydbd size 2.2 GiB changed* by +558.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6261c52 merge: 1086d73 diff diff %
ydbd size 2 313 762 408 Bytes 2 314 334 120 Bytes +558.3 KiB +0.025%
ydbd stripped size 484 329 248 Bytes 484 356 288 Bytes +26.4 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 21, 2025

2025-03-21 09:50:37 UTC Pre-commit check linux-x86_64-release-asan for 1086d73 has started.
2025-03-21 09:50:52 UTC Artifacts will be uploaded here
2025-03-21 09:53:44 UTC ya make is running...
🟡 2025-03-21 10:59:35 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12251 12150 0 48 18 35

2025-03-21 11:00:40 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-21 11:13:19 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
148 (only retried tests) 106 0 1 8 33

2025-03-21 11:13:28 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-21 11:25:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
69 (only retried tests) 28 0 1 8 32

🟢 2025-03-21 11:25:41 UTC Build successful.
🟡 2025-03-21 11:26:13 UTC ydbd size 3.8 GiB changed* by +994.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6261c52 merge: 1086d73 diff diff %
ydbd size 4 072 832 984 Bytes 4 073 851 008 Bytes +994.2 KiB +0.025%
ydbd stripped size 1 408 946 280 Bytes 1 409 032 104 Bytes +83.8 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

, Issues(std::move(issues))
{}

TEvUploadError(const TString& message, const TString& requestId, const TString& responseBody, const IHTTPGateway::TResult& result)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

зачем передавать responseBody отдельным аргументом, если он всегда равен response.Content.Extract()?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Сделал так, потому что response.Content.Extract() можно вызывать лишь раз, там делается std::move от контента (а он ещё нужен при обработке, чтобы s3Result создать)

}

if (result.Content.HttpResponseCode >= 300) {
actorSystem->Send(new IEventHandle(parentId, selfId, new TEvPrivate::TEvUploadError(TStringBuilder() << "Upload operation failed, url: " << url, requestId, body, result)));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

в этом случае ошибка не проглотится? если CurlResponseCode == CURLE_OK, а Issues пустые

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Тут дело в том, что в случае успеха приходит http ответ с пустым body, по этому так валидируется

@GrigoriyPA GrigoriyPA merged commit 22e1472 into ydb-platform:main Mar 21, 2025
12 checks passed
@GrigoriyPA GrigoriyPA deleted the YQ-4186-refactor-errors-in-s3-write-actor branch March 21, 2025 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants