Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(clustering/sync): validate deltas #14127

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

feat(clustering/sync): validate deltas #14127

wants to merge 27 commits into from
+389 −25

Conversation

chobits
Copy link
Contributor

@chobits chobits commented Jan 10, 2025 •
edited
Loading

Summary

  1. validate entity schema
    • run original full sync logic
  2. validate entity's references (foreign references)
    • re-implemented logic: traverse delta -> try to find delta's foreign entity in config & LMDB

Checklist

  • The Pull Request has tests
  • A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
  • There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

Fix KAG-5897

@chobits chobits requested a review from chronolaw January 10, 2025 06:29
@chronolaw chronolaw changed the title feat(sync): validate deltas feat(clustering/sync): validate deltas Jan 10, 2025
chronolaw
chronolaw reviewed Jan 10, 2025
View reviewed changes
kong/clustering/services/sync/validate.lua Outdated Show resolved Hide resolved
kong/clustering/services/sync/validate.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
@github-actions github-actions bot added core/clustering core/db schema-change-noteworthy cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee labels Jan 13, 2025
@chobits chobits force-pushed the feat/sync_validate_deltas branch from 888f1a4 to 281683a Compare January 13, 2025 08:06
chronolaw
chronolaw reviewed Jan 13, 2025
View reviewed changes
spec/fixtures/dc_blueprints.lua Show resolved Hide resolved
spec/internal/db.lua Show resolved Hide resolved
chronolaw
chronolaw reviewed Jan 13, 2025
View reviewed changes
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
spec/02-integration/18-hybrid_rpc/06-validate_deltas_spec.lua Outdated Show resolved Hide resolved
@chobits chobits force-pushed the feat/sync_validate_deltas branch from 1311558 to 34a04dd Compare January 14, 2025 02:14
@chronolaw chronolaw requested a review from Water-Melon January 14, 2025 02:22
StarlightIbuki
StarlightIbuki reviewed Jan 14, 2025
View reviewed changes
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
chronolaw
chronolaw reviewed Jan 14, 2025
View reviewed changes
kong/db/declarative/init.lua Outdated Show resolved Hide resolved
kong/db/schema/others/declarative_config.lua Outdated Show resolved Hide resolved
kong/init.lua Outdated Show resolved Hide resolved
chobits and others added 22 commits January 15, 2025 10:44
@chobits
ngx.null -> null
1b5011b
@chobits
fix lint error of test case
1a63d10
@chobits
fix typo: generate
d31a2b6
@chobits
coding style
a19f5bc
@chobits
add comment
bf13811
@chobits
add comment for expand_foreigns
980029c
@chobits @chronolaw
Update kong/db/schema/others/declarative_config.lua
089d82e 
Co-authored-by: Chrono <[email protected]>
@chobits @chronolaw
Update kong/db/schema/others/declarative_config.lua
4526fd9 
Co-authored-by: Chrono <[email protected]>
@chobits
fix comments
8f8de42
@chobits
select return fix
bdb40b8
@chobits
fixed tests: assert -> assert.is_true
36e4701
@chobits
improve loop logic
ab513d0
@chobits
improve perf of test cases
e0db810
@chobits
generaete full schema of declarative config for sync
e3494c0
@chobits
add blank to fix coding style
1bca8c8
@chobits
simplify logic: we dont validate full sync, all run vallidate sync
f77ca9b
@chobits
Revert "generaete full schema of declarative config for sync"
9fec3e6 
This reverts commit 4329314.
@chobits
1
134e2c6
@chobits
fix 09-node-id-persistence_spec.lua
88992ff
@chobits
rename: declarative_config.validate -> declarative_config.validate_sc…
bf38ec1 
…hema
@chobits
remove DeclarativeConfig.validate_references_full
c2c58d5
@chobits
fix title of 09-node-id-persistence_spec.lua
ec2e241
@chobits chobits force-pushed the feat/sync_validate_deltas branch from 022b490 to ec2e241 Compare January 15, 2025 02:45
@chobits
Copy link
Contributor Author

chobits commented Jan 15, 2025

cherry-pick in advance: https://github.com/Kong/kong-ee/pull/11065

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@StarlightIbuki StarlightIbuki StarlightIbuki left review comments

@chronolaw chronolaw chronolaw left review comments

@Water-Melon Water-Melon Awaiting requested review from Water-Melon

At least 2 approving reviews are required to merge this pull request.

Assignees

@chobits chobits

Labels
cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee core/clustering core/db schema-change-noteworthy size/L skip-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@chobits @chronolaw @Water-Melon @StarlightIbuki