MANTA-5504 buckets-mdapi should provide discovery endpoints for sharkspotter#66
Draft
cneira wants to merge 5 commits into
Draft
MANTA-5504 buckets-mdapi should provide discovery endpoints for sharkspotter#66cneira wants to merge 5 commits into
cneira wants to merge 5 commits into
Conversation
New Fast RPC endpoint that accepts multiple UpdateObjectPayload items, groups them by vnode, and executes each vnode group within a single PostgreSQL transaction. Objects in the same vnode succeed or fail atomically. === AI-METADATA === AI-Role: code generation Human-Role: code review, implementation guidance, design decisions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix syntax error (missing dot and semicolon) on the 'which pg_tmp' expect call and replace generic "failed to execute process" messages with descriptive ones that name the command and suggest what to check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add discovery RPCs for vnode enumeration and per-vnode owner listing. Includes functional, concurrency, randomized, and performance integration tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We need to be able to enumerate all the buckets across a data center to evacuate objects to a new storage node.
Meta-data in manta-buckets-api is partitioned into vnodes. The metadata of interest in this case are buckets and their objects. The proposed solution is to have two new RPC calls that will enable callers to map all existing buckets and objects in deployment.
The new RPC calls are :
listvnodes() returns a vector of integers that represent the current vnodes that exist.
listowners(vnode: u64) returns UUIDs for bucket owners in a target vnode.
The caller should call first listvnodes to get the list of vnodes, then use that list to enumerate all the existing owners, which are the accounts that have created buckets and objects.