Skip to content

Speed up shard evacuation process #3454

@roman-khimov

Description

@roman-khimov

Is your feature request related to a problem? Please describe.

I'm always frustrated when shard evacuation takes more time than it could. We're pulling one object after another and writing them one after another, it's not very productive.

Describe the solution you'd like

Pull objects and push them into write queues (when pushing into multiple shards), accumulate small objects into batches and write batches of objects. Source shard iteration can be done by size if metabase is available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    I3Minimal impactS4RoutineU4Nothing urgentenhancementImproving existing functionalityneofs-storageStorage node application issuesperformanceMore of something per second

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions