Skip to content

About how to calculate total_batch_size #9

@xwg0508

Description

@xwg0508

Thanks for your code! I'd like to know how to calculate the total_batch_size? Is total_batch_size = batch_size(config.yaml) * gpus * accumulation_steps?
But in your example of trainning for SSV2, you set nproc_per_node=8, accumulation-steps =8, and in config.yaml batch_size=8, it can not calculate 256 for total_batch_size

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions