Skip to content

[feature request] --num-experts-offload-per-rank is gone #433

Description

@connectionRst

the MoE Expert CPU offload (EP, expert parallel) is introduced in #130, with argument --num-experts-offload-per-rank for use
then in #203 it changed to environment variable NUM_EXPERTS_OFFLOAD_PER_RANK
then in #403, a big refactor let the feature not available anymore

ref: guoqingbao/xinfer#161

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions