docs: describe training launches in terms of primus-cli - #976
Open
WangLingxun wants to merge 2 commits into
Open
docs: describe training launches in terms of primus-cli#976WangLingxun wants to merge 2 commits into
WangLingxun wants to merge 2 commits into
Conversation
WangLingxun
force-pushed
the
refactor/primus-cli-launchers
branch
10 times, most recently
from
August 14, 2026 09:14
5d583ef to
3016b30
Compare
WangLingxun
marked this pull request as ready for review
August 14, 2026 10:24
WangLingxun
requested review from
Xiaoming-AMD,
limou102 and
wenxie-amd
as code owners
August 14, 2026 10:24
WangLingxun
force-pushed
the
refactor/primus-cli-launchers
branch
from
August 17, 2026 03:07
3016b30 to
25ade5c
Compare
Every guide, config header and example still told users to export EXP and call examples/run_*.sh, even though `primus-cli` has been the supported entry point for a while. The instructions now use the runner modes and reference the hooks that own the behavior. - Rewrite launch snippets to direct/container/slurm forms and keep container environment options at the container layer, with a smoke test preventing the `-- --env DATA_PATH` parsing regression. - Document the HipBLASLt tuning master switch: the stage variable is only honored when `PRIMUS_HIPBLASLT_TUNING=1` and deterministic mode is off, which the prepare_experiment.sh hook enforces. - Point environment, determinism and ANP/NCCL_NET_PLUGIN references at the hooks and env files that actually set them instead of at the launcher script. - Add a docs test that fails when a Markdown command invokes a script the tree does not contain. The test matches only bash/sh/source invocations, so prose that merely names a script stays out of scope. The launcher scripts themselves are not removed here; retiring them, and the documentation that only becomes true once they are gone, follows in a stacked PR on top of this one.
WangLingxun
force-pushed
the
refactor/primus-cli-launchers
branch
from
August 20, 2026 07:25
25ade5c to
3196102
Compare
18 tasks
The repository root ships a `primus-cli` wrapper that execs `runner/primus-cli`, and the top-level README already spells launches as `./primus-cli`. Point the migrated examples at that entry point too, so the documentation does not carry two spellings of the same command. The directory tree in the diffusion structure guide keeps `runner/primus-cli`: it records where the implementation lives rather than how to invoke it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documentation-only. Rewrites every guide, example README and config header comment that still told users to
export EXPand runexamples/run_*.sh, so the whole repository describes launches in terms ofrunner/primus-cli.No behaviour changes and no scripts deleted here.
examples/run_pretrain.shand friends still exist on this branch and still work; this PR only stops the documentation from presenting them as the way to launch training. Retiring the scripts, migrating their downstream callers, and the fixes that came out of that migration are in the stacked follow-up #999, which is based on this branch.primus-cli direct,primus-cli container,primus-cli slurm srun— acrossdocs/,examples/, and the header comments of the diffusion and SFT configs. Container environment options stay at the container layer rather than leaking past the--into the Primus CLI.examples/README.mdgains a mode table explaining when to usedirect/container/slurm, the--argument shape, and where the environment defaults and setup hooks live (runner/helpers/envs/,runner/helpers/hooks/,runner/.primus.yaml).runner/helpers/hooks/train/pretrain/prepare_experiment.sh, deterministic mode atrunner/helpers/hooks/05_deterministic.shandrunner/helpers/envs/base_env.sh, and ANP /NCCL_NET_PLUGINselection atrunner/helpers/hooks/03_enable_ainic.sh. The HipBLASLt section previously omitted that the stage is ignored unlessPRIMUS_HIPBLASLT_TUNING=1; that is now stated where users hit it.tests/runner/test_docs_cli_examples.pyfails when a Markdown command invokes a.shthat is not in the tree, so the next launcher move cannot leave stale instructions behind. It matches onlybash/sh/sourceinvocations, so prose that merely names a script stays out of scope. A second assertion pins the hybrid-models guide's--volume/--env DATA_PATHplacement, which is the shape that is easy to get wrong when translating a snippet.Scope boundary
Documentation whose statements only become true once the launchers are actually gone is deliberately not in this PR — it lives in #999 alongside the code that makes it true:
examples/run_k8s_pretrain.sh;runner/helpers/launch/slurm_pretrain.sh, a helper refactor(runner): retire the examples launchers in favour of primus-cli #999 creates;primus-cli;TOKENIZED_*_DATA_PATH/ split-seed / rank-coordination sections, which describe prepare-hook behaviour refactor(runner): retire the examples launchers in favour of primus-cli #999 restores.The rule applied throughout: this branch must be self-consistent on its own. Nothing here references a file that does not exist yet, and nothing contradicts the tree as it stands.
Test plan
pre-commit run --from-ref origin/main --to-ref HEAD— clean.tests/runner/run_all_tests.sh— 10/10 suites.pytest tests/runner/— 2 passed, includingtest_docs_cli_examples.py, which confirms every documented script invocation resolves against this branch's tree (the oldexamples/run_*.shscripts are still present here and still referenced where they have not been rewritten).