Issues

• [Bug] FSDP1 block statistics are not reduced when each rank sees only one GPU

  zyushun/Adam-mini#42 · Filter by author gss10282023 opened on Sep 17, 2026

• No data folder inside the examples/gpt2

  zyushun/Adam-mini#41 · Filter by author ndvbd opened on Apr 19, 2026
  2

• Can't convert the checkpoint to huggingface format for inference

  zyushun/Adam-mini#40 · Filter by author kv-wang opened on Oct 13, 2025
  1

• Adam-mini Fails to Recognize Qwen 2.5 Math Model Architecture

  zyushun/Adam-mini#38 · Filter by author Ting011 opened on Feb 28, 2025
  2

• Why the distributed communication only used when tmp_lr is scalar

  zyushun/Adam-mini#37 · Filter by author Wenhua-Hu opened on Feb 15, 2025
  1

• FSDP and tensor-paralllel compatibility

  zyushun/Adam-mini#36 · Filter by author asahniSC opened on Dec 16, 2024
  10

• Default process group has not been initialized

  zyushun/Adam-mini#35 · Filter by author yi249 opened on Dec 11, 2024
  2

• No module named 'torch.distributed._tensor'

  zyushun/Adam-mini#33 · Filter by author SK-HC opened on Nov 29, 2024
  4

• No Query or Key Found when using nn.TransformerEncoderLayer

  zyushun/Adam-mini#30 · Filter by author ACautomata opened on Sep 25, 2024
  25

• Hessian Calculation

  zyushun/Adam-mini#29 · Filter by author lucasliunju opened on Sep 25, 2024
  1

• RuntimeError: No backend type associated with device type cpu

  zyushun/Adam-mini#28 · Filter by author minienglish1 opened on Sep 18, 2024
  3

• does not support bf16-only training

  zyushun/Adam-mini#26 · Filter by author bghira opened on Aug 18, 2024
  1

• Qwen2-0.5B cannot be Adam-mini-optimized in 4 shards (Deepspeed Zero-3)

  zyushun/Adam-mini#25 · Filter by author xiningnlp opened on Aug 16, 2024
  3

• Megatron support, please

  zyushun/Adam-mini#23 · Filter by author zhuango opened on Aug 15, 2024
  6

• ValueError: algorithm not supported when running gpt2 example

  zyushun/Adam-mini#20 · Filter by author Benjamin-eecs opened on Jul 25, 2024
  2

• 集成到transformer的trainer后会会爆显存，用adamW不会

  zyushun/Adam-mini#15 · Filter by author Panda-eat-meat opened on Jul 16, 2024
  1

• 使用Adam-mini，全参量微调7B模型需要多少显存？

  zyushun/Adam-mini#13 · Filter by author NTDXYG opened on Jul 14, 2024
  4

• Memory saving only in checkpoint size, not during training

  zyushun/Adam-mini#8 · Filter by author aditya2331 opened on Jul 6, 2024
  7

• works well on smaller models, updates for torchtitan and 8B size

  zyushun/Adam-mini#7 · Filter by author lessw2020 opened on Jul 4, 2024
  23

• Adam mini can't save when using with FSDP in Huggingface Trainer

  zyushun/Adam-mini#5 · Filter by author hahuyhoang411 opened on Jun 27, 2024
  19

• Is there any plant to implement Quantized Adam-mini?

  zyushun/Adam-mini#4 · Filter by author Kyeongpil opened on Jun 27, 2024
  4