Score-based Generative Model with Adaptive Momentum Sampling

This repo contains the official implementation for the paper Score-based Generative Model with Adaptive Momentum Sampling, and is highly build upon the excellent previous work by Yang Song in Score-Based Generative Modeling through Stochastic Differential Equations and Jaehyeong Jo in Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations. For any problems, please contact us.

Reproducibility

For the image generation tasks, please run the following

cd /ImageGeneration

Our experiments were mainly conducted by Python 3.8.18 with a CUDA version 11.8. See the requirements.txt, run

pip install -r requirements

For the pre-trained checkpoints, please refer to Score-Based Generative Modeling through Stochastic Differential Equations, Google drive.

We provide our generated data in Onedrive, cd the folder and change the config file by setting

evaluate.enable_sampling = False

All the used parameters can found in the Appendix.

For the graph generation, please create a new python environment and run

cd /GraphGeneration
pip install -r requirements

For a more detailed usage and graph checkpoints, see the original code base.

Usage

Please refer to the Score-Based Generative Modeling through Stochastic Differential Equations, and change the folder. If you want to use the NCSN2 score net, use the original code by Song with our AMS sampler.

References

If you find the code useful for your research, please consider citing

@inproceedings{
  song2021scorebased,
  title={Score-Based Generative Modeling through Stochastic Differential Equations},
  author={Yang Song and Jascha Sohl-Dickstein and Diederik P Kingma and Abhishek Kumar and Stefano Ermon and Ben Poole},
  booktitle={International Conference on Learning Representations},
  year={2021},
  url={https://openreview.net/forum?id=PxTIG12RRHS}
}

@inproceedings{Jo2022ScorebasedGM,
  title={Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations},
  author={Jaehyeong Jo and Seul Lee and Sung Ju Hwang},
  booktitle={International Conference on Machine Learning},
  year={2022},
  url={https://api.semanticscholar.org/CorpusID:246634850}
}

and

{
@inproceedings{Wen2023NormalizedSH,
  title={Normalized Stochastic Heavy Ball with Adaptive Momentum},
  author={Ziqing Wen and Xiaoge Deng and Tao Sun and Dongsheng Li},
  booktitle={European Conference on Artificial Intelligence},
  year={2023},
  url={http://dx.doi.org/10.3233/FAIA230568}
}

@article{Wen2024ScorebasedGM,
  title={Score-based Generative Models with Adaptive Momentum},
  author={Ziqing Wen and Xiaoge Deng and Ping Luo and Tao Sun and Dongsheng Li},
  journal={ArXiv},
  year={2024},
  volume={abs/2405.13726},
  url={https://api.semanticscholar.org/CorpusID:269982664}
}

Samples (check the paper for more samples)

This work is built upon some previous papers you might interest:

• Yang Song, and Stefano Ermon. "Generative Modeling by Estimating Gradients of the Data Distribution." Proceedings of the 33rd Annual Conference on Neural Information Processing Systems. 2019.
• Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole "Score-Based Generative Modeling through Stochastic Differential Equations." International Conference on Learning Representations. 2021.
• Jaehyeong Jo, Seul Lee, and Sung Ju Hwang"Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations." International Conference on Machine Learning. 2022.
• Ziqing Wen, Xiaoge Deng, Tao Sun, and Dongsheng Li. "Normalized Stochastic Heavy Ball with Adaptive Momentum." European Conference on Artificial Intelligence. 2023.