Skip to content

Commit cdf18c9

Browse files
author
mmardani
committed
v0
0 parents  commit cdf18c9

File tree

251 files changed

+72519
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

251 files changed

+72519
-0
lines changed

‎.gitignore

+139
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Custom
2+
outputs/
3+
_exp
4+
_data
5+
dist/
6+
!_configs/exp/
7+
!_configs/dist/
8+
9+
src/
10+
11+
# Byte-compiled / optimized / DLL files
12+
__pycache__/
13+
*.py[cod]
14+
*$py.class
15+
*.npy
16+
17+
# C extensions
18+
*.so
19+
20+
# Distribution / packaging
21+
.Python
22+
build/
23+
develop-eggs/
24+
downloads/
25+
eggs/
26+
.eggs/
27+
lib/
28+
lib64/
29+
parts/
30+
sdist/
31+
var/
32+
wheels/
33+
pip-wheel-metadata/
34+
share/python-wheels/
35+
*.egg-info/
36+
.installed.cfg
37+
*.egg
38+
MANIFEST
39+
40+
# PyInstaller
41+
# Usually these files are written by a python script from a template
42+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
43+
*.manifest
44+
*.spec
45+
46+
# Installer logs
47+
pip-log.txt
48+
pip-delete-this-directory.txt
49+
50+
# Unit test / coverage reports
51+
htmlcov/
52+
.tox/
53+
.nox/
54+
.coverage
55+
.coverage.*
56+
.cache
57+
nosetests.xml
58+
coverage.xml
59+
*.cover
60+
*.py,cover
61+
.hypothesis/
62+
.pytest_cache/
63+
64+
# Translations
65+
*.mo
66+
*.pot
67+
68+
# Django stuff:
69+
*.log
70+
local_settings.py
71+
db.sqlite3
72+
db.sqlite3-journal
73+
74+
# Flask stuff:
75+
instance/
76+
.webassets-cache
77+
78+
# Scrapy stuff:
79+
.scrapy
80+
81+
# Sphinx documentation
82+
docs/_build/
83+
84+
# PyBuilder
85+
target/
86+
87+
# Jupyter Notebook
88+
.ipynb_checkpoints
89+
90+
# IPython
91+
profile_default/
92+
ipython_config.py
93+
94+
# pyenv
95+
.python-version
96+
97+
# pipenv
98+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
99+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
100+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
101+
# install all needed dependencies.
102+
#Pipfile.lock
103+
104+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
105+
__pypackages__/
106+
107+
# Celery stuff
108+
celerybeat-schedule
109+
celerybeat.pid
110+
111+
# SageMath parsed files
112+
*.sage.py
113+
114+
# Environments
115+
.env
116+
.venv
117+
env/
118+
venv/
119+
ENV/
120+
env.bak/
121+
venv.bak/
122+
123+
# Spyder project settings
124+
.spyderproject
125+
.spyproject
126+
127+
# Rope project settings
128+
.ropeproject
129+
130+
# mkdocs documentation
131+
/site
132+
133+
# mypy
134+
.mypy_cache/
135+
.dmypy.json
136+
dmypy.json
137+
138+
# Pyre type checker
139+
.pyre/

‎LICENSE

+95
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
Copyright (c) 2023-2024, NVIDIA Corporation & affiliates. All rights reserved.
2+
3+
4+
NVIDIA Source Code License
5+
6+
7+
=======================================================================
8+
9+
1. Definitions
10+
11+
"Licensor" means any person or entity that distributes its Work.
12+
13+
"Software" means the original work of authorship made available under
14+
this License.
15+
16+
"Work" means the Software and any additions to or derivative works of
17+
the Software that are made available under this License.
18+
19+
The terms "reproduce," "reproduction," "derivative works," and
20+
"distribution" have the meaning as provided under U.S. copyright law;
21+
provided, however, that for the purposes of this License, derivative
22+
works shall not include works that remain separable from, or merely
23+
link (or bind by name) to the interfaces of, the Work.
24+
25+
Works, including the Software, are "made available" under this License
26+
by including in or with the Work either (a) a copyright notice
27+
referencing the applicability of this License to the Work, or (b) a
28+
copy of this License.
29+
30+
2. License Grants
31+
32+
2.1 Copyright Grant. Subject to the terms and conditions of this
33+
License, each Licensor grants to you a perpetual, worldwide,
34+
non-exclusive, royalty-free, copyright license to reproduce,
35+
prepare derivative works of, publicly display, publicly perform,
36+
sublicense and distribute its Work and any resulting derivative
37+
works in any form.
38+
39+
3. Limitations
40+
41+
3.1 Redistribution. You may reproduce or distribute the Work only
42+
if (a) you do so under this License, (b) you include a complete
43+
copy of this License with your distribution, and (c) you retain
44+
without modification any copyright, patent, trademark, or
45+
attribution notices that are present in the Work.
46+
47+
3.2 Derivative Works. You may specify that additional or different
48+
terms apply to the use, reproduction, and distribution of your
49+
derivative works of the Work ("Your Terms") only if (a) Your Terms
50+
provide that the use limitation in Section 3.3 applies to your
51+
derivative works, and (b) you identify the specific derivative
52+
works that are subject to Your Terms. Notwithstanding Your Terms,
53+
this License (including the redistribution requirements in Section
54+
3.1) will continue to apply to the Work itself.
55+
56+
3.3 Use Limitation. The Work and any derivative works thereof only
57+
may be used or intended for use non-commercially. Notwithstanding
58+
the foregoing, NVIDIA and its affiliates may use the Work and any
59+
derivative works commercially. As used herein, "non-commercially"
60+
means for research or evaluation purposes only.
61+
62+
3.4 Patent Claims. If you bring or threaten to bring a patent claim
63+
against any Licensor (including any claim, cross-claim or
64+
counterclaim in a lawsuit) to enforce any patents that you allege
65+
are infringed by any Work, then your rights under this License from
66+
such Licensor (including the grant in Section 2.1) will terminate
67+
immediately.
68+
69+
3.5 Trademarks. This License does not grant any rights to use any
70+
Licensor’s or its affiliates’ names, logos, or trademarks, except
71+
as necessary to reproduce the notices described in this License.
72+
73+
3.6 Termination. If you violate any term of this License, then your
74+
rights under this License (including the grant in Section 2.1) will
75+
terminate immediately.
76+
77+
4. Disclaimer of Warranty.
78+
79+
THE WORK IS PROVIDED "AS IS" WITHOUT WARRANTIES OR CONDITIONS OF ANY
80+
KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF
81+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR
82+
NON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER
83+
THIS LICENSE.
84+
85+
5. Limitation of Liability.
86+
87+
EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL
88+
THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE
89+
SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,
90+
INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF
91+
OR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK
92+
(INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,
93+
LOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER
94+
COMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF
95+
THE POSSIBILITY OF SUCH DAMAGES.

‎README.md

+152
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Regularization by denoising diffusion process (RED-diff)
2+
3+
This is the README file for the implementaiton of variaitonal smapling in the paper https://arxiv.org/abs/2305.04391. It provides an overview of the project, instructions for installation and usage, and other relevant information.
4+
5+
6+
## Description
7+
8+
This codebase implements a variational posterior sampling method termed RED-diff. It is orginated from varaitional inference by minimizing the KL divergence between the data distribution p(x) and a variational approximaiton q(x). We show that this leads to score matching regulrization, and assuming dirac distribution for q(x), it yields very simple updates that resemble regulrization by denoising diffusion process.
9+
10+
This codebase includes experiments for various linear and nonlinear scenarios such as image inpainting, superresolution, Gaussian deblurring, motion debluring, high dynamic range (HDR), phase retreival, and nonlinear deblurring. We use Palette dataset for evaluation. The animation shows forward noising process ({$x_t$}) [left] and backward denoising prpcess ({$\mu_t$}) [right] based on RED-diff optimization.
11+
12+
13+
![Animated GIF](demo/output.gif)
14+
15+
## Installation
16+
17+
1. Clone the repository:
18+
19+
```
20+
cd <root>
21+
git clone https://gitlab-master.nvidia.com/mmardani/red-diff.git .
22+
```
23+
24+
2. Download pretrained checkpoints: checkpoints available at _exp/ckpt
25+
26+
<!-- ```
27+
##unconditional guided diffusion for ImageNet
28+
imagenet/256x256_diffusion_uncond.pt
29+
##class conditional guided diffusion for ImageNet
30+
imagenet/256x256_diffusion.pt
31+
imagenet/256x256_classifier.pt
32+
##guided diffusion with ffhq data
33+
ffhq/ffhq_10m.pt
34+
``` -->
35+
36+
37+
| Description | File |
38+
|-----------------------------------------------------|-------------------------------------------|
39+
| Unconditional guided diffusion for ImageNet | imagenet/256x256_diffusion_uncond.pt |
40+
| Class conditional guided diffusion for ImageNet | imagenet/256x256_diffusion.pt |
41+
| Classifier for ImageNet (256x256) | imagenet/256x256_classifier.pt |
42+
| Guided diffusion with FFHQ data | ffhq/ffhq_10m.pt |
43+
44+
45+
3. Install the dependencies:
46+
47+
```
48+
pip install -r requirements.txt
49+
```
50+
51+
4. Add external dependencies: we use external codes for motion-blurring and non-linear deblurring.
52+
53+
```
54+
git clone https://github.com/VinAIResearch/blur-kernel-space-exploring bkse
55+
git clone https://github.com/LeviBorodenko/motionblur motionblur
56+
```
57+
58+
59+
## Usage
60+
61+
1. Choose the degradation. The degradation.py covers several linear and nolinear tasks
62+
63+
```
64+
deg=in2_20ff #[sr4, deblur, deblur_gauss, deblur_uni, phase_retrieval, deblur_nl, hdr]
65+
```
66+
67+
2. Choose the algorithm.
68+
69+
```
70+
algo=reddiff #[ddrm, pgdm, reddiff, dps, mcg]
71+
```
72+
73+
3. Tune the hyperparamaters interactively using sampling script:
74+
75+
76+
```
77+
sh sample_test.sh
78+
```
79+
80+
An instance of sampling script is as fllows
81+
82+
```
83+
samples_root=<root>/_exp
84+
model.ckpt=imagenet256_uncond
85+
#-cn imagenet256_cond #for conditional imagenent ckpt with classifier guidance
86+
algo=reddiff
87+
deg=in2_20ff
88+
num_steps=100 #number of diffusion steps
89+
sigma_y=0.0 #measurement noise std
90+
batch_size=10
91+
lambda=0.25
92+
lr=0.5
93+
94+
python main.py algo=$algo algo.deg=$deg exp.num_steps=$num_steps algo.sigma_y=$sigma_y loader.batch_size=$batch_size loader=imagenet256_ddrmpp dist.num_processes_per_node=1 exp.name=debug exp.save_ori=True exp.smoke_test=1 exp.samples_root=$samples_root exp.save_evolution=True algo.lr=$lr algo.grad_term_weight=$lambda
95+
```
96+
97+
4. For more data run the batch job
98+
99+
```
100+
sh sample_batch.sh #for batch jobs
101+
```
102+
103+
104+
105+
5. Run the evaluation script:
106+
107+
```
108+
sh run_eval.sh
109+
```
110+
111+
112+
## Citation
113+
If you find our work interesting, please consider citing
114+
115+
```
116+
@article{mardani2023variational,
117+
title={A Variational Perspective on Solving Inverse Problems with Diffusion Models},
118+
author={Mardani, Morteza and Song, Jiaming and Kautz, Jan and Vahdat, Arash},
119+
journal={arXiv preprint arXiv:2305.04391},
120+
year={2023}
121+
}
122+
```
123+
124+
```
125+
@inproceedings{song2023pseudoinverse,
126+
title={Pseudoinverse-guided diffusion models for inverse problems},
127+
author={Song, Jiaming and Vahdat, Arash and Mardani, Morteza and Kautz, Jan},
128+
booktitle={International Conference on Learning Representations},
129+
year={2023}
130+
}
131+
```
132+
133+
## License
134+
135+
This project is licensed under the [NVIDIA Source Code License](LICENSE).
136+
137+
## Contact
138+
139+
For any inquiries, please contact [[email protected]](mailto:[email protected]).
140+
141+
<!-- ## Acknowledgement
142+
143+
We want to thank Jiaming Song for his contributions to the codebase. -->
144+
145+
146+
147+
148+
149+
150+
151+
152+

‎_configs/algo/ddim.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved
2+
3+
name: 'ddim'
4+
eta: 0.5
5+
sdedit: False
6+
cond_awd: False

0 commit comments

Comments
 (0)