Skip to content

Commit

Permalink
1. diffusers support Sana;
Browse files Browse the repository at this point in the history
2. update README.md;

Signed-off-by: lawrence-cj <[email protected]>
  • Loading branch information
lawrence-cj committed Dec 13, 2024
1 parent 41dcbe9 commit 407f99a
Show file tree
Hide file tree
Showing 2 changed files with 67 additions and 2 deletions.
68 changes: 67 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ As a result, Sana-0.6B is very competitive with modern giant diffusion model (e.

## 🔥🔥 News

- (🔥 New) \[2024/12/13\] `diffusers` has Sana! [All Sana models in diffusers safetensors](https://huggingface.co/collections/Efficient-Large-Model/sana-673efba2a57ed99843f11f9e) are released and diffusers pipeline `SanaPipeline`, `SanaPAGPipeline`, `DPMSolverMultistepScheduler(with FlowMatching)` are all supported now.
- (🔥 New) \[2024/12/10\] 1.6B BF16 [Sana model](https://huggingface.co/Efficient-Large-Model/Sana_1600M_1024px_BF16) is released for stable fine-tuning.
- (🔥 New) \[2024/12/9\] We release the [ComfyUI node](https://github.com/Efficient-Large-Model/ComfyUI_ExtraModels) for Sana. [\[Guidance\]](asset/docs/ComfyUI/comfyui.md)
- (🔥 New) \[2024/11\] All multi-linguistic (Emoji & Chinese & English) SFT models are released: [1.6B-512px](https://huggingface.co/Efficient-Large-Model/Sana_1600M_512px_MultiLing), [1.6B-1024px](https://huggingface.co/Efficient-Large-Model/Sana_1600M_1024px_MultiLing), [600M-512px](https://huggingface.co/Efficient-Large-Model/Sana_600M_512px), [600M-1024px](https://huggingface.co/Efficient-Large-Model/Sana_600M_1024px). The metric performance is shown [here](#performance)
Expand Down Expand Up @@ -123,6 +124,69 @@ python app/app_sana.py \
--model_path=hf://Efficient-Large-Model/Sana_1600M_1024px/checkpoints/Sana_1600M_1024px.pth
```

### 1. How to use `SanaPipeline` with `🧨diffusers`

run `pip install -U diffusers` before use Sana in diffusers

```python
import torch
from diffusers import SanaPipeline

pipe = SanaPipeline.from_pretrained(
"Efficient-Large-Model/Sana_1600M_1024px_diffusers",
variant="fp16",
torch_dtype=torch.float16,
)
pipe.to("cuda")

pipe.vae.to(torch.bfloat16)
pipe.text_encoder.to(torch.bfloat16)

prompt = 'a cyberpunk cat with a neon sign that says "Sana"'
image = pipe(
prompt=prompt,
height=1024,
width=1024,
guidance_scale=5.0,
num_inference_steps=20,
generator=torch.Generator(device="cuda").manual_seed(42),
)[0]

image[0].save("sana.png")
```

### 2. How to use `SanaPAGPipeline` with `🧨diffusers`

```python
# run `pip install -U diffusers` before use Sana in diffusers
import torch
from diffusers import SanaPAGPipeline

pipe = SanaPAGPipeline.from_pretrained(
"Efficient-Large-Model/Sana_1600M_1024px_diffusers",
variant="fp16",
torch_dtype=torch.float16,
pag_applied_layers="transformer_blocks.8"
)
pipe.to("cuda")

pipe.text_encoder.to(torch.bfloat16)
pipe.vae.to(torch.bfloat16)

prompt = 'a cyberpunk cat with a neon sign that says "Sana"'
image = pipe(
prompt=prompt,
guidance_scale=5.0,
pag_scale=2.0,
num_inference_steps=20,
generator=torch.Generator(device="cuda").manual_seed(42),
)[0]
image[0].save('sana.png')
```

<details>
<summary><h3>3. How to use Sana in this repo</h3></summary>

```python
import torch
from app.sana_pipeline import SanaPipeline
Expand All @@ -147,8 +211,10 @@ image = sana(
save_image(image, 'output/sana.png', nrow=1, normalize=True, value_range=(-1, 1))
```

</details>

<details>
<summary><h2>Run Sana (Inference) with Docker</h2></summary>
<summary><h3>4. Run Sana (Inference) with Docker</h3></summary>

```
# Pull related models
Expand Down
1 change: 0 additions & 1 deletion asset/docs/ComfyUI/comfyui.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@

1. All the checkpoints will be downloaded automatically.
1. KSampler(Flow Euler) is available for now; Flow DPM-Solver will be available soon.
1. For more information, check the [original city96/ComfyUI_ExtraModels](https://github.com/city96/ComfyUI_ExtraModels).

```bash
git clone https://github.com/comfyanonymous/ComfyUI.git
Expand Down

0 comments on commit 407f99a

Please sign in to comment.