Skip to content

Commit 9f5a723

Browse files
committed
Add Veo video generation documentation
1 parent 34aadbf commit 9f5a723

1 file changed

Lines changed: 138 additions & 4 deletions

File tree

README.md

Lines changed: 138 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ OpenTryOn is an open-source AI toolkit designed for fashion technology and virtu
2929
- GPT-Image-1 & GPT-Image-1.5 (OpenAI): High-quality image generation with strong prompt understanding, consistent composition, and reliable visual accuracy. GPT-Image-1.5 offers enhanced quality and better consistency
3030
- **Video Generation**:
3131
- Luma AI Video Generation Model (Dream Machine): High-quality video generation with text-to-image and image-to-video modes.
32+
- Google Veo 3 Video Generation Model: Generate high-quality, cinematic videos from text or images with realistic motion, temporal consistency, and fine-grained control over style and camera dynamics.
3233
- **Datasets Module**:
3334
- Fashion-MNIST dataset loader with automatic download
3435
- VITON-HD dataset loader with lazy loading via PyTorch DataLoader
@@ -61,6 +62,7 @@ OpenTryOn is an open-source AI toolkit designed for fashion technology and virtu
6162
- [Image Generation with Luma AI](#luma-ai-image-generation)
6263
- [Image Generation with OpenAI](#image-generation-with-gpt-image-1)
6364
- [Video Generation with Luma AI](#video-generation-with-luma-ai)
65+
- [Video Generation with Google Veo 3](#video-generation-with-google-veo-3)
6466
- [Preprocessing Functions](#preprocessing-functions)
6567
- [Demos](#demos)
6668
- [Project Structure](#project-structure)
@@ -138,13 +140,13 @@ KLING_AI_BASE_URL=https://api-singapore.klingai.com # Optional, defaults to Sin
138140
# Segmind Credentials (required for Segmind virtual try-on)
139141
SEGMIND_API_KEY=your_segmind_api_key
140142
141-
# Google Gemini Credentials (required for Nano Banana image generation)
143+
# Google Gemini Credentials (required for Nano Banana image generation and Google Veo 3 Video generation)
142144
GEMINI_API_KEY=your_gemini_api_key
143145
144146
# BFL API Credentials (required for FLUX.2 image generation)
145147
BFL_API_KEY=your_bfl_api_key
146148
147-
# Luma AI Credentials (required for Luma AI image generation)
149+
# Luma AI Credentials (required for Luma AI image generation and Luma AI Video generation)
148150
LUMA_AI_API_KEY=your_luma_ai_api_key
149151
150152
# OpenAI Credentials (required for OpenAI GPT-Image-1 image generation)
@@ -164,7 +166,7 @@ GOOGLE_API_KEY=your_google_api_key # For Google Gemini
164166
- For Kling AI, obtain your API key and secret key from the [Kling AI Developer Portal](https://app.klingai.com/global/dev/document-api/apiReference/model/functionalityTry)
165167

166168
- For Segmind, obtain your API key from the [Segmind API Portal](https://www.segmind.com/models/try-on-diffusion/api)
167-
- For Nano Banana, obtain your API key from the [Google AI Studio](https://aistudio.google.com/app/apikey)
169+
- For Nano Banana and Google Veo 3, obtain your API key from the [Google AI Studio](https://aistudio.google.com/app/apikey)
168170
- For FLUX.2 models, obtain your API key from [BFL AI](https://docs.bfl.ai/)
169171

170172
- For FLUX.2 models, obtain your API key from [BFL AI](https://docs.bfl.ai/)
@@ -1621,7 +1623,7 @@ for idx, vid_bytes in enumerate(video_list):
16211623

16221624
#### Supported Features
16231625

1624-
- **Text to Video**: Generate videos using test descriptions.
1626+
- **Text to Video**: Generate videos using text descriptions.
16251627
- **Image to Video**: Generate videos using keyframes.
16261628
- **Keyframe Generation**: Generate videos using a start keyframe or an end keyframe or both.
16271629
- **Duration**: Durations in seconds (5s, 9s, 10s)
@@ -1639,6 +1641,138 @@ for idx, vid_bytes in enumerate(video_list):
16391641

16401642
**Reference**: [Luma AI Video Generation Documentation](https://docs.lumalabs.ai/docs/video-generation)
16411643

1644+
### Video Generation with Google Veo 3
1645+
1646+
Generate high-quality, cinematic videos using Google’s Veo 3 models (Veo 3.0 and Veo 3.1), including (veo-3.1-generate-preview, veo-3.1-fast-generate-preview, veo-3.0-generate-001, and veo-3.0-fast-generate-001). These models support text-to-video, image-to-video, reference-images-to-video, and frames-to-video generation for controlled motion, realistic dynamics, and consistent visual quality.
1647+
1648+
#### Prerequisites
1649+
1650+
1. **Google Gemini Account Setup**:
1651+
- Sign up for a Google AI Studio account at [Google AI Studio](https://aistudio.google.com/)
1652+
- Obtain your API key from the [API Keys page](https://aistudio.google.com/app/apikey)
1653+
- Configure credentials in your `.env` file (see Environment Variables section)
1654+
1655+
2. **Model Selection**:
1656+
- **veo-3.1-generate-preview**: Generate high-quality cinematic videos with enhanced motion realism and temporal consistency using the latest Veo 3.1 model.
1657+
- **veo-3.1-fast-generate-preview**: Create videos quickly with optimized inference speed while retaining strong visual quality and motion coherence.
1658+
- **veo-3.0-generate-001**: Produce stable, high-fidelity videos using the proven Veo 3.0 generation model with reliable motion and style control.
1659+
- **veo-3.0-fast-generate-001**: Generate videos faster with the Veo 3.0 fast variant, balancing speed and visual quality for rapid iteration.
1660+
1661+
#### Command Line Usage
1662+
1663+
```bash
1664+
# Text to Video with Google Veo 3
1665+
python veo_video.py --provider veo-3.1-generate-preview --mode text --prompt "model at a fashion show" --aspect 16:9 --duration 8 --resolution 1080p --output_dir outputs
1666+
1667+
# Video generation with negative prompt
1668+
python veo_video.py --provider veo-3.1-generate-preview --mode text --prompt "person with a hat" --resolution 1080p --negative_prompt "cartoon, anime, kids"
1669+
1670+
# Image to Video
1671+
python veo_video.py --provider veo-3.1-generate-preview --mode image --prompt "model at a fashion show" --images person.jpg --aspect 16:9 --duration 8 --resolution 1080p
1672+
1673+
# Video generation with reference images (up to 3)
1674+
python veo_video.py --provider veo-3.1-generate-preview --mode reference --prompt "create a fashion week video" --images person1.jpg person2.jpg person3.jpg --resolution 1080p
1675+
1676+
# Video generation with frames
1677+
python veo_video.py --provider veo-3.1-generate-preview --mode frames --prompt "create a cinematic video" --start_image person1.jpg --end_image person2.jpg --aspect 16:9 --resolution 720p
1678+
```
1679+
1680+
#### Python API Usage
1681+
1682+
**Google Veo 3**
1683+
1684+
```python
1685+
1686+
from dotenv import load_dotenv
1687+
load_dotenv()
1688+
1689+
from pathlib import Path
1690+
from tryon.api.veo import VeoAdapter
1691+
1692+
adapter = VeoAdapter()
1693+
video_list = []
1694+
1695+
1696+
def save_video(video_bytes: bytes, idx: int):
1697+
Path("outputs").mkdir(exist_ok=True)
1698+
out_path = Path("outputs") / f"generated_{idx}.mp4"
1699+
with open(out_path, "wb") as f:
1700+
f.write(video_bytes)
1701+
print(f"[SAVED] {out_path}")
1702+
1703+
1704+
# TEXT → VIDEO
1705+
video = adapter.generate_text_to_video(
1706+
prompt="A cinematic neon city with cars moving at night",
1707+
duration_seconds="4",
1708+
aspect_ratio="16:9",
1709+
resolution="720p",
1710+
model="veo-3.1-generate-preview",
1711+
)
1712+
video_list.append(video)
1713+
1714+
1715+
# IMAGE → VIDEO
1716+
video = adapter.generate_image_to_video(
1717+
image="model.jpg",
1718+
prompt="Two monsters fighting with each other",
1719+
duration_seconds="4",
1720+
aspect_ratio="16:9",
1721+
resolution="720p",
1722+
model="veo-3.1-generate-preview",
1723+
negative_prompt="cartoon, anime, for kids",
1724+
)
1725+
video_list.append(video)
1726+
1727+
1728+
# REFERENCE IMAGES → VIDEO
1729+
video = adapter.generate_video_with_references(
1730+
prompt="A fashion model walking on a runway",
1731+
reference_images=[
1732+
"test_assets/ref1.jpg",
1733+
"test_assets/ref2.jpg",
1734+
],
1735+
duration_seconds="8",
1736+
aspect_ratio="16:9",
1737+
resolution="720p",
1738+
model="veo-3.1-generate-preview",
1739+
)
1740+
video_list.append(video)
1741+
1742+
1743+
# FIRST + LAST FRAME → VIDEO
1744+
video = adapter.generate_video_with_frames(
1745+
prompt="Smooth cinematic transition from grizzly bear to polar bear",
1746+
first_image="person1.jpg",
1747+
last_image="person2.jpg",
1748+
duration_seconds="8",
1749+
aspect_ratio="16:9",
1750+
resolution="720p",
1751+
model="veo-3.1-generate-preview",
1752+
negative_prompt="cartoon, anime, kids",
1753+
)
1754+
video_list.append(video)
1755+
1756+
1757+
# SAVE ALL RESULTS
1758+
for idx, vid_bytes in enumerate(video_list):
1759+
save_video(vid_bytes, idx)
1760+
1761+
```
1762+
1763+
#### Supported Features
1764+
1765+
- **Text to Video**: Generate Video using text descriptions.
1766+
- **Image to Video**: Generate Video using a single image.
1767+
- **Video Generation with Reference Images**: Generate Video using reference Images (up to 3).
1768+
- **Video Generation with Frames**: Video Generation with first frame and last frame.
1769+
- **Duration**: Durations in seconds (4s, 6s, 8s)
1770+
- **Resolution**: Quality of the video (720p, 1080p)
1771+
- **Aspect Ratio**: Aspect Ratio of videos (16:9, 9:16)
1772+
- **Negative Prompt**: Negative Prompt tells the Veo model what to avoid generating in the video.
1773+
1774+
**Reference**: [Google Veo 3 Video Generation Documentation](https://ai.google.dev/gemini-api/docs/video)
1775+
16421776
### Preprocessing Functions
16431777

16441778
#### Segment Garment

0 commit comments

Comments
 (0)