-
Notifications
You must be signed in to change notification settings - Fork 89
Push streaming audioseal #98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds streaming support to AudioSeal by integrating the Moshi SEANet backend for Python 3.10+, enabling causal training and real-time watermarking. The changes also make models TorchScript-compatible, deprecate internal audio resampling (shifting responsibility to users), and fix bugs in message handling.
Key changes:
- Add streaming-capable models (
audioseal_wm_streaming,audioseal_detector_streaming) with context manager API - Refactor for TorchScript compatibility by removing weight normalization hooks before scripting
- Deprecate internal 16kHz resampling - users must now provide correctly sampled audio
Reviewed changes
Copilot reviewed 27 out of 31 changed files in this pull request and generated 16 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_models.py | Adds JIT scripting test for generator and detector models |
| src/audioseal/models.py | Adds NormalizationProcessor, streaming context manager, TorchScript exports, deprecation warnings |
| src/audioseal/loader.py | Implements state dict conversion for Moshi models, improved weight norm handling |
| src/audioseal/builder.py | Adds version-conditional imports for SEANet (Moshi vs Audiocraft) |
| src/scripts/checkpoints.py | Improves checkpoint conversion with prefix support and safer torch.load |
| src/audioseal/libs/moshi/* | Integrates Moshi library modules (SEANet, streaming, transformer, etc.) |
| src/audioseal/cards/*.yaml | Adds configuration cards for streaming models |
| README.md | Documents streaming API usage and requirement changes |
| CHANGELOG.md | Documents version 0.2.0 changes |
| requirements.txt, pyproject.toml | Removes julius, adds einops for Python 3.10+ |
| .github/workflows/lint_and_test.yaml | Updates CI to Python 3.10 and removes julius dependency |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
hadyelsahar
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Why ?
This big PR pushes the internal changes of AudioSeal inside Meta to the public repository. It consists of several improvements, fixes and new features:
audioseal_wm_streaming,audioseal_detector_streaming)How ?
Test plan