Skip to content

v0.11.0

Latest
Latest

Choose a tag to compare

View all tags
@hannahwestra25 hannahwestra25 released this 06 Feb 06:15
· 27 commits to main since this release
v0.11.0
2eacdd9

What's Changed

⚠️ Breaking Changes

  • Attacks and executors now operate on Message instead of SeedPromptGroup
  • Scorer evaluation and registry refactors introduce new protocols and identifiers
  • Scenario names and configuration APIs have been renamed for consistency
  • PrependedConversationConfig and attack parameter handling have been aligned
  • Message normalization and registry metadata were refactored

Please review the deprecation notes and migration guidance before upgrading.

🎯 Targets

  • Added WebSocketCopilotTarget, enabling WebSocket-based prompt execution against Microsoft Copilot
  • Refactored ImageTarget, including image download support
  • Added image edit/remix support to OpenAIImageTarget
  • Introduced target identifiers (including underlying model and version metadata) across all target classes
  • Added audio and tool support to chat completions

📚 Datasets

  • Added VLSU Multimodal Dataset
  • Added 30 jailbreak attack templates, spanning:
    • Authority & institutional framing (6)
    • Philosophical / decision-theory exploits (5)
    • Identity / persona attacks (4)
    • Context manipulation (4)
    • Few-shot priming (3)
    • Fictional / narrative framing (3)
    • Technical exploits (3)
    • Emotional / scenario-based attacks (2)
  • Restored the Transphobia Awareness Dataset

🔄 Converters

  • Added NegationTrapConverter which frames requests as negations
  • Added ConverterIdentifier and standardized identifiable behavior
  • Reorganized and expanded converter documentation
  • Fixed edge cases in word-selection converters and perturbation loops

⚙️ Executors & Attacks

  • Aligned attack parameters across executors
  • Updated attack interface to use Message
  • Added ChunkedRequestAttack which extracts data by requesting it in small chunks
  • Added support for simulated conversations in attacks
  • Improved attack reliability, error reporting, and maintainability

📊 Scoring

  • Enabled multi-modal scoring support for SelfAskTrueFalseScorer, allowing image- and multimodal-aware evaluations
  • Refactored scorer evaluation flow and registry integration
  • Added scorer identifiers and improved metadata consistency
  • Introduced stricter typing and clearer scorer interfaces

🧪 Scanners & Scenarios

  • Added new scenarios:
    • Scams
    • Leakage
    • Psychosocial
  • Added ScenarioDatasetConfiguration allowing custom dataset configuration
  • Enabled baseline-only execution for scenarios
  • Renamed scenarios for clarity and consistency
  • Improved scenario documentation and example notebooks

🧰 Setup & Tooling

  • Added UV support for dependency management
  • Improved devcontainer experience:
    • ARM64 / Apple Silicon support
    • Simplified virtual environment handling
    • Environment file configurability
  • Consolidated linting under ruff
  • Enabled strict mypy checking across the repository
  • Added skeleton frontend and backend for the GUI

🧩 Other

  • Added new binary_path data type to support binary artifacts and richer schema definitions
  • Added identifiers across targets, scorers, and converters
  • Multiple reliability and integration test improvements

🐛 Fixes & Maintenance

  • Numerous fixes across:
    • Image handling and integration tests
    • Docker and devcontainer setup
    • Environment activation and permissions
    • Retry configuration and pipelines
  • Improved type hinting across authentication and analytics modules
  • Added py.typed for better downstream type checking

🆕 New Contributors

A big thank you to our new contributors! 🎉

Full List of Changes

📜 Full Changelog

https://github.com/Azure/PyRIT/commits/v0.11.0

Contributors

fitzpr, romanlutz, and 15 other contributors
Assets 2
Loading