NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
-
Updated
Jun 9, 2026 - Python
#
llm-safety
Here are 229 public repositories matching this topic...
DeepTeam is a framework to red team LLMs and AI agents.
-
Updated
Jun 9, 2026 - Python
UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection
uncertainty-quantification uncertainty-estimation ai-safety confidence-score hallucination confidence-estimation ai-evaluation llm llm-evaluation llm-safety hallucination-evaluation hallucination-detection hallucination-mitigation llm-hallucination
-
Updated
Jun 8, 2026 - Python
Internal Safety Collapse (ISC): Turning the LLM or an AI Agent into a sensitive data generator.
benchmark jailbreak ai-safety red-teaming large-language-models llm-safety safety-evaluation agent-safety
-
Updated
Jun 7, 2026 - Python
A resource repository for machine unlearning in large language models
nlp machine-learning awesome privacy evaluation alignment awesome-list ai-safety right-to-be-forgotten machine-unlearning paper-list responsible-ai unlearning llm model-editing large-language-model llm-safety llm-unlearning selective-forgetting knowledge-erasure
-
Updated
Jun 10, 2026
Agent trace and tool-use safety evaluation lab.
-
Updated
May 2, 2026 - Python
Decrypted Generative Model safety files for Apple Intelligence containing filters
-
Updated
Jan 26, 2026 - Python
[NDSS'25 Best Technical Poster] A collection of automated evaluators for assessing jailbreak attempts.
-
Updated
Apr 1, 2025 - Python
Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 51 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.
python cli security mcp static-analysis owasp taint-analysis vulnerability-detection vulnerability-scanner ai-security ai-agent langchain prompt-injection llm-security llm-safety crewai ai-security-tool langchain-security-
-
Updated
Jun 7, 2026 - Python
Papers about red teaming LLMs and Multimodal models.
-
Updated
May 28, 2025
Attack to induce LLMs within hallucinations
-
Updated
May 17, 2024 - Python
Reading list for adversarial perspective and robustness in deep reinforcement learning.
deep-reinforcement-learning ai-safety machine-learning-security ai-security robust-machine-learning ai-alignment safe-reinforcement-learning robust-reinforcement-learning responsible-ai adversarial-reinforcement-learning secure-ai ml-security llm-security reinforcement-learning-safety llm-safety artificial-intelligence-security artificial-intelligence-alignment robust-deep-reinforcement-learning reinforcement-learning-security large-language-model-safety
-
Updated
Mar 2, 2026
Open Source Reliability Harness: Make your agents follow rules. One line of code to enforce, trace, and improve.
rule-engine ai-safety policy-engine ai-security guardrails responsible-ai ai-policy ai-compliance ai-governance prompt-injection llm-security llm-proxy llm-safety content-safety llm-monitoring llm-guardrails agentic-ai ai-guardrails ai-firewall ai-audit
-
Updated
May 23, 2026 - Python
Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses | 500+ Papers | Perception, Cognition, Planning, Interaction, Agentic System
robotics jailbreak survey awesome-list autonomous-driving ai-safety adversarial-attacks red-teaming backdoor-attacks adversarial-robustness embodied-ai trustworthy-ai large-language-models prompt-injection world-model llm-safety vision-language-action embodied-agents agent-safety world-action-model
-
Updated
Jun 3, 2026 - Shell
[NeurIPS 2025] SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
-
Updated
Dec 10, 2025 - Python
Official repository for the paper "ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming"
nlp benchmark ai artificial-intelligence nlp-machine-learning red-teaming bias-detection safety-monitoring transformers-models llm llm-evaluation llm-safety llm-safety-benchmark
-
Updated
Sep 20, 2024 - Python
AISecOps (AI Security Operations) framework for deterministic verification of AI systems. QWED verifies LLM outputs using math, logic, and symbolic execution — creating an auditable trust boundary for agentic AI systems. Not generation. Verification.
python nlp machine-learning sympy smt-solver formal-verification ai-safety ai-security hallucination code-security ai-accuracy generative-ai enterprise-ai neurosymbolic-ai llm-safety hallucination-detection z3-prover deterministic-ai aisecops llm-verification
-
Updated
Jun 6, 2026 - Python
Papers from our SoK on Red-Teaming (Accepted at TMLR)
-
Updated
May 2, 2026
lintlang is a static linter for AI agent configs, tool descriptions, and system prompts that runs zero-LLM quality gating in CI. Catches language-level failures (vague tool descriptions, missing stop conditions, schema gaps) before they reach runtime, with deterministic regex + structural detectors and no model calls.
linter static-analysis ci-cd developer-tools ai-safety ai-agents quality-gate llm prompt-engineering llm-ops llm-safety system-prompts ai-audit ai-reliability agent-config prompt-linter tool-descriptions zero-llm hermes-labs agent-linter
-
Updated
Jun 2, 2026 - Python
NeurIPS'24 - LLM Safety Landscape
-
Updated
Oct 21, 2025 - Python
Improve this page
Add a description, image, and links to the llm-safety topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-safety topic, visit your repo's landing page and select "manage topics."