Major Strix Upgrade: Core Agent Framework, Structured Exports, Bot Integration, and Expanded Tooling

.env.example

@@ -0,0 +1,19 @@
+BOT_TOKEN=telegram-bot-token
+WEBHOOK_URL=https://example.com/telegram/webhook
+ALLOWLIST_IDS=12345,67890
+
+# Optional settings
+BOT_TOKEN_FILE=/path/to/token.txt
+BOT_DB_PATH=bot_state.sqlite
+STRIX_ROOT=.
+BOT_HTTP_HOST=0.0.0.0
+BOT_HTTP_PORT=8081
+BOT_HTTP_TOKEN=changeme
+BOT_ALERT_WEBHOOK=https://example.com/alert-endpoint
+BOT_RATE_LIMIT=1.0
+BOT_GLOBAL_RATE_LIMIT=0.5
+BOT_DEFAULT_VERBOSITY=high-only
+# LLM config for Strix core
+STRIX_LLM=gpt-4o
+LLM_API_KEY=your-llm-key
+LLM_API_BASE=https://api.openai.com/v1  # optional for proxies/custom base

README.md

@@ -74,6 +74,7 @@ pipx install strix-agent
 # Configure your AI provider
 export STRIX_LLM="openai/gpt-5"
 export LLM_API_KEY="your-api-key"
+# Alternatively place STRIX_LLM/LLM_API_KEY in a .env file alongside the repo.
 
 # Run your first security assessment
 strix --target ./app-directory
@@ -234,10 +235,6 @@ Have questions? Found a bug? Want to contribute? **[Join our Discord!](https://d
 ## 🌟 Support the Project
 
 **Love Strix?** Give us a ⭐ on GitHub!
-## 🙏 Acknowledgements
-
-Strix builds on the incredible work of open-source projects like [LiteLLM](https://github.com/BerriAI/litellm), [Caido](https://github.com/caido/caido), [ProjectDiscovery](https://github.com/projectdiscovery), [Playwright](https://github.com/microsoft/playwright), and [Textual](https://github.com/Textualize/textual). Huge thanks to their maintainers!
-
 
 > [!WARNING]
 > Only test apps you own or have permission to test. You are responsible for using Strix ethically and legally.

docs/README.md

@@ -0,0 +1,36 @@
+# Strix Documentation Hub
+
+Audience: AI and human developers extending Strix. Start here to find deep references, workflows, and extension guides.
+
+- **Quick start**: see `docs/setup-and-running.md`.
+- **Architecture**: high-level map in `docs/architecture.md`.
+- **Agent loop**: internals in `docs/agent-loop.md`.
+- **Tools**: contract and extensions in `docs/tools-and-extensions.md`.
+- **Runtime**: sandbox and docker flow in `docs/runtime-and-sandbox.md`.
+- **LLM config**: provider setup and tuning in `docs/llm-config.md`.
+- **Prompts**: taxonomy and conventions in `docs/prompts.md`.
+- **Interface**: CLI/TUI behaviors in `docs/interface.md`.
+- **Telemetry**: tracing and events in `docs/telemetry-and-observability.md`.
+- **Testing/QA**: strategies in `docs/testing-and-qa.md`.
+- **Security/Privacy**: guardrails in `docs/security-and-privacy.md`.
+- **Release**: versioning and publishing in `docs/release-and-versioning.md`.
+- **Troubleshooting**: fixes in `docs/troubleshooting.md`.
+- **Glossary**: definitions in `docs/glossary.md`.
+- **Roadmap templates**: RFC/ADR formats in `docs/roadmap-templates.md`.
+
+Minimum environment
+- Python 3.12, Docker running, Playwright browsers installed.
+- STRIX_LLM + LLM_API_KEY (or litellm proxy) exported.
+- Local write access for `strix_runs/` outputs.
+
+Flow for new contributors
+1) Read `architecture.md` and `agent-loop.md` for mental model.  
+2) Run a local scan following `setup-and-running.md`.  
+3) Review `development.md` + `testing-and-qa.md` before changes.  
+4) Extend tools/prompts/runtime using relevant docs.  
+5) Update docs and add tests with every feature or bugfix.  
+
+Maintenance
+- Keep links valid when files move.  
+- Update dependency minimums when `pyproject.toml` changes.  
+- Refresh examples and flags when CLI/TUI arguments change.  

docs/agent-loop.md

@@ -0,0 +1,46 @@
+# Agent Loop Internals
+
+Primary files: `agents/base_agent.py`, `agents/state.py`, `agents/StrixAgent/strix_agent.py`, `agents/StrixAgent/system_prompt.jinja`.
+
+## Lifecycle
+1) Agent instantiated with config: llm config, max iterations (default 300; adaptive budget via `agents/iteration_policy.py` based on targets + LLM timeout recorded in tracer), non-interactive flag, optional state/local sources.
+2) `AgentMeta` wires Jinja environment per agent folder for prompts.
+3) `AgentState` tracks messages, tasks, wait states, agent graph ids.
+4) Main loop (in `BaseAgent.run`): fetches/creates state, processes queued messages, updates tracer, calls LLM with prompt, dispatches tool invocations, handles waits/finishes.
+5) Completion when finish tool invoked, max iterations hit, or fatal error.
+6) State persistence: snapshots saved as JSON (default in `strix_runs/<run>/<agent_id>_state.json`); can resume via config `load_state_from`.
+
+## Message handling
+- Messages stored in `AgentState`; inter-agent messages include metadata and are added as user messages with delivery notice.
+- Tracer updates agent status on message receipt/resume.
+
+## Tool selection and execution
+- LLM output parsed for tool calls → `tools.process_tool_invocations` → `tools/executor.py`.
+- Tool executions logged via tracer, results added back into state/context for next iteration.
+
+## Memory and limits
+- `llm/memory_compressor.py` trims context to fit provider limits.
+- `llm/request_queue.py` manages concurrency/ordering; `llm/llm.py` handles retries/backoff (tenacity).
+- Configurable `max_iterations` per agent instance via config.
+
+## Error handling
+- LLM errors wrapped in `LLMRequestFailedError`; retries applied.
+- Tool errors logged and surfaced in state; agents can adapt prompts accordingly.
+
+## Vulnerability propagation
+- Tracer `vulnerability_found_callback` (set in CLI) renders findings immediately; tracer records IDs, severity, content.
+
+## Extending the agent
+- Adjust prompts in `agents/StrixAgent/system_prompt.jinja`.
+- Modify decision logic in `StrixAgent/strix_agent.py`.
+- Add new state fields carefully; ensure serialization if persisted; update tracer calls to include new metadata.
+
+## ASCII loop snapshot
+```
+State -> Prompt render -> LLM -> Tool calls -> Results -> State update
+   ^                                                   |
+   +--------------------Tracer/events------------------+
+```
+
+## Maintenance
+- Revise when state fields or loop control change; keep diagram aligned with actual steps; update when new hooks are added.

docs/architecture.md

@@ -0,0 +1,50 @@
+# Architecture
+
+Strix runs coordinated agents that drive tools inside a dockerized sandbox, orchestrated via CLI/TUI, with telemetry and prompt packs guiding behavior.
+
+## System map
+- Entry: CLI/TUI (`interface/main.py`, `cli.py`, `tui.py`) parses args, sets scan config, starts tracer callbacks.
+- Agent loop: `agents/base_agent.py`, `agents/state.py`, `agents/StrixAgent/strix_agent.py` manage iterations, messages, tool calls, memory compression.
+- Tools: `tools/*` define XML action schemas + Python implementations; `tools/registry.py` registers; `tools/executor.py` dispatches; `interface/tool_components/*` render outputs.
+- Runtime: `runtime/docker_runtime.py`, `runtime/tool_server.py`, `runtime/runtime.py` manage sandbox containers and tool execution endpoints.
+- LLM: `llm/llm.py`, `llm/config.py`, `llm/request_queue.py`, `llm/memory_compressor.py` handle provider routing, retries, queueing, token budgeting.
+- Prompts: `prompts/**/*.jinja`, `agents/StrixAgent/system_prompt.jinja`, `prompts/coordination/root_agent.jinja` supply structured instructions.
+- Telemetry: `telemetry/tracer.py` captures agent lifecycle, tool executions, vulnerabilities; `interface/utils.py` renders stats.
+- Outputs: run artifacts under `strix_runs/<run_name>` (reports/logs).
+
+## Data flow (simplified)
+1) User invokes `strix` -> `interface/main.py` builds args, `cli.py`/`tui.py` start UI.  
+2) Scan config + tracer created -> `StrixAgent` instantiated with `LLMConfig`.  
+3) Agent loop requests LLM completions; responses trigger tool invocations via `tools/executor.py`.  
+4) Tools call `runtime/tool_server.py` (docker sandbox) for side effects (browser, proxy, terminal, python, file edits, etc.).  
+5) Tool results and tracer events propagate to UI renderers; vulnerabilities emitted to console and saved.  
+6) Loop continues until max iterations, finish action, or user stop; results stored in `strix_runs/`.
+
+## ASCII data flow
+```
+CLI/TUI -> Tracer -> StrixAgent -> LLM -> Tool Executor -> Runtime (Docker) -> Tool Server
+             ^                                           |
+             |-------------------------------------------+
+```
+
+## Extension seams
+- Add tools: new folder under `tools/`, schema XML, implementation, registry entry, renderer in `interface/tool_components/`.
+- Add prompts: new Jinja in `prompts/*` or agent prompt folder; wire selection logic where consumed.
+- Add telemetry: emit via `telemetry/tracer.py` helper methods; extend UI renderers to display.
+- Add providers: extend `llm/config.py` + `llm/llm.py` to create client, auth, and request path.
+- Adjust runtime: modify `runtime/docker_runtime.py` for images/limits, `tool_server.py` for endpoints.
+
+## Persistence
+- Runs: `strix_runs/<run_name>` contains reports and logs (non-interactive mode prints to stdout too).
+- Agent graph: managed in-memory via `tools/agents_graph/agents_graph_actions.py`, rendered by interface. Graph definitions can be loaded/validated from YAML/JSON via `agents/graph_builder.py` (unique ids, single root, parent/child checks) before instantiation.
+
+## Non-interactive mode
+- Enabled via `-n/--non-interactive`; suppresses interactive UI, streams findings to stdout; still uses tracer callbacks for vulnerability events.
+
+## Reliability and limits
+- Max iterations default 300 (`BaseAgent.max_iterations`), configurable via agent config.
+- Request queue/backoff in `llm/request_queue.py`; retries in `llm/llm.py` using tenacity.
+- Memory compression in `llm/memory_compressor.py` to stay within context limits.
+
+## Maintenance
+- Update module paths if files move; refresh diagram when flows change; align with CLI flag changes.

docs/development.md

@@ -0,0 +1,46 @@
+# Development Guide
+
+## Layout primer
+- Core agent logic: `strix/agents/*`
+- Tools and action schemas: `strix/tools/*`
+- Runtime sandbox: `strix/runtime/*`
+- LLM layer: `strix/llm/*`
+- Prompts: `strix/prompts/*` and `strix/agents/StrixAgent/system_prompt.jinja`
+- Interface (CLI/TUI + renderers): `strix/interface/*`
+- Telemetry: `strix/telemetry/*`
+
+## Standards
+- Python 3.12, strict typing (see `pyproject.toml` mypy config).
+- Lint/format: `ruff`, `black`, `isort`.
+- Security/static: `bandit`, `pylint`.
+- Keep docstrings concise; prefer clear variable names over comments.
+
+## Commands
+```bash
+# Format + lint
+poetry run ruff check .
+poetry run black .
+poetry run isort .
+
+# Type check
+poetry run mypy .
+poetry run pyright
+
+# Tests
+poetry run pytest
+poetry run pytest --cov
+```
+
+## Workflow
+- Create feature branches; keep commits scoped.
+- Run format + lint + tests before PR.
+- Update relevant docs when changing behavior, flags, prompts, or outputs.
+- Add regression coverage for new tools/prompt changes/runtime adjustments.
+
+## Performance tips
+- Reuse docker images; avoid repeated pulls.
+- Cache provider auth where possible; tune LLM parallelism in config.
+- Use smaller prompt packs for targeted testing when iterating quickly.
+
+## Maintenance
+- Revise commands/tools when linters/types/test stacks change; align with `pyproject.toml`.

docs/glossary.md

@@ -0,0 +1,15 @@
+# Glossary
+
+- Agent loop: Iterative cycle in `agents/base_agent.py` driving LLM calls and tool executions.
+- Agent graph: In-memory map of agents managed via `tools/agents_graph/agents_graph_actions.py`, rendered in UI.
+- Action schema: XML definition (`*_actions_schema.xml`) describing tool actions/args.
+- Renderer: UI component in `interface/tool_components/*` mapping tool outputs to panels.
+- Runtime: Docker-based sandbox managed by `runtime/docker_runtime.py` and `tool_server.py`.
+- Tracer: Telemetry recorder in `telemetry/tracer.py` logging agent and tool events.
+- Run name: Unique id for a scan; names output directory `strix_runs/<run_name>`.
+- Request queue: LLM request coordinator in `llm/request_queue.py`.
+- Memory compressor: Context trimming utility in `llm/memory_compressor.py`.
+- Non-interactive mode: Headless CLI mode (`-n`) emitting findings to stdout without TUI.
+
+## Maintenance
+- Add new terms as components are introduced; keep paths accurate after refactors.

docs/interface.md

@@ -0,0 +1,30 @@
+# Interface (CLI/TUI)
+
+## Entrypoints
+- `interface/main.py`: CLI entry, argument parser wiring, dispatch to CLI/TUI.
+- `interface/cli.py`: non-interactive flow; renders startup panel and vulnerability panels via tracer callbacks.
+- `interface/tui.py`: textual-based interactive UI; panels for tools, logs, stats.
+- `interface/utils.py`: stats builders, severity colors, shared helpers.
+
+## Arguments (key)
+- `--target/-t`: target path/URL (multi allowed).
+- `--instruction`: freeform guidance to agent.
+- `--non-interactive/-n`: headless mode; prints findings to stdout.
+- `--run-name`: optional custom run id (otherwise generated).
+- Provider/env vars read separately; ensure `STRIX_LLM`/`LLM_API_KEY` set.
+
+## Rendering
+- Tool outputs mapped via `interface/tool_components/*` renderers (browser, proxy, terminal, file edits, reports, notes, thinking, etc.).
+- Live stats and final stats built in `utils.py` and displayed in panels.
+- Vulnerabilities emitted from tracer callback in CLI mode; TUI shows panes with updates.
+
+## Customization
+- Styles in `interface/assets/tui_styles.tcss`.
+- Add new renderers by extending `tool_components/base_renderer.py` and registering in `registry.py`.
+
+## Non-interactive behavior
+- Skips TUI; logs findings immediately.
+- Still writes outputs under `strix_runs/<run_name>`.
+
+## Maintenance
+- Update argument list when CLI flags change; refresh renderer mapping when new tools are added.

docs/llm-config.md

@@ -0,0 +1,33 @@
+# LLM Configuration
+
+Core files: `llm/config.py`, `llm/llm.py`, `llm/request_queue.py`, `llm/memory_compressor.py`, `llm/utils.py`.
+
+## Providers and models
+- `LLMConfig` defines provider/model id (e.g., `openai/gpt-5`) and auth.
+- Extend providers by adding client setup and request paths in `llm/llm.py`; expose config knobs in `config.py`.
+
+## Request handling
+- Requests queued via `request_queue.py` to manage concurrency and order.
+- Retries/backoff handled in `llm.py` using tenacity; errors wrapped in `LLMRequestFailedError`.
+- Streaming support depends on provider implementation in `llm.py`.
+
+## Context management
+- `memory_compressor.py` trims conversation/state to fit provider token limits.
+- `llm/utils.py` cleans content before sending to providers.
+
+## Tuning
+- Control parallelism and rate limits in request queue.
+- Adjust model choice to balance cost vs. latency.
+- Customize temperature/other params in `LLMConfig`.
+
+## Telemetry
+- LLM calls can be logged via tracer; ensure sensitive data is redacted before emission.
+
+## Adding a new provider
+1) Define config fields in `config.py`.
+2) Add client creation and request method in `llm.py`.
+3) Wire retries/backoff and error normalization.
+4) Update docs and examples in `setup-and-running.md`.
+
+## Maintenance
+- Revise when providers/models or retry/queue logic change; ensure env var expectations are documented.

docs/prompts.md

@@ -0,0 +1,33 @@
+# Prompts
+
+## Taxonomy
+- Coordination: `prompts/coordination/root_agent.jinja`
+- Frameworks: `prompts/frameworks/*.jinja` (e.g., `fastapi`, `nextjs`)
+- Technologies: `prompts/technologies/*.jinja` (e.g., `firebase_firestore`, `supabase`)
+- Vulnerabilities: `prompts/vulnerabilities/*.jinja` (e.g., `sql_injection`, `xss`, `rce`)
+- Auth playbooks: `prompts/auth/oidc_saml_sso.jinja`
+- Cloud/custom/recon placeholders: `prompts/cloud`, `prompts/custom`, `prompts/reconnaissance`
+- Agent system prompt: `agents/StrixAgent/system_prompt.jinja`
+
+## Conventions
+- Jinja templates with explicit placeholders; avoid hidden assumptions.
+- Keep titles and sections consistent for downstream parsing.
+- Prefer actionable guidance (steps, checks, PoC ideas) and explicit do/don’t lists.
+
+## Selection/combination
+- Agent selects relevant prompt packs based on target metadata; templates rendered via Jinja environment set in `AgentMeta`.
+- Root coordination prompt guides multi-agent behavior; specialized prompts augment depending on framework/tech/vuln focus.
+
+## Safe testing
+- Dry-run new prompts in non-interactive mode against test targets.
+- Check for prompt injection surfaces; ensure instructions avoid unsafe actions outside sandbox.
+- Validate output format expected by tools (e.g., when tool calls must be produced).
+
+## Adding a prompt pack
+1) Create `.jinja` file in appropriate folder with descriptive name.
+2) Document variables required; keep defaults sensible.
+3) Add regression test or fixture to ensure rendering works and key strings exist.
+4) Update this doc and any selection logic if needed.
+
+## Maintenance
+- Refresh taxonomy when adding/removing prompt packs; ensure variable names stay consistent with agent code.

docs/release-and-versioning.md

@@ -0,0 +1,26 @@
+# Release and Versioning
+
+Current version: 0.4.0 (`pyproject.toml`).
+
+## Versioning
+- Follow semantic-ish bumps: increment patch for fixes, minor for features, major for breaking changes.
+- Update `pyproject.toml` version and any surfaced docs.
+
+## Packaging
+```bash
+poetry build
+poetry publish  # requires credentials
+```
+- Ensure `README.md` and license included (listed in `[tool.poetry]` include).
+- Verify wheels/sdist contain `.jinja`, `.xml`, `.tcss` assets (declared in `include`).
+
+## Changelog
+- Maintain a changelog (add file if missing) summarizing features, fixes, breaking changes.
+- Reference PRs/issues; highlight security-impacting changes.
+
+## Compatibility
+- Python 3.12 only (per `pyproject.toml`).
+- Document any deprecated flags or behaviors and provide migration notes.
+
+## Maintenance
+- Update version numbers and commands when packaging flow changes; ensure asset include lists stay correct.