diff --git a/.github/workflows/AGENTS.md b/.github/workflows/AGENTS.md index fa3f6c4..494a647 100644 --- a/.github/workflows/AGENTS.md +++ b/.github/workflows/AGENTS.md @@ -127,6 +127,26 @@ Simple workflow for testing the self-hosted runner setup. 2. Echoes test messages 3. Verifies runner connectivity and permissions +## Privacy Scan Workflow + +### privacy-scan + +Scans incoming changes for secrets and PII before merge/deploy. + +**Triggers:** +- Push +- Pull request (non-fork PRs only) +- Manual (`workflow_dispatch`) + +**Behavior:** +1. Runs on self-hosted runner +2. Executes gitleaks for secret detection +3. Runs `scripts/pii_guard.py` on added lines in diff range +4. Fails when medium/high-confidence PII is detected (email/phone/cpf/cnpj/credit-card) + +**Allowlist:** +- Use `.pii-allowlist` with regex entries for explicit, reviewed exceptions. + ## Deployment Workflow ### deploy-to-home-server diff --git a/.github/workflows/privacy-scan.yml b/.github/workflows/privacy-scan.yml new file mode 100644 index 0000000..157e241 --- /dev/null +++ b/.github/workflows/privacy-scan.yml @@ -0,0 +1,59 @@ +name: Privacy Scan + +on: + push: + pull_request: + workflow_dispatch: + +jobs: + scan: + if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false + runs-on: self-hosted + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Cleanup stale gitleaks temp file + run: | + sudo rm -f /tmp/gitleaks.tmp || rm -f /tmp/gitleaks.tmp || true + + - name: Run gitleaks + uses: gitleaks/gitleaks-action@v2 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Determine diff range + id: range + run: | + if [ "${{ github.event_name }}" = "pull_request" ]; then + echo "base=${{ github.event.pull_request.base.sha }}" >> "$GITHUB_OUTPUT" + echo "head=${{ github.event.pull_request.head.sha }}" >> "$GITHUB_OUTPUT" + echo "three_dot=true" >> "$GITHUB_OUTPUT" + exit 0 + fi + + BEFORE="${{ github.event.before }}" + if [ -z "$BEFORE" ] || [ "$BEFORE" = "0000000000000000000000000000000000000000" ]; then + BEFORE=$(git rev-list --max-parents=0 HEAD) + fi + + echo "base=$BEFORE" >> "$GITHUB_OUTPUT" + echo "head=${{ github.sha }}" >> "$GITHUB_OUTPUT" + echo "three_dot=false" >> "$GITHUB_OUTPUT" + + - name: Run PII guard on added lines + run: | + if [ "${{ steps.range.outputs.three_dot }}" = "true" ]; then + python3 scripts/pii_guard.py \ + --base "${{ steps.range.outputs.base }}" \ + --head "${{ steps.range.outputs.head }}" \ + --three-dot \ + --fail-on medium + else + python3 scripts/pii_guard.py \ + --base "${{ steps.range.outputs.base }}" \ + --head "${{ steps.range.outputs.head }}" \ + --fail-on medium + fi diff --git a/.opencode/plugins/pii-commit-guard.mjs b/.opencode/plugins/pii-commit-guard.mjs new file mode 100644 index 0000000..29d5061 --- /dev/null +++ b/.opencode/plugins/pii-commit-guard.mjs @@ -0,0 +1,36 @@ +import { execFileSync } from "node:child_process" +import { existsSync } from "node:fs" +import path from "node:path" + +function isGitCommitCommand(command) { + if (!command || typeof command !== "string") return false + const compact = command.replace(/\s+/g, " ").trim().toLowerCase() + return /(^|[;&|]\s*|&&\s*)git commit(\s|$)/.test(compact) +} + +export const PiiCommitGuard = async ({ directory }) => { + const root = directory + + return { + "tool.execute.before": async (input, output) => { + if (input.tool !== "bash") return + + const command = String(output?.args?.command || "") + if (!isGitCommitCommand(command)) return + + const guardScript = path.join(root, "scripts", "pii_guard.py") + if (!existsSync(guardScript)) { + throw new Error("PII guard script not found at scripts/pii_guard.py") + } + + try { + execFileSync("python3", [guardScript, "--staged", "--fail-on", "medium"], { + cwd: root, + stdio: "inherit", + }) + } catch { + throw new Error("Commit blocked by deterministic PII guard (scripts/pii_guard.py)") + } + }, + } +} diff --git a/.opencode/skills/pii-commit-check/SKILL.md b/.opencode/skills/pii-commit-check/SKILL.md new file mode 100644 index 0000000..0f24c20 --- /dev/null +++ b/.opencode/skills/pii-commit-check/SKILL.md @@ -0,0 +1,30 @@ +--- +name: pii-commit-check +description: Run staged PII checks before creating commits. +compatibility: opencode +--- + +# PII Commit Check + +Use this skill whenever preparing a commit. + +## Workflow + +1. Run deterministic scan on staged changes: + +```bash +python3 scripts/pii_guard.py --staged --fail-on medium +``` + +2. Run manual agentic review in the current OpenCode session (uses current configured model/provider, no extra API key): + +- Ask OpenCode to inspect staged diff for PII risks before commit. +- Example prompt: + - "Review `git diff --cached` for possible PII leaks (CPF/CNPJ/phone/email/card/personal identifiers). Flag high-confidence risks and suggest redactions." + +3. If any check fails: +- Block commit. +- Show offending lines/findings. +- Ask for redaction or explicit allowlist update in `.pii-allowlist`. + +4. Only proceed with `git commit` after all checks pass. diff --git a/.pii-allowlist b/.pii-allowlist new file mode 100644 index 0000000..52a8632 --- /dev/null +++ b/.pii-allowlist @@ -0,0 +1,5 @@ +# Regex allowlist for pii_guard.py findings. +# One regex per line. Matches against: +# file_path:line_number:kind:value:line_text +# +# Keep this file minimal and documented when adding exceptions. diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index b8a004b..78227b4 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -3,3 +3,11 @@ repos: rev: v8.24.2 hooks: - id: gitleaks + + - repo: local + hooks: + - id: pii-guard + name: pii-guard + entry: python3 scripts/pii_guard.py --staged --fail-on medium + language: system + pass_filenames: false diff --git a/AGENTS.md b/AGENTS.md index 6bae189..c6dde64 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -405,12 +405,15 @@ On container start, the workspace sync script (`scripts/workspace-sync.sh`) clon ## Security: Secret Scanning -This repository uses **gitleaks** to prevent accidental commits of secrets (API keys, tokens, passwords, etc.). +This repository uses **gitleaks** and a **PII guard** to reduce accidental exposure of secrets and personal data. ### Automated Scanning -- **CI/CD:** Every push to any branch is scanned via GitHub Actions (`.github/workflows/gitleaks.yml`) +- **CI/CD:** Every push to any branch is scanned via GitHub Actions (`.github/workflows/privacy-scan.yml`) - **Local:** Pre-commit hooks scan staged changes before each commit (optional but recommended) +- **Privacy guard:** Added-line PII checks run in pre-commit and in CI via `.github/workflows/privacy-scan.yml` +- **OpenCode runtime guard:** `opencode.json` loads `.opencode/plugins/pii-commit-guard.mjs`, which intercepts `git commit` bash calls and runs staged PII checks before allowing commit execution +- **OpenCode commit behavior (mandatory):** whenever a commit is requested in this repo, OpenCode must run the `pii-commit-check` skill before creating the commit ### Setup Pre-commit Hooks (One-Time) @@ -426,9 +429,12 @@ This script creates `venv/` if needed, installs pre-commit, and installs git hoo # Normal commit - gitleaks runs automatically git commit -m "your message" -# Skip gitleaks (emergency only) -SKIP=gitleaks git commit -m "your message" +# Skip checks (emergency only) +SKIP=gitleaks,pii-guard git commit -m "your message" # Manual full scan source venv/bin/activate && pre-commit run gitleaks --all-files + +# Manual staged PII scan +python3 scripts/pii_guard.py --staged --fail-on medium ``` diff --git a/opencode.json b/opencode.json new file mode 100644 index 0000000..2df2d0e --- /dev/null +++ b/opencode.json @@ -0,0 +1,6 @@ +{ + "$schema": "https://opencode.ai/config.json", + "plugin": [ + "./.opencode/plugins/pii-commit-guard.mjs" + ] +} diff --git a/scripts/pii_guard.py b/scripts/pii_guard.py new file mode 100755 index 0000000..bd956d4 --- /dev/null +++ b/scripts/pii_guard.py @@ -0,0 +1,284 @@ +#!/usr/bin/env python3 +"""PII guard for git diffs (staged or commit range).""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + + +EMAIL_RE = re.compile(r"\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b", re.IGNORECASE) +PHONE_BR_RE = re.compile( + r"(? str: + cmd = ["git", "diff", "--no-color", "--unified=0"] + if staged: + cmd.append("--cached") + if base and head: + separator = "..." if three_dot else ".." + cmd.append(f"{base}{separator}{head}") + + completed = subprocess.run(cmd, check=False, capture_output=True, text=True) + if completed.returncode != 0: + raise RuntimeError(completed.stderr.strip() or "git diff failed") + return completed.stdout + + +def _parse_added_lines(diff_text: str) -> list[tuple[str, int, str]]: + findings: list[tuple[str, int, str]] = [] + current_file = "" + new_line_number = 0 + + for raw_line in diff_text.splitlines(): + if raw_line.startswith("+++ b/"): + current_file = raw_line[len("+++ b/") :] + continue + + if raw_line.startswith("@@"): + match = re.search(r"\+(\d+)(?:,(\d+))?", raw_line) + if match: + new_line_number = int(match.group(1)) + continue + + if raw_line.startswith("+") and not raw_line.startswith("+++"): + findings.append((current_file, new_line_number, raw_line[1:])) + new_line_number += 1 + continue + + if raw_line.startswith(" "): + new_line_number += 1 + + return findings + + +def _digits(value: str) -> str: + return "".join(ch for ch in value if ch.isdigit()) + + +def _valid_cpf(value: str) -> bool: + digits = _digits(value) + if len(digits) != 11: + return False + if len(set(digits)) == 1: + return False + + weights_1 = list(range(10, 1, -1)) + total_1 = sum(int(digits[i]) * weights_1[i] for i in range(9)) + dv1 = (total_1 * 10 % 11) % 10 + if dv1 != int(digits[9]): + return False + + weights_2 = list(range(11, 1, -1)) + total_2 = sum(int(digits[i]) * weights_2[i] for i in range(10)) + dv2 = (total_2 * 10 % 11) % 10 + return dv2 == int(digits[10]) + + +def _valid_cnpj(value: str) -> bool: + digits = _digits(value) + if len(digits) != 14: + return False + if len(set(digits)) == 1: + return False + + weights_1 = [5, 4, 3, 2, 9, 8, 7, 6, 5, 4, 3, 2] + total_1 = sum(int(digits[i]) * weights_1[i] for i in range(12)) + rem_1 = total_1 % 11 + dv1 = 0 if rem_1 < 2 else 11 - rem_1 + if dv1 != int(digits[12]): + return False + + weights_2 = [6] + weights_1 + total_2 = sum(int(digits[i]) * weights_2[i] for i in range(13)) + rem_2 = total_2 % 11 + dv2 = 0 if rem_2 < 2 else 11 - rem_2 + return dv2 == int(digits[13]) + + +def _luhn_valid(value: str) -> bool: + digits = _digits(value) + if len(digits) < 13 or len(digits) > 19: + return False + checksum = 0 + parity = len(digits) % 2 + for index, ch in enumerate(digits): + number = int(ch) + if index % 2 == parity: + number *= 2 + if number > 9: + number -= 9 + checksum += number + return checksum % 10 == 0 + + +def _is_example_email(value: str) -> bool: + _, _, domain = value.lower().rpartition("@") + return domain in EXAMPLE_EMAIL_DOMAINS + + +def _load_allowlist(path: Path) -> list[re.Pattern[str]]: + if not path.exists(): + return [] + + patterns: list[re.Pattern[str]] = [] + for line in path.read_text(encoding="utf-8").splitlines(): + raw = line.strip() + if not raw or raw.startswith("#"): + continue + patterns.append(re.compile(raw)) + return patterns + + +def _matches_allowlist(finding: Finding, allow_patterns: list[re.Pattern[str]]) -> bool: + if not allow_patterns: + return False + subject = f"{finding.file_path}:{finding.line_number}:{finding.kind}:{finding.value}:{finding.line_text}" + return any(pattern.search(subject) for pattern in allow_patterns) + + +def _collect_findings(added_lines: list[tuple[str, int, str]]) -> list[Finding]: + findings: list[Finding] = [] + + for file_path, line_number, line_text in added_lines: + if not file_path or file_path == "/dev/null": + continue + + for match in EMAIL_RE.finditer(line_text): + value = match.group(0) + if _is_example_email(value): + continue + findings.append(Finding("email", "medium", file_path, line_number, value, line_text.strip())) + + for match in PHONE_BR_RE.finditer(line_text): + value = match.group(0) + if len(_digits(value)) < 10: + continue + findings.append(Finding("phone", "medium", file_path, line_number, value, line_text.strip())) + + for match in CPF_RE.finditer(line_text): + value = match.group(0) + if _valid_cpf(value): + findings.append(Finding("cpf", "high", file_path, line_number, value, line_text.strip())) + + for match in CNPJ_RE.finditer(line_text): + value = match.group(0) + if _valid_cnpj(value): + findings.append(Finding("cnpj", "high", file_path, line_number, value, line_text.strip())) + + for match in CARD_RE.finditer(line_text): + value = match.group(0) + if _luhn_valid(value): + findings.append(Finding("credit_card", "high", file_path, line_number, value, line_text.strip())) + + deduped: dict[tuple[str, int, str, str], Finding] = {} + for item in findings: + key = (item.file_path, item.line_number, item.kind, item.value) + deduped[key] = item + return sorted(deduped.values(), key=lambda item: (item.file_path, item.line_number, item.kind, item.value)) + + +def main() -> int: + parser = argparse.ArgumentParser(description="Scan added git diff lines for potential PII.") + parser.add_argument("--staged", action="store_true", help="Scan staged diff (`git diff --cached`).") + parser.add_argument("--base", help="Base git ref/sha for range scan.") + parser.add_argument("--head", help="Head git ref/sha for range scan.") + parser.add_argument("--three-dot", action="store_true", help="Use three-dot diff (base...head).") + parser.add_argument( + "--fail-on", + choices=["none", "medium", "high"], + default="high", + help="Fail when finding severity is at or above this threshold.", + ) + parser.add_argument( + "--allowlist", + default=".pii-allowlist", + help="Path to regex allowlist file.", + ) + parser.add_argument("--json", action="store_true", help="Emit JSON output.") + args = parser.parse_args() + + if (args.base and not args.head) or (args.head and not args.base): + parser.error("--base and --head must be provided together") + + try: + diff_text = _run_git_diff( + staged=args.staged, + base=args.base, + head=args.head, + three_dot=args.three_dot, + ) + except RuntimeError as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 2 + + added_lines = _parse_added_lines(diff_text) + findings = _collect_findings(added_lines) + allow_patterns = _load_allowlist(Path(args.allowlist)) + filtered = [item for item in findings if not _matches_allowlist(item, allow_patterns)] + + threshold = SEVERITY_RANK[args.fail_on] + failing = [item for item in filtered if SEVERITY_RANK[item.severity] >= threshold] + + if args.json: + payload = { + "scanned_added_lines": len(added_lines), + "findings": [item.__dict__ for item in filtered], + "failing": [item.__dict__ for item in failing], + "fail_on": args.fail_on, + } + print(json.dumps(payload, ensure_ascii=True, indent=2)) + else: + print(f"PII guard scanned {len(added_lines)} added line(s)") + if not filtered: + print("PII guard: no findings") + else: + print(f"PII guard: {len(filtered)} finding(s)") + for item in filtered: + flag = "BLOCK" if item in failing else "WARN" + print( + f"[{flag}] {item.severity.upper():6} {item.kind:11} " + f"{item.file_path}:{item.line_number} -> {item.value}" + ) + + if failing: + print( + "PII guard blocked this change. Remove/redact data or add an explicit regex exception in .pii-allowlist.", + file=sys.stderr, + ) + return 1 + + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/setup-pre-commit.sh b/scripts/setup-pre-commit.sh index 5a0a22a..6577def 100755 --- a/scripts/setup-pre-commit.sh +++ b/scripts/setup-pre-commit.sh @@ -23,6 +23,6 @@ pre-commit install echo "" echo "=== Setup Complete ===" -echo " Pre-commit hooks installed. Gitleaks will run on every commit." -echo " To skip: SKIP=gitleaks git commit -m 'message'" +echo " Pre-commit hooks installed. Gitleaks + PII guard will run on every commit." +echo " To skip (emergency only): SKIP=gitleaks,pii-guard git commit -m 'message'" echo " To update hooks: source $VENV_DIR/bin/activate && pre-commit run --all-files" diff --git a/skills-factory/vault-gateway/README.md b/skills-factory/vault-gateway/README.md index 78f2b52..649ac8f 100644 --- a/skills-factory/vault-gateway/README.md +++ b/skills-factory/vault-gateway/README.md @@ -26,7 +26,8 @@ This directory contains the repo-shipped MBIFC vault bundle for Josemar. - `onboarding` supports deterministic new-vault and port-existing flows. - `onboarding` requires a `state_key` in payload for multi-turn isolation. - Port flow includes destructive confirmation gates with backup warning. -- `note.capture`, `note.read`, `note.update` (supports append, prepend, replace with frontmatter auto-preserve, and surgical frontmatter mode), `note.search`, `note.link`, and `note.file` provide flexible day-to-day vault manipulation. +- `note.capture`, `note.read`, `note.update` (supports append, prepend, replace with frontmatter auto-preserve, surgical frontmatter mode, and section-targeted append/prepend), `note.search`, `note.link`, and `note.file` provide flexible day-to-day vault manipulation. +- Section-intent updates follow a read-first policy (`note.read` before `note.update`) and avoid silent fallback to raw append/prepend when the heading is missing or ambiguous. - Read/write note routes ingest folder context from nearest `_index.md` (including `## Working Rules`) plus `Meta/vault-structure.md` managed snapshot when available. - After write routes (`note.capture`, `note.update`, `note.file`), managed context blocks are refreshed in `Meta/vault-structure.md` and folder `_index.md`; human-authored sections remain untouched. - `inbox.triage`, `vault.defrag`, `vault.audit`, `vault.deep-clean`, and `tags.garden` return structured maintenance summaries. diff --git a/skills-factory/vault-gateway/SKILL.md b/skills-factory/vault-gateway/SKILL.md index f13bab0..1b8caf1 100644 --- a/skills-factory/vault-gateway/SKILL.md +++ b/skills-factory/vault-gateway/SKILL.md @@ -97,6 +97,20 @@ Frontmatter surgical update example: } ``` +Section-targeted update example: + +```json +{ + "route": "note.update", + "payload": { + "path": "07-Daily/2026-04-26.md", + "mode": "section_append", + "section_heading": "Tasks", + "text": "- [ ] Confirmar fechamento com cliente" + } +} +``` + Contract rules: - Top-level keys: only `route` and `payload` - Route params must be inside `payload` @@ -108,6 +122,9 @@ Contract rules: - For "port existing vault", always propose a safe plan before execution. - If user requests destructive mode, display a strong warning and strongly recommend a vault backup before continuing. - When using `note.update` with `mode: replace`, existing frontmatter is auto-preserved if the replacement text has no YAML block. This prevents accidental frontmatter loss during read-then-replace edits. +- For "add item to existing section" edits, prefer `note.update` with `mode: section_append` (or `section_prepend`) and explicit `section_heading` to avoid duplicate section creation. +- For section-intent writes, run a mandatory read-first flow: `note.read` -> verify heading -> `note.update` with `section_append/section_prepend`. +- If target heading is missing or duplicated, do not silently fallback to raw append/prepend; ask one focused clarification before writing. - Read/write note routes ingest contextual guidance from nearest folder `_index.md` (including `## Working Rules`) and from managed snapshot in `Meta/vault-structure.md` when present. - After vault write routes (`note.capture`, `note.update`, `note.file`), gateway refreshes managed context blocks in `Meta/vault-structure.md` and folder `_index.md` files while preserving human-authored sections. diff --git a/skills-factory/vault-gateway/contracts/routes.json b/skills-factory/vault-gateway/contracts/routes.json index 93bf063..66bcf30 100644 --- a/skills-factory/vault-gateway/contracts/routes.json +++ b/skills-factory/vault-gateway/contracts/routes.json @@ -96,13 +96,14 @@ "playbook": "playbooks/note-update/PLAYBOOK.md", "source": "agents/seeker.md", "summary": "Update an existing markdown note", - "payload": { - "path": "required relative path", - "text": "optional string (required unless mode is frontmatter)", - "mode": "optional: append|prepend|replace|frontmatter (default append)", - "frontmatter_fields": "optional object (required when mode is frontmatter)" - } - }, + "payload": { + "path": "required relative path", + "text": "optional string (required unless mode is frontmatter)", + "mode": "optional: append|prepend|replace|frontmatter|section_append|section_prepend (default append)", + "frontmatter_fields": "optional object (required when mode is frontmatter)", + "section_heading": "optional string (required when mode is section_append or section_prepend)" + } + }, "note.search": { "status": "active", "playbook": "playbooks/note-search/PLAYBOOK.md", diff --git a/skills-factory/vault-gateway/lib/handlers.py b/skills-factory/vault-gateway/lib/handlers.py index 7cab52e..587e6b4 100644 --- a/skills-factory/vault-gateway/lib/handlers.py +++ b/skills-factory/vault-gateway/lib/handlers.py @@ -460,6 +460,7 @@ def handle_route(route: str, payload: dict, metadata: dict) -> dict: path=payload.get("path"), mode=str(payload.get("mode") or "append"), frontmatter_fields=fm_fields if isinstance(fm_fields, dict) else None, + section_heading=payload.get("section_heading"), ) return { "message": "Nota atualizada com sucesso." + _maintenance_suffix(result), diff --git a/skills-factory/vault-gateway/lib/vault_ops.py b/skills-factory/vault-gateway/lib/vault_ops.py index 27e7cec..a2c5fad 100644 --- a/skills-factory/vault-gateway/lib/vault_ops.py +++ b/skills-factory/vault-gateway/lib/vault_ops.py @@ -142,6 +142,59 @@ def _extract_markdown_section(content: str, heading: str) -> str: return match.group(1).strip() +def _normalize_heading_label(value: str) -> str: + label = (value or "").strip() + label = re.sub(r"\s+#*$", "", label).strip() + label = re.sub(r"\s+", " ", label) + return label.casefold() + + +def _update_markdown_section(body_text: str, heading: str, payload_text: str, prepend: bool = False) -> str: + normalized_target = _normalize_heading_label(heading) + if not normalized_target: + raise ValueError("section_heading is required when using section mode") + + heading_matches = list(re.finditer(r"(?m)^(#{1,6})[ \t]+(.+?)\s*$", body_text)) + matching_indices = [ + index + for index, match in enumerate(heading_matches) + if _normalize_heading_label(match.group(2)) == normalized_target + ] + + if not matching_indices: + raise ValueError(f"Section '{heading}' was not found in note body") + if len(matching_indices) > 1: + raise ValueError( + f"Multiple sections named '{heading}' were found; use note.read + replace for an explicit edit" + ) + + target_index = matching_indices[0] + target_match = heading_matches[target_index] + target_level = len(target_match.group(1)) + + section_start = target_match.end() + section_end = len(body_text) + for candidate in heading_matches[target_index + 1 :]: + if len(candidate.group(1)) <= target_level: + section_end = candidate.start() + break + + current_section = body_text[section_start:section_end].lstrip("\n") + insertion = payload_text.strip() + if prepend: + if current_section.strip(): + rebuilt_section = "\n" + insertion + "\n\n" + current_section.rstrip() + "\n" + else: + rebuilt_section = "\n" + insertion + "\n" + else: + if current_section.strip(): + rebuilt_section = "\n" + current_section.rstrip() + "\n" + insertion + "\n" + else: + rebuilt_section = "\n" + insertion + "\n" + + return body_text[:section_start] + rebuilt_section + body_text[section_end:] + + def _truncate_text(text: str, limit: int = 1200) -> str: normalized = (text or "").strip() if len(normalized) <= limit: @@ -1405,9 +1458,10 @@ def update_note( path: str | None = None, mode: str = "append", frontmatter_fields: dict | None = None, + section_heading: str | None = None, ) -> dict: normalized_mode = (mode or "append").strip().lower() - valid_modes = {"append", "prepend", "replace", "frontmatter"} + valid_modes = {"append", "prepend", "replace", "frontmatter", "section_append", "section_prepend"} if normalized_mode not in valid_modes: raise ValueError(f"Invalid mode. Expected one of: {', '.join(sorted(valid_modes))}") @@ -1421,6 +1475,9 @@ def update_note( if normalized_mode != "frontmatter" and not payload_text: raise ValueError("Field 'text' is required") + if normalized_mode in {"section_append", "section_prepend"} and not str(section_heading or "").strip(): + raise ValueError("section_heading is required when mode is section_append or section_prepend") + note_path = _resolve_note_path(vault_root, path=path) existing = _safe_read_text(note_path) existing_frontmatter, existing_body = _extract_frontmatter(existing) @@ -1452,6 +1509,19 @@ def update_note( else: updated = payload_text + "\n\n" + existing + elif normalized_mode in {"section_append", "section_prepend"}: + updated_body = _update_markdown_section( + existing_body, + heading=str(section_heading or ""), + payload_text=payload_text, + prepend=normalized_mode == "section_prepend", + ) + body_text = updated_body.strip("\n") + if existing_frontmatter: + updated = _serialize_frontmatter(existing_frontmatter) + "\n\n" + body_text + "\n" + else: + updated = body_text + "\n" + else: separator = "\n\n" if existing.strip() else "" updated = existing.rstrip() + separator + payload_text + "\n" @@ -1466,6 +1536,8 @@ def update_note( "maintenance_updates": maintenance_updates, "context": operation_context, } + if normalized_mode in {"section_append", "section_prepend"}: + result["section_heading"] = str(section_heading or "").strip() if warnings: result["warnings"] = warnings _append_log(vault_root, "note.update", result) diff --git a/skills-factory/vault-gateway/playbooks/note-update/PLAYBOOK.md b/skills-factory/vault-gateway/playbooks/note-update/PLAYBOOK.md index 4aa5932..ed6f3ce 100644 --- a/skills-factory/vault-gateway/playbooks/note-update/PLAYBOOK.md +++ b/skills-factory/vault-gateway/playbooks/note-update/PLAYBOOK.md @@ -9,7 +9,7 @@ ## Route Mapping - Route: `note.update` -- Payload: `path` (required), `text` (required unless mode is frontmatter), `mode` (optional: append|prepend|replace|frontmatter), `frontmatter_fields` (required when mode is frontmatter) +- Payload: `path` (required), `text` (required unless mode is frontmatter), `mode` (optional: append|prepend|replace|frontmatter|section_append|section_prepend), `frontmatter_fields` (required when mode is frontmatter), `section_heading` (required when mode is section_append/section_prepend) ## Purpose @@ -62,6 +62,43 @@ This is how MBIFC's Seeker works in practice: the LLM does the intelligent editi - The note body remains unchanged. - Example: set `status: active` and add `updated: 2026-04-15` without re-reading or re-writing the body. +### Section Append / Section Prepend + +- Use `mode: section_append` to insert content at the end of an existing markdown section. +- Use `mode: section_prepend` to insert content at the start of an existing markdown section. +- Provide `section_heading` with the target heading text (example: `Tasks`). +- The section must already exist; the handler does not create a new heading. +- If the heading does not exist (or appears more than once), the operation fails with validation guidance. + +## Section-Intent Policy (Mandatory) + +For any user request that implies "add content inside an existing section" (tasks, decisions, action items, notes, wins, blockers, etc.), follow this decision flow: + +1. **Read first (mandatory)** + - Call `note.read` for the target note before writing. + - Identify whether the target heading exists and whether it is unique. + +2. **If heading exists exactly once** + - Use `note.update` with `mode: section_append` (default) or `section_prepend`. + - Pass explicit `section_heading`. + +3. **If heading does not exist** + - Do **not** silently fallback to `append`/`prepend`. + - Ask one focused question to confirm creating a new section heading. + - After explicit confirmation, use read-then-replace to add the new heading in the right location. + +4. **If heading exists more than once** + - Do **not** guess. + - Ask which heading instance should receive the content. + +5. **Never create duplicate headings by default** + - For section-intent requests, raw `append`/`prepend` is disallowed unless the user explicitly asks for free-form insertion. + +Default interpretation examples: +- "crie X tarefas" in a daily note -> section intent targeting `Tasks` +- "adicione em decisões" -> section intent targeting `Decisões` +- "anota isso no final da nota" -> free-form append (not section intent) + ## Frontmatter Safety (Auto-Preserve) When using `mode: replace` with content that lacks a YAML frontmatter block, the gateway automatically preserves the existing frontmatter. This prevents the most common read-then-replace mistake: the LLM reads a note with `note.read`, edits the body only, and forgets to include the frontmatter in the replacement text. @@ -83,14 +120,14 @@ Current deterministic handler behavior: - Resolves note by relative path. - Enforces `.md` target and existence. - Applies `append`, `prepend`, or `replace` text operation. +- Supports `section_append` and `section_prepend` to edit a single existing section without creating duplicate headings. - Supports `frontmatter` mode for surgical YAML field updates. - Auto-preserves existing frontmatter on `replace` when replacement text has no YAML block. - Refreshes structural context files after write: updates `Meta/vault-structure.md` managed block and updates folder `_index.md` managed summary. - Logs operation in `Meta/vault-gateway-log.md`. Current handler does not: -- Parse or merge frontmatter fields. -- Rebuild sections semantically. +- Rebuild sections semantically beyond deterministic section insertion. - Auto-fix links, tags, or MOC references. Those richer MBIFC refinements should happen in additional turns when needed.