Skip to content

feat(keycloak-docker): complete Path C + Layer 2 migration (v4.1.1.3) - #46

Closed
makeroftools wants to merge 1 commit into
mainfrom
fix/mot-keycloak-docker-migration
Closed

feat(keycloak-docker): complete Path C + Layer 2 migration (v4.1.1.3)#46
makeroftools wants to merge 1 commit into
mainfrom
fix/mot-keycloak-docker-migration

Conversation

@makeroftools

Copy link
Copy Markdown
Contributor

Summary

Complete migration of keycloak-docker template to match the anythingllm-docker reference implementation, implementing all 11 steps of the Path C + Layer 2 bootstrap pattern.

Changes

Layer 1: DO Spaces Remote State Backend (Fixed)

  • Fixed backend.tf.jinja to not use var.spaces_* (backend can't reference vars)
  • Added init.sh.jinja to pass credentials via -backend-config flags
  • Added spaces_access_key, spaces_secret_key, spaces_encryption_key variables

Layer 2: Bootstrap-Secret Rotation

  • Embedded rotate-bootstrap-secret.sh in cloud-init
  • Automatic v1 → v2 Machine Identity secret rotation on first boot
  • v1 invalidated within minutes of provisioning

Path C: Thin Cloud-Init + Ansible

  • Slimmed cloud-init to first-boot bootstrap only
  • Rewrote deploy.yml.jinja to own all app-layer state (compose, Caddyfile, backup, cron)
  • Added pre-tasks for bootstrap verification
  • Added DO droplet tagging (skinny-backup, commit-sha)
  • Rewrote deploy.sh as thin ansible-playbook wrapper

Terraform Alignment

  • Added ignore_changes = [tags] to droplet lifecycle
  • Slimmed user_data templatefile vars (only project_name + 4 Infisical vars)
  • Added ssh_source_cidrs variable for SSH firewall restriction
  • Switched monitoring from Jinja conditionals to count = pattern

Infisical CLI Upgrade

  • Switched from legacy install-cli.sh (capped at v0.38) to current artifacts-cli.infisical.com apt repo
  • Replaced infisical-auth.sh with .infisical-auth.env (0600 permissions)

Backup Script Upgrade

  • Remote mode now invokes droplet-local backup.sh inside infisical run (heredoc pattern)
  • Avoids shell quoting issues with docker ps --format directives

Configuration Updates

  • Renamed .gitignore.gitignore.jinja
  • Un-ignored .terraform.lock.hcl (commit for reproducibility)

Documentation

  • Updated README migration status table (all 4 layers now Done/Current)
  • Added comprehensive CHANGELOG with all 11 migration steps
  • Bumped #WeOwnVer to v4.1.1.3 (2026-06-05)

Compliance

  • All pre-commit hooks pass (gitleaks, yamllint, markdownlint, etc.)
  • No secrets committed
  • Branch naming follows GitHub Flow (fix/*)
  • CHANGELOG updated with #WeOwnVer
  • README migration status table updated

Testing

  • Pre-commit hooks validated on all 7 commits
  • Template syntax verified via copier
  • Terraform/OpenTofu syntax valid
  • Docker Compose syntax valid
  • Ansible playbook syntax valid

Migration Impact

Breaking changes:

  • deploy.sh now requires INFISICAL_PROJECT_ID env var
  • Cloud-init no longer starts the stack (ansible handles it)
  • backend.tf no longer uses var references (use init.sh instead)

New requirements:

  • INFISICAL_PROJECT_ID env var required when running deploy.sh
  • community.docker Ansible collection auto-installed by deploy.sh
  • Run ./init.sh before tofu plan to configure backend credentials

Related

  • Reference implementation: anythingllm-docker template
  • Bootstrap pattern: docs/INFRA_BOOTSTRAP_PATTERN.md
  • Version: #WeOwnVer v4.1.1.3

Copilot AI review requested due to automatic review settings June 5, 2026 18:29
@makeroftools
makeroftools requested a review from ncimino as a code owner June 5, 2026 18:29
@makeroftools

Copy link
Copy Markdown
Contributor Author

@github/copilot review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR completes the keycloak-docker template migration toward the Path C + Layer 2 bootstrap pattern used by anythingllm-docker, shifting app-layer reconciliation to Ansible and introducing first-boot bootstrap-secret rotation for the Infisical Machine Identity.

Changes:

  • Adds Layer 1 remote-state alignment for DO Spaces (backend no-vars + init.sh forwarding credentials).
  • Refactors cloud-init to a thin first-boot bootstrap (Docker + Infisical CLI + auth file + bootstrap-secret rotation), moving compose/Caddy/backup/cron responsibilities into a new Ansible playbook.
  • Adds SSH firewall CIDR restriction and updates monitoring resources to use count instead of Jinja conditionals.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
keycloak-docker/template/terraform/variables.tf.jinja Adds SSH CIDR allowlist variable and backend credential variables.
keycloak-docker/template/terraform/templates/cloud-init.yaml.jinja Slims cloud-init to bootstrap-only and embeds Layer 2 bootstrap-secret rotation logic.
keycloak-docker/template/terraform/monitoring.tf.jinja Switches monitoring resources to count gating.
keycloak-docker/template/terraform/main.tf.jinja Slims user_data inputs, ignores tag drift, and applies SSH CIDR variable to firewall rule.
keycloak-docker/template/terraform/init.sh.jinja Adds backend init helper to pass Spaces credentials via -backend-config.
keycloak-docker/template/terraform/backend.tf.jinja Removes variable references from backend block and adds S3-compatible backend flags for DO Spaces.
keycloak-docker/template/scripts/deploy.sh.jinja Rewrites deploy wrapper to call the new Ansible playbook and install required collections.
keycloak-docker/template/scripts/backup.sh.jinja Updates remote backup execution to run droplet-local backup under infisical run via heredoc.
keycloak-docker/template/CHANGELOG.md.jinja Adds a detailed migration entry and bumps #WeOwnVer to v4.1.1.3.
keycloak-docker/template/ansible/deploy.yml.jinja Introduces new Path C app-layer deployment playbook (compose/Caddy/backup/cron/tagging/health).
keycloak-docker/template/.gitignore.jinja Stops ignoring .terraform.lock.hcl to improve reproducibility.
keycloak-docker/README.md Updates the migration status table to reflect the completed layers/pattern.

# `tojson` emits a valid JSON array (double-quoted strings) which HCL parses
# as a list. Without it, Copier renders Python's list-repr ('a', 'b') and
# `tofu plan` fails with "Invalid character" on the single quotes.
default = {{ ssh_source_cidrs | tojson }}
Comment on lines +115 to +119
source {{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
Comment on lines +166 to +170
source {{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
Comment on lines +228 to +232
source {{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
Comment on lines +160 to +164
source "/opt/$PROJECT_NAME/.infisical-auth.env"
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
Comment on lines +66 to +78
- name: Verify Infisical auth file exists (created by cloud-init)
ansible.builtin.stat:
path: "{{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env"
register: auth_file

- name: Fail clearly if auth file is missing
ansible.builtin.fail:
msg: >-
{{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env is missing —
Infisical Machine Identity bootstrap failed. Check
/var/log/cloud-init-output.log and
/var/log/{{ project_name | replace('-', '') }}-rotation.log.
when: not auth_file.stat.exists
Comment on lines 26 to 29
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
REMOTE="${1:-}"
APP_DIR="/opt/{{ project_name | replace('-', '') }}"
PLAYBOOK="$PROJECT_DIR/ansible/deploy.yml"

Comment on lines 270 to 275
# Caddy log dir: the hardened (non-root) caddy image writes its access log to
# the /var/log/caddy host bind mount and crash-loops with "permission denied"
# if the dir does not exist writable. 0777 is fine for a single-tenant log dir
# (otel-agent also reads these files).
- [mkdir, -p, /var/log/caddy]
- [chmod, "0777", /var/log/caddy]
Copilot AI review requested due to automatic review settings June 5, 2026 18:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.

Comment on lines +51 to +58
variable "ssh_source_cidrs" {
description = "CIDR list allowed to reach port 22 — PRODUCTION: restrict to admin IP/32 or VPN range"
type = list(string)
# `tojson` emits a valid JSON array (double-quoted strings) which HCL parses
# as a list. Without it, Copier renders Python's list-repr ('a', 'b') and
# `tofu plan` fails with "Invalid character" on the single quotes.
default = {{ ssh_source_cidrs | tojson }}
}
Comment on lines +26 to +33
get_tfvar() {
# Anchor both ends of the key so e.g. `spaces_access_key` doesn't also
# match `spaces_access_key_v2 = ...` if a future config grows variations.
local var_name="$1"
grep -E "^${var_name}[[:space:]]*=" terraform.tfvars \
| sed 's/.*= *"\(.*\)"/\1/' \
| tr -d ' '
}
Comment on lines +161 to +166
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
exec infisical run --projectId="$INFISICAL_PROJECT_ID" --env="$INFISICAL_ENV" \
-- "/opt/$PROJECT_NAME/backup.sh"
Comment on lines +110 to +124
#!/bin/bash
# Daily backup with Infisical runtime secret injection. Managed by
# {{ project_name }}/ansible/deploy.yml — edit there, not here.
set -euo pipefail
# shellcheck disable=SC1091
source {{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
infisical run \
--projectId={{ '{{' }} infisical_project_id {{ '}}' }} \
--env={{ '{{' }} infisical_env {{ '}}' }} \
-- {{ '{{' }} app_dir {{ '}}' }}/backup.sh \
>> /var/log/{{ project_name | replace('-', '') }}-backup.log 2>&1
Comment on lines +164 to +171
set -euo pipefail
# shellcheck disable=SC1091
source {{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
cd {{ '{{' }} app_dir {{ '}}' }}
Comment on lines +226 to +233
set -euo pipefail
# shellcheck disable=SC1091
source {{ '{{' }} app_dir {{ '}}' }}/.infisical-auth.env
infisical login --method=universal-auth \
--clientId="$INFISICAL_CLIENT_ID" \
--clientSecret="$INFISICAL_CLIENT_SECRET" \
--silent
cd {{ '{{' }} app_dir {{ '}}' }}
Complete migration of keycloak-docker template to match anythingllm-docker
reference implementation with all 11 security improvements:

Layer 1: DO Spaces remote state backend with init.sh
Layer 2: Bootstrap-secret rotation (v1 → v2 auto-rotation)
Path C: Thin cloud-init + ansible playbook architecture
- Slimmed cloud-init to bootstrap-only (removed app-layer content)
- Ansible playbook owns all post-bootstrap state
- deploy.sh rewritten as thin ansible-playbook wrapper

Security improvements:
- .infisical-auth.env with 0600 permissions (not .sh)
- artifacts-cli.infisical.com apt repo (not legacy install-cli.sh)
- .bootstrap-complete marker for ansible pre-flight verification
- ignore_changes = [user_data, tags] to prevent tag drift
- ssh_source_cidrs variable for SSH firewall restriction
- Monitoring using count = pattern (not Jinja conditionals)
- deploy.sh requires INFISICAL_PROJECT_ID env var
- Ansible playbook has pre_tasks for bootstrap verification
- Backup script uses infisical run for remote mode
- Grandfather-father-son retention policy

Path consistency: replace('-', '') across all files (matches original template)

#WeOwnVer: v4.1.1.3 (2026-06-05)
@makeroftools
makeroftools force-pushed the fix/mot-keycloak-docker-migration branch from 4335e42 to f0ac137 Compare June 5, 2026 19:34
@makeroftools

Copy link
Copy Markdown
Contributor Author

Closing to consolidate. This change is part of a larger batched PR that combines all site.conf implementations across docker templates. Will resubmit as a single consolidated PR for easier review.

@makeroftools
makeroftools deleted the fix/mot-keycloak-docker-migration branch June 6, 2026 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants